Computerized adaptive testing (CAT)

What is the alternative to the hitherto known tools for assessing candidates in selection?

The technological progress of society and the development of digital solutions have affected society at all levels. Changes are manifested in various spheres, from some common habits and behaviours to working conditions and business. Given the changes in working conditions, more and more technological advances are also affecting the field of human resource management.

In addition to the usual challenges of finding the most capable candidates and assessing their characteristics, the area of human resources has been hitherto unseen. Like all industries in the world, COVID-19 has imposed the need for a complete reorganization of work. Accordingly, the testing of potential candidates as we have known them so far has been forgotten. New technologies have introduced innovations such as computerized adaptive testing.

When it comes to abilities and assessments in an organizational context, the predictive value of cognitive abilities is also of great importance (Schmidt & Hunter, 1998). Intelligence tests are an unavoidable aspect of a quality selection process and can be a key factor for various organizational decisions. Therefore, due to the availability and ease of application, digital psychological tests have become a necessary tool for quality assessment. Also, automatic scoring and interpretation of results reduces time and necessary human resources which is why we cannot give them up. But, it was necessary to create new instruments in digital form that would slowly replace traditional paper-pencil tests.

It is not new that in line with the development of technology, digital tests are becoming more and more sophisticated. But the complete revolution in digital testing has been brought by computerized adaptive testing.  If you have not heard of this test yet, you have missed a lot. Computerized adaptive testing or abbreviated CAT is a sophisticated method of measuring a certain ability, which is based on the idea of adapting the test to each individual. Due to the optimization of the test, the time required to solve the test is also reduced. This makes the job much easier for the candidates, in terms of time and energy.

What is computerized adaptive testing and how does it work?

How is it possible that adaptive testing is individually optimized and still has all the metric characteristics of the test met? Adaptive testing involves applying only the most appropriate tasks to each individual by presenting the next one, depending on the answer a person offers on one task. The difficulty of the tasks that a person solves is constantly adjusted during the test to the development of his abilities.

The test begins with an averagely difficult task, after which the branching starts according to the results of the individual. For example, if the first task is solved incorrectly, the next offered task will be from the category of slightly easier tasks. Depending on the next answer, branching will continue. By the same logic, if a person solves another task wrong, he or she will get an even easier task. If the second task is correct, then the candidate is given a slightly more difficult task until his ability is determined. As a rule, most candidates should complete the test through about 15 tasks.

This method allows accurate computer estimation of one’s level of ability, using a significantly smaller number of tasks than is the case with classical tests. Of course, the very development of a computerized adaptive test requires a whole range of psychometric analyses to create a smaller group of clearly defined tasks, selection and presentation algorithms.

Therefore, it is not possible to create this test in paper form. In collaboration with a professor at the Faculty of Philosophy, University of Zagreb and the company Selekcija d.o.o. started the project of developing the first Croatian computerized adaptive test. The joint work developed from a detailed analysis of tasks that have been applied for years in the selection of staff and the work of the Selekcija.hr team to the development of the final database of adaptive test tasks and the algorithm for their application.

By creating a digital testing tool, the selection process has accelerated, but it has also become more economical than it was before. Thus, adaptive testing imposed new trends in assessment, but also provided insight into the bright future of the selection process.

 

Advantages of adaptive testing in candidate selection

When we look back to the beginnings of computer-based testing of cognitive abilities, we can notice that in the not-so-distant past, it was tested with very simple tests. Due to the unavailability of computers, the tests had to be applied on paper. Today, almost every household owns a computer. The adaptive test on Selekcija.hr platform can even be solved on a mobile phone, so there is almost no excuse for candidates about not being able to solve the test.

Furthermore, what we often see in selection is the presence of a candidate discouragement effect when solving aptitude tests on paper. Since they are unfamiliar with standardizing test, respondents may feel discouraged and frightened by solving more difficult tasks. Then the effect of test anxiety is more pronounced. On the other hand, for some candidates, the same tests may be too easy and they may sometimes lose initial interest in solving the test.

In this way, the final result of the examinee on the test depends on the level of difficulty of the question to which he answered correctly. Thus different respondents may have the same number of correct answers, but those who answered the more difficult questions correctly ultimately have a better result. An additional advantage of adaptive testing lies in the reduced ability to rewrite or transmit test information because it generates a much larger amount of tasks that can be different each time.

The progress provided by adaptive testing benefits both, the employer and the candidate. Namely, we often encounter the problem that candidates are employed and cannot attend testing during working hours or are therefore generally unable to participate in a lengthy process. We are also sometimes inclined to test a larger number of candidates in order to select the best one. With adaptive testing, the time frame is greatly reduced and provides more time to get to know the candidate and his or her behavioural characteristics in more detail.

 

Conclusion

Ultimately, we can notice that the COVID-19 has shaken all the industries, but the challenges have forced us to make progress, especially in business and candidate selection. We can only imagine in which direction CAT technology will develop, but we can agree that the quality of testing has increased rapidly in the age of digitalization and adaptive testing. Given the current advances in testing, general digitization and its benefits, we’ll hardly ever go back to paper-pencil tests again.