What is Computerized Adaptive Testing (CAT)?

Let’s start by defining what Computerized Adaptive Testing (CAT) is. CAT is a form of computerized testing that adapts based on the answers provided by the test-taker. The algorithm commonly starts with a medium or medium-easy difficulty level as a first question. Based on the examinee’s first answer or first set of answers, it will then adjust the difficulty of the questions that follow.

The main purpose of CAT is to maximize the precision of the test. This method of testing also requires less testing items while at the same time having the same or better level of accuracy as traditional multiple-choice assessments.

Pros

1. CAT is on average 50% shorter than traditional fixed versions, while at the same time maintaining a high accuracy level.
2. Because Computerized Adaptive Testing is typically shorter, there’s a time-saving component that benefits the examinee.
3. For the organization conducting the testing, shorter tests translate to savings in time and money by reducing seat times in half.
4. Computerized Adaptive Testing (CAT) provides uniformly accurate scores for most test-takers.
5. Test results are immediate.

 

Cons

1. The development of Computerized Adaptive Testing (CAT) can sometimes be costly and benefits mostly industries with large populations such as companies in scientific or research-based fields.
2. CAT does not apply to every type of situation. For example, CAT is not ideal for a small sample group or real-time items.
3. For the algorithm to be able to select the right type of questions for the test-taker, all the questions first need to go through a “pre-testing” phase. This phase requires using a fairly large test group of up to 1,000 test-takers.
4. In a Computerized Adaptive Test, the ability to review past items during testing is generally disabled. This is due to a vulnerability where a test-taker could purposely answer the first questions wrong to potentially receive an easier test and then go back to correct the initial answers.
5. Another disadvantage is that some of the test items in CAT could become overexposed over time. At the same time, other items may never be used at all.

To conclude, when trying to decide whether CAT is right for your organization, it is best to consider consulting with a firm that has years of experience in the field. Not only could this help you make better choices but it could also save time and money by minimizing mistakes.

 

Interested in discussing how A Pass can help with creating your next assessment?