心理学报Issue(9)：1188-1198,11.DOI:10.3724/SP.J.1041.2015.01188
允许CAT题目检查的区块题目袋方法
The Block Item Pocket Method to Allow Item Review in CAT
摘要
Abstract
Most computerized adaptive testing (CAT) do not allow examinees to review items because it will drastically decrease measurement precision and bring about extra cheating strategies (Wainer, 1993; Wise, 1996). Allowing item review is essential to make CAT comparable with traditional tests. It also matters in application. Item review enables examinees to correct mistakes due to carelessness, which can further improve the precision of ability estimation. No such option may cause some negative consequences for their overall performance especially in high-stake examinations, such as tension or anxiety (Vispoel, Henderickson, & Bleiler, 2000). Therefore, it is worth trying if allowing item review could alleviate problems mentioned at the beginning (Wise, 1996; Vispoel, 2000, 2005). Several methods have been proposed, including the successive block method (Stocking, 1997) and the item pocket (IP) method (Han, 2013). However, both methods are limited in some ways. Stocking’s method does not allow examinees to skip items and requires a large number of blocks which may bring about some extra adverse effects because of frequent decision to go to next block. Han’s method can avoid limitations of Stocking’s. But it requires an appropriate IP size and may result in high bias in large IP size situation. The present study proposed the block item pocket (BIP) method which sets fewer but larger blocks with a proper total IP size. This method keeps advantages of Stocking’s and Han’s and overcomes their disadvantages. Two simulation studies of two response strategies were conducted to evaluate validity of the BIP method. Item parameters were randomly drawn from uniform distribution (b~ U (-3, 3)) and (α ~ U (0, 2)). Each examinee was administered a fixed-length CAT with 30 items. The initial item for each examinee was randomly drawn from θ ~ U (-0.5, 0.5). For the CAT administration, the Maximum Fisher Information method was adopted to select items. The interim and final scores were estimated using MLE method in most conditions. When responses were less than 5 or when all answers were correct or wrong, EAP method was adopted. Each study contained five conditions: non-review, 1 blocks IP method, 2 blocks, 3 blocks and 6 blocks BIP method. Statistics like BIAS, MAE, and RMSE were used as evaluation criteria. Results indicated that: (1) BIP method had better estimate precision than IP method at low ability level under normal strategy; (2) When dealing with Wainer-like strategy, BIP method was far more precise than item pocket method at all ability levels; (3) As the number of blocks increased, estimate precision got closer to non-review condition. Advantages of this new method and future directions were discussed.关键词
计算机化自适应测验/题目检查/题目袋/题目修改/区块题目袋Key words
computerized adaptive testing/item review/item pocket method/answer change/block item pocket method分类
社会科学引用本文复制引用
林喆,陈平,辛涛..允许CAT题目检查的区块题目袋方法[J].心理学报,2015,(9):1188-1198,11.基金项目
国家自然科学基金面上项目(31371047) （31371047）
国家自然科学基金青年科学基金项目(31300862) （31300862）
高等学校博士学科点专项科研基金项目(20130003120002) （20130003120002）
中央高校基本科研业务费专项资金资助(2013YB26)。 （2013YB26）
