Abstract：
OBJECTIVE: To determine the impact of item-writing flaws and cognitive level on student performance metrics in 1 course series across 2 semesters at a single institution.
METHODS: Four investigators reviewed 928 multiple-choice items from an integrated therapeutics course series. Differences in performance metrics were examined between flawed and standard items, flawed stems and flawed answer choices, and cognitive levels.
RESULTS: Reviewers found that 80% of the items were flawed, with the most common types being implausible distractors and unfocused stems. Flawed items were generally easier than standard ones, but the type of flaw significantly impacted the difficulty. Items with flawed stems had the same difficulty as standard items; however, those with flawed answer choices were significantly easier. Most items tested lower-level skills and have more flaws than higher-level items. There was no significant difference in difficulty between lower- and higher-level cognitive items, and higher-level items were more likely to have answer flaws than item flaws.
CONCLUSIONS: Item-writing flaws differently impact student performance. Implausible distractors artificially lower the difficulty of questions, even those designed to assess higher-level skills. This effect contributes to a lack of significant difference in difficulty between higher- and lower-level items. Unfocused stems, on the other hand, likely increase confusion and hinder performance, regardless of the question\'s cognitive complexity.
METHODS: Four investigators reviewed 928 multiple-choice items from an integrated therapeutics course series. Differences in performance metrics were examined between flawed and standard items, flawed stems and flawed answer choices, and cognitive levels.
RESULTS: Reviewers found that 80% of the items were flawed, with the most common types being implausible distractors and unfocused stems. Flawed items were generally easier than standard ones, but the type of flaw significantly impacted the difficulty. Items with flawed stems had the same difficulty as standard items; however, those with flawed answer choices were significantly easier. Most items tested lower-level skills and have more flaws than higher-level items. There was no significant difference in difficulty between lower- and higher-level cognitive items, and higher-level items were more likely to have answer flaws than item flaws.
CONCLUSIONS: Item-writing flaws differently impact student performance. Implausible distractors artificially lower the difficulty of questions, even those designed to assess higher-level skills. This effect contributes to a lack of significant difference in difficulty between higher- and lower-level items. Unfocused stems, on the other hand, likely increase confusion and hinder performance, regardless of the question\'s cognitive complexity.
摘要：
目的：确定在一个机构的两个学期中，一个课程系列中的项目编写缺陷和认知水平对学生表现指标的影响。
方法：四位研究者回顾了综合治疗课程系列的928项多项选择项目。检查了有缺陷的项目和标准项目之间的绩效指标差异，有缺陷的茎和有缺陷的答案选择，和认知水平。
结果：审核员发现80%的项目有缺陷，最常见的类型是令人难以置信的干扰物和不集中的茎。有缺陷的物品比标准的容易，项目变得更容易，因为他们积累了更多的缺陷。这些缺陷使表现较低的学生比表现较高的学生受益更多，特别是两个缺点。茎有缺陷的项目与标准项目具有相同的难度，但是那些有错误答案的人要容易得多。大多数项目测试较低级别的技能，并且比高级别的项目有更多的缺陷。低级和高级认知项目之间的难度没有显着差异，较高级别的项目比茎缺陷更有可能具有答案缺陷。
结论：项目写作缺陷对学生表现的影响不同。令人难以置信的干扰因素人为地降低了问题的难度，即使是那些旨在评估高级技能的人。这种影响导致较高和较低级别项目之间的难度缺乏显着差异。不集中的茎，另一方面,无论问题的认知复杂性如何，都可能增加混乱并阻碍表现。
方法：四位研究者回顾了综合治疗课程系列的928项多项选择项目。检查了有缺陷的项目和标准项目之间的绩效指标差异，有缺陷的茎和有缺陷的答案选择，和认知水平。
结果：审核员发现80%的项目有缺陷，最常见的类型是令人难以置信的干扰物和不集中的茎。有缺陷的物品比标准的容易，项目变得更容易，因为他们积累了更多的缺陷。这些缺陷使表现较低的学生比表现较高的学生受益更多，特别是两个缺点。茎有缺陷的项目与标准项目具有相同的难度，但是那些有错误答案的人要容易得多。大多数项目测试较低级别的技能，并且比高级别的项目有更多的缺陷。低级和高级认知项目之间的难度没有显着差异，较高级别的项目比茎缺陷更有可能具有答案缺陷。
结论：项目写作缺陷对学生表现的影响不同。令人难以置信的干扰因素人为地降低了问题的难度，即使是那些旨在评估高级技能的人。这种影响导致较高和较低级别项目之间的难度缺乏显着差异。不集中的茎，另一方面,无论问题的认知复杂性如何，都可能增加混乱并阻碍表现。
