High‑Quality Item Writing Matters

Designing True/False Questions That Assess Knowledge, Not Test& 8209;Wiseness A question bank is only as good as its questions. Even with excellent explanations, analytics, and adaptive learning, poorly written items undermine learning and assessment validity . Bad questions do not measure physics understanding – they reward guessing strategies, pattern recognition, or test& 8209;wiseness. For FRCR Part 1 Physics, which uses True/False items , item quality is especially critical. Binary formats magnify flaws: ambiguity, absolutes, and poorly framed statements can distort both learning and performance. High& 8209;quality item writing is therefore not a cosmetic concern – it is a core educational design requirement . Why Item Quality Matters The purpose of an exam& 8209;aligned question bank is to sample a trainee’s underlying knowledge accurately. When items are poorly constructed, they introduce construct& 8209;irrelevant variance – performance differences driven by factors other than the intended construct. This problem has been extensively documented in health professions education. The Gold Standard: NBME Item Writing Guidance The NBME Item Writing Guide 6th edition  is widely regarded as the gold standard for health& 8209;profession assessment design. Its core message is simple: most item flaws are preventable , and most damage validity. Common violations include: ·           ambiguous wording ·           unfocused stems ·           grammatical cues ·           testing trivia instead of concepts While the NBME guidance is often discussed in the context of MCQs, its principles apply directly – and arguably more stringently – to True/False formats , where every word carries weight. Guide, NBME Item Writing. Constructing written test questions for the health sciences . 2021. High& 8209;quality items ensure that a correct answer reflects knowledge of physics , not recognition of exam tricks. Best Practices in Health& 8209;Professional Question Writing These principles are reinforced in discipline& 8209;specific research. In a review of best practices, Rudolph et& 8239;al. 2019  emphasise that poorly written items shift assessment away from knowledge and toward cue utilisation and elimination strategies . Learners who are “good at exams” but weak in content may outperform stronger candidates simply because of item flaws. The authors stress that clarity, focus, and plausibility  are not stylistic preferences – they are essential to fairness and educational value. Rudolph, Michael J., Kimberly K. Daugherty, Mary Elizabeth Ray, Veronica P. Shuford, Lisa Lebovitz, and Margarita V. DiVall. "Best practices related to examination item construction and post-hoc review." American journal of pharmaceutical education  83, no. 7 2019 : 7204. In formative contexts such as question banks, these flaws also distort learning by reinforcing incorrect heuristics. Threats to Validity From Poor Items The consequences of bad item writing are well described by Steven Downing 2005 , whose work on assessment validity remains foundational. Downing identifies item writing flaws  as a major threat to validity in MCQ& 8209;based examinations. These flaws: ·           reduce reliability ·           inflate or deflate scores unpredictably ·           weaken the link between performance and competence Critically, Downing argues that validity threats are often systematic rather than random , meaning that some learners are consistently advantaged or disadvantaged by poor design. Downing, Steven M. "The effects of violating standard item writing principles on tests and students: the consequences of using flawed test items on achievement examinations in medical education." Advances in health sciences education  10, no. 2 2005 : 133-143. When revising for high& 8209;stakes exams like FRCR Part 1, the questions you practice with must be valid – otherwise your performance in practice won’t match your exam results. Why True/False Items Demand Extra Care True/False questions are often dismissed as “simple,” but this is misleading. In reality, they demand: ·           precise language ·           absolute correctness ·           careful handling of qualifiers ·           clear boundaries between true and false Poorly written True/False items encourage: ·           guessing based on wording style ·           avoidance of absolutes “always,” “never” rather than physics reasoning ·           memorisation of examiner habits rather than concepts High& 8209;quality True/False design, by contrast, rewards accurate conceptual discrimination , which is exactly what FRCR Physics aims to assess. Design Principle The evidence leads to a clear principle: High& 8209;quality items assess knowledge; poor items assess test& 8209;wiseness. For an FRCR Part 1 Physics question bank, this means: ·           Each statement tests one clear, meaningful concept ·           Language is precise, unambiguous, and exam& 8209;authentic ·           True and false statements are equally plausible on first reading ·           Errors reflect genuine misconceptions, not wording traps ·           Item difficulty arises from physics understanding, not linguistic tricks When item quality is high, learners can trust that getting a question wrong means something – and fixing it matters . Our Approach Our FRCR Part 1 Physics question bank follows established item& 8209;writing standards , informed by NBME guidance and validity research. Every True/False item is reviewed not just for accuracy, but for clarity, fairness, and educational signal . Because practice questions should teach physics – Not how to game the exam.