MCQ CONSTRUCTION & DESIGN Dr. Nadia Al Wardy

MCQ CONSTRUCTION & DESIGN Dr. Nadia Al Wardy

MCQ CONSTRUCTION & DESIGN Dr. Nadia Al Wardy College of Medicine & Health Sciences Medical Education Unit College of Nursing Workshop March 2014 A Simple Model of Competence 2 Does Performance or hands on assessment

Shows how Knows how Knows Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. Written, Oral or Computer based assessme nt 2 3 Does Does Shows

how Shows how Knows how Knows how Knows Knows Performance assessment in vivo: Masked SPs, Video, Audits.. Performance assessment in vitro: OSCE, SP-based test.. (Clinical) Context based tests: MCQ, essay type, oral.. Factual tests: MCQ, essay type, oral.. 3 Instruments of Written Assessment

4 Open-ended question types Modified Essay Questions (MEQs) Short-answer questions (SAQs) . Multiple-choice question types

Single-best answer True/false Extended-matching items (EMIs) Key features . Multiple-Choice Question Formats A-type 5 One-best-answer Questions Multiple-choice Questions R-type

Extended matching Standard True-false-type True-false Questions K-type Complex True False Chapter 2 6 Multiple-Choice Item Formats Page 13 From Case & Swanson: Constructing Written Test Questions f Basic and Clinical Sciences http://www.NBME.ORG/about/itemwriting.asp MULTIPLE-CHOICE

QUESTIONS A-Type (Single Best Answer) Components of an A-type Question 8 A 35-year old man has had a stomach ache all afternoon. He ate the following lunch: two big McDonalds hamburgers, an ice cream shake, and large fries. What is the most likely diagnosis? A. Abdominal aneurysm B. Appendicitis C. Bowel obstruction D. Cholecystitis E. Colon cancer F. Pancreatitis G. Too much lunch

Vignette, Scenario, Or Stem Lead-in Options A, B, C, D, E & F a distractors Option G is the key Sample Good Question & Cover The Options Rule 9 A 32-year old man has a 4-day history of progressive weakness in his extremities. He has been healthy except for an upper respiratory tract infection 10 days ago. His temperature is 100 F, BP 130/80, pulse 94,

respirations 42 and shallow. He has symmetric weakness of both sides of the face and the proximal and distal muscles of the extremities. Sensation is intact. No deep tendon reflexes can be elicited; the plantar responses are flexor. What is the most likely diagnosis? A. Acute disseminated encephalomyelitis Diagram of Rank-Ordered Options for a Good One-Best-Answer Item 10 D C

Least Correct Answer A E B Most Correct Answer Note: Options must be homogeneous (eg, all diagnoses, all muscles). Sample Good Question & Cover The Options Rule 11

A 40 year old man is admitted to the Emergency Room with renal calculi and is complaining of moderate to severe flank pain and nausea. His temperature is 38.20C. What is the priority nursing goal for this patient? A. maintain fluid and electrolyte balance B. Control nausea C. Manage pain D. Prevent urinary tract infection Sample Flawed Question & Cover The Options Rule 12 Which of the following is true about pseudogout? A. It occurs frequently in women. B. It is seldom associated with acute pain in a joint.

C. It may be associated with a finding of chondrocalcinosis. D. It is clearly hereditary in most cases. E. It responds well to treatment with allopurinol. Sample Flawed One-Best-Answer Item 13 Which of the following is true about pseudogout? A. It occurs frequently in women. B. It is seldom associated with acute pain in a joint. C. It may be associated with chondrocalcinosis. Gendercases. A D. It is clearly hereditary in most E. It responds well to treatment with

allopurinol. D Inheritance Rx E B C Associations False True The #1 Rule For One-Best-Answer Questions 14 A 24-year old man comes to the

urgent care unit.. What is the most likely cause of his pain? A.cholecystitis B.diverticulitis Cover The Options Rule C.kidney stone D. pancreatitis E. peptic ulcer disease 15 If you want your students to go beyond memorizing facts, your

tests must require more Two Rules 16 1. 2. Ensure that each question satisfies the cover the options rule. Assess examinee ability to apply their knowledge rather than simply their ability to recall isolated facts. Item Assessing Recall of an Isolated Fact

17 What arterial blood gas analysis is most likely to show in patients with cardiogenic shock? A. B. C. D. E. Hypoxemia with normal pH Metabolic acidosis Metabolic alkalosis Respiratory acidosis Respiratory alkalosis 18 Alternate Item Assessing Application of Knowledge:

Diagnosis A 74-year-old woman is brought to the emergency department because of crushing chest pain. She is restless, confused, and diaphoretic. On admission, temperature is 36.7 C, blood pressure is 148/78 mm Hg, pulse is 90/min, and resp are 24/min. During the next hour, she becomes increasingly stuporous, blood pressure decreases to 80/40 mm Hg, pulse increases to 120/min, and respirations increase to 40/min. Her skin is cool and clammy. An ECG shows sinus rhythm 19 Alternate Item Assessing Application of Knowledge: Diagnosis A 74-year-old woman is brought to the emergency department because of crushing

chest pain. She is restless, confused, and diaphoretic.. Arterial blood gas analysis is most likely to show A. Hypoxemia with normal pH B. Metabolic acidosis C. Metabolic alkalosis D. Respiratory acidosis 20 Sample MCQ Items: Isolated Fact vs Case-Based Question What is the most likely renal abnormality in children with nephrotic syndrome and normal renal function? OR A 2-year-old boy developed swelling of his eyes and

ankles over the past week. BP 100/60; pulse 110; R 28. PE shows eyelid swelling; 2+ pitting edema of his ankles; abdominal distension with positive fluid wave. Serum: creatinine 0.4; albumin 1.4; cholesterol 569 mg/dL. Urinalysis: 4+ protein and How To Develop Better MultipleChoice Questions 21 A 29-yr-old woman has a What are one-day history of (insert the signs description of presenting and symptoms complaint, history, physical exam here). of multiple sclerosis?

What is the most likely diagnosis? How To Develop Better MultipleChoice Questions 22 A 6-yr-old boy has a oneday history of (insert What is the best description of presenting complaint, history, drug to treat physical exam here). Otitis media? What is the most appropriate therapy? 23 How To Develop Better MultipleChoice Questions A 37-yr-old man comes to

the physician for a routine health What immunizations maintenance visit (insert relevant history, physical should be given exam here). to adults? What immunizations should be administered? Structure of a Patient Vignette 24 Age, Gender (eg, A 45-year-old man)

Site of Care emergency dept.) (eg, comes to the Presenting Complaint headache) Duration days) Patient History Physical Findings (eg, because of (eg, that has continued for 2 (with Family History ?) Stages in writing MCQs 25

1. Plan the question 2. Write the question 3. Test the question 1. Plan the question 27 Ensure question deals with an important and useful aspect of the subject

Avoid trivia Include: Common issues Serious conditions

Important or frequent misunderstandings Appropriate to expected level Test knowledge around a central theme 28 Be clear about exactly what fact, concept or generalization is being tested This central theme should be clearly defined in the stem

All options should refer to this theme Decide on the ability you wish to test 29 Recall or recognition Interpretation Problem-solving

Consider how the information should be presented 30 Word description Photograph Radiograph Tracing (e.g. ECG, visual field)

Diagram (e.g. anatomical) Movie film Amplified sound (e.g. heart sounds) 2. Write the Question Use simple and clear language 32

Avoid unfamiliar technical or unusual words Check for ambiguity amongst fellow examiners or students Anxious examinees imagine traps in questions Write stem as a question or incomplete statement 33 If a question, each option must be a

possible answer If an incomplete statement, each response must follow grammatically and logically from the stem; i.e. stem + each option must be a complete statement Avoid stems with just one or two words: Hair: A. growth in man shows the phenomenon of asynchronous cyclic growth.

34 Keep reading time to a minimum Include only material or explanations necessary for answering the question In considering your management of a 28-year-old fashion designer with bleeding from the nose, you would be aware that the most frequent cause of epistaxis at any age is A. 35 Make the responses clear

and concise If a response requires a large number of words to qualify what is meant by it, the questions is probably too complex for an MCQ Include all common elements in the stem The third part of the duodenum A. is anterior to the uncinate process of the pancreas B. is anterior to the common bile duct C. is anterior to the right ureter D. is anterior to the superior mesenteric artery

36 Avoid clues to the correct answer Use of stereotyped or standard phraseology the pituitary gland is the master of the endocrine orchestra Unequal length of alternatives Obvious inconsistencies between stem and a response An intercostal tube may be placed in a patient with a flail chest but

without a pneumothorax or haemothorax to A. remove air B. remove blood C. prevent onset of pulmonary contusion D. prevent development of a tension pneumothorax when Avoid clues to the correct answer [contd] 37 Using opposite of the correct answer as a distractor Paralysis of a vocal cord in a patient with bronchogenic carcinoma is most likely to be: A. bilateral B. on the right side

C. on the left side D. due to metastasis to the larynx E. due to cigarette smoking Use of absolutes (e.g. always, only, never) Adamantinoma of the mandible Avoid clues to the correct answer [contd] 38 Use of synonyms or overlapping alternatives

Use of similar distractors, making the correct response more conspicuous Antidiuretic hormone is synthesized in A. the anterior lobe of the pituitary gland B. the posterior lobe of the pituitary gland C. the pars intermedia of the pituitary gland Use plausible and logical distractors 39 Obvious distractors increase chances of guessing

Derive distractors from common errors made by students Derive distractors from common misconceptions held by students If legitimate distractors cannot be found, recast question in a different format 40 Use eponyms, acronyms and abbreviations only if you expect students to know them

Eponym: Named after a person (e.g. pouch of Douglas; Meckels diverticulum) Acronym: Combination of first letters (e.g. ACTH, CK) Abbreviation: Shorter form of a word (e.g. ht. sd.) Specify units 41

Particularly when using laboratory units, as they can vary from one laboratory to another The normal range for a given laboratory may have to be quoted 42 Indicate opinion or authority if dealing with unestablished facts Indicate whose opinion or what authority the question is based on

According to the passive equilibrium model of Kokko (1975) the main factor responsible for the concentration of urine in the loop of Henle is A. the relative impermeability of the descending limb of the loop of Henle to sodium chloride B. the movement of urea out of the apex of the loop of Henle Minimize negatively phrased questions 43 Avoid double negatives

They confuse examinees If absolutely necessary, emphasize the word(s) conveying a negative aspect by using capitals or underlining Haematemesis is not unexpected in each of the following, except A. B. 44 Beware of unintentionally asking two questions in the same option

Avoid especially if the answer to one is true and to the other is false. Psychological process and gastrointestinal function are A. clearly separable and rarely correlated B. Avoid all of the above and beware of none of the above 45 After partial gastrectomy, gastric emptying is abnormal because A. the pylorus is removed B. there is interference with the pacemaker

C. the gastric remnant is smaller D. all of the above E. none of the above 3. Test the question Get agreement from fellow examiners that: 47 the question is worth asking it is clearly and unambiguously stated

distractors are reasonable and plausible Test the question on students if possible 48 may represent a different interpretation of the question may reveal what the question was actually testing

may have an opinion on importance and validity can indicate their answering strategy may point out errors in the question MULTIPLE-CHOICE QUESTIONS R-Type (Extended-Matching Items, EMIs) Chapter 6

50 Extended Matching Page 69 From Case & Swanson: Constructing Written Test Questions f Basic and Clinical Sciences http://www.NBME.ORG/about/itemwriting.asp Components of an R-type Question 51 Theme: Options: Cardiovascular drugs A. Alpha agonists B. Alpha blockers C. ACE inhibitors D. Beta agonists.

K. Vasoconstrictors Lead-in: For each patient, select the most appropriate drug class Item stems: 1. A 40-year-old man with sudden onset of severe headache, dizziness and vomiting, BP 260/130 mmHg; encephalopathy and grade IV retinopathy 2. A 55-year-old man with mild, chronic hypertension, tachycardia and other symptoms of mild congestive cardiac failure. Previous history of asthma and gout. 52 Writing R-Set Extended Matching Items

1. Decide on the theme of your set (eg, Establishing a diagnosis of patients with abdominal pain) 2. Develop the lead-in for your set 3. Write the options for your set (eg, appendicitis, cholecystitis) 4. Write a patient description for each of the diagnoses in the list 5. Ask a colleague to review your items (patient descriptions) without the correct answers noted Topic: Diagnosis of Patient With Abdominal Pain 53 Lead In: For each patient with abdominal pain, select the most likely diagnosis. 54

Options and Lead-in: Diagnosis A. Abdominal aneurysm K. Kidney stone B. Appendicitis L. Mesenteric adenitis C. Bowel obstruction M. Mesenteric artery thrombosis D. Cholecystitis N. Ovarian cyst - ruptured E. Colon cancer O. Pancreatitis F. Constipation P. Pelvic inflammatory disease G. Diverticulitis Q. Peptic ulcer disease H. Ectopic pregnancy - ruptured R. Perforated peptic

ulcer I. Endometriosis S. Pyelonephritis J. Hernia T. Torsion For each patient with abdominal pain, select the most likely diagnosis. Structure of a Patient Vignette 55 Age, Gender (eg, A 45-year-old man) Site of Care emergency dept.) (eg, comes to the

Presenting Complaint headache) Duration days) Patient History Physical Findings (eg, because of (eg, that has continued for 2 (with Family History ?) Sample Diagnosis Items: Abdominal Pain 56 1.

A 25-year-old woman has sudden onset of persistent right lower abdominal pain that is increasing in severity. She has nausea without vomiting. She had a normal bowel movement just before onset of pain. Examination shows exquisite deep tenderness to palpation in right lower abdomen with guarding but no rebound; bowel sounds are present. Pelvic examination shows a 7-cm, exquisitely tender right sided mass. Hematocrit is 32%. WBC is 18,000/mm3. Serum amylase activity is within normal limits. Test of the stool for Sample Diagnosis Items: Abdominal Pain 57 2.

A 27-year-old woman has had abdominal pain for the past 18 hours. The pain was first noted in epigastrium and has moved to the right lower quadrant of the abdomen. She has mild nausea and anorexia, but no vomiting. She has a low-grade fever. Her last bowel movement was normal yesterday. Her last menstrual period was 4 weeks ago. Examination shows marked right lower quadrant abdominal tenderness with guarding and rebound. Pelvic exam reveals marked tenderness on right side without Ans: C masses. Hematocrit is 37%. WBC is 14,500/ Sample Diagnosis Items: Abdominal Pain 58 3.

An 84-year-old man in a nursing home has increasing poorly localized lower abdominal pain recurring every 3-4 hours over the past 3 days. He has no nausea or vomiting; the last bowel movement was not recorded. Examination shows a soft abdomen with a palpable, slightly tender, lower left abdominal mass. Hematocrit is 28%. WBC is 10,000/mm3. Serum amylase activity is within normal limits. Test of the stool for occult blood is positive. Rules for Use in Reviewing Sets 59 Options should be single terms, or very short phrases. Make sure the options are homogeneous.

A lead-in must be specified for each set to make the examinee task explicit by specifying the relationship between the items and the options. Items should assess application of knowledge, not just one-word associations. Distractors should be plentiful. Add distractors Sample Extended Matching Set: Flawed 60 A. is motion sickness I. are completely controlled B. have no effects on people J. cause plant & eye damage Cover The Options Rule

C. indirectly increase CO2 K. are negligible D. cause death L. increase risk of skin cancer E. increased odor sensitivity M. cannot be controlled F. is a reduction in visibility N. excess acute respiratory G. esthetics, economics, health illness among children H. products of fossil fuel O. contrary to public opinion Options and Lead-in: Microbiology 61

A. Adenovirus L. influenzae B. Aspergillus fumigatus capsulatum C. Bacillus anthracis tuberculosis D. Candida albicans pneumoniae E. Chlamydia psittaci gonorrhoeae F. Coccidioides immitis meningitidis G. Coronavirus R. H. Corynebacterium diphtheriae I. Coxiella burnetii pneumoniae Haemophilus

M. Histoplasma N. Mycobacterium O. Mycoplasma P. Neisseria Q. Neisseria Pneumocystis carinii S. Rhinovirus T. Streptococcus Sample Poor Item: Microbiology 62 An encapsulated gram-positive organism that usually grows in pairs or short chains. T This stem assesses recall of isolated facts, rather than application of knowledge. It is more like a crossword puzzle question than an item for a medical school examination.

Sample Good Items: Microbiology 63 1. 2. A 7-year-old girl has a high fever and a sore throat. There is pharyngeal redness, a swollen right tonsil with creamy exudate, and painful right submandibular lymphadenopathy. Throat culture on blood agar yields numerous smallUbhemolytic colonies that are inhibited by bacitracin. For the past week, an 18-year-old man has had fever, sore throat, and malaise with bilaterally enlarged tonsils, tonsillar exudate, diffuse K cervical lymphadenopathy, and splenomegaly. Defining Content Domain Using

Chief Complaints 64 Wheezing Chest Pain Palpitations Itching Easy Bruising Fatigue Lymphadenopathy Diarrhea Anemia Jaundice Confusion Back Pain

Ascites Hematemesis/Melena Joint Pain Cough Weight Loss 65 Are There Advantages of Extended Matching R-Sets over Traditional 5-Option Single Items? 66 Percentage of Students Selecting Each Option When Item

Presented in a 5-Option and 15-Option Format A B K C D M F G H I J L O 5-option

81 * 4 0 14 * 1 15-option 2 59 2 13 1 2 1 0 1 2 2 8 0 0 E N 7

Frequency Distribution of Scores Comparing Sets with 5 or 15 -Options 67 Percent of Students 20 5-Option 15-Option 15 10 5 0 30

40 50 60 70 80 90 Percent Correct Score 100 Case SM, Swanson DB. Evaluating diagnostic pattern recognition: a psychometric comparison

of items with 15, 5, and 2 options. Paper presentation, Annual Meeting of the AERA. San ACTIVITY Checklist 69 Does it assess application of knowledge? Does it assess key concepts and principles that are essential information?

Does it assess relevant material? Does it satisfy the cover the options rule? Checklist 70 1. Decide on the theme of your set (eg, Establishing a diagnosis of patients with abdominal pain) 2. Develop the lead-in for your set 3. Write the options for your set (eg, appendicitis, cholecystitis) 4. Write a patient description for each of the

diagnoses in the list 5. Ask a colleague to review your items (patient descriptions) without the correct answers noted

Recently Viewed Presentations

  • General Embryology

    General Embryology

    Primordial Germ Cells . Development begins with fertilization, the process. by which the male gamete, the sperm, and the. female gamete, the oocyte, unite to give rise to a zygote.
  • Victorian Curriculum F-10

    Victorian Curriculum F-10

    The Victorian English Curriculum 7-10. ... tenor. mode. field. knowledge of the English language and how it works. Planning discussion . What kinds of language knowledge do teachers most need in order to teach this part of the curriculum effectively?
  • APA Format and Writing Style

    APA Format and Writing Style

    APA Citation and References Format The Writing Center If you have any questions about APA style or any other concerns about your writing, please come by the Writing Center and one of our consultants will be happy to meet with...
  • Training - TUT

    Training - TUT

    Practical Techniques for Improving Backpropagation Newton's method converges quickly asymtotically and does not exhibit the zigzagging behavior. Newton's method requires that the Hessian H(n) has to be a positive definite matrix for all n!
  • Présentation PowerPoint - www.cfa.harvard.edu/

    Présentation PowerPoint - www.cfa.harvard.edu/

    HITRAN provides almost all needed data (except T dep of shift and self broad, broadening by H2O) Refined Isolated line shape: speed dependence and Dicke narrowing Effects of the speed dependences of collisional width and shift and of velocity changes....
  • http://www.phdcomics.com/comics/archive.php?comicid=847 Advice for creating slides: Advice #1 Dont

    http://www.phdcomics.com/comics/archive.php?comicid=847 Advice for creating slides: Advice #1 Dont

    Use powerpoint and start with blank sides (no formatting) Create math formulas using latex and use screen capture of . pdf opened in Adobe (not e.g. TexWorks) to include image . in your powerpoint slides . Advice #2. Ignore all...
  • Prosecution Luncheon - U.S. Patent

    Prosecution Luncheon - U.S. Patent

    TTAB Reversed: "a trademark ... need not be displayed in any particular size or degree of prominence." Question is, will it be understood as indicating the origin of the goods - yes, especially b/c coined term. In . re Eli...
  • Fractures in Abused Children Mechanisms and Estimates of

    Fractures in Abused Children Mechanisms and Estimates of

    Osteoblasts become active 7 days after injury with bone resorption occurring at areas of necrosis. According to some reports [1, 2], this stage peaks at 2-3 weeks after injury and is defined by a loss of fracture line definition. Bone...