Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Tools
Abstract Background Examination is the traditional assessment tool, and are used to guide the improvement of academic programs. Accreditation committees’ emphasis on the implementation of standardized examinations. The aims of the current study are to evaluate the quality of assessment tools of sixth year PharmD students as a function of assessment item format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items. Methods. A total of 173 assessment items used in the examinations of sixth year rotations of PharmD program were included. Items were classified as case based or non-case based and as MCQs or open-ended. The psychometric characteristics of examinations were studied as a function of the level of the Bloom’s levels addressed by an item, item format, and number of choices in MCQs. Results. Items addressing intellectual and analysis skills were more difficult, while items associated with multiple cognitive levels were more discriminative. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the letter. Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier and more discriminative. Open-ended questions were significantly easier and more discriminative in comparison to MCQs as case based items while they were more difficult and more discriminative as non-case based items. Conclusion. Item formats, structure, and number of options in MCQs, affected students’ performance and overall examination quality. The discrimination of items associated with multiple Bloom’s levels was significantly higher than those associated with a single level. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively..
| Medienart: |
|
|---|
Preprint| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
ResearchSquare.com - (2023) vom: 23. Sept. Zur Gesamtaufnahme - year:2023 |
|---|
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Muhaissen, Suha Al [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
|---|
| doi: |
10.21203/rs.2.17768/v1 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
XRA034646434 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | XRA034646434 | ||
| 003 | DE-627 | ||
| 005 | 20231205150227.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 220113s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.21203/rs.2.17768/v1 |2 doi | |
| 035 | |a (DE-627)XRA034646434 | ||
| 035 | |a (ResearchSquare)10.21203/rs.2.17768/v1 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Muhaissen, Suha Al |e verfasserin |0 (orcid)0000-0002-5177-1222 |4 aut | |
| 245 | 1 | 0 | |a Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Tools |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 520 | |a Abstract Background Examination is the traditional assessment tool, and are used to guide the improvement of academic programs. Accreditation committees’ emphasis on the implementation of standardized examinations. The aims of the current study are to evaluate the quality of assessment tools of sixth year PharmD students as a function of assessment item format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items. Methods. A total of 173 assessment items used in the examinations of sixth year rotations of PharmD program were included. Items were classified as case based or non-case based and as MCQs or open-ended. The psychometric characteristics of examinations were studied as a function of the level of the Bloom’s levels addressed by an item, item format, and number of choices in MCQs. Results. Items addressing intellectual and analysis skills were more difficult, while items associated with multiple cognitive levels were more discriminative. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the letter. Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier and more discriminative. Open-ended questions were significantly easier and more discriminative in comparison to MCQs as case based items while they were more difficult and more discriminative as non-case based items. Conclusion. Item formats, structure, and number of options in MCQs, affected students’ performance and overall examination quality. The discrimination of items associated with multiple Bloom’s levels was significantly higher than those associated with a single level. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively. | ||
| 650 | 4 | |a Biology |7 (dpeaa)DE-84 | |
| 650 | 4 | |a 570 |7 (dpeaa)DE-84 | |
| 700 | 1 | |a AlKhatib, Hatim S. |4 aut | |
| 700 | 1 | |a Brazeau, Gayle |4 aut | |
| 700 | 1 | |a Akour, Amal |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t ResearchSquare.com |g (2023) vom: 23. Sept. |
| 773 | 1 | 8 | |g year:2023 |g day:23 |g month:09 |
| 856 | 4 | 0 | |u https://doi.org/10.1186/s12909-020-02107-3 |z lizenzpflichtig |3 Volltext |
| 856 | 4 | 0 | |u http://dx.doi.org/10.21203/rs.2.17768/v1 |z kostenfrei |3 Volltext |
| 912 | |a GBV_XRA | ||
| 951 | |a AR | ||
| 952 | |j 2023 |b 23 |c 09 | ||