Measuring and Predicting Faculty Consensus Rankings of Standardized Letters of Evaluation
Background Standardized letters of evaluation (SLOE) are becoming more widely incorporated into the residency application process to make the letter of recommendation, an already critical component in a residency application packet, more objective. However, it is not currently known if the reviewers of these letters share consensus regarding the strength of an applicant determined by their SLOE. Objective We measured the level of faculty agreement regarding applicant competitiveness as determined by SLOEs and the ability of 2 algorithms to predict faculty consensus rankings. Methods Using data from the 2021-2022 Match cycle from the Council of Residency Directors in Emergency Medicine SLOE Database as a blueprint, authors created 50 fictional SLOEs representative of the national data. Seven faculty then rated these SLOEs in order of applicant competitiveness, defined as suggested rank position. Consensus was evaluated using cutoffs established a priori, and 2 prediction models, a point-based system and a linear regression model, were tested to determine their ability to predict consensus rankings. Results There was strong faculty consensus regarding the interpretation of SLOEs. Within narrow windows of agreement, faculty demonstrated similar ranking patterns with 83% and 93% agreement for "close" and "loose" agreement, respectively. Predictive models yielded a strong correlation with the consensus ranking (point-based system r=0.97, linear regression r=0.97). Conclusions Faculty displayed strong consensus regarding the competitiveness of applicants via SLOEs, adding further support to the use of SLOEs for selection and advising. Two models predicted consensus competitiveness rankings with a high degree of accuracy.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:16 |
|---|---|
| Enthalten in: |
Journal of graduate medical education - 16(2024), 1 vom: 02. Feb., Seite 51-58 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Sehdev, Morgan [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
|---|
| Anmerkungen: |
Date Completed 05.02.2024 Date Revised 05.02.2024 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.4300/JGME-D-22-00901.1 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM367939665 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM367939665 | ||
| 003 | DE-627 | ||
| 005 | 20240205232127.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240202s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.4300/JGME-D-22-00901.1 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1281.xml |
| 035 | |a (DE-627)NLM367939665 | ||
| 035 | |a (NLM)38304605 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Sehdev, Morgan |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Measuring and Predicting Faculty Consensus Rankings of Standardized Letters of Evaluation |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 05.02.2024 | ||
| 500 | |a Date Revised 05.02.2024 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Background Standardized letters of evaluation (SLOE) are becoming more widely incorporated into the residency application process to make the letter of recommendation, an already critical component in a residency application packet, more objective. However, it is not currently known if the reviewers of these letters share consensus regarding the strength of an applicant determined by their SLOE. Objective We measured the level of faculty agreement regarding applicant competitiveness as determined by SLOEs and the ability of 2 algorithms to predict faculty consensus rankings. Methods Using data from the 2021-2022 Match cycle from the Council of Residency Directors in Emergency Medicine SLOE Database as a blueprint, authors created 50 fictional SLOEs representative of the national data. Seven faculty then rated these SLOEs in order of applicant competitiveness, defined as suggested rank position. Consensus was evaluated using cutoffs established a priori, and 2 prediction models, a point-based system and a linear regression model, were tested to determine their ability to predict consensus rankings. Results There was strong faculty consensus regarding the interpretation of SLOEs. Within narrow windows of agreement, faculty demonstrated similar ranking patterns with 83% and 93% agreement for "close" and "loose" agreement, respectively. Predictive models yielded a strong correlation with the consensus ranking (point-based system r=0.97, linear regression r=0.97). Conclusions Faculty displayed strong consensus regarding the competitiveness of applicants via SLOEs, adding further support to the use of SLOEs for selection and advising. Two models predicted consensus competitiveness rankings with a high degree of accuracy | ||
| 650 | 4 | |a Journal Article | |
| 700 | 1 | |a Schnapp, Benjamin |e verfasserin |4 aut | |
| 700 | 1 | |a Dubosh, Nicole M |e verfasserin |4 aut | |
| 700 | 1 | |a Alvarez, Al'ai |e verfasserin |4 aut | |
| 700 | 1 | |a Pelletier-Bui, Alexis |e verfasserin |4 aut | |
| 700 | 1 | |a Bord, Sharon |e verfasserin |4 aut | |
| 700 | 1 | |a Schrepel, Caitlin |e verfasserin |4 aut | |
| 700 | 1 | |a Park, Yoon Soo |e verfasserin |4 aut | |
| 700 | 1 | |a Shappell, Eric |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Journal of graduate medical education |d 2009 |g 16(2024), 1 vom: 02. Feb., Seite 51-58 |w (DE-627)NLM197775551 |x 1949-8357 |7 nnns |
| 773 | 1 | 8 | |g volume:16 |g year:2024 |g number:1 |g day:02 |g month:02 |g pages:51-58 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.4300/JGME-D-22-00901.1 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 16 |j 2024 |e 1 |b 02 |c 02 |h 51-58 | ||