Development and validation of an objective virtual reality tool for assessing technical aptitude among potential candidates for surgical training
© 2024. The Author(s)..
BACKGROUND: Good technical skills are crucial for surgeons. Yet although surgical training programs strive to assess technical aptitude when selecting surgical residents, valid assessments of such aptitude are still lacking. Surgical simulators have been proposed as a potentially effective tool for this purpose. The current study aims to develop a technical aptitude test using a virtual reality surgical simulator, and to validate its use for the selection of surgical residents.
METHODS: The study had three phases. In Phase 1, we developed an initial version of the technical aptitude test using the Lap-X-VR laparoscopic simulator. In Phases 2 and 3 we refined the test and collected empirical data to evaluate four main sources of validity evidence (content, response process, internal structure, and relationships with other variables), and to evaluate the feasibility and acceptability of the test. Specifically, Phase 2 comprised a review of the test by 30 senior surgeons, and in Phase 3 a revised version of the test was administered to 152 interns to determine its psychometric properties.
RESULTS: Both the surgeons and interns rated the test as highly relevant for selecting surgical residents. Analyses of the data obtained from the trial administration of the test supported the appropriateness of the score calculation process and showed good psychometric properties, including reliability (α = 0.83) and task discrimination (mean discrimination = 0.5, SD = 0.1). The correlations between test scores and background variables revealed significant correlations with gender, surgical simulator experience, and video game experience (ps < 0.001). These variables, however, explained together only 10% of the variance in test scores.
CONCLUSIONS: We describe the systematic development of an innovative virtual reality test for assessing technical aptitude in candidates for surgical training, and present evidence for its validity, feasibility and acceptability. Further validation is required to support the application of the test for selection, as well as to discern the impact of gender, surgical simulator experience, and video game experience on the fairness of test results. However, the test appears to be a promising tool that may help training programs assess the suitability of candidates for surgical training.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:24 |
|---|---|
| Enthalten in: |
BMC medical education - 24(2024), 1 vom: 14. März, Seite 286 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Gazit, Noa [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Aptitude |
|---|
| Anmerkungen: |
Date Completed 18.03.2024 Date Revised 18.03.2024 published: Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1186/s12909-024-05228-1 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM369758560 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM369758560 | ||
| 003 | DE-627 | ||
| 005 | 20240318234811.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240315s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1186/s12909-024-05228-1 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1335.xml |
| 035 | |a (DE-627)NLM369758560 | ||
| 035 | |a (NLM)38486166 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Gazit, Noa |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Development and validation of an objective virtual reality tool for assessing technical aptitude among potential candidates for surgical training |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 18.03.2024 | ||
| 500 | |a Date Revised 18.03.2024 | ||
| 500 | |a published: Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a © 2024. The Author(s). | ||
| 520 | |a BACKGROUND: Good technical skills are crucial for surgeons. Yet although surgical training programs strive to assess technical aptitude when selecting surgical residents, valid assessments of such aptitude are still lacking. Surgical simulators have been proposed as a potentially effective tool for this purpose. The current study aims to develop a technical aptitude test using a virtual reality surgical simulator, and to validate its use for the selection of surgical residents | ||
| 520 | |a METHODS: The study had three phases. In Phase 1, we developed an initial version of the technical aptitude test using the Lap-X-VR laparoscopic simulator. In Phases 2 and 3 we refined the test and collected empirical data to evaluate four main sources of validity evidence (content, response process, internal structure, and relationships with other variables), and to evaluate the feasibility and acceptability of the test. Specifically, Phase 2 comprised a review of the test by 30 senior surgeons, and in Phase 3 a revised version of the test was administered to 152 interns to determine its psychometric properties | ||
| 520 | |a RESULTS: Both the surgeons and interns rated the test as highly relevant for selecting surgical residents. Analyses of the data obtained from the trial administration of the test supported the appropriateness of the score calculation process and showed good psychometric properties, including reliability (α = 0.83) and task discrimination (mean discrimination = 0.5, SD = 0.1). The correlations between test scores and background variables revealed significant correlations with gender, surgical simulator experience, and video game experience (ps < 0.001). These variables, however, explained together only 10% of the variance in test scores | ||
| 520 | |a CONCLUSIONS: We describe the systematic development of an innovative virtual reality test for assessing technical aptitude in candidates for surgical training, and present evidence for its validity, feasibility and acceptability. Further validation is required to support the application of the test for selection, as well as to discern the impact of gender, surgical simulator experience, and video game experience on the fairness of test results. However, the test appears to be a promising tool that may help training programs assess the suitability of candidates for surgical training | ||
| 650 | 4 | |a Review | |
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Aptitude | |
| 650 | 4 | |a Assessment | |
| 650 | 4 | |a Selection | |
| 650 | 4 | |a Surgical training | |
| 650 | 4 | |a Technical skills | |
| 650 | 4 | |a Validation | |
| 700 | 1 | |a Ben-Gal, Gilad |e verfasserin |4 aut | |
| 700 | 1 | |a Eliashar, Ron |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t BMC medical education |d 2001 |g 24(2024), 1 vom: 14. März, Seite 286 |w (DE-627)NLM112079636 |x 1472-6920 |7 nnns |
| 773 | 1 | 8 | |g volume:24 |g year:2024 |g number:1 |g day:14 |g month:03 |g pages:286 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1186/s12909-024-05228-1 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 24 |j 2024 |e 1 |b 14 |c 03 |h 286 | ||