Understanding metric-related pitfalls in image analysis validation
© 2024. Springer Nature America, Inc..
Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.
| Errataetall: | |
|---|---|
| Medienart: |
E-Artikel |
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:21 |
|---|---|
| Enthalten in: |
Nature methods - 21(2024), 2 vom: 10. Feb., Seite 182-194 |
| Sprache: |
Englisch |
|---|
| Links: |
|---|
| Themen: |
|---|
| Anmerkungen: |
Date Completed 15.02.2024 Date Revised 02.03.2024 published: Print-Electronic UpdateOf: ArXiv. 2024 Feb 23;:. - PMID 36945687 Citation Status MEDLINE |
|---|
| doi: |
10.1038/s41592-023-02150-0 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM368372715 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM368372715 | ||
| 003 | DE-627 | ||
| 005 | 20240302232541.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240213s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1038/s41592-023-02150-0 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1314.xml |
| 035 | |a (DE-627)NLM368372715 | ||
| 035 | |a (NLM)38347140 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Reinke, Annika |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Understanding metric-related pitfalls in image analysis validation |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 15.02.2024 | ||
| 500 | |a Date Revised 02.03.2024 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a UpdateOf: ArXiv. 2024 Feb 23;:. - PMID 36945687 | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a © 2024. Springer Nature America, Inc. | ||
| 520 | |a Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Review | |
| 700 | 1 | |a Tizabi, Minu D |e verfasserin |4 aut | |
| 700 | 1 | |a Baumgartner, Michael |e verfasserin |4 aut | |
| 700 | 1 | |a Eisenmann, Matthias |e verfasserin |4 aut | |
| 700 | 1 | |a Heckmann-Nötzel, Doreen |e verfasserin |4 aut | |
| 700 | 1 | |a Kavur, A Emre |e verfasserin |4 aut | |
| 700 | 1 | |a Rädsch, Tim |e verfasserin |4 aut | |
| 700 | 1 | |a Sudre, Carole H |e verfasserin |4 aut | |
| 700 | 1 | |a Acion, Laura |e verfasserin |4 aut | |
| 700 | 1 | |a Antonelli, Michela |e verfasserin |4 aut | |
| 700 | 1 | |a Arbel, Tal |e verfasserin |4 aut | |
| 700 | 1 | |a Bakas, Spyridon |e verfasserin |4 aut | |
| 700 | 1 | |a Benis, Arriel |e verfasserin |4 aut | |
| 700 | 1 | |a Buettner, Florian |e verfasserin |4 aut | |
| 700 | 1 | |a Cardoso, M Jorge |e verfasserin |4 aut | |
| 700 | 1 | |a Cheplygina, Veronika |e verfasserin |4 aut | |
| 700 | 1 | |a Chen, Jianxu |e verfasserin |4 aut | |
| 700 | 1 | |a Christodoulou, Evangelia |e verfasserin |4 aut | |
| 700 | 1 | |a Cimini, Beth A |e verfasserin |4 aut | |
| 700 | 1 | |a Farahani, Keyvan |e verfasserin |4 aut | |
| 700 | 1 | |a Ferrer, Luciana |e verfasserin |4 aut | |
| 700 | 1 | |a Galdran, Adrian |e verfasserin |4 aut | |
| 700 | 1 | |a van Ginneken, Bram |e verfasserin |4 aut | |
| 700 | 1 | |a Glocker, Ben |e verfasserin |4 aut | |
| 700 | 1 | |a Godau, Patrick |e verfasserin |4 aut | |
| 700 | 1 | |a Hashimoto, Daniel A |e verfasserin |4 aut | |
| 700 | 1 | |a Hoffman, Michael M |e verfasserin |4 aut | |
| 700 | 1 | |a Huisman, Merel |e verfasserin |4 aut | |
| 700 | 1 | |a Isensee, Fabian |e verfasserin |4 aut | |
| 700 | 1 | |a Jannin, Pierre |e verfasserin |4 aut | |
| 700 | 1 | |a Kahn, Charles E |e verfasserin |4 aut | |
| 700 | 1 | |a Kainmueller, Dagmar |e verfasserin |4 aut | |
| 700 | 1 | |a Kainz, Bernhard |e verfasserin |4 aut | |
| 700 | 1 | |a Karargyris, Alexandros |e verfasserin |4 aut | |
| 700 | 1 | |a Kleesiek, Jens |e verfasserin |4 aut | |
| 700 | 1 | |a Kofler, Florian |e verfasserin |4 aut | |
| 700 | 1 | |a Kooi, Thijs |e verfasserin |4 aut | |
| 700 | 1 | |a Kopp-Schneider, Annette |e verfasserin |4 aut | |
| 700 | 1 | |a Kozubek, Michal |e verfasserin |4 aut | |
| 700 | 1 | |a Kreshuk, Anna |e verfasserin |4 aut | |
| 700 | 1 | |a Kurc, Tahsin |e verfasserin |4 aut | |
| 700 | 1 | |a Landman, Bennett A |e verfasserin |4 aut | |
| 700 | 1 | |a Litjens, Geert |e verfasserin |4 aut | |
| 700 | 1 | |a Madani, Amin |e verfasserin |4 aut | |
| 700 | 1 | |a Maier-Hein, Klaus |e verfasserin |4 aut | |
| 700 | 1 | |a Martel, Anne L |e verfasserin |4 aut | |
| 700 | 1 | |a Meijering, Erik |e verfasserin |4 aut | |
| 700 | 1 | |a Menze, Bjoern |e verfasserin |4 aut | |
| 700 | 1 | |a Moons, Karel G M |e verfasserin |4 aut | |
| 700 | 1 | |a Müller, Henning |e verfasserin |4 aut | |
| 700 | 1 | |a Nichyporuk, Brennan |e verfasserin |4 aut | |
| 700 | 1 | |a Nickel, Felix |e verfasserin |4 aut | |
| 700 | 1 | |a Petersen, Jens |e verfasserin |4 aut | |
| 700 | 1 | |a Rafelski, Susanne M |e verfasserin |4 aut | |
| 700 | 1 | |a Rajpoot, Nasir |e verfasserin |4 aut | |
| 700 | 1 | |a Reyes, Mauricio |e verfasserin |4 aut | |
| 700 | 1 | |a Riegler, Michael A |e verfasserin |4 aut | |
| 700 | 1 | |a Rieke, Nicola |e verfasserin |4 aut | |
| 700 | 1 | |a Saez-Rodriguez, Julio |e verfasserin |4 aut | |
| 700 | 1 | |a Sánchez, Clara I |e verfasserin |4 aut | |
| 700 | 1 | |a Shetty, Shravya |e verfasserin |4 aut | |
| 700 | 1 | |a Summers, Ronald M |e verfasserin |4 aut | |
| 700 | 1 | |a Taha, Abdel A |e verfasserin |4 aut | |
| 700 | 1 | |a Tiulpin, Aleksei |e verfasserin |4 aut | |
| 700 | 1 | |a Tsaftaris, Sotirios A |e verfasserin |4 aut | |
| 700 | 1 | |a Van Calster, Ben |e verfasserin |4 aut | |
| 700 | 1 | |a Varoquaux, Gaël |e verfasserin |4 aut | |
| 700 | 1 | |a Yaniv, Ziv R |e verfasserin |4 aut | |
| 700 | 1 | |a Jäger, Paul F |e verfasserin |4 aut | |
| 700 | 1 | |a Maier-Hein, Lena |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Nature methods |d 2004 |g 21(2024), 2 vom: 10. Feb., Seite 182-194 |w (DE-627)NLM154352349 |x 1548-7105 |7 nnns |
| 773 | 1 | 8 | |g volume:21 |g year:2024 |g number:2 |g day:10 |g month:02 |g pages:182-194 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1038/s41592-023-02150-0 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 21 |j 2024 |e 2 |b 10 |c 02 |h 182-194 | ||