Commonly used software tools produce conflicting and overly-optimistic AUPRC values
Abstract The precision-recall curve (PRC) and the area under it (AUPRC) are useful for quantifying classification performance. They are commonly used in situations with imbalanced classes, such as cancer diagnosis and cell type annotation. We evaluated 10 popular tools for plotting PRC and computing AUPRC, which were collectively used in>3,000 published studies. We found the AUPRC values computed by the tools rank classifiers differently and some tools produce overly-optimistic results..
| Medienart: |
|
|---|
Preprint| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
bioRxiv.org - (2024) vom: 10. Feb. Zur Gesamtaufnahme - year:2024 |
|---|
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Chen, Wenyu [VerfasserIn] |
|---|
| Links: |
Volltext [kostenfrei] |
|---|
| Themen: |
|---|
| doi: |
10.1101/2024.02.02.578654 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
XBI042434599 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | XBI042434599 | ||
| 003 | DE-627 | ||
| 005 | 20240211090417.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240208s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1101/2024.02.02.578654 |2 doi | |
| 035 | |a (DE-627)XBI042434599 | ||
| 035 | |a (biorXiv)10.1101/2024.02.02.578654 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Chen, Wenyu |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Commonly used software tools produce conflicting and overly-optimistic AUPRC values |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 520 | |a Abstract The precision-recall curve (PRC) and the area under it (AUPRC) are useful for quantifying classification performance. They are commonly used in situations with imbalanced classes, such as cancer diagnosis and cell type annotation. We evaluated 10 popular tools for plotting PRC and computing AUPRC, which were collectively used in>3,000 published studies. We found the AUPRC values computed by the tools rank classifiers differently and some tools produce overly-optimistic results. | ||
| 650 | 4 | |a Biology |7 (dpeaa)DE-84 | |
| 650 | 4 | |a 570 |7 (dpeaa)DE-84 | |
| 700 | 1 | |a Miao, Chen |4 aut | |
| 700 | 1 | |a Zhang, Zhenghao |4 aut | |
| 700 | 1 | |a Fung, Cathy Sin-Hang |4 aut | |
| 700 | 1 | |a Wang, Ran |4 aut | |
| 700 | 1 | |a Chen, Yizhen |4 aut | |
| 700 | 1 | |a Qian, Yan |4 aut | |
| 700 | 1 | |a Cheng, Lixin |0 (orcid)0000-0002-9427-383X |4 aut | |
| 700 | 1 | |a Yip, Kevin Y. |0 (orcid)0000-0001-5516-9944 |4 aut | |
| 700 | 1 | |a Tsui, Stephen Kwok-Wing |4 aut | |
| 700 | 1 | |a Cao, Qin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t bioRxiv.org |g (2024) vom: 10. Feb. |
| 773 | 1 | 8 | |g year:2024 |g day:10 |g month:02 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1101/2024.02.02.578654 |z kostenfrei |3 Volltext |
| 912 | |a GBV_XBI | ||
| 951 | |a AR | ||
| 952 | |j 2024 |b 10 |c 02 | ||