Applying Natural Language Processing to Single-Report Prediction of Metastatic Disease Response Using the OR-RADS Lexicon
Generating Real World Evidence (RWE) on disease responses from radiological reports is important for understanding cancer treatment effectiveness and developing personalized treatment. A lack of standardization in reporting among radiologists impacts the feasibility of large-scale interpretation of disease response. This study examines the utility of applying natural language processing (NLP) to the large-scale interpretation of disease responses using a standardized oncologic response lexicon (OR-RADS) to facilitate RWE collection. Radiologists annotated 3503 retrospectively collected clinical impressions from radiological reports across several cancer types with one of seven OR-RADS categories. A Bidirectional Encoder Representations from Transformers (BERT) model was trained on this dataset with an 80-20% train/test split to perform multiclass and single-class classification tasks using the OR-RADS. Radiologists also performed the classification to compare human and model performance. The model achieved accuracies from 95 to 99% across all classification tasks, performing better in single-class tasks compared to the multiclass task and producing minimal misclassifications, which pertained mostly to overpredicting the equivocal and mixed OR-RADS labels. Human accuracy ranged from 74 to 93% across all classification tasks, performing better on single-class tasks. This study demonstrates the feasibility of the BERT NLP model in predicting disease response in cancer patients, exceeding human performance, and encourages the use of the standardized OR-RADS lexicon to improve large-scale prediction accuracy.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:15 |
|---|---|
| Enthalten in: |
Cancers - 15(2023), 20 vom: 10. Okt. |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Elbatarny, Lydia [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Computed tomography |
|---|
| Anmerkungen: |
Date Revised 10.02.2024 published: Electronic Citation Status PubMed-not-MEDLINE |
|---|
| doi: |
10.3390/cancers15204909 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM363862935 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM363862935 | ||
| 003 | DE-627 | ||
| 005 | 20240210232823.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.3390/cancers15204909 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1287.xml |
| 035 | |a (DE-627)NLM363862935 | ||
| 035 | |a (NLM)37894276 | ||
| 035 | |a (PII)4909 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Elbatarny, Lydia |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Applying Natural Language Processing to Single-Report Prediction of Metastatic Disease Response Using the OR-RADS Lexicon |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Revised 10.02.2024 | ||
| 500 | |a published: Electronic | ||
| 500 | |a Citation Status PubMed-not-MEDLINE | ||
| 520 | |a Generating Real World Evidence (RWE) on disease responses from radiological reports is important for understanding cancer treatment effectiveness and developing personalized treatment. A lack of standardization in reporting among radiologists impacts the feasibility of large-scale interpretation of disease response. This study examines the utility of applying natural language processing (NLP) to the large-scale interpretation of disease responses using a standardized oncologic response lexicon (OR-RADS) to facilitate RWE collection. Radiologists annotated 3503 retrospectively collected clinical impressions from radiological reports across several cancer types with one of seven OR-RADS categories. A Bidirectional Encoder Representations from Transformers (BERT) model was trained on this dataset with an 80-20% train/test split to perform multiclass and single-class classification tasks using the OR-RADS. Radiologists also performed the classification to compare human and model performance. The model achieved accuracies from 95 to 99% across all classification tasks, performing better in single-class tasks compared to the multiclass task and producing minimal misclassifications, which pertained mostly to overpredicting the equivocal and mixed OR-RADS labels. Human accuracy ranged from 74 to 93% across all classification tasks, performing better on single-class tasks. This study demonstrates the feasibility of the BERT NLP model in predicting disease response in cancer patients, exceeding human performance, and encourages the use of the standardized OR-RADS lexicon to improve large-scale prediction accuracy | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a computed tomography | |
| 650 | 4 | |a disease progression | |
| 650 | 4 | |a metastasis | |
| 650 | 4 | |a natural language processing | |
| 650 | 4 | |a radiology | |
| 700 | 1 | |a Do, Richard K G |e verfasserin |4 aut | |
| 700 | 1 | |a Gangai, Natalie |e verfasserin |4 aut | |
| 700 | 1 | |a Ahmed, Firas |e verfasserin |4 aut | |
| 700 | 1 | |a Chhabra, Shalini |e verfasserin |4 aut | |
| 700 | 1 | |a Simpson, Amber L |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Cancers |d 2009 |g 15(2023), 20 vom: 10. Okt. |w (DE-627)NLM198667213 |x 2072-6694 |7 nnns |
| 773 | 1 | 8 | |g volume:15 |g year:2023 |g number:20 |g day:10 |g month:10 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.3390/cancers15204909 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 15 |j 2023 |e 20 |b 10 |c 10 | ||