Automated Radiology-Arthroscopy Correlation of Knee Meniscal Tears Using Natural Language Processing Algorithms
Copyright © 2021 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved..
RATIONALE AND OBJECTIVES: Train and apply natural language processing (NLP) algorithms for automated radiology-arthroscopy correlation of meniscal tears.
MATERIALS AND METHODS: In this retrospective single-institution study, we trained supervised machine learning models (logistic regression, support vector machine, and random forest) to detect medial or lateral meniscus tears on free-text MRI reports. We trained and evaluated model performances with cross-validation using 3593 manually annotated knee MRI reports. To assess radiology-arthroscopy correlation, we then randomly partitioned this dataset 80:20 for training and testing, where 108 test set MRIs were followed by knee arthroscopy within 1 year. These free-text arthroscopy reports were also manually annotated. The NLP algorithms trained on the knee MRI training dataset were then evaluated on the MRI and arthroscopy report test datasets. We assessed radiology-arthroscopy agreement using the ensembled NLP-extracted findings versus manually annotated findings.
RESULTS: The NLP models showed high cross-validation performance for meniscal tear detection on knee MRI reports (medial meniscus F1 scores 0.93-0.94, lateral meniscus F1 scores 0.86-0.88). When these algorithms were evaluated on arthroscopy reports, despite never training on arthroscopy reports, performance was similar, though higher with model ensembling (medial meniscus F1 score 0.97, lateral meniscus F1 score 0.99). However, ensembling did not improve performance on knee MRI reports. In the radiology-arthroscopy test set, the ensembled NLP models were able to detect mismatches between MRI and arthroscopy reports with sensitivity 79% and specificity 87%.
CONCLUSION: Radiology-arthroscopy correlation can be automated for knee meniscal tears using NLP algorithms, which shows promise for education and quality improvement.
| Errataetall: |
CommentIn: Acad Radiol. 2022 Apr;29(4):488-489. - PMID 34996688 |
|---|---|
| Medienart: |
E-Artikel |
| Erscheinungsjahr: |
2022 |
|---|---|
| Erschienen: |
2022 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:29 |
|---|---|
| Enthalten in: |
Academic radiology - 29(2022), 4 vom: 21. Apr., Seite 479-487 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Li, Matthew D [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Journal Article |
|---|
| Anmerkungen: |
Date Completed 10.03.2022 Date Revised 02.04.2023 published: Print-Electronic CommentIn: Acad Radiol. 2022 Apr;29(4):488-489. - PMID 34996688 Citation Status MEDLINE |
|---|
| doi: |
10.1016/j.acra.2021.01.017 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM321433467 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM321433467 | ||
| 003 | DE-627 | ||
| 005 | 20231225175810.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231225s2022 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1016/j.acra.2021.01.017 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1071.xml |
| 035 | |a (DE-627)NLM321433467 | ||
| 035 | |a (NLM)33583713 | ||
| 035 | |a (PII)S1076-6332(21)00026-X | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Li, Matthew D |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Automated Radiology-Arthroscopy Correlation of Knee Meniscal Tears Using Natural Language Processing Algorithms |
| 264 | 1 | |c 2022 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 10.03.2022 | ||
| 500 | |a Date Revised 02.04.2023 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a CommentIn: Acad Radiol. 2022 Apr;29(4):488-489. - PMID 34996688 | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Copyright © 2021 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved. | ||
| 520 | |a RATIONALE AND OBJECTIVES: Train and apply natural language processing (NLP) algorithms for automated radiology-arthroscopy correlation of meniscal tears | ||
| 520 | |a MATERIALS AND METHODS: In this retrospective single-institution study, we trained supervised machine learning models (logistic regression, support vector machine, and random forest) to detect medial or lateral meniscus tears on free-text MRI reports. We trained and evaluated model performances with cross-validation using 3593 manually annotated knee MRI reports. To assess radiology-arthroscopy correlation, we then randomly partitioned this dataset 80:20 for training and testing, where 108 test set MRIs were followed by knee arthroscopy within 1 year. These free-text arthroscopy reports were also manually annotated. The NLP algorithms trained on the knee MRI training dataset were then evaluated on the MRI and arthroscopy report test datasets. We assessed radiology-arthroscopy agreement using the ensembled NLP-extracted findings versus manually annotated findings | ||
| 520 | |a RESULTS: The NLP models showed high cross-validation performance for meniscal tear detection on knee MRI reports (medial meniscus F1 scores 0.93-0.94, lateral meniscus F1 scores 0.86-0.88). When these algorithms were evaluated on arthroscopy reports, despite never training on arthroscopy reports, performance was similar, though higher with model ensembling (medial meniscus F1 score 0.97, lateral meniscus F1 score 0.99). However, ensembling did not improve performance on knee MRI reports. In the radiology-arthroscopy test set, the ensembled NLP models were able to detect mismatches between MRI and arthroscopy reports with sensitivity 79% and specificity 87% | ||
| 520 | |a CONCLUSION: Radiology-arthroscopy correlation can be automated for knee meniscal tears using NLP algorithms, which shows promise for education and quality improvement | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Research Support, N.I.H., Extramural | |
| 650 | 4 | |a Knee MRI | |
| 650 | 4 | |a Machine learning | |
| 650 | 4 | |a Meniscal tear | |
| 650 | 4 | |a Natural language processing | |
| 650 | 4 | |a Radiology-arthroscopy correlation | |
| 700 | 1 | |a Deng, Francis |e verfasserin |4 aut | |
| 700 | 1 | |a Chang, Ken |e verfasserin |4 aut | |
| 700 | 1 | |a Kalpathy-Cramer, Jayashree |e verfasserin |4 aut | |
| 700 | 1 | |a Huang, Ambrose J |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Academic radiology |d 1995 |g 29(2022), 4 vom: 21. Apr., Seite 479-487 |w (DE-627)NLM087676818 |x 1878-4046 |7 nnns |
| 773 | 1 | 8 | |g volume:29 |g year:2022 |g number:4 |g day:21 |g month:04 |g pages:479-487 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1016/j.acra.2021.01.017 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 29 |j 2022 |e 4 |b 21 |c 04 |h 479-487 | ||