Validation of RUBY for Breast Cancer Knowledge Extraction From a Large French Electronic Medical Record System
PURPOSE: RUBY is a tool for extracting clinical data on breast cancer from French medical records on the basis of named entity recognition models combined with keyword extraction and postprocessing rules. Although initial results showed a high precision of the system in extracting clinical information from surgery, pathology, and biopsy reports (≥92.7%) and good precision in extracting data from consultation reports (81.8%), its validation is needed before its use in routine practice.
METHODS: In this work, we analyzed RUBY's performance compared with the manual entry and we evaluated the generalizability of the approach on different sets of reports collected on a span of 40 years.
RESULTS: RUBY performed similarly or better than the manual entry for 15 of 27 variables. It showed similar performances when structuring newer reports but failed to extract entities for which changes in terminology appeared. Finally, our tool could automatically structure 15,990 reports in 77 minutes.
CONCLUSION: RUBY can automate the data entry process of a set of variables and reduce its burden, but a continuous evaluation of the format and structure of the reports and a subsequent update of the system is necessary to ensure its robustness.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:7 |
|---|---|
| Enthalten in: |
JCO clinical cancer informatics - 7(2023) vom: 10. Mai, Seite e2200130 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Schiappa, Renaud [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
|---|
| Anmerkungen: |
Date Completed 29.05.2023 Date Revised 30.05.2023 published: Print Citation Status MEDLINE |
|---|
| doi: |
10.1200/CCI.22.00130 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM357380991 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM357380991 | ||
| 003 | DE-627 | ||
| 005 | 20231226072453.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1200/CCI.22.00130 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1191.xml |
| 035 | |a (DE-627)NLM357380991 | ||
| 035 | |a (NLM)37235837 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Schiappa, Renaud |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Validation of RUBY for Breast Cancer Knowledge Extraction From a Large French Electronic Medical Record System |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 29.05.2023 | ||
| 500 | |a Date Revised 30.05.2023 | ||
| 500 | |a published: Print | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a PURPOSE: RUBY is a tool for extracting clinical data on breast cancer from French medical records on the basis of named entity recognition models combined with keyword extraction and postprocessing rules. Although initial results showed a high precision of the system in extracting clinical information from surgery, pathology, and biopsy reports (≥92.7%) and good precision in extracting data from consultation reports (81.8%), its validation is needed before its use in routine practice | ||
| 520 | |a METHODS: In this work, we analyzed RUBY's performance compared with the manual entry and we evaluated the generalizability of the approach on different sets of reports collected on a span of 40 years | ||
| 520 | |a RESULTS: RUBY performed similarly or better than the manual entry for 15 of 27 variables. It showed similar performances when structuring newer reports but failed to extract entities for which changes in terminology appeared. Finally, our tool could automatically structure 15,990 reports in 77 minutes | ||
| 520 | |a CONCLUSION: RUBY can automate the data entry process of a set of variables and reduce its burden, but a continuous evaluation of the format and structure of the reports and a subsequent update of the system is necessary to ensure its robustness | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Research Support, Non-U.S. Gov't | |
| 700 | 1 | |a Contu, Sara |e verfasserin |4 aut | |
| 700 | 1 | |a Culie, Dorian |e verfasserin |4 aut | |
| 700 | 1 | |a Chateau, Yann |e verfasserin |4 aut | |
| 700 | 1 | |a Gal, Jocelyn |e verfasserin |4 aut | |
| 700 | 1 | |a Pace-Loscos, Tanguy |e verfasserin |4 aut | |
| 700 | 1 | |a Bailleux, Caroline |e verfasserin |4 aut | |
| 700 | 1 | |a Haudebourg, Juliette |e verfasserin |4 aut | |
| 700 | 1 | |a Ferrero, Jean-Marc |e verfasserin |4 aut | |
| 700 | 1 | |a Barranger, Emmanuel |e verfasserin |4 aut | |
| 700 | 1 | |a Chamorey, Emmanuel |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t JCO clinical cancer informatics |d 2017 |g 7(2023) vom: 10. Mai, Seite e2200130 |w (DE-627)NLM275406369 |x 2473-4276 |7 nnns |
| 773 | 1 | 8 | |g volume:7 |g year:2023 |g day:10 |g month:05 |g pages:e2200130 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1200/CCI.22.00130 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 7 |j 2023 |b 10 |c 05 |h e2200130 | ||