Linking Biomedical Data Warehouse Records With the National Mortality Database in France : Large-scale Matching Algorithm
©Vianney Guardiolle, Adrien Bazoge, Emmanuel Morin, Béatrice Daille, Delphine Toublant, Guillaume Bouzillé, Youenn Merel, Morgane Pierre-Jean, Alexandre Filiot, Marc Cuggia, Matthieu Wargny, Antoine Lamer, Pierre-Antoine Gourraud. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 01.11.2022..
BACKGROUND: Often missing from or uncertain in a biomedical data warehouse (BDW), vital status after discharge is central to the value of a BDW in medical research. The French National Mortality Database (FNMD) offers open-source nominative records of every death. Matching large-scale BDWs records with the FNMD combines multiple challenges: absence of unique common identifiers between the 2 databases, names changing over life, clerical errors, and the exponential growth of the number of comparisons to compute.
OBJECTIVE: We aimed to develop a new algorithm for matching BDW records to the FNMD and evaluated its performance.
METHODS: We developed a deterministic algorithm based on advanced data cleaning and knowledge of the naming system and the Damerau-Levenshtein distance (DLD). The algorithm's performance was independently assessed using BDW data of 3 university hospitals: Lille, Nantes, and Rennes. Specificity was evaluated with living patients on January 1, 2016 (ie, patients with at least 1 hospital encounter before and after this date). Sensitivity was evaluated with patients recorded as deceased between January 1, 2001, and December 31, 2020. The DLD-based algorithm was compared to a direct matching algorithm with minimal data cleaning as a reference.
RESULTS: All centers combined, sensitivity was 11% higher for the DLD-based algorithm (93.3%, 95% CI 92.8-93.9) than for the direct algorithm (82.7%, 95% CI 81.8-83.6; P<.001). Sensitivity was superior for men at 2 centers (Nantes: 87%, 95% CI 85.1-89 vs 83.6%, 95% CI 81.4-85.8; P=.006; Rennes: 98.6%, 95% CI 98.1-99.2 vs 96%, 95% CI 94.9-97.1; P<.001) and for patients born in France at all centers (Nantes: 85.8%, 95% CI 84.3-87.3 vs 74.9%, 95% CI 72.8-77.0; P<.001). The DLD-based algorithm revealed significant differences in sensitivity among centers (Nantes, 85.3% vs Lille and Rennes, 97.3%, P<.001). Specificity was >98% in all subgroups. Our algorithm matched tens of millions of death records from BDWs, with parallel computing capabilities and low RAM requirements. We used the Inseehop open-source R script for this measurement.
CONCLUSIONS: Overall, sensitivity/recall was 11% higher using the DLD-based algorithm than that using the direct algorithm. This shows the importance of advanced data cleaning and knowledge of a naming system through DLD use. Statistically significant differences in sensitivity between groups could be found and must be considered when performing an analysis to avoid differential biases. Our algorithm, originally conceived for linking a BDW with the FNMD, can be used to match any large-scale databases. While matching operations using names are considered sensitive computational operations, the Inseehop package released here is easy to run on premises, thereby facilitating compliance with cybersecurity local framework. The use of an advanced deterministic matching algorithm such as the DLD-based algorithm is an insightful example of combining open-source external data to improve the usage value of BDWs.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2022 |
|---|---|
| Erschienen: |
2022 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:10 |
|---|---|
| Enthalten in: |
JMIR medical informatics - 10(2022), 11 vom: 01. Nov., Seite e36711 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Guardiolle, Vianney [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Clinical data warehouse |
|---|
| Anmerkungen: |
Date Revised 18.11.2022 published: Electronic Citation Status PubMed-not-MEDLINE |
|---|
| doi: |
10.2196/36711 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM348322615 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM348322615 | ||
| 003 | DE-627 | ||
| 005 | 20231226035912.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2022 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.2196/36711 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1161.xml |
| 035 | |a (DE-627)NLM348322615 | ||
| 035 | |a (NLM)36318244 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Guardiolle, Vianney |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Linking Biomedical Data Warehouse Records With the National Mortality Database in France |b Large-scale Matching Algorithm |
| 264 | 1 | |c 2022 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Revised 18.11.2022 | ||
| 500 | |a published: Electronic | ||
| 500 | |a Citation Status PubMed-not-MEDLINE | ||
| 520 | |a ©Vianney Guardiolle, Adrien Bazoge, Emmanuel Morin, Béatrice Daille, Delphine Toublant, Guillaume Bouzillé, Youenn Merel, Morgane Pierre-Jean, Alexandre Filiot, Marc Cuggia, Matthieu Wargny, Antoine Lamer, Pierre-Antoine Gourraud. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 01.11.2022. | ||
| 520 | |a BACKGROUND: Often missing from or uncertain in a biomedical data warehouse (BDW), vital status after discharge is central to the value of a BDW in medical research. The French National Mortality Database (FNMD) offers open-source nominative records of every death. Matching large-scale BDWs records with the FNMD combines multiple challenges: absence of unique common identifiers between the 2 databases, names changing over life, clerical errors, and the exponential growth of the number of comparisons to compute | ||
| 520 | |a OBJECTIVE: We aimed to develop a new algorithm for matching BDW records to the FNMD and evaluated its performance | ||
| 520 | |a METHODS: We developed a deterministic algorithm based on advanced data cleaning and knowledge of the naming system and the Damerau-Levenshtein distance (DLD). The algorithm's performance was independently assessed using BDW data of 3 university hospitals: Lille, Nantes, and Rennes. Specificity was evaluated with living patients on January 1, 2016 (ie, patients with at least 1 hospital encounter before and after this date). Sensitivity was evaluated with patients recorded as deceased between January 1, 2001, and December 31, 2020. The DLD-based algorithm was compared to a direct matching algorithm with minimal data cleaning as a reference | ||
| 520 | |a RESULTS: All centers combined, sensitivity was 11% higher for the DLD-based algorithm (93.3%, 95% CI 92.8-93.9) than for the direct algorithm (82.7%, 95% CI 81.8-83.6; P<.001). Sensitivity was superior for men at 2 centers (Nantes: 87%, 95% CI 85.1-89 vs 83.6%, 95% CI 81.4-85.8; P=.006; Rennes: 98.6%, 95% CI 98.1-99.2 vs 96%, 95% CI 94.9-97.1; P<.001) and for patients born in France at all centers (Nantes: 85.8%, 95% CI 84.3-87.3 vs 74.9%, 95% CI 72.8-77.0; P<.001). The DLD-based algorithm revealed significant differences in sensitivity among centers (Nantes, 85.3% vs Lille and Rennes, 97.3%, P<.001). Specificity was >98% in all subgroups. Our algorithm matched tens of millions of death records from BDWs, with parallel computing capabilities and low RAM requirements. We used the Inseehop open-source R script for this measurement | ||
| 520 | |a CONCLUSIONS: Overall, sensitivity/recall was 11% higher using the DLD-based algorithm than that using the direct algorithm. This shows the importance of advanced data cleaning and knowledge of a naming system through DLD use. Statistically significant differences in sensitivity between groups could be found and must be considered when performing an analysis to avoid differential biases. Our algorithm, originally conceived for linking a BDW with the FNMD, can be used to match any large-scale databases. While matching operations using names are considered sensitive computational operations, the Inseehop package released here is easy to run on premises, thereby facilitating compliance with cybersecurity local framework. The use of an advanced deterministic matching algorithm such as the DLD-based algorithm is an insightful example of combining open-source external data to improve the usage value of BDWs | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a French National Mortality Database | |
| 650 | 4 | |a clinical data warehouse | |
| 650 | 4 | |a clinical informatics | |
| 650 | 4 | |a data reuse | |
| 650 | 4 | |a data warehousing | |
| 650 | 4 | |a medical informatics applications | |
| 650 | 4 | |a medical record linkage | |
| 650 | 4 | |a open data, R | |
| 700 | 1 | |a Bazoge, Adrien |e verfasserin |4 aut | |
| 700 | 1 | |a Morin, Emmanuel |e verfasserin |4 aut | |
| 700 | 1 | |a Daille, Béatrice |e verfasserin |4 aut | |
| 700 | 1 | |a Toublant, Delphine |e verfasserin |4 aut | |
| 700 | 1 | |a Bouzillé, Guillaume |e verfasserin |4 aut | |
| 700 | 1 | |a Merel, Youenn |e verfasserin |4 aut | |
| 700 | 1 | |a Pierre-Jean, Morgane |e verfasserin |4 aut | |
| 700 | 1 | |a Filiot, Alexandre |e verfasserin |4 aut | |
| 700 | 1 | |a Cuggia, Marc |e verfasserin |4 aut | |
| 700 | 1 | |a Wargny, Matthieu |e verfasserin |4 aut | |
| 700 | 1 | |a Lamer, Antoine |e verfasserin |4 aut | |
| 700 | 1 | |a Gourraud, Pierre-Antoine |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t JMIR medical informatics |d 2013 |g 10(2022), 11 vom: 01. Nov., Seite e36711 |w (DE-627)NLM245263071 |x 2291-9694 |7 nnns |
| 773 | 1 | 8 | |g volume:10 |g year:2022 |g number:11 |g day:01 |g month:11 |g pages:e36711 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.2196/36711 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 10 |j 2022 |e 11 |b 01 |c 11 |h e36711 | ||