An empirical comparison of statistical methods for multiple cut-off diagnostic test accuracy meta-analysis of the Edinburgh postnatal depression scale (EPDS) depression screening tool using published results vs individual participant data
© 2024. The Author(s)..
BACKGROUND: Selective reporting of results from only well-performing cut-offs leads to biased estimates of accuracy in primary studies of questionnaire-based screening tools and in meta-analyses that synthesize results. Individual participant data meta-analysis (IPDMA) of sensitivity and specificity at each cut-off via bivariate random-effects models (BREMs) can overcome this problem. However, IPDMA is laborious and depends on the ability to successfully obtain primary datasets, and BREMs ignore the correlation between cut-offs within primary studies.
METHODS: We compared the performance of three recent multiple cut-off models developed by Steinhauser et al., Jones et al., and Hoyer and Kuss, that account for missing cut-offs when meta-analyzing diagnostic accuracy studies with multiple cut-offs, to BREMs fitted at each cut-off. We used data from 22 studies of the accuracy of the Edinburgh Postnatal Depression Scale (EPDS; 4475 participants, 758 major depression cases). We fitted each of the three multiple cut-off models and BREMs to a dataset with results from only published cut-offs from each study (published data) and an IPD dataset with results for all cut-offs (full IPD data). We estimated pooled sensitivity and specificity with 95% confidence intervals (CIs) for each cut-off and the area under the curve.
RESULTS: Compared to the BREMs fitted to the full IPD data, the Steinhauser et al., Jones et al., and Hoyer and Kuss models fitted to the published data produced similar receiver operating characteristic curves; though, the Hoyer and Kuss model had lower area under the curve, mainly due to estimating slightly lower sensitivity at lower cut-offs. When fitting the three multiple cut-off models to the full IPD data, a similar pattern of results was observed. Importantly, all models had similar 95% CIs for sensitivity and specificity, and the CI width increased with cut-off levels for sensitivity and decreased with an increasing cut-off for specificity, even the BREMs which treat each cut-off separately.
CONCLUSIONS: Multiple cut-off models appear to be the favorable methods when only published data are available. While collecting IPD is expensive and time consuming, IPD can facilitate subgroup analyses that cannot be conducted with published data only.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:24 |
|---|---|
| Enthalten in: |
BMC medical research methodology - 24(2024), 1 vom: 01. Feb., Seite 28 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Negeri, Zelalem F [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Depression screening accuracy |
|---|
| Anmerkungen: |
Date Completed 05.02.2024 Date Revised 05.02.2024 published: Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1186/s12874-023-02134-w |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM367922932 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM367922932 | ||
| 003 | DE-627 | ||
| 005 | 20240205232117.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240202s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1186/s12874-023-02134-w |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1281.xml |
| 035 | |a (DE-627)NLM367922932 | ||
| 035 | |a (NLM)38302928 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Negeri, Zelalem F |e verfasserin |4 aut | |
| 245 | 1 | 3 | |a An empirical comparison of statistical methods for multiple cut-off diagnostic test accuracy meta-analysis of the Edinburgh postnatal depression scale (EPDS) depression screening tool using published results vs individual participant data |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 05.02.2024 | ||
| 500 | |a Date Revised 05.02.2024 | ||
| 500 | |a published: Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a © 2024. The Author(s). | ||
| 520 | |a BACKGROUND: Selective reporting of results from only well-performing cut-offs leads to biased estimates of accuracy in primary studies of questionnaire-based screening tools and in meta-analyses that synthesize results. Individual participant data meta-analysis (IPDMA) of sensitivity and specificity at each cut-off via bivariate random-effects models (BREMs) can overcome this problem. However, IPDMA is laborious and depends on the ability to successfully obtain primary datasets, and BREMs ignore the correlation between cut-offs within primary studies | ||
| 520 | |a METHODS: We compared the performance of three recent multiple cut-off models developed by Steinhauser et al., Jones et al., and Hoyer and Kuss, that account for missing cut-offs when meta-analyzing diagnostic accuracy studies with multiple cut-offs, to BREMs fitted at each cut-off. We used data from 22 studies of the accuracy of the Edinburgh Postnatal Depression Scale (EPDS; 4475 participants, 758 major depression cases). We fitted each of the three multiple cut-off models and BREMs to a dataset with results from only published cut-offs from each study (published data) and an IPD dataset with results for all cut-offs (full IPD data). We estimated pooled sensitivity and specificity with 95% confidence intervals (CIs) for each cut-off and the area under the curve | ||
| 520 | |a RESULTS: Compared to the BREMs fitted to the full IPD data, the Steinhauser et al., Jones et al., and Hoyer and Kuss models fitted to the published data produced similar receiver operating characteristic curves; though, the Hoyer and Kuss model had lower area under the curve, mainly due to estimating slightly lower sensitivity at lower cut-offs. When fitting the three multiple cut-off models to the full IPD data, a similar pattern of results was observed. Importantly, all models had similar 95% CIs for sensitivity and specificity, and the CI width increased with cut-off levels for sensitivity and decreased with an increasing cut-off for specificity, even the BREMs which treat each cut-off separately | ||
| 520 | |a CONCLUSIONS: Multiple cut-off models appear to be the favorable methods when only published data are available. While collecting IPD is expensive and time consuming, IPD can facilitate subgroup analyses that cannot be conducted with published data only | ||
| 650 | 4 | |a Meta-Analysis | |
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Depression screening accuracy | |
| 650 | 4 | |a Individual participant data | |
| 650 | 4 | |a Multiple cut-offs meta-analysis | |
| 650 | 4 | |a Selective reporting bias | |
| 650 | 4 | |a Sensitivity | |
| 650 | 4 | |a Specificity | |
| 700 | 1 | |a Levis, Brooke |e verfasserin |4 aut | |
| 700 | 1 | |a Ioannidis, John P A |e verfasserin |4 aut | |
| 700 | 1 | |a Thombs, Brett D |e verfasserin |4 aut | |
| 700 | 1 | |a Benedetti, Andrea |e verfasserin |4 aut | |
| 700 | 0 | |a DEPRESsion Screening Data (DEPRESSD) EPDS Group |e verfasserin |4 aut | |
| 700 | 1 | |a Sun, Ying |e investigator |4 oth | |
| 700 | 1 | |a He, Chen |e investigator |4 oth | |
| 700 | 1 | |a Krishnan, Ankur |e investigator |4 oth | |
| 700 | 1 | |a Wu, Yin |e investigator |4 oth | |
| 700 | 1 | |a Bhandari, Parash Mani |e investigator |4 oth | |
| 700 | 1 | |a Neupane, Dipika |e investigator |4 oth | |
| 700 | 1 | |a Imran, Mahrukh |e investigator |4 oth | |
| 700 | 1 | |a Rice, Danielle B |e investigator |4 oth | |
| 700 | 1 | |a Azar, Marleine |e investigator |4 oth | |
| 700 | 1 | |a Chiovitti, Matthew J |e investigator |4 oth | |
| 700 | 1 | |a Riehm, Kira E |e investigator |4 oth | |
| 700 | 1 | |a Boruff, Jill T |e investigator |4 oth | |
| 700 | 1 | |a Cuijpers, Pim |e investigator |4 oth | |
| 700 | 1 | |a Gilbody, Simon |e investigator |4 oth | |
| 700 | 1 | |a Kloda, Lorie A |e investigator |4 oth | |
| 700 | 1 | |a Patten, Scott B |e investigator |4 oth | |
| 700 | 1 | |a Ziegelstein, Roy C |e investigator |4 oth | |
| 700 | 1 | |a Markham, Sarah |e investigator |4 oth | |
| 700 | 1 | |a Comeau, Liane |e investigator |4 oth | |
| 700 | 1 | |a Mitchell, Nicholas D |e investigator |4 oth | |
| 700 | 1 | |a Vigod, Simone N |e investigator |4 oth | |
| 700 | 1 | |a Bakare, Muideen O |e investigator |4 oth | |
| 700 | 1 | |a Beck, Cheryl Tatano |e investigator |4 oth | |
| 700 | 1 | |a Bunevicius, Adomas |e investigator |4 oth | |
| 700 | 1 | |a Couto, Tiago Castro E |e investigator |4 oth | |
| 700 | 1 | |a Chorwe-Sungani, Genesis |e investigator |4 oth | |
| 700 | 1 | |a Favez, Nicolas |e investigator |4 oth | |
| 700 | 1 | |a Field, Sally |e investigator |4 oth | |
| 700 | 1 | |a Garcia-Esteve, Lluïsa |e investigator |4 oth | |
| 700 | 1 | |a Honikman, Simone |e investigator |4 oth | |
| 700 | 1 | |a Khalifa, Dina Sami |e investigator |4 oth | |
| 700 | 1 | |a Kohlhoff, Jane |e investigator |4 oth | |
| 700 | 1 | |a Kusminskas, Laima |e investigator |4 oth | |
| 700 | 1 | |a Kozinszky, Zoltán |e investigator |4 oth | |
| 700 | 1 | |a Nakić Radoš, Sandra |e investigator |4 oth | |
| 700 | 1 | |a Pawlby, Susan J |e investigator |4 oth | |
| 700 | 1 | |a Rochat, Tamsen J |e investigator |4 oth | |
| 700 | 1 | |a Sharp, Deborah J |e investigator |4 oth | |
| 700 | 1 | |a Smith-Nielsen, Johanne |e investigator |4 oth | |
| 700 | 1 | |a Su, Kuan-Pin |e investigator |4 oth | |
| 700 | 1 | |a Tadinac, Meri |e investigator |4 oth | |
| 700 | 1 | |a Tandon, S Darius |e investigator |4 oth | |
| 700 | 1 | |a Thiagayson, Pavaani |e investigator |4 oth | |
| 700 | 1 | |a Töreki, Annamária |e investigator |4 oth | |
| 700 | 1 | |a Torres-Giménez, Anna |e investigator |4 oth | |
| 700 | 1 | |a van Heyningen, Thandi |e investigator |4 oth | |
| 700 | 1 | |a Vega-Dienstmaier, Johann M |e investigator |4 oth | |
| 773 | 0 | 8 | |i Enthalten in |t BMC medical research methodology |d 2001 |g 24(2024), 1 vom: 01. Feb., Seite 28 |w (DE-627)NLM111431921 |x 1471-2288 |7 nnns |
| 773 | 1 | 8 | |g volume:24 |g year:2024 |g number:1 |g day:01 |g month:02 |g pages:28 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1186/s12874-023-02134-w |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 24 |j 2024 |e 1 |b 01 |c 02 |h 28 | ||