Effective sample size : A measure of individual uncertainty in predictions
© 2024 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd..
Clinical prediction models are estimated using a sample of limited size from the target population, leading to uncertainty in predictions, even when the model is correctly specified. Generally, not all patient profiles are observed uniformly in model development. As a result, sampling uncertainty varies between individual patients' predictions. We aimed to develop an intuitive measure of individual prediction uncertainty. The variance of a patient's prediction can be equated to the variance of the sample mean outcome in n ∗ $$ {n}_{\ast } $$ hypothetical patients with the same predictor values. This hypothetical sample size n ∗ $$ {n}_{\ast } $$ can be interpreted as the number of similar patients n eff $$ {n}_{\mathrm{eff}} $$ that the prediction is effectively based on, given that the model is correct. For generalized linear models, we derived analytical expressions for the effective sample size. In addition, we illustrated the concept in patients with acute myocardial infarction. In model development, n eff $$ {n}_{\mathrm{eff}} $$ can be used to balance accuracy versus uncertainty of predictions. In a validation sample, the distribution of n eff $$ {n}_{\mathrm{eff}} $$ indicates which patients were more and less represented in the development data, and whether predictions might be too uncertain for some to be practically meaningful. In a clinical setting, the effective sample size may facilitate communication of uncertainty about predictions. We propose the effective sample size as a clinically interpretable measure of uncertainty in individual predictions. Its implications should be explored further for the development, validation and clinical implementation of prediction models.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:43 |
|---|---|
| Enthalten in: |
Statistics in medicine - 43(2024), 7 vom: 30. März, Seite 1384-1396 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Thomassen, Doranne [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Journal Article |
|---|
| Anmerkungen: |
Date Completed 18.03.2024 Date Revised 18.03.2024 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1002/sim.10018 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM36786763X |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM36786763X | ||
| 003 | DE-627 | ||
| 005 | 20240318234156.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240201s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1002/sim.10018 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1334.xml |
| 035 | |a (DE-627)NLM36786763X | ||
| 035 | |a (NLM)38297411 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Thomassen, Doranne |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Effective sample size |b A measure of individual uncertainty in predictions |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 18.03.2024 | ||
| 500 | |a Date Revised 18.03.2024 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a © 2024 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd. | ||
| 520 | |a Clinical prediction models are estimated using a sample of limited size from the target population, leading to uncertainty in predictions, even when the model is correctly specified. Generally, not all patient profiles are observed uniformly in model development. As a result, sampling uncertainty varies between individual patients' predictions. We aimed to develop an intuitive measure of individual prediction uncertainty. The variance of a patient's prediction can be equated to the variance of the sample mean outcome in n ∗ $$ {n}_{\ast } $$ hypothetical patients with the same predictor values. This hypothetical sample size n ∗ $$ {n}_{\ast } $$ can be interpreted as the number of similar patients n eff $$ {n}_{\mathrm{eff}} $$ that the prediction is effectively based on, given that the model is correct. For generalized linear models, we derived analytical expressions for the effective sample size. In addition, we illustrated the concept in patients with acute myocardial infarction. In model development, n eff $$ {n}_{\mathrm{eff}} $$ can be used to balance accuracy versus uncertainty of predictions. In a validation sample, the distribution of n eff $$ {n}_{\mathrm{eff}} $$ indicates which patients were more and less represented in the development data, and whether predictions might be too uncertain for some to be practically meaningful. In a clinical setting, the effective sample size may facilitate communication of uncertainty about predictions. We propose the effective sample size as a clinically interpretable measure of uncertainty in individual predictions. Its implications should be explored further for the development, validation and clinical implementation of prediction models | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a prediction modeling | |
| 650 | 4 | |a risk communication | |
| 650 | 4 | |a uncertainty quantification | |
| 700 | 1 | |a le Cessie, Saskia |e verfasserin |4 aut | |
| 700 | 1 | |a van Houwelingen, Hans C |e verfasserin |4 aut | |
| 700 | 1 | |a Steyerberg, Ewout W |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Statistics in medicine |d 1984 |g 43(2024), 7 vom: 30. März, Seite 1384-1396 |w (DE-627)NLM012664596 |x 1097-0258 |7 nnns |
| 773 | 1 | 8 | |g volume:43 |g year:2024 |g number:7 |g day:30 |g month:03 |g pages:1384-1396 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1002/sim.10018 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 43 |j 2024 |e 7 |b 30 |c 03 |h 1384-1396 | ||