Concordance of Hospital Ranks and Category Ratings Using the Current Technical Specification of US Hospital Star Ratings and Reasonable Alternative Specifications
Copyright 2022 Barclay ME et al. JAMA Health Forum..
Importance: The Centers for Medicare & Medicaid Services (CMS) Hospital Compare star ratings are widely used summaries of hospital quality that can influence patient choice and organizational reputation.
Objective: To identify the changes in hospital ratings and rankings associated with alternative methodological choices in the calculation of the 2021 CMS Hospital Compare star ratings.
Design Setting and Participants: This cross-sectional study used publicly available 2021 Hospital Compare data for 3339 US hospitals from the October 2020 data release. Change in apparent hospital performance was assessed, arising when plausible alternatives to current methods are used for calculating star ratings in relation to individual measure standardization, domain derivation, and domain weighting. Three example changes were examined in detail, with more comprehensive changes considered using Monte Carlo simulation. Changes in centile of hospital ranks and in star rating overall were examined, as well as separately in CMS peer groupings defined by the number of reported quality domains.
Main Outcomes and Measures: Proportion of hospitals receiving a different star rating under the alternative technical specifications than under the current (2021) CMS approach and mean absolute change in centile rank of hospitals under alternative technical specifications compared with the current (2021) CMS approach.
Results: A total of 3339 US hospitals were included in the analysis. Of the specific changes to current specifications considered, the alternative method of standardization was associated with the most substantial changes, with 55.4% (95% CI, 53.7%-57.1%) of hospitals having their star rating reclassified. The change in domain weights was associated with the smallest differences, but even that resulted in reclassification of the star rating in approximately 1 in 4 (24.5%; 95% CI, 23.0%-26.0%) hospitals. In the simulation study, which covered a range of possible changes, on average half of hospitals (51.8%; IQR, 44.2%-59.7%) were assigned a different star rating from that produced by the current specification, usually into an adjacent star category.
Conclusions and Relevance: In this cross-sectional study of data on US hospitals, CMS Hospital Compare star ratings were found to be highly sensitive to how performance ratings are calculated, demonstrating the need for transparent justification of the technical approaches used in calculating composite performance ratings.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2022 |
|---|---|
| Erschienen: |
2022 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:3 |
|---|---|
| Enthalten in: |
JAMA health forum - 3(2022), 5 vom: 13. Mai, Seite e221006 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Barclay, Matthew E [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
|---|
| Anmerkungen: |
Date Completed 05.09.2022 Date Revised 13.03.2024 published: Electronic-eCollection Citation Status MEDLINE |
|---|
| doi: |
10.1001/jamahealthforum.2022.1006 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM344958760 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM344958760 | ||
| 003 | DE-627 | ||
| 005 | 20240313233231.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2022 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1001/jamahealthforum.2022.1006 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1326.xml |
| 035 | |a (DE-627)NLM344958760 | ||
| 035 | |a (NLM)35977255 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Barclay, Matthew E |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Concordance of Hospital Ranks and Category Ratings Using the Current Technical Specification of US Hospital Star Ratings and Reasonable Alternative Specifications |
| 264 | 1 | |c 2022 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 05.09.2022 | ||
| 500 | |a Date Revised 13.03.2024 | ||
| 500 | |a published: Electronic-eCollection | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Copyright 2022 Barclay ME et al. JAMA Health Forum. | ||
| 520 | |a Importance: The Centers for Medicare & Medicaid Services (CMS) Hospital Compare star ratings are widely used summaries of hospital quality that can influence patient choice and organizational reputation | ||
| 520 | |a Objective: To identify the changes in hospital ratings and rankings associated with alternative methodological choices in the calculation of the 2021 CMS Hospital Compare star ratings | ||
| 520 | |a Design Setting and Participants: This cross-sectional study used publicly available 2021 Hospital Compare data for 3339 US hospitals from the October 2020 data release. Change in apparent hospital performance was assessed, arising when plausible alternatives to current methods are used for calculating star ratings in relation to individual measure standardization, domain derivation, and domain weighting. Three example changes were examined in detail, with more comprehensive changes considered using Monte Carlo simulation. Changes in centile of hospital ranks and in star rating overall were examined, as well as separately in CMS peer groupings defined by the number of reported quality domains | ||
| 520 | |a Main Outcomes and Measures: Proportion of hospitals receiving a different star rating under the alternative technical specifications than under the current (2021) CMS approach and mean absolute change in centile rank of hospitals under alternative technical specifications compared with the current (2021) CMS approach | ||
| 520 | |a Results: A total of 3339 US hospitals were included in the analysis. Of the specific changes to current specifications considered, the alternative method of standardization was associated with the most substantial changes, with 55.4% (95% CI, 53.7%-57.1%) of hospitals having their star rating reclassified. The change in domain weights was associated with the smallest differences, but even that resulted in reclassification of the star rating in approximately 1 in 4 (24.5%; 95% CI, 23.0%-26.0%) hospitals. In the simulation study, which covered a range of possible changes, on average half of hospitals (51.8%; IQR, 44.2%-59.7%) were assigned a different star rating from that produced by the current specification, usually into an adjacent star category | ||
| 520 | |a Conclusions and Relevance: In this cross-sectional study of data on US hospitals, CMS Hospital Compare star ratings were found to be highly sensitive to how performance ratings are calculated, demonstrating the need for transparent justification of the technical approaches used in calculating composite performance ratings | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Research Support, Non-U.S. Gov't | |
| 700 | 1 | |a Dixon-Woods, Mary |e verfasserin |4 aut | |
| 700 | 1 | |a Lyratzopoulos, Georgios |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t JAMA health forum |d 2020 |g 3(2022), 5 vom: 13. Mai, Seite e221006 |w (DE-627)NLM322850525 |x 2689-0186 |7 nnns |
| 773 | 1 | 8 | |g volume:3 |g year:2022 |g number:5 |g day:13 |g month:05 |g pages:e221006 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1001/jamahealthforum.2022.1006 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 3 |j 2022 |e 5 |b 13 |c 05 |h e221006 | ||