What Do CMS?s Hospital Star Ratings Really Tell Us?

Thursday, October 13, 2016 | Larry Sobal

In an effort to help patients choose hospitals based on quality, the Centers for Medicare and Medicaid Services (CMS) recently introduced a five star hospital rating system for more than 3,600 U.S. hospitals. In this system, one star signals the lowest level of performance and five stars indicate the highest quality of care; the ratings will be updated quarterly.

I was intrigued, so I looked up all my local hospitals and found they were rated either three or four stars. If I were to need hospitalization any time soon (hopefully not), does this information have any value? Let’s find out by digging deeper into the star rating methodology.

The new Hospital Quality Star Rating summarizes data from 64 existing quality measures publicly reported on the Hospital Compare website into a uniform star rating for each hospital. The goal is to make it easier for consumers to compare hospitals and interpret complex quality information. The measures include death rates, readmissions, safety indicators and patient satisfaction scores. This is an attempt to offer consumers a simpler measure of hospital quality than the full listing of more than 100 separate measures reported by the government. 

This overall star rating supplements the star ratings currently posted for hospitals on their patient experience of care data, based on information from the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) Survey. The patient experience of care hospital star ratings were first publicly reported in April 2015.

Back to my question: Do the star ratings have any value to me as a consumer, in terms of making a wise choice if I needed hospitalization? 

To start with, any individual hospital’s star rating is calculated using only the measures for which data is available. That means hospitals' star ratings could be based on as few as nine measures, or on as many as 64; the average is roughly 40. A recent article in Health Affairs pointed out that “summary scores for different hospitals may be constructed from very different measures, making comparisons difficult to understand. For example, within the Effectiveness of Care domain, 25% of hospitals are scored on two or fewer measures, while 32% are scored on 27, 28, 29, or 30 measures.” This implies that there is dangerously little uniformity to the scoring methodology. 

Upon review of the data, it appears hospitals that reported on the majority of metrics tended to get one, two or three stars. The hospitals that consistently scored worst were typically teaching and safety-net hospitals that arguably treat the sickest patients with the most complex care. In fact, of the 102 hospitals that received five stars, there are a mixture of some large, medium and small hospitals. A few are considered the nation’s best by private ratings sources such as U.S. News & World Report, or viewed as the most elite within the medical profession. Likewise, among the 102 were relatively obscure rural hospitals and at least 40 hospitals that specialize in just a few types of surgery, such as knee replacements. 

Medicare awarded the lowest rating of one star to 129 hospitals. A sizable proportion of the nation’s major academic medical centers, which train doctors, scored poorly. Out of 288 hospitals that teach significant numbers of residents, six in 10 received below average scores. Teaching hospitals comprised one-third of the facilities receiving one star.

Hospitals, and health care in general, are far too complex to simplify into a one-to-five rating system.

In addition, although Medicare does not consider social or financial situations in the ratings, an analysis released by Medicare found hospitals that treated large numbers of low-income patients tended to score lower.

This suggests that the star ratings are flawed; however, others feel differently. A recent research letter published in JAMA Internal Medicine indicates that consumers could choose 5-star hospitals without a concern for their clinical quality—and a 5-star rating may in fact guide patients to better institutions. The letter highlighted whether, holding all other factors constant, picking a 5-star hospital would lead patients to a hospital with lower mortality rates than that of a one-star hospital. It turns out that it does. There is a 1.4 percentage point difference in mortality between a 5-star hospital (mortality rate 9.8%) and a one-star hospital (mortality rate 11.2%). 

Despite that information, my conclusion is that Medicare’s star ratings are generally not that useful. The measures, the underlying methodology, lack of consideration for socioeconomic factors, and discounting of a patient’s clinical needs, are all factors that muddle the Hospital Star Ratings results. I would not recommend judging a 5-star hospital to be clearly superior to a one-star hospital, let alone trying to judge whether a four is better than a three. Hospitals, and health care in general, are far too complex to simplify into a one-to-five ratings system.

My advice to anyone who wants to know where to go for health care is the same: ask individuals who work in the profession. Nurses, allied health professionals and physicians all know that there are good and not-so-good aspects to individual hospitals, just as there are good and not-so- good individual nurses and physicians. The trick is getting them to share what they know.  

What do CMS’s star ratings really tell us? Not much of real value.

Illustration: Lee Sauer

---

 

Larry Sobal is Executive Vice President of Business Development at MedAxiom. He has a 35-year background as a senior executive in medical group leadership, hospital leadership and insurance. As part of his current role, Larry consults, writes and presents on topics relevant to transforming physician practices and health systems.

 

 

 

 

---