Measure for Measure: C A L I FOR N I A Analyzing California Hospital Characteristics and Performance H EALTH C ARE F OU NDATION Introduction improvement. (One exception was the somewhat In recent years, consumers and stakeholders poorer performance, in a few measures, of increasingly have been offered access to hospitals with Disproportionate Share Hospital standardized information comparing clinical [DSH] status.) These findings suggest that all performance among hospitals. With the advent hospitals, regardless of type, can become high Issue Brief of this information about individual hospitals performers by implementing effective practices. comes a potentially significant second-generation It also suggests that researchers, policymakers, question: Are there specific institutional and others should discontinue the practice of characteristics or attributes that lead to higher or separating hospitals by characteristics or type when lower clinical performance in some hospitals? If attempting to assess performance or acting to such attributes can be identified, other hospitals improve it. might adopt or alter them in an effort to improve their own performance. Methodology The project used data from CHART measures as The few studies that previously examined this the dependent variables in its analysis: figures on question had mixed, inconclusive results regarding coronary artery bypass graft (CABG) mortality, a limited number of characteristics (hospital size intensive care unit (ICU) mortality, hospital- and teaching status,1 Leapfrog Group standards,2 acquired pressure ulcer (HAPU) rates, and patient and ownership and rural/urban characteristics 3). experience. The project also used structural, In 2008, the California HealthCare Foundation financial, and operational measures data from partnered with Convergence Health Consulting, OSHPD, as well as data from the Accreditation Inc. to examine more comprehensively whether Council for Graduate Medical Education and hospital characteristics were independently other sources. associated with hospital performance. The project used California Hospital Assessment Thirty hospital characteristics were analyzed to and Reporting Taskforce (CHART) data, plus determine if a relationship existed between any information from the California Office of characteristic and any of the CHART clinical Statewide Health Planning and Development measures (see Table 1). Once the relationships (OSHPD), to assess the impact of various between hospital attributes and clinical hospital characteristics on clinical outcomes and performance were identified, the analysis built improvement and to determine whether any regression models to determine whether such patterns emerged. relationships were independent of other variables, and whether other independent relationships The project’s primary findings were surprising: emerged. Then the project tested whether the No major category of hospital characteristics, variables had any impact on improvement with including financial health, was determined to be regard to selected core measures and two outcome D ecember an independent driver of clinical performance or measures. Finally, the project examined the 2008 characteristics of those hospitals with the greatest and Most Hospital Characteristics Are Non-Factors least improvement to determine whether any patterns in Performance could be identified. Nine major domains (representing 15 variables) did not factor in any of the 11 hospital performance or Key Findings improvement models. Another five major hospital The project’s most striking finding was that a large domains (representing 12 variables) were present in only majority of hospital characteristics never or rarely one of the 11 models — a relationship much too tenuous appeared in any way related to performance. Of the to consider them performance predictors. The hospital 30 hospital characteristics the project examined against attributes that bore no relation at all to performance or 11 primary performance and improvement measures, improvement are: the data suggested — using presence in at least three K Teaching status; statistical models as the threshold — that only four were independent predictors of performance: K Rural hospital status; K DSH status (a significant proportion of indigent care K Measures of hospital value (net PPE, net PPE/bed, and Medi-Cal patients); and bad debt); K Membership in a large (more than ten hospitals) K Length of stay; system; K Presence of hospitalists; K Percentage of gross revenue collected; and K CABG volume; K Initial starting value (in the improvement analyses). K Proportion of managed care revenue and managed care days; K Proportion of minority discharges; Table 1. Hospital Characteristics Used as Independent Variables This table groups, into five basic categories, the 30 hospital characteristics or attributes used as independent variables in this study. Income Va l u e P r o fit a b i l it y S t r uctu r e Oth e r Pre-tax net income Net PPE Total margin Size Region (property, plant, and equipment [PPE] and construction-in-progress) Net income Net PPE per bed Operating margin Teaching status Rural Gross inpatient revenue Bad debt Cost-per-charge ratio Ownership Total nursing level (city/county, district, investor, non-profit) Percentage of gross System membership Staffed bed occupancy; revenue collected licensed bed occupancy Disproportionate Share System size Adjusted length of stay Hospital (DSH) status (number of hospitals) System gross revenue Leapfrog computerized Presence of hospitalists physician order entry standard Proportion of managed Leapfrog intensivist Proportion of… care revenue physician standard • minority discharges • managed care days CABG volume 2  |  California HealthCare Foundation K Performance on the Leapfrog computerized provider association with improvement regarding one surgical order entry standard; infection prevention measure (SIP 3). K Performance on the Leapfrog intensivist physician DSH status is a marker for high levels of indigent and standard; Medi-Cal-funded care, and reflects a patient population K System membership (two or more hospitals); with socio-demographics that differ from populations in non-DSH facilities. DSH patients may have a higher K Profitability (total margin, operating margin, and severity of illness, but the risk-adjustment models control cost-per-charge ratio); for much of this clinical severity. DSH status also might K Staffing measures; be a marker regarding hospital financial characteristics, but these factors showed no significance in the models. K Occupancy measures; and Further, the proportion of minority discharges was not a K Pre-tax net income. factor in any of the models. DSH status, then, probably is a marker for socio-demographic or clinical factors that Three major hospital domains were present in two models are not represented by any of the 30 variables tested in but were inconsistent in the direction or type of measure, this study. Whether the negative performance and weaker and thus produce no reliable performance prediction: improvement associated with DSH status is a result of K Region was a factor in HAPU and ICU mortality, poorer hospital care, more challenging and unmeasured but no discernable pattern existed; patient characteristics, or both requires more in-depth analysis using data sets that include socio-demographic K Ownership status produced two strikingly measures. different results: Investor-owned hospitals had less improvement in one surgical improvement measure, Membership in a Large System while tax-exempt hospitals were weakly associated Membership in a health system of more than ten with improved patient experience; and hospitals — regardless of profitability — is significantly K Gross system revenue was related to better related to one heart failure care and two surgical performance in CABG mortality and CABG improvement measures. Also, membership in a medium- improvement but is not indicated in any other to-large system (four or more hospitals) is positively clinical area. associated with improvement in the pneumonia quality measure. However, system size was not in the final model While statistically these three domains have some role for either performance or improvement in any of the in each of the above-noted clinical conditions, they may outcome measures. This suggests that while large health well be markers for other, unreported aspects of hospital systems may be better able to improve process measures, performance and quite probably are not factors in the large size has no effect on overall clinical outcomes. larger view of hospital performance. This modest relationship between large system size DSH Status and a few improvement measures could stem from DSH designation was a significant negative factor in several factors outside this study’s data: 1) including four models: CABG mortality, ICU mortality, patient core measure performance in executive compensation experience, and improvement in heart failure care. On formulas (data that is not publicly available); 2) putting the other hand, DSH status had a significant positive corporate resources toward improvement at the expense Measure for Measure: Analyzing California Hospital Characteristics and Performance  |  3 of other priorities; 3) sharing effective improvement better performance and safety (the Healthcare Leadership practices among system hospitals; and 4) using corporate and Quality Assessment Tool), as well as examination of information systems and other technology to drive executive compensation programs that reward improved improvement. These and other possible factors require performance on items like CHART measures. Similarly, further study and analysis. physician engagement may be crucial with regard to transparency and public reporting, but this factor is Percentage of Gross Revenue Collected very difficult to assess because standardized tools and The percentage of gross revenue collected reflects the approaches are not available. actual amount received from patients and third-party payers. In this study, a higher percentage of gross revenue Staffing levels in hospital quality departments also may be collected correlated with better performance on four a driver of improved clinical performance. However, the quality indicators: ICU mortality, patient satisfaction, exact makeup and responsibilities of a quality department and improvement in patient satisfaction and one surgical are difficult to categorize. Some hospitals have begun to infection prevention composite (SIP 3). On the other move quality department responsibility for specific clinical hand, for unknown reasons, this variable is negatively measures toward the operational unit involved, and associated with improvement in a different surgical differences in these practices would make comparisons infection prevention measure (SIP 1). among hospitals difficult. It is noteworthy that the data demonstrated that gross Specific hospital departmental structures and practices revenue collected was more important with respect to within operational units were not studied in this project performance and improvement than any studied measure (except for the Leapfrog group standard involving of the level of revenue — gross inpatient revenue, pre-tax intensivists). There are now standardized measures with net income, and net income. which to examine such practices and structures across a large group of hospitals, however, and other studies Initial Performance Level should take up this task. Finally, this study included For all improvement measures, initial starting value was only one proxy (from Leapfrog, related to computerized the dominant element in every one of this study’s models. physician order entry) regarding the impact of health In several cases, this factor alone explained between 25 information technology. Since this technology is now a percent and 50 percent of the variance. Of course, this is major focus in both hospital market and health policy not surprising (and has been described elsewhere) since circles, it should be studied against actual performance. the absolute opportunity to improve is the greatest with the weakest performers, while the strongest performers Implications have a cap (at 100 percent) on their ability to improve.4 The study’s analysis of hospital characteristics against a range of clinical performance measures may have Study Limitations significant implications regarding public reporting of The study’s list of hospital attributes is by no means such measures. The findings also may have specific public exhaustive. With regard to improvement, many experts policy implications regarding DSH facilities. believe that leadership participation and skill play a role. Future study could include a survey instrument Performance Improvement Strategies recently adopted by the Centers for Medicare & Medicaid This study found that clinical performance and Services to assess leadership practices associated with improvement are predominantly driven not by structural 4  |  California HealthCare Foundation hospital attributes but by the reliable implementation of that the categories will reveal differences in performance. effective practices. This includes attention to execution, (DSH status may yet prove to be an exception, but such and an understanding of work flow abetted by techniques stratification should be withheld unless and until it is gleaned from the growing field of improvement science.5 clear that performance differences are due primarily to Some organizational structures may yield higher or lower poorer care and not to unmeasured patient characteristics; levels of reliable implementation, but these structures are see below.) Policymakers, too, should adjust their likely to be directly related to specific practices rather than approach regarding matters of performance, ignoring to broad constructs such as hospital size, teaching status, these structural categories when creating legislation or ownership, hospital financial health, or the many other regulation to improve California hospital quality. attributes evaluated in this study. Source of Low Performance Based on Because of this lack of correlation between structural DSH Status characteristics and performance measures, hospitals This study’s findings suggest an urgent need to explore should emphasize, when addressing performance, the why DSH status often correlates with poor performance. execution of specific improvement practices rather The study determined that DSH facilities’ poorer than major structural or transformational strategies. performance is not associated with their financial picture. Moreover, since any hospital can improve, regardless of its If, instead, the poorer outcomes and lesser improvement structural characteristics or attributes, different types of in DSH facilities result mostly from unmeasured patient hospitals and hospital systems can adopt others’ effective socio-demographic characteristics, then risk-adjustment improvement practices with greater confidence that the models will need modification to reflect these. In practices can be transferrable and broadly effective. addition, public reporting programs might consider altering their stratification of DSH facilities to account Reporting of Improvement Measures for the differences in patient population. The study revealed several problems with the public reporting of improvement measures as information for There is a need to investigate whether and to what extent consumers or stakeholders. First, several improvement DSH facilities perform worse as the result of poorer care measures are nearly topped off and so provide little delivery. If additional research uncovers deficiencies in actionable information. Also, since improvement is often care practices, specific policies will need to attend to those most strongly predicted by a low initial starting value, by deficiencies. Ultimately, poorer care at DSH facilities definition significant improvement will slow as starting needs to be addressed regardless of its sources, whether value rises. As a result, public reporting entities may unmeasured differences in patient factors, deficient care want to alter the use of topped-off measures, or instead practices, or other undetermined causes. simply report those hospitals that perform below a chosen threshold. Conclusion The study’s findings are somewhat surprising and for This study’s overarching finding is that there appears the most part encouraging. The surprise is their clear to be no relationship between most major hospital indication that, with only a few partial exceptions, attributes — such as teaching status, hospital size, and major hospital characteristics are not significant drivers ownership — and clinical performance. Therefore, of clinical performance or improvement. Given these continuing to stratify hospitals by these categories tends findings, those who establish hospital performance to set up an unrealizable expectation among stakeholders measures may want to reconsider using structural Measure for Measure: Analyzing California Hospital Characteristics and Performance  |  5 characteristics in their rating systems. And policymakers Endnotes may do well to ignore such characteristics when 1.Kroch, E.A., M. Duan, S. Silow-Carroll, and J. Meyer. establishing legislative programs intended to improve the April 2007. “Hospital Performance Improvement: quality of hospital patient care. Overall, these findings Trends in Quality and Efficiency.” The Commonwealth are encouraging, suggesting that a hospital with any mix Fund. www.commonwealthfund.org/publications/ of characteristics may be able to perform at a high level publications_show.htm?doc_id=471264; Thornlow, as to any measure. The one set of findings that raises D.K. and G. Stukenborg. March 2006. “The Association concern relates to DSH facilities, which score poorly on Between Hospital Characteristics and Rates of Preventable several measures. Establishing the causes of this poorer Complications and Adverse Events.” Medical Care 44(3): performance — whether rooted in socio-demographic 265 – 269. factors in the patient population or in poor care 2.Jha, A.K. June 2008. “Does the Leapfrog Program Help delivery — urgently requires further study. Identify High-Quality Hospitals?” Journal on Quality and Patient Safety 34(6): 318 – 325. About the R e s e ar c h P art n e r s 3.Hines, S. and M. Joshi. June 2008. “Variation in Quality Convergence Health Consulting, Inc. is a multidisciplinary of Care Within Health Systems.” Journal on Quality and team of professionals supporting health care organizations to Patient Safety 34: 6; 326-332; Kahn, C. January/February advance patient safety, improve quality, strengthen leadership, 2006. “Snapshot of Hospital Quality Reporting and promote sustainable change, and facilitate a solution- Pay-for-Performance Under Medicare.” Health Affairs, oriented dialogue among physicians, hospitals, health plans, 25(1): 148 – 162. purchasers, the public, and other health care stakeholders. 4.Kroch, E.A. et al. Op. cit. “Hospital Performance Bruce Spurlock, M.D., president and chief executive officer Improvement.” of Convergence Health, is a national expert in quality and 5.Berwick, D.M. March 2008. “The Science of patient safety and serves as the executive director for the Improvement.” Journal of the American Medical Association California Hospital Assessment and Reporting Taskforce and 299(10): 1182 – 1184. as the executive director, clinical acceleration, for the Bay Area Patient Safety Collaborative. Rebecca Abravanel, Ph.D., is a statistician, demographer, and sociologist with research clients in several industries, including health care. Sarah Spurlock is a research assistant and student of political science and history at the University of California, Davis. About the F o u n d at i o n The California HealthCare Foundation is an independent philanthropy committed to improving the way health care is delivered and financed in California. By promoting innovations in care and broader access to information, our goal is to ensure that all Californians can get the care they need, when they need it, at a price they can afford. For more information, visit www.chcf.org. 6  |  California HealthCare Foundation