® Reducing Low Value Care: Research Questions Identified by Researchers, Patients, Physicians, and Stakeholders Laura Esmail, PhD, MSc Daniel Wolfson, MHSA Lisa Simpson, MB, BCh, MPH, FAAP April 2016 Reducing Low Value Care: Research Questions Identified by Researchers, Patients, Physicians, and Stakeholders Acknowledgements This white paper is based on discussions conducted at a multistakeholder meeting that took place on May 12, 2015. The discussions explored how researchers and stakeholders can partner to reduce unnecessary care. Sixty-six participants rep- resenting patients, researchers, health systems, purchasers, policymakers, and practitioners discussed the current evidence base and identified future research priorities. AcademyHealth hosted the meeting in partnership with the ABIM Foundation, the Veterans Affairs Health Services Research and Development Branch (HSR&D), Kaiser Permanente, and the Hartford Foundation. Laura Esmail (AcademyHealth) and Lisa Simpson (AcademyHealth) authored the paper; Daniel Wolfson (ABIM Foundation) and Timothy Lynch (ABIM Foundation) reviewed and commented on the paper. ® 2 Reducing Low Value Care: Research Questions Identified by Researchers, Patients, Physicians, and Stakeholders Introduction 3.Explore the barriers to research into needed interventions and The problem of overuse, unnecessary care, or “low value” care is how partnerships among stakeholders can address such barriers increasingly a focus of the national health policy conversation. Esti- 4.Develop ongoing partnerships and dialogue among funders, mates suggest that as much as one-third of health care spending in payers, health plans, government agencies, providers, patients/ the United States is of low value (1). Many factors contribute to the consumers, and researchers committed to reducing unnecessary problem, including increased patient demand, information asym- care metry, perverse financial incentives for providers, and a culture of “more is better than less”(2). The complexity and multifaceted Two commissioned papers provided background for the meeting: a nature of the problem warrant a wide range of solutions. In fact, systematic review of interventions aimed at reducing the use of low health systems, policymakers, and health services researchers have value health services and a paper articulating the role of the patient been developing and testing a variety of interventions that address in reducing low value care (3,4). By focusing on the two papers’ the problem at various levels and from different perspectives. findings and engaging an expert group, the meeting built on lessons learned from earlier initiatives and looked to capitalize on opportu- To complicate the problem further, there is no common definition nities for synergy among stakeholders to overcome barriers to low of low value care, which embodies a host of factors, including the value care research and implementation. patient population, potential benefits and harms, existing alterna- tives, and financial and other costs (such as opportunity, time, The Current Evidence Base effort, and physical harm) (3). Finally, poor or limited evidence A systematic review conducted by Carrie Colla, Ph.D., and col- makes it difficult to identify low value care. For purposes of this leagues summarized what is known about the effectiveness of report, the terms unnecessary care and low value care are inter- interventions aimed at reducing the use of low value health services changeable. (3). Clinical decision support, performance feedback, and provider education, coupled with other interventions, demonstrate a suf- Several organizations have launched initiatives to address one or ficient and promising evidence base. Overall, multicomponent in- more aspects of low value care. For example, since 2012, the ABIM terventions directed at patients and providers appear to be more ef- Foundation has been coordinating the Choosing Wisely campaign, fective than single-component interventions. Pay-for-performance which promotes patient-physician conversations about unnecessary and risk-sharing contracts show some evidence of their potential medical tests and procedures. The campaign now includes more effectiveness but require additional study. At the same time, Colla than 70 specialty societies whose reach exceeds 1 million clinicians. and colleagues’ review pointed to several gaps in the literature, such While these and other initiatives show promise, the pace of change as the identification of outcomes important to patients; an examina- is slow, necessitating a better understanding of what is—and is tion of chronic longitudinal care; development of broad indicators not—succeeding. of overuse; classification of effective versus low value care in high- cost settings (i.e., inpatient, emergency department, and intensive Stakeholders and Researchers Working Together care); and the role of payer-provider-patient collaborations. to Reduce Low Value Care To focus attention on the emerging and needed evidence base, The systematic review also emphasized that advances in reducing AcademyHealth partnered with the ABIM Foundation, the Vet- low value care will require supportive systems, including (1) action- erans Affairs Health Services Research and Development Branch able definitions of low value care; (2) effective patient education (HSR&D), Kaiser Permanente, and the Hartford Foundation to campaigns; (3) evidence of effective implementation approaches; convene a meeting in May 2015. Sixty-six participants represent- (4) aligned incentives; and (5) access to data sufficiently rich to ing patients, researchers, health systems, purchasers, policymakers, permit valid measurement (i.e., a shift from a focus on claims data and practitioners gathered to discuss the current evidence base and to a look at the individual provider). prioritize future research needs. Specifically, the goals of the meet- ing were to: The Patient’s Role in Reducing Low Value Care 1.Determine what is known and what needs to be learned about Understanding the perspectives of various stakeholders with ways to reduce unnecessary care respect to low value care is essential to developing and testing new interventions. From the patient and consumer side, Amy Berman 2.Identify critical research questions associated with different types and Shannon Brownlee’s paper discussed the untapped role of the of interventions as well as ways to incorporate various stakehold- patient in reducing low value care (4). For patients, high-value care er perspectives into the development of interventions means taking into account patient values, goals, and life circum- 3 Reducing Low Value Care: Research Questions Identified by Researchers, Patients, Physicians, and Stakeholders stances. Yet, patients and the public have been noticeably absent known about the initiatives’ impact or effectiveness. Over the last from most discussions of how to define value and related research 20 years, several interventions have focused on improving patient- efforts. Brownlee and Berman identified five categories or permuta- physician communication and facilitating shared decision making, tions of low value services: (1) the use of an effective treatment, test, which involves patient education and patients’ active engagement or technology on an inappropriate patient; (2) the use of an ineffec- in decisions about their care. While existing evidence suggests tive or low value treatment, test, or technology on any patient; (3) that some patient education initiatives hold promise, the current the use of an unproven treatment, test, or technology; (4) the use of evidence base addresses only a narrow range of interventions as re- an effective but unwanted treatment (preference-sensitive care); and flected in a mere handful of published studies (3). Given the poten- (5) unnecessary discretionary hospitalization. tial impact of patient demand on the use of low value care services (1), information interventions represent an area ripe for research. Some of the major barriers to achieving value-based care include the public perception that more is better; the insufficient evidence What are the most effective mechanisms for integrating patient base for many treatments; the role of marketing in driving many values into care planning decisions? clinical decisions; the lack of a strong relationship between prices and outcomes; the absence of a patient role in the research enter- The discussion participants addressed the need for research to prise; and the lack of opportunities for public deliberation around investigate the most effective mechanisms for integrating patient value. Berman and Brownlee’s recommendations call for (1) foster- values into care planning decisions. In particular, the “patient- ing public deliberation on value to encourage transparency and the redesigned personal care plan” would identify and incorporate development of measures that matter; (2) engaging patients and patient values to shape care in complicated illness. As a practical the public in deliberations and decisions on appropriate payment matter, patients have a tendency both to overestimate the benefits amounts (with government and private payers); and (3) increas- of treatment and to underestimate adverse effects (4), and the lack ing communication and shared decision making at the point of of information and discussion about the benefit/harm trade-off un- care. They write, “There is no greater value than care and outcomes dermines appropriate care planning decisions. The use of a frame- concordant with a patient’s goals and values and yet the perspec- work or system to structure and integrate discussions of trade-offs tives of patients, their families and the public are often absent from and ensure patient engagement in care planning may yield treat- the national conversation about value in health care policy and ment decisions that accord with patient goals, improve the patient practice”(4). The engagement of patients and the public should experience, and reduce unwanted service utilization. occur at all levels—research, policy, and payer—of decision making 2. Patient Incentives and Disincentives and discussion about value in health care. Patient incentives and disincentives can reduce the delivery of low The Need for Research to Test Interventions That value care by making patients more price-sensitive and, as a result, Reduce Low Value Care encouraging them to consider the value of the care or service that Using the framework presented in the systematic review (3), meet- they may receive. Interventions related to incentives/disincentives ing participants engaged in a tabletop exercise to identify priority include forms of patient cost sharing such as value-based insurance interventions to be developed and evaluated. They discussed mea- design (1). Overall, the evidence suggests that patient cost sharing surement, data, and contextual issues related to those interventions, can be effective in reducing low value care, with effect sizes ranging along with overall barriers to and facilitators of the interventions. from 6 percent (combined colonoscopy, flexible sigmoidoscopy, Each table included a patient/consumer representative to ensure and double contrast barium enemas) to 67 percent (medical tests) that the role of patients and/or consumers was discussed. depending on the service studied (3). As Colla (3) reports, “Greater patient cost sharing decreases patient-driven utilization, including Demand-Side Mechanisms office visits, number of prescriptions, and use of outpatient and Demand-side interventions are those that seek to reduce patient inpatient services, but whether cost sharing has the potential to dif- and consumer demand for low value services. Throughout the ferentially affect low value care while maintaining use of high-value discussions, demand-side strategies focused on information-based services is incompletely understood.” Value-based insurance design approaches (i.e., patient/consumer education) and incentive-based (VBID) uses differential cost sharing to communicate the relative approaches. value of services. Evidence suggests that VBID can reduce service 1. Information Interventions utilization (3), although the effects on patient outcomes and costs Information interventions may be directed to patients and/or the are less clear (6)Recent studies, such as the EMPOWER study (Or- broader public. For example, initiatives have aimed to generate egon), showed that the offer of waivers for copayments for medi- dialogue about and explain value in health care (5), though little is cines and visits related to diabetes control led to a greater reduction 4 Reducing Low Value Care: Research Questions Identified by Researchers, Patients, Physicians, and Stakeholders in HbA1C values and mean LDL cholesterol compared to a control make trade-offs. The employer market is large and predominantly group (3,7). Additional research is warranted to understand more a fee-for-service market that offers incentives, and many employers fully how benefit design can target low value care and encourage want to self-fund. appropriate use of high-value care while granting the flexibility needed to ensure preference-sensitive care decisions. The group also discussed some highly innovative interventions. In looking to capitated or fee-for-service systems, participants What interventions are effective in reducing low value care in a proposed investment in a geographic region or provider group to high-deductible health plan environment? encourage high value/effective care. If the provider group saves money, it could reinvest its funds, sustain the innovation cycle, and Research designed to identify interventions that reduce low value deliver high value care. While designing and implementing such care in a high-deductible health plan environment would help iden- an investment model would pose a considerable challenge, the tify “levers”—mechanisms that work to reduce inappropriate care approach—yet to be tried in the health care sector—is nonetheless in a variety of contexts—and reduce health care disparities. Such promising. research would require a large sample, with randomization at the level of the plan recipient as well as consideration of the provider What is the best payment reform to encourage appropriate use of payment model. The selected interventions could target primary low value diagnostic tests for patients with chronic conditions? care clinicians who provide high levels of low value care and deliver education directed to both providers and patients. Participants discussed what type of payment reform would most likely encourage the appropriate use of low value diagnostic tests B. Supply-Side Mechanisms for patients with chronic conditions. Such reform could lead to 1. Incentives and Pay for Performance fewer false positives, fewer downstream interventions, and de- Pay for performance is a financial incentive that ties provider pay- creased overuse of services and therefore lower costs. However, ment to the achievement of specified goals. It aims to reduce inap- unintended outcomes could include missed diagnoses. A study of propriate care or increase the provision of high value care (3). The payment reform would compare shared savings versus a fee-for- limited evidence on the success of pay-for-performance initiatives service approach, and cluster randomize at the practice/clinic level shows mixed results. Of two studies reviewed by Dr. Colla and col- within markets across a single payer. Participants pointed to the leagues, one showed no effect on test ordering, whereas the other importance of measuring and accounting for contextual issues and study found a 47 percent reduction in testing (3). As payment re- variation within a practice. They also noted that a potential barrier form continues to evolve and expand in both the public and private to payment reform could be payers’ unwillingness to adopt a new sectors, more evaluation is needed to determine what models might payment mechanism. Other interventions that could be tested work in which settings to reduce low value care. concurrently include patient decision aids and feedback on patient self-reported health status and patients’ care experiences. What combination of provider payment models and insurance systems has the greatest impact on encouraging high value/effec- What is the impact of risk-sharing contracts on reducing inap- tive care? propriate care? Payment reform was a popular topic among discussion partici- Colla and colleagues (3) highlighted the need for more research on pants and the focus of two groups’ discussions. One group talked how risk-bearing contracts may produce unintended consequences. about what combination of provider payment models and insur- “Regardless of the form financial risk sharing takes, it has been ance systems has the greatest impact on encouraging high value/ shown to reduce utilization, and in some cases to reduce low value effective care. Participants discussed a comparative study across care. However, further research is necessary to determine whether three natural market segments: public insurance, exchanges, and reductions in utilization are due to reductions in inappropriate or employer self-insured. The three segments respond to different appropriate care” (3). Discussion participants did not directly ad- provider and patient incentives. Increasingly, Medicare Advantage dress risk-sharing contracts. and Medicaid are evolving into managed care models (with full or partial capitation), making it important to learn how these public 2. Clinician Information sector payment models operate. The exchanges involve narrower Interventions that use information directed to providers have been networks and more disruptive innovations in payment. Consumers the subject of considerable research and include (1) clinical deci- are highly price-sensitive in these contexts and may be willing to sion support, (2) clinician education, and (3) clinician feedback. 5 Reducing Low Value Care: Research Questions Identified by Researchers, Patients, Physicians, and Stakeholders Studies of clinical information interventions are often multicompo- make providers “hold” certain care decisions until discussion with nent, and the interventions appear to be highly effective. Dr. Colla’s their peers (otherwise known as “third-party consultation or over- review found that “interventions that combine clinical education with sight”) (3). By delaying the patient care decision, the intervention en- either decision support or feedback are most common and frequently courages the clinician to shift the decision-making process from the effective at reducing overuse” (1). Furthermore, the authors found fast-paced, intuitive environment to a slower, more thoughtful and that multicomponent interventions targeting “both the patient and deliberate moment (9). Participants would like to see a study match provider roles in overuse have the greatest potential to reduce low specialty practices and compare them to usual care. Providers would value care.” self-identify issues that require a group discussion. Payers would be involved in the study and require previous authorization if the patient What factors should determine the design of paired interventions care decision is diverted to peer discussion. Study outcomes would that include both provider-facing and patient-facing components? largely focus on the provider: the proportion of decisions made by the care team, models of positive behavior, and physician satisfac- Discussion participants emphasized that the design of paired in- tion/distress. In addition, the study would measure patient satisfac- terventions needs to embody certain considerations. For example, tion and the number of low value care decisions. Contextual issues pairing clinician-facing and patient-facing interventions is important, would include culture and infrastructure. The discussion participants but the interventions must not be symmetric. Clinician interven- recognized that the intervention could potentially reinforce biases or tions are more important when consensus exists. On the other hand, culture in an unwanted direction and therefore needs to be evaluated. patient interventions are appropriate when communication/patient 4. Clinician Education expectations/preferences are more important (i.e., in the “grey zone”). What are the best approaches to educating clinicians to change the Another consideration raised by participants is the question of how culture of overuse/low value care? to match the level of an intervention’s intrusiveness to the level of practice variation while remaining mindful of likely consequences. A Clinician education about low value care mainly involves teaching further issue relates to autonomy and where, along the continuum of clinicians to recognize and reduce low value care (3). Its effectiveness intrusiveness, an intervention should be targeted, particularly when varies with the intensity of the clinician education program. “Passive providing clinicians with data and feedback. For example, should a educational interventions, those with a narrow scope, or those with clinician information intervention be coupled with financial conse- only one educational tactic are often less successful at reducing low quences (e.g., making the physician pay for an unnecessary MRI)? value care” (3). 3. Clinical Decision Support What are the most effective collaborative decision-making models What are patient preferences for the timing and content of provider for reducing low value care? initiatives for end-of-life care, and how can we train providers to implement these initiatives effectively? Clinical decision support includes a range of interventions such as integration of information within electronic health records, admin- Participants discussed an intervention that would improve care man- istrative restrictions on test orders, third-party support, and gen- agement of the costliest 5 percent of patients who are at the end of life eral promotion of clinical pathways or guidelines through decision (last 12 months) and continuously high cost over three to five years. support tools such as apps/decision trees (i.e., point-of-care decision First, participants discussed the need for an observational study to support, administrative decision support, and staff-level decision understand patient preferences for the timing and content of provider support). Clinical decision support is the most widely studied and initiatives (including consumer segmentation to determine the appro- demonstrates enormous variation in results, but the evidence suggests priate target group). Subsequently, they proposed a four-arm, multi- that the intervention can offer an effective method of reducing low site cluster randomized controlled trial at several levels of systems and value care (3). Several groups discussed the use of clinical decision providers. The intervention would involve different types and levels support mechanisms to reduce low value care. of provider training and would look at the time spent by providers on end-of-life care planning and management. Expected outcomes How can we increase both patient and provider comfort with would include an understanding of patient end-of-life preferences uncertainty? and goals, the preferred site of death, preferences for hospice care, financial outcomes, and outcomes that matter to the patient. Several A group of discussion participants focused on the use of clinical deci- contextual issues need to be taken into account for the observational sion support strategies to increase provider comfort with uncertainty. study, including the health information technology infrastructure, The strategy of greatest interest was referral boards or protocols that provider and patient financial incentives, practice culture, specialists 6 Reducing Low Value Care: Research Questions Identified by Researchers, Patients, Physicians, and Stakeholders who may be involved in the patients’ care, and regional variations Participants discussed a provider feedback intervention that would in spending patterns. Potential barriers include a lack of data on develop profiles on a range of value measures coupled with patient- outcomes, particularly longer-term outcomes. facing reports. The participants’ research question focused on how to reduce unnecessary routine care for chronic disease. The partici- 5. Clinician Feedback pants discussed the design of a randomized controlled trial in public Clinician feedback generally involves the provision of information sector group practices. The study would investigate the propor- to physicians on their use of unnecessary care, along with sugges- tion of inappropriate care, the changes in how low value providers tions for change, information on achievable benchmarks, and tools practice, and the quality of the patient-physician relationship. Some for improvement (3). Clinician feedback is often coupled with some potentially unintended outcomes might include patients’ decision to form of clinician education (3). Overall, studies show that clinician leave a practice or a decrease in appropriate care. Among the several feedback is an effective intervention, with results ranging from an 8 contextual issues in need of consideration are financial incentives, percent reduction in avoidable laboratory tests to a 78 percent reduc- the provider contract, and the degree of managed care penetration. tion in carotid endarterectomy (CEA) surgery (3). Multicomponent Barriers to conduct of the study relate to access to clinical data, data interventions that involve performance feedback have also demon- beyond the group level, and global measures of value incorporated strated success (3). into the electronic health record. What are the best combinations of interventions that effectively C. Cross-Cutting Issues present clinician feedback alongside another intervention? 1. Measurement Direct measures of low value care need to be developed and in- As a type of nonfinancial incentive, clinician feedback works in the corporated into electronic health records. The measures will help context of professionalism, which can be an effective incentive to characterize the potential extent of low value care; identify such encourage clinicians to improve their performance. For example, care at the practice, provider, and individual levels; and guide policy some unified practices with salaried providers in the Kaiser Perma- interventions that reduce low value care (11). The development of nente health system found that benchmarking—providing clinicians the needed measures poses a challenge because the value of care is with feedback on their performance against that of their peers in the inherently tied to the clinical context in which care is delivered as same practice—was sufficient to motivate improvement in clinicians’ well as to the patient’s values and goals. performance (10). What are the most promising indicators for measuring value ac- A group of participants discussed the benefits of feedback and curately and reliably across diverse settings? provider champions/leaders. They proposed a study that would randomize control and experimental groups at the site level across While administrative data are useful in benchmarking care—pro- five regions and a range of specialties. Outcomes would look at a viding high-level snapshots that characterize low value service reduction in practice variation and an increase in evidence-based use—they do not provide the detailed clinical data needed to assess care, a reduction in costs and utilization, an increase in access to whether an individual care decision was a high value decision. care, a change in culture and attitudes, and a potential increase in As Colla describes, “[R]esearchers have developed algorithms to professional group cohesion. Other interventions could be tested identify low value service use in health services datasets, such as concurrently, including patient education, the training of facilitators, claims or electronic health records. Some have aggregated clusters and communication training for physicians. The study would need of overuse measures and begun to explore patterns and correlates to account for several contextual issues, such as provider financial of overuse” (1). Participants, however, highlighted the importance incentives, provider contracts, practice characteristics, and practice of understanding practice variation and how such variation may be culture. The discussion participants identified several barriers to the used to identify and investigate potential overuse. proposed study, including the challenge of characterizing medi- cal groups by degree of risk-bearing, sample selection, the need for How can we best incorporate the patient perspective into measures trained (and respected) facilitators to lead group discussions, and of low value care? variation in the conditions targeted for study (e.g., diabetes, conges- tive heart failure). Participants across several groups emphasized the need to develop patient-reported outcome (PRO) measures that focus on outcomes How can unnecessary routine chronic disease care be reduced? that matter to patients and accurately reflect patient values and goals. To that end, the patient’s voice needs to be integrated into all stages of the research process, but especially early in the process as topics are considered and concepts are defined (12). 7 Reducing Low Value Care: Research Questions Identified by Researchers, Patients, Physicians, and Stakeholders 2. Data Infrastructure and Rapid Learning Systems What factors are most important to the successful scale-up of an The development and improvement of outcome measures goes hand effective intervention to reduce low value care? in hand with the improvement of the data infrastructure needed to create a learning health system that permits the real-time collec- One challenge to the scale-up of any effective intervention is the tion and analysis of and access to data. To date, most systems do not organizational will to introduce and champion change. Leaders facilitate sophisticated and real-time data capture and analysis. and clinical champions play an important role in influencing an organization’s culture and encouraging the achievement of positive What are the best practices in using electronic health records to outcomes. Evidence across a variety of business sectors suggests that target low value care? a grassroots, peer-driven approach to developing a culture of quality (e.g., provider-driven quality improvement initiatives) is highly ef- While electronic health records offer an excellent data source for fective. It is equally important to obtain buy-in from on-site prac- identifying and targeting reductions in low value services, they are titioners. It is essential to engage practitioners as early as possible. not without limitations. One promising area is natural language pro- Their involvement in the design of implementation strategies may cessing that converts audio recordings of patient-physician interac- help encourage their ownership of the process. Finally, aligning qual- tions into data, which might be more reliable than some parts of the ity measurement and payment and reporting structures spreads the electronic health record in that the recordings do not rely on recall. use of effective strategies. Conclusions How can we improve electronic data infrastructure to facilitate real- Despite a growing focus on low value care, much remains to be time feedback on value to providers, patients, and health systems? learned about which interventions are effective in reducing such care across a variety of settings, conditions, and patient populations. A group of participants discussed a vision for a transparent data set The evidence to date suggests that the most promising interventions or registry that could answer clinician and patient questions about are multicomponent and target both patient and the provider (3). comparative effectiveness at the point of care. The group proposed Future efforts will need to integrate the patient perspective into the a multicomponent intervention comprised of patient education design, implementation, and evaluation of interventions that aim information coupled with a shared decision-making approach to reduce low value care. A robust measurement and data infra- that integrates a real-time and personalized comparative effective- structure will be critical for continuous learning, dissemination, and ness research (CER) data set. The proposed study would evaluate adaptation. whether clinicians and patients use the database and, if so, the extent to which care plans are matched to patients’ desired and achievable Suggested Citation outcomes and the extent to which the intervention lowered costs. Esmail LC, Wolfson DB, and Simpson LA. “Reducing Low Value Such an approach would need to overcome several obstacles. First, Care: Research Questions Identified by Researchers, Patients, Physi- are the available data of sufficient specificity and granularity to guide cians, and Stakeholders,” AcademyHealth, April 2016. care decisions adequately for individual patients? Even if so, can clinicians and patients make sense of the data as part of a clini- References cal encounter? A more fundamental challenge is sufficient a priori 1. Colla CH. “Swimming against the Current—What Might Work to Reduce Low- identification and articulation of patient goals such that outcomes Value Care?” New England Journal of Medicine, Vol. 371, No.14, October 2, 2014, associated with those goals may be incorporated into the data set. In pp. 1280-3. addition, patient goals are dynamic such that a data set would need to measure goals and track them over time. 2. Pearl R. “Why So Much of the Health Care We Deliver Is Unnecessary— And What We Can Do about It” [Internet] [cited November 11, 2015]. Forbes/Pharma 3. Identifying, Understanding, and Scaling Up Best & Healthcare. Available at http://www.forbes.com/sites/robertpearl/2015/07/09/ Practices and Positive Outliers why-so-much-of-the-health-care-we-deliver-is-unnecessary-and-what-we-can- Once interventions are identified as effective in reducing or eliminat- do-about-it/ ing low value care in a specific setting, the next challenge is to scale up and spread or adapt the interventions. To do so, it is critical to 3. Colla CH, AJ Mainor, C Hargreaves, T Sequist, and N Morden. “Interventions understand the context for success and to identify the factors con- Aimed at Reducing Use of Low-Value Health Services: A Systematic Review.” tributing to that success, thus guiding the needed changes across an (under peer review). organization, health system, or regions. 4. Brownlee S and A Berman. Healthcare Resource Utilization: Articulating the Role of the Patient, 2015. 8 Reducing Low Value Care: Research Questions Identified by Researchers, Patients, Physicians, and Stakeholders 5. The RightCare Alliance—Lown Institute [Internet] [cited November 11, 2015]. 9. Kullgren J. “Decreasing Overuse of Low-Value Services in Clinical Care by Shift- Available at http://lowninstitute.org/take-action/join-the-right-care-alliance/ ing Physicians’ Decisions to a Pre-Encounter Period” [Internet] [cited November 11, 2015 ]. Robert Wood Johnson Foundation. Available at http://www.rwjf.org/ 6. Choudhry NK, MB Rosenthal, and A Milstein. “Assessing the Evidence for Value- en/library/grants/2013/11/decreasing-overuse-of-low-value-services-in-clinical- Based Insurance Design.” Health Affairs (Millwood), Vol. 29, No. 11, November 1, care-by-shi.html 2010, pp. 1988-94. 10. Cassel CK, PH Conway, SF Delbanco, AK Jha, RS Saunders, and TH Lee. “Getting 7. Kraemer DF, WA Kradjan, TM Bianco, and JA Low. “A Randomized Study to More Performance from Performance Measurement.” New England Journal of Assess the Impact of Pharmacist Counseling of Employer-Based Health Plan Medicine, Vol. 371, No. 23, December 4, 2014, pp. 2145-7. Beneficiaries with Diabetes. The EMPOWER Study.” J Pharm Pract, Vol. 25, No. 2, April 1, 2012, pp. 169-79. 11. Schwartz AL, BE Landon, AG Elshaug, ME Chernew, and JM McWilliams. “Mea- suring Low-Value Care in Medicare.” JAMA Intern Med, Vol. 174, No. 7, July 2014, 8. Toobin J. “How to Stop Mass Incarceration.” The New Yorker [Internet] [cited pp. 1067-76. November 11, 2015]. May 11, 2015]. Available at http://www.newyorker.com/ magazine/2015/05/11/the-milwaukee-experiment 12. Rein A, E Holve, M Hamilton Lopez, and J Winkler. “A Framework for Patient and Consumer Engagement in Evidence Generation.” iBooks [Internet] [cited November 11, 2015]. Available at https://itunes.apple.com/us/book/framework- for-patient-consumer/id569273673 9