Evaluation of the California HealthCare Foundation’s Ambulatory Care Redesign Collaborative Final Report Prepared for The California HealthCare Foundation Prepared by The Lewin Group, Inc. July, 2006 387610 Table of Contents I. Introduction 2 II. Evaluation Methodology 2 A. Environmental Scan...................................................................................................... 2 B. Dashboard of Key Performance Indicators................................................................... 3 C. Interviews with Participating Clinic Staff Members ....................................................... 3 D. Site Visits to Promising Practice Sites .......................................................................... 4 III. Overview of the CAPH/SNI Ambulatory Care Process Redesign Collaborative 5 IV. Outcomes of Redesign Initiatives 7 A. Overview of Findings .................................................................................................... 7 B. Impact of Redesign on Patient Cycle Time................................................................... 8 C. Impact of Redesign on Provider Productivity .............................................................. 11 D. Impact of Redesign on Patient Satisfaction ................................................................ 13 E. Impact of Clinic Redesign on Staff Job Satisfaction Levels ........................................ 16 V. Factors Influencing Sustainability and Spread of Gains Achieved through Process Redesign 18 A. Introduction ................................................................................................................. 18 B. Factors Influencing Sustainability of Redesign Performance Improvements.............. 19 C. Factors Influencing Spread of the Redesign Model.................................................... 21 D. Supporting Sustainability and Spread beyond the Formal Redesign Training Period ........................................................................................................... 22 VI. Recommendations 23 VII. Conclusion VIII. Forms Used in Evaluation 29 Appendix A: Evaluation of the CHCF Ambulatory Care Redesign Collaborative Key Informant Discussion Guide 29 Appendix B: Evaluation of the CHCF Ambulatory Care Collaborative Focus Group Facilitation Guide 34 Appendix C: Dashboard of Key Performance Indicators Template 39 IX. Endnotes 40 i 387610 I. Introduction The past 20 years have seen significant changes in the organization and delivery of health care, including a greater focus on quality of patient care and growth in the availability and use of sophisticated medical technology. These and other changes have spurred development of care systems designed to move patients to the most clinically appropriate treatment setting, making it possible to deliver a growing proportion of medical care in ambulatory care settings.1 In response to these developments, both public and private hospitals and health systems have increasingly shifted care from inpatient to outpatient settings, built new outpatient capacity, and moved into the communities they serve to improve access and care coordination.2 As an important access point of care for the medically underserved, ambulatory care clinics operated by the nation’s safety-net providers face considerable challenges responding to these developments. These include increasing service demand as the numbers of uninsured and underinsured continue to rise. In response, safety-net hospitals and health systems provide significant amounts of outpatient care. In 2002, 81 of the National Association of Public Hospitals and Health Systems (NAPH) members provided more than 33 million outpatient visits.3 Furthermore, the average number of outpatient visits for NAPH member hospitals has increased steadily since 1993.4 Other challenges faced by safety-net providers include limited access to resources and public payment systems that have historically underfunded outpatient centers serving a high proportion of Medicaid and uninsured patients.2 Within many safety-net clinics, both patients and providers have become increasingly dissatisfied with inefficiencies frequently built into traditional ambulatory care visits. These include long wait times to obtain an appointment, frequent delays, and non-productive interactions with the clinic staff before being seen by a clinician. Staff workloads in many ambulatory care settings have also increased, often creating stress and fatigue for health care professionals and leading to lower job satisfaction and increased staff turnover.2 In the past, many safety-net providers assumed that delays in timely access to hospital or community-based primary care were an inevitable result of high service demand coupled with limited capacity. Today, there is a growing body of evidence that through ambulatory care process redesign, service demand can be predicted and the patient experience transformed through organizational commitment to clearly defined—and consistently applied—processes of care that create a more efficient infrastructure. The California HealthCare Foundation (CHCF) commissioned The Lewin Group (Lewin) to conduct an evaluation of an Ambulatory Care Redesign Collaborative, whose objectives included improving patient cycle time, provider productivity, and provider and patient satisfaction levels at nine clinic sites affiliated with five public hospital and health care systems in California. This report presents the findings of our external evaluation of this initiative. The remainder of the report is organized into the following sections: • A description of the evaluation methodology ii 387610 • An overview of the California Association of Public Hospitals and Health Care Systems (CAPH)/California Health Care Safety Net Institute (SNI) Ambulatory Care Process Redesign Collaborative • Outcomes of the CAPH/SNI Ambulatory Care Process Redesign Collaborative • The impact of clinic redesign on patient cycle time Impact of clinic redesign on provider productivity Impact of clinic redesign on patient satisfaction levels Impact of clinic redesign on staff satisfaction levels • Factors influencing sustainability and spread of gains achieved through process redesign • Recommendations and conclusion II. Evaluation Methodology This evaluation was conducted between May 2005 and February 2006. It consisted of four primary tasks: • Conducting an environmental scan to determine the nature and outcomes of ambulatory care redesign efforts nationwide • Developing a dashboard of key indicators in collaboration with an evaluation advisory committee and analyzing performance measure data periodically • Conducting 45-to-60-minute semi-structure interviews with clinic administrators, medical providers, and administrative staff members who were involved in the redesign collaborative • Carrying out site visits to four promising practice sites with demonstrated success redesigning ambulatory care processes A. Environmental Scan Lewin conducted a focused literature review and a series of semi-structured interviews to better understand what is known about ambulatory care process redesign in other settings, the impact on health care practice and productivity, how these outcomes have influenced patient and staff satisfaction levels and factors influencing sustainability. Completed in August 2005, our environmental scan indicated that a number of ambulatory care redesign models have emerged and evolved over the past decade. According to our findings, there is a continuum of redesign models from the first-generation open-access models that “carve out” a portion of the saturated patient visit schedule to respond to a rising demand for urgent care, to more comprehensive redesign collaboratives such as the models developed by Roger Coleman Associates and the Institute for Healthcare Improvement (IHI). Figure 1 graphically depicts the continuum of redesign models that were described in the report from the least to most comprehensive. 2 387610 Figure 1. Continuum of Clinic Redesign Models Most Comprehensive Comprehensive Comprehensive Redesign Redesign 3 Collaboratives Collaboratives Second Second 2 Generation Generation First First Generation Generation 1 Least Comprehensive Traditional Open Advanced Coleman Associate Model Access Access IHI Our environmental scan also indicated that many redesign initiatives are focused on improving cycle time and patient satisfaction and that the driving force for addressing cycle time in clinics was feedback from patients regarding their discontent with long wait times. Fewer clinics focused on improving provider productivity, and we identified very few articles or respondents that systematically sought to measure changes in staff satisfaction stemming from redesign initiatives. B. Dashboard of Key Performance Indicators In collaboration with the evaluation advisory committee, Lewin developed a dashboard of key performance indicators to measure changes in cycle time, productivity, and patient satisfaction resulting from the redesign initiative. Our evaluation involved periodically collecting and analyzing the data to identify trends or improvements in our key indicators. Appendix C provides the dashboard of key indicators we collected and analyzed during this evaluation. C. Interviews with Participating Clinic Staff Members In December 2005, Lewin conducted semi-structured interviews with 11 staff members from participating redesign collaborative sites. Interview topics included identifying the key components of the redesign process, outcomes to date, challenges encountered and overcome, unexpected outcomes, and the extent to which the site developed or implemented plans to sustain and spread process improvements over time. Lewin spoke with a cross-section of professionals to learn more about unique challenges or opportunities that physicians, clinic administrators, nurses, and clerical staff members may have faced during the redesign process. Figure 2 summarizes the number of individuals we interviewed by professional affiliation. 3 387610 Figure 2. Summary of Who Was Interviewed Professional Affiliation Interviewed (%) Clinic administrator 4 (36.4%) Medical provider 6 (54.5%) • Physician 1 • Nurse 5 Clerical staff member 1 (9.1%) Total 11 (100%) D. Site Visits to Promising Practice Sites During the week of January 8, 2006, Lewin conducted site visits to four promising practice sites with demonstrated success redesigning ambulatory care processes. The purpose of the site visits was to allow a concentrated period of time to be spent fully exploring the extent to which team members have been able to translate skills and knowledge acquired through the collaborative to improve patient flow, provider productivity, patient and staff satisfaction, and other aspects of organizational effectiveness and efficiency. Additionally, the site visits allowed us to identify any lessons learned from the experience, as well as any preliminary plans at the sites for ensuring ongoing sustainability and possible spread of gains achieved. Each site visit included four hour-long interviews with clinic administrators and physicians involved with the redesign initiative. We also facilitated hour-long focus groups composed of six to nine individuals from professionally diverse backgrounds (nurses, clerical/administrative staff members, physicians, and administrators). This concentrated effort allowed us to interact with a total of 10 to 14 participants at each site. Lewin visited the following promising practice sites: • Arrowhead Regional Medical Center, McKee Family Health Center • Harbor-UCLA Medical Center, Wilmington Family Medicine • LAC+USC Healthcare Network, Roybal Comprehensive Health Center • Riverside County Regional Medical Center, Pediatrics Clinic Study Limitations Although we were able to capture and analyze quantitative performance indicators and qualitative evidence as to the impact and outcome of the redesign collaborative, we encountered challenges and limitations in collecting robust outcomes data relating to changes in levels of patient satisfaction. Lewin planned to conduct a quantitative analysis of early outcomes data related to changes in patient-satisfaction levels by comparing pre-and post- redesign patient experiences. Because of the evaluation time line, we were unable to collect pre-redesign primary data. Instead, we sought to collect data as early as possible during the redesign training period—designated as Time One—to compare to post-redesign outcomes (Time Two). However, only two of the nine sites participating in the redesign collaborative collected patient satisfaction data for both Time One and Time Two. Therefore, we were unable to generalize valid 4 387610 conclusions regarding changes in patient satisfaction levels stemming from the redesign initiative across all sites. Lewin was, however, able to perform this analysis for two sites, and the findings are presented below. We were also able to qualitatively assess early changes in patient satisfaction stemming from redesign through interviews and focus groups with program stakeholders. III. Overview of the CAPH/SNI Ambulatory Care Process Redesign Collaborative The California HealthCare Foundation awarded grants to the Primary Care Development Corporation and the California Health Care Safety Net Institute to redesign ambulatory care processes and improve the patient experience. The training was conducted by Coleman Associates, a national leader in patient visit redesign. Nine primary care clinics representing five California safety-net health care delivery systems participated in the collaborative. System Clinic Sites McKee Family Health Center Arrowhead Regional Medical Center ARMC Women’s Health Clinic Roybal Comprehensive Health Center LAC+USC Healthcare Network LAC+USC Medical Center Family Care Clinic Riverside County Regional Medical Center Pediatrics Clinic San Francisco Department of Health Tom Waddell Health Center Medical Clinic Harbor-UCLA Medical Center Wilmington Family Medicine The patient visit redesign collaborative seeks to transform the patient experience, the health care workplace, and the organization as a whole through rethinking and redesigning clinic processes to achieve dramatic improvements in performance. The program is termed a collaborative because the redesign involves creating effective teams. Teams participating in this collaborative adopted a structured approach to develop and test a model to redesign the patient-visit experience in their clinics. When asked to describe the primary purpose of their redesign initiatives, most participants pointed to reducing patient visit times by eliminating most of the non-productive wait time from the patient visit. Achieving this goal was seen as a vital step toward other desired outcomes among participating safety-net clinics, including: • Increases in provider productivity and operational efficiency • Improved cycle time • Improved patient and staff satisfaction • Less frustrating and more streamlined patient visits 5 387610 Participating clinics embarked on a highly structured eight-month voyage that featured these 11 landmarks: • They began by obtaining management commitment to the goals and methods of the redesign collaborative. • They created effective cross-functional redesign teams by selecting a mix of managers, clinicians, and support staff members committed to permanently changing current practices and assuming broader work roles. • Management allowed redesign team members eight hours weekly to dedicate to redesign work. • Clinic teams tracked and mapped patient visits firsthand to understand and dissect every step of the current visit process. • Teams came together in a 1.5-day learning session where they learned how to redesign the patient visit and test their new model. • Teams returned to their clinics where, guided by coaches, they conducted eight Rapid Redesign Tests to pilot and measure model performance. • Teams came together in a second learning session to learn how to implement the model. • Teams finalized the redesigned visit process to reduce waiting times and other non-productive patient time, and ran final tests. • After final testing, teams presented the final redesign model to senior managers and received approvals and resource commitments to implement the desired clinic redesign. • Teams and managers orient, train, and coach all staff members to work together according to the new model. • Clinic redesign teams concluded their voyages with a final one-day learning session in which they learned how to identify and resolve any remaining barriers to full implementation of the new model. Arrowhead McKee Family Health Center, a San Bernardino-based safety-net clinic, is an example of a site that significantly improved patient cycle time and clinic productivity through participation in the redesign collaborative. Table 1 on the following page summarizes the salient features of the health center’s new, streamlined redesigned visit as seen from a patient’s perspective. 6 387610 Table 1. Arrowhead McKee Family Health Center’s Redesigned Patient Visit Clinic site of Patient flow care and staff Pre-visit Medical records pulled and reviewed by clinical staff before patient’s visit. activities Site: Welcome desk Patient arrives, is greeted by the flow master, who verifies appointment and radios Staff: Flow the patient’s primary care team. master Site: Exam room Patient taken to exam room by CA, who takes vital signs and prepares patient. Staff: Clinic Second CA registers patient in the exam room. assistant (CA) Site: Exam Provider enters room, reviews electronic medical record displayed on laptop, and room treats patient. Provider discharges patient or radios CA or nurse to handle follow- Staff: Provider up appointments, immunizations, etc. EXIT Patient exits clinic with all medications, appointments, etc. completed. Cycle Time Results In Minutes Productivity Results (Patients Seen/Hour) Average before redesign: 86 minutes Average before redesign: 2.96 Average after redesign: 44 minutes Average after redesign: 3.58 IV. Outcomes of Redesign Initiatives A. Overview of Findings As described in detail below, we evaluated the outcomes of the redesign initiative across the following performance indicators: • Cycle time • Provider productivity • Patient and staff satisfaction Overall, analysis of key performance indicators concludes that this initiative has been a success to date. Participating clinic staff members attributed improvements in cycle time, productivity, and patient and staff satisfaction to the redesign model and acknowledged that they probably would not have seen such dramatic improvements in the absence of the impact of redesign. Performance highlights include: • All sites observed improvements in one or more of the key performance indicators. • Participating clinic sites observed dramatic improvements in cycle time, averaging 45 percent improvement across all sites. 7 387610 • On average, provider productivity increased 36 percent, facilitated by the efforts of care teams, an important model component. • All sites indicated that patients reported being happier with the redesigned clinic experience. Patients were finding it easier to get an appointment, they were seen in a shorter period of time, and they appreciated having someone call them to remind them to come in for appointments. Data analysis from the two sites who administered surveys to measure changes in patient satisfaction indicated improved patient satisfaction scores after the redesign was implemented. • Levels of staff satisfaction varied by profession. Nurses and clinic assistants appear most challenged by the demands of the redesign model, and most medical providers reported being happier with the redesign. Although all sites saw improvements in key indicators, the results were not homogenous. Variability in cycle times, productivity, and patient and staff satisfaction may be attributed to several factors, including the clinic staffing model, access to sufficient resources (e.g., staff, exam rooms) to meet patient visit volume demands, as well as the extent to which facilities were able to fully implement the design features of the model. In addition, all sites experienced some resistance from clinic staff members during redesign implementation. Sites that made all staff members aware of the redesign, regardless of whether they were a part of the initial team, were generally more successful in diffusing the negative feedback from the resistors to change. Other successful strategies for overcoming resistance included strong and committed leadership, rewarding appropriate behavior, empowering the staff and developing a system for holding staff members accountable to the model. B. Impact of Redesign on Patient Cycle Time Improving cycle time was the single most widely cited reason for engaging in ambulatory care redesign. When asked why they wanted to improve cycle time, virtually every site acknowledged that patients and staff members were unhappy with the time spent waiting for care in the clinic and sought to create a more patient-centered and efficient process. All of the sites improved cycle time, ranging from 13 percent to 69 percent. The average across all sites was a 45 percent decrease in wait times, with most sites experiencing a 40 to 50 percent decrease. Figure 3 illustrates baseline and final cycle time performance for all redesign clinics. 8 387610 Figure 3. Trends in Cycle Time Performance (in minutes) 200 180 160 140 120 Baseline Performance 100 Final Performance 80 60 40 20 0 McKee Women Roybal OPD Medical Wilmington Family PEDS Waddell ARROWHEAD LAC-USC LAC-UCLA RIVERSIDE SF HEALTH Variations in improved cycle times may be attributed to a number of important success factors and challenges, including: • The impact of a resident versus an attending physician staffing model significantly influenced cycle time performance. Sites that experienced the most dramatic improvements in cycle time were staffed by a consistent group of attending physicians who were continually engaged in the redesign model and understood the goals of the initiative. For example, Harbor-Wilmington reduced its cycle time by 69 percent, the most significant improvement in cycle time across all clinics. The attending model at Harbor-Wilmington enabled providers to conduct patient visits efficiently without affecting quality of care. In addition, these providers were engaged in the redesign daily and fully understood what was expected of them in the new model. In contrast, clinics staffed by residents under attending supervision found that these providers were less efficient because of a lack of experience and acknowledged that there are limitations on the number of patients that residents can see during each clinic session. In addition, residents are constantly rotating in and out of the clinic setting, and this lack of continuity requires that time be spent retraining them on the basic principles of the redesign model. • An appropriate balance between patient visit volume and clinic resources is needed to ensure that cycle time goals can be achieved. Sites that were more successful in reducing patient wait times were generally more satisfied with the timely availability of resources at their facility. For instance, Arrowhead McKee’s newly designed facility offers sufficient exam rooms, a centralized nursing station, portable computer workstations, conference rooms for group visits, as well as a high-tech library for ongoing provider education and training. 9 387610 Clinics that were less successful improving cycle time frequently stated that the lack of resources (e.g., exam room equipment, computers, staff) was a contributing factor. Several clinics with a large patient visit volume reported that they did not have enough clerical staff members to review patient charts in a timely manner before each visit and that this directly affected cycle time results. • Sites that stayed true to the redesign model and fully implemented the design elements saw greater improvements in cycle time than sites that deviated from the model. Roybal Comprehensive Health Center (Roybal) decreased its cycle time from 103 minutes before the redesign to 45 minutes after the redesign, surpassing the 50- minute goal set by Coleman Associates. Roybal was one of several sites that successfully integrated all important team functions, such as flow masters or “floating nurses,” into its redesign model. However, factors such as a lack of resources, an inability to hire staff members in a timely manner, and space constraints affected some sites’ ability to fully implement the various features of the redesign model. • Making all staff members aware of redesign goals and objectives at the beginning of the initiative can reduce the amount of resistance from the clinic’s staff. Virtually every site experienced some resistance from the staff when implementing the redesigned patient-visit process. To reduce resistance, several sites held staff meetings early on in the planning and implementation process to communicate to all staff members that the redesign would ultimately be implemented in every clinic and would affect all staff roles, even though some staff members were not part of the initial redesign team. In addition, leaders at the sites consistently communicated to resistors, and all staff members, that redesign was not optional but was a new, permanent patient-visit experience. One of the challenges in holding the staff accountable to the redesign goals and objectives in public safety-net settings is the presence of unions. Several staff members reported that this factor can complicate ensuring that all staff members are committed to the redesign. They noted, however, that this challenge can be minimized by early and frequent communication with union representatives regarding the staffing requirements of the redesign initiative. The Harbor Wilmington case study (see sidebar) illustrates some of the factors that contributed to Harbor Wilmington’s success as well as the aspects of the model that the clinic’s staff found most useful in reducing cycle time. 10 387610 Improving Cycle Time at Harbor Wilmington What were Harbor Wilmington’s baseline and final cycle time results? Harbor Wilmington saw dramatic improvements in cycle time. Before redesign, the average cycle time was 107 minutes. After the redesign, cycle time dropped by 74 minutes to 33 minutes, a 69 percent improvement in cycle time. What factors contributed to the success? First, the overwhelming majority of the staff was dedicated to the redesign model and felt empowered to make changes without awaiting approval from the administrative staff, who might not have been as familiar with the redesign model. For instance, staff members purchased fax machines, headsets, and scales for the exam rooms because they knew it would take a considerable amount of time to receive approval for these purchases. Second, their attending-physician-based staffing model promoted efficiency. Another contributing factor was that team members were open and honest with one another regarding the benefits and weaknesses of the redesign model. Although it was clear that certain team members were ultimately responsible for making decisions, all members were empowered to share their opinions, creating a team atmosphere in which everyone contributed to the success of the redesign. In addition, team members were constantly willing to seek new and better ways of improving the patient visit, such as by placing the nursing stations closer to each team’s exam rooms to improve efficiency. In short, they boldly faced the difficult change process. What aspects of the model did Harbor Wilmington find most useful in reducing cycle time? The huddle sheets provided team members with an opportunity to prepare for visits in advance and identify the necessary tests and procedures before patients arrived for their visits. In addition to improving the providers’ ability to ensure all necessary tests were performed, the huddle sheets also improved the quality of care that was delivered to patients. C. Impact of Redesign on Provider Productivity The impact of the redesign collaborative on provider productivity also varied considerably among participating clinic sites. On average, provider productivity increased by 36 percent with site-specific results ranging from a 13 percent decrease to a 340 percent increase in productivity. The lowest-performing site went from seeing 3.0 patients per provider hour before the redesign to 2.6 patients per provider hour after the redesign. In comparison, the highest performing site reportedly went from seeing 1.0 patient per provider hour before redesign to 4.4 patients per provider hour after the redesign. Figure 4 summarizes changes in provider productivity for all of the participating sites measured in number of patients per provider hour. 11 387610 Figure 4. Trends in Provider Productivity 5 4.5 4 3.5 3 Baseline Performance 2.5 Final Performance 2 1.5 1 0.5 0 McKee Women Roybal OPD Medical Wilmington Family PEDS Waddell ARROWHEAD LAC-USC LAC-UCLA RIVERSIDE SF HEALTH Provider productivity results varied depending on the individual characteristics of participating clinic sites as well as the features of their redesign model. Here are some of the most frequently cited factors influencing productivity. • Care teams are integral in improving provider productivity. All of our informants spoke to the advantages of creating care teams in order to increase the efficiency of the patient visit. Care teams create a sense of shared accountability for redesign and demonstrate that improved productivity stems from the efforts of a team, not a single provider or staff member. • Physician expectations should be clear from the beginning. Resident-staffed clinics whose physicians rotate in and out of the facility said that they approached improving productivity by determining how much time providers would actually spend in the clinic and defining clear expectations of provider productivity. By clearly stating expectations, physicians could be held accountable for meeting productivity targets. Stating expectations also holds providers accountable to the model and helps prevent them from digressing to the “old ways” of conducting patient visits. • Improved communication decreases the number of hand-offs and can increase staff efficiency. The redesign staff at most redesign clinics found that technology that enhances communication, such as headsets, was helpful because staff members could request supplies or staff attention without leaving the patient exam room. Other tools such as “flags” on the doors and walkie-talkies decreased time spent traveling to and from exam rooms and ultimately led to more efficient patient visits. 12 387610 • Clinic utilization may increase because of external factors, which can affect improvements in provider productivity. Some clinic sites acknowledged that they saw dramatic increases in provider productivity but that they were skeptical about attributing the increases to improved efficiency as opposed to the increased demand for care in public hospitals resulting from local population growth, shifts in insurance status, demographics, and other external variables. The LAC+USC Medical Center case study (see sidebar) illustrates improved productivity in the face of staffing and resource challenges. Improving Productivity at LAC+USC Medical Center What was the motivation for improving productivity? LAC+USC Medical Center wanted to ensure that operations were working efficiently because the clinic was preparing for a move to a new facility. The move entailed consolidating services into one building, so organizers had to be thoughtful about how they used their limited space. Did provider productivity increase as a result of the redesign? LAC+USC Medical Center saw modest improvements in provider productivity. Before the redesign, providers were seeing 1.3 patients per hour, and after the redesign, provider productivity increased by 8 percent to 1.4 patients an hour. Although the increase was not substantial, LAC+USC Medical Center’s results demonstrate that even under challenging circumstances (i.e., resident model, high patient-visit volume, space constraints) gains from redesign are possible. How did LAC+USC Medical Center leaders overcome the inherent challenges to redesign at their facility? LAC+USC Medical Center relies heavily on residents. The resident model posed challenges for the facility because the providers’ dual priorities of education and patient care can limit productivity gains. One way that LAC+USC Medical Center addressed this issue was to get the attending staff’s support for teaching the residents ways of conducting a more efficient patient visit. The medical center also gained support from the hospital CEO and other officers. LAC+USC Medical Center’s staffing model before the redesign consisted primarily of part-time staff members. Recognizing the importance of retaining staff members that were already trained in redesign, the clinic’s leadership worked hard to obtain the necessary management approval to hire those individuals full time. As we discovered during our environmental scan, increasing productivity may be a “double- edged sword” for safety-net clinics. Improving productivity may lead to better access to care because it is easier for patients to obtain timely and convenient appointments. However, improving access may also worsen safety-net clinics’ financial viability because of the disproportionate share of unreimbursed indigent and under-reimbursed Medicaid patients served by many safety-net providers. We found that, in general, sites that saw an increase in productivity were also able to improve their ability to enroll patients in various public programs. However, they were not able to determine at this juncture whether improved clinic financial health followed. D. Impact of Redesign on Patient Satisfaction Redesign reportedly benefits the staff and organizations alike. However, the ultimate goal of redesign is to create a more patient-centered process. To that end, improving patient satisfaction is an integral part of redesign. Virtually all of our informants reported that, as a result of the redesign, patients were finding it easier to get appointments, they were seen in a shorter period of time, and they appreciated having someone call them to remind them to come in for important appointments. 13 387610 Those who administered their own internal surveys reported improved patient satisfaction scores after their redesign initiative. However, as described in our discussion of study limitations above, we found mixed success in systematically measuring changes in patient satisfaction levels. Most important, very few sites completed both the initial and final patient satisfaction surveys, so the results of our analysis are based on a small sample size. Therefore, with the exception of the two clinics at Arrowhead Regional Medical Center that administered both requested patient satisfaction surveys, most information regarding the impact of redesign on patient satisfaction is incomplete or anecdotal. (See sidebar for a sample of the most common comments from patients at the participating clinics.) Comments from Patients about the Redesign • I love the change; keep it please! • Excellent! Please continue with this! • You mean I’m finished?! • Is this a redesign day? I want to have my appointment on a redesign day! • My visit was so fast today! • Not having to stand in any lines was great! • What a surprise to have someone at the door waiting for me! • The visit seemed much more personal! • I love not having to check in at the office! • You guys are getting high-tech! • This is better than a private office! • I was not satisfied before today, but [now] I like the staff. • I hope more clinics do this. To complement anecdotal data, Lewin created a patient satisfaction survey to better understand whether there were noticeable improvements in patient satisfaction resulting from the redesign. The following closed-ended questions, in both English and Spanish, were included in the patient satisfaction survey: Question 1: How would you rate your satisfaction with getting through to the office by phone? Question 2: How would you rate your satisfaction with the length of time you waited to get your appointment today? Question 3: How long (in minutes) did you have to wait today to be seen by a provider? Question 4: How would you rate your satisfaction with the length of time you waited in the clinic today to see a provider? Question 5: Did you see the clinician, or team member, that you wanted to see today? Question 6: How would you rate your satisfaction with the personal manner of the clinician, or team member, today (courtesy, respect, sensitivity, friendliness)? Question 7: How would you rate your satisfaction with the personal manner of other clinic staff you interacted with today? Question 8: How would you rate your satisfaction with the amount of time you spent with the person you saw today? Question 9: How would you rate your overall level of satisfaction with your clinic visit today? 14 387610 Both of Arrowhead Regional Medical Center’s clinics completed the initial (Time One) and final patient satisfaction surveys (Time Two). Table 2 summarizes the patient satisfaction survey results at Arrowhead McKee Family Health Center. Table 3 summarizes the patient satisfaction survey results at Arrowhead Women’s. Table 2. Trends in Patient Satisfaction Survey Results (McKee Family Health) Initial Final (Time (Time Question One) Two) Percent Change How would you rate your satisfaction with getting 2.65 2.85 7.55% through to the office by phone? How would you rate your satisfaction with the length of 3.08 3.53 14.61% time you waited to get your appointment today? How long (in minutes) did you have to wait today to be 3.65 4.56 24.93% seen by a provider? How would you rate your satisfaction with the length of 3.05 3.74 22.62% time you waited in the clinic today to see a provider? Did you see the clinician, or team member, that you 3.65 4.60 26.03% wanted to see today? How would you rate your satisfaction with the personal 4.24 4.25 0.24% manner of the clinician, or team member, today? How would you rate your satisfaction with the personal 3.79 4.30 13.46% manner of other clinic staff you interacted with today? How would you rate your satisfaction with the amount 4.16 4.11 -1.20% of time you spent with the person you saw today? How would you rate your overall level of satisfaction 3.58 4.10 14.53% with your clinic visit today? AVERAGE 3.54 4.00 13.16% On a scale from one (poor) to five (excellent), patients at Arrowhead McKee Family Health Center reported a 13 percent improvement overall in satisfaction with the visit experience. Patients appeared particularly pleased with issues related to cycle time. This suggests that McKee’s excellent performance in reducing the cycle from 86 to 45 minutes has already had a positive influence on patient satisfaction. 15 387610 Table 3. Trends in Patient Satisfaction Survey Results (Arrowhead Women’s) Initial Final (Time (Time Question One) Two) Percent Change How would you rate your satisfaction with getting 3.23 3.46 7.12% through to the office by phone? How would you rate your satisfaction with the length of 3.08 3.00 -2.60% time you waited to get your appointment today? How long (in minutes) did you have to wait today to be 3.80 3.08 -18.95% seen by a provider? How would you rate your satisfaction with the length of 3.04 3.00 -1.32% time you waited in the clinic today to see a provider? Did you see the clinician, or team member, that you 2.76 4.47 61.96% wanted to see today? How would you rate your satisfaction with the personal 3.79 4.38 15.57% manner of the clinician, or team member, today? How would you rate your satisfaction with the personal 3.72 4.13 11.02% manner of other clinic staff you interacted with today? How would you rate your satisfaction with the amount 3.64 4.20 15.38% of time you spent with the person you saw today? How would you rate your overall level of satisfaction 3.55 3.86 8.73% with your clinic visit today? AVERAGE 3.40 3.73 9.70% The Arrowhead McKee Family Health Center saw greater overall improvements in patient satisfaction compared with Arrowhead Women’s (13.16 percent and 9.70 percent, respectively). Patients at Arrowhead Women’s reported growing dissatisfaction with their wait times in general. To some extent, this may be correlated to this clinic’s relatively modest improvement in cycle time compared to other sites, falling from 150 to 130 minutes. We also observed sizable improvements at this clinic in patient satisfaction with the personal manner of clinicians and other staff members. These findings, although only illustrative, suggest a relationship between improved cycle time and patient satisfaction. They may also suggest that modest gains in cycle time alone are not enough to improve patient satisfaction with time factors involved in their clinic experience. There may be a performance threshold that needs to be crossed before patients translate improved performance to heightened satisfaction with the visit. This is an area worthy of further research. E. Impact of Clinic Redesign on Staff Job Satisfaction Levels Very few redesign sites have formal methods of collecting data regarding changes in staff satisfaction. Most of the data currently available is qualitative and anecdotal. (See sidebar for staff comments.) 16 387610 Comments from Redesign Staff Members’ about Redesign • [Redesign] makes my day go faster! • I thought it would be harder, but it was easier! • When can we implement redesign in our own clinic? • It was helpful to review charts in advance so we could prepare for the visit. • I liked seeing the patient without having to hear about how long they had to wait. • The increased accountability makes for less errors. • It’s great to have the special attention and support from the nurses. • I’ve never seen this many patients in this amount of time! Several themes emerged during our informant interviews and site visits that reflect staff perceptions of the redesign model and its impact on their job satisfaction levels. • Redesign improves staff accountability, and underperforming staff members are generally the least satisfied with the redesign. We discovered during our environmental scan that it is difficult to identify appropriate measures of staff satisfaction. For example, several sites originally used changes in staff turnover as a measure of staff satisfaction. However, they soon realized the fallacy of assuming that increases in staff turnover are always a negative outcome of redesign, after finding that it was “low-performing” staff members that were leaving in response to redesign-related higher standards of performance and accountability. Our key informants confirmed this finding. In particular, staff members found that the redesign leads to an increase in accountability for staff members, and those that were not working efficiently before the redesign were the least satisfied with the outcomes of redesign because their workloads increased. • Staff satisfaction varies within (and across) sites because of staffing level issues. Several facilities reported lacking the key staff members necessary to optimize the redesign experience, such as greeters or clerks. This resulted in increased workloads for other staff members and led to some discontent with the redesign model. For example, as reported by a number of sites, the clinic assistants (CAs) are the “workhorses” that help make the model work efficiently. Insufficient numbers of CAs increased the workload for the remainder of the staff. In addition, some staff members reported that there were few monetary or other incentives for working harder and assuming broader work roles, as called for by the model. • Staff members were generally pleased with the impact of the redesign on patient satisfaction. Staff members consistently stated during informant interviews and site visits that the redesign dramatically improved patient satisfaction, and they were happy that there were fewer complaints from patients. In general, despite some reservations, few staff members indicated that they wanted to go back to the old model. The Tom Waddell Health Center case study (see sidebar) highlights what staff members liked and disliked about the redesign model. 17 387610 Tom Waddell Health Center What did the staff like about redesign? The Tom Waddell Health Center followed the Coleman Associates model and moved all patients to one exam room, created care teams, and implemented chart review/huddle time before the clinic started. Staff members were very happy with the time spent preparing for the visit and thought the huddle sheets were very successful. They also learned to work well together in their care teams and felt that everyone was involved in the redesign. Staff members also acknowledged that they now have a better way of accommodating drop-ins, and they like having someone dedicated to problem-solving when patients are not being seen efficiently. What didn’t they like about redesign? Nurses seemed the most disgruntled with the redesign because they felt their roles changed the most. Nurses also did not like the headsets and walkie-talkies because they felt they were offensive to patients and were somewhat distracting. What do they wish they had done differently to decrease the level of staff resistance to redesign? In general, they wish they had involved all of the clinic staff members right from the beginning to give them an opportunity to express their concerns and alleviate fears about the process. However, they also acknowledged that it would have been difficult to involve the staff earlier because the redesign leaders had to learn more about the model themselves and leave sufficient time to create their teams. V. Factors Influencing Sustainability and Spread of Gains Achieved through Process Redesign A. Introduction The gains reported by participants in the ambulatory care redesign collaborative have been impressive and were achieved in a short period of time, across varied settings, and in the face of challenges common to safety-net health systems. Achieving the goals of improved cycle time, provider productivity, and patient and staff satisfaction required strong leadership, dedicated staffs, resources, and a commitment to the process of changing how care is provided to patients. However, virtually all of our key informants agreed that sustaining these gains and spreading the model to other clinic settings is an altogether different and more daunting challenge. We consistently found throughout our interviews and site visits an undercurrent of concern that the gains achieved by sites participating in the redesign initiative are tenuous and may require varying levels of ongoing support and commitment. It is important to acknowledge the influence of time on our ability to assess factors influencing the sustainability of this redesign initiative. This evaluation was completed only four months after the final redesign collaborative learning session concluded in October 2005. During this short period, many sites worked feverishly to carry out the challenging task of implementing their redesign models while simultaneously continuing to deliver quality health care to their patient panels. As a result, many are only now focusing on developing plans to sustain their achievements. Given these factors, we were encouraged by the commitment to sustainability we encountered among those who have implemented their respective change initiatives. We also anticipate that as the remaining sites finish implementing their redesign work and participate in the planned 18 387610 redesign follow-on activities described below, sustainability planning will emerge as a priority for them as well. B. Factors Influencing Sustainability of Redesign Performance Improvements Based on our interviews and focus groups with teams and individuals participating in the redesign collaborative, several factors emerged that are likely to influence the sustainability of gains achieved through process redesign. A discussion of each follows. Leadership and Organizational Commitment to Sustainability Leaders must clearly communicate the goals of redesign to all staff members to ensure they understand the value of redesign to the organization. We found general consensus that without consistent, engaged leadership commitment, teams will be challenged to sustain the gains they reached during the redesign process. We found that sites that have achieved the most impressive gains in performance to date are led by change agents that are planning and proactively engaging in activities to institutionalize key design features and permanently alter individual and collective work behaviors. These include incorporating redesign into clinic administrative structures and activities such as holding weekly staff meetings to keep the staff engaged in redesign, and creating committees that focus on different aspects of the redesign, including promotional activities. Another promising practice we found for sustaining redesign results is to rotate staff members that are not part of the redesign through the clinic. This allows individuals who are not familiar with redesign to learn more about the principles and, ultimately, help spread it to other clinic sites. We found that at several sites, a lack of consistently committed senior leadership at the health system level and beyond was seen as an obstacle impeding the rapid uptake of redesign initiatives and sustainability planning. Examples ranged from county boards of supervisors reluctant to expedite the resources needed to implement change, to health system administrators whose original enthusiasm subsequently dimmed. “It would be nice to have some local authority to make decisions. Everything has to go through the board of supervisors and they don’t understand the sense of urgency.” As a result, several participants in the collaborative have been unable to focus on sustainability, largely because they have encountered delays in completing their redesign work. Typical challenges they report encountering include delays in gaining approval to hire new or replacement staff members, purchase IT and other equipment, implement staff training, and undertake needed clinic renovations. These findings demonstrate the importance of clinic leaders who can proactively market and disseminate the value of the redesign work to internal and external stakeholders. For example, contacting, disseminating information to, and delivering presentations to external stakeholders such as county commissioners may help lift the profile and highlight the importance of the redesign work. The literature, as well as our key informant interviews and focus groups, also highlighted the importance of clinical and administrative leaders effectively communicating and establishing 19 387610 accountability for performance. This helps ground the model organizationally and ensures that staff members understand the importance of their roles in the redesign. Other common internal barriers encountered and overcome by effective redesign leaders were staff resistance to change and insufficient time spent obtaining support from clinic and hospital administrative staff members before implementing change initiatives. One site addressed the problem of resistors by pairing those resistors with staff members who support the model. Virtually all of our key informants also emphasized the importance of obtaining physician buy- in before implementing a redesign initiative. In general, redesign leaders appeared effective in demonstrating the value of the model to physicians. Once physicians immersed themselves in the model, they generally appreciated the richer level of staff support and the greater resemblance of their redesigned workplace to an office-based medical practice environment. Another challenge to sustainability faced by redesign leaders stems from the nature of clinical staffing in safety-net clinics participating in the redesign collaborative. As described above, in most clinics care is organized around a resident-based system under attending supervision. This creates two challenges in achieving and sustaining redesign goals: • Residents rotate into and out of redesign clinics as part of their clinical training program. The resulting limited exposure to, and lack of continuity with, the elements of redesign can complicate the important process of achieving buy-in and effective teamwork. And as soon as residents become familiar with the redesign model, it is often time for them to rotate out to other areas or to graduate from their residency training programs. • The resident-driven staffing model emphasizes education in partnership with direct patient care. These dual priorities, coupled with resident’s relative inexperience, often increases the average amount of time spent evaluating and treating patients compared to more experienced physicians. These factors present challenges in meeting and maintaining cycle time and productivity performance goals. “A resident-driven model slows you down about 40 percent because of a lack of experience, especially among PGY-1 and PGY-2 levels. They see two to three patients per hour compared to more for the more experienced PGY-3s.” Despite these challenges, the following quote typifies the responses from sites that have not yet focused on sustainability planning and points to the importance of leadership commitment to the process. “Developing a sustainability plan is on the list of things to do. We need administration to keep redesign as a priority, need the support of higher level administration and need to plan for sustainability once we have the redesign up and running.” Ongoing Performance Measurement Some sites acknowledged the importance of ongoing performance measurement to regularly track performance trends in relation to benchmarks. Other sites, however, reported that they stopped collecting trends in cycle time and productivity measures after reaching their redesign goals. They also acknowledged that failure to continue ongoing performance measurement may contribute to possible future declines in performance. 20 387610 “By documenting how we are doing, we can know this is a priority and people want this to continue working, as opposed to many times when people work for two months and then go back to the way they were.” Actively focusing on maintaining ongoing performance measurement will enable redesigned clinics to determine whether early gains continue to be sustained and advanced and take remedial action when necessary. As we found in examining other redesign initiatives, it is clear that longer-term sustainability of redesign gains requires a commitment to ongoing performance measurement and evaluation to avoid entropy. Continuous Staff Training and Empowerment We found that adherence to a cross-functional team model helps prevent the departmental isolation that often impaired effective teamwork and communication. It also avoids unnecessary staff specialization and encourages the emergence and empowerment of leaders and other high- performing staff members. However, staff members require some level of ongoing training to compensate for turnover, redesign-related changes in their roles and responsibilities and, in some cases, a tendency to revert to “the old way of doing things.” Empowering redesign staff members at all levels is also extremely important in sustaining effective redesign teams. Sites stressed allowing staff members to be change agents through ongoing involvement in making suggestions and sharing and testing proposals for improving clinic performance. Staff Incentive/Support Programs Tying compensation to care teams’ performance, or individual staff member’s performance, can be effective in sustaining high performance levels in the long run. However, our respondents repeatedly pointed out that public safety-net systems are challenged in financially rewarding outstanding performance. In most cases, these are currently limited to more intangible forms of appreciation and acknowledgement of superior performance. Over time, this feature, common in safety-net settings, may dampen staff creativity and enthusiasm for sustaining a model that for many is more demanding than the traditional way of treating patients. C. Factors Influencing Spread of the Redesign Model Plans for spreading the redesign model to other clinics are underway at several participating sites. The initial targets for replication include a variety of primary care settings, including pediatrics, internal medicine, and family health. Thereafter, there is some interest in spreading to specialty- care clinics, although the model may have to be modified to reflect differences in staffing mix and other factors in these settings. These sites plan to stage the spread one site at a time, provide the necessary technology, such as portable computers and walkie-talkies, and work with the new partner throughout the entire process. If sufficient resources become available and leadership commitment remains strong, this staged “train-the-trainers” approach may prove effective. “We have a multidisciplinary group called the outpatient flow redesign committee. It looks at activities inside and outside the clinic, whether it’s a medical records issue, or how referrals are coming in and getting scheduled, looking at the flow, looking at the processes. This was great because it gave us the backbone to build on for future clinics.” 21 387610 Factors influencing spread of the redesign model are generally similar to those influencing sustainability of gains achieved at current sites. Respondents pointed to these factors for successfully spreading this model: • Mobilizing the support of key stakeholders from the beginning. • Having a plan to guide the process. The plan should describe up-front goals and objectives, milestones, and performance targets and should indicate required staffing and IT resources. • Having current staff members familiar with the model available to provide advice and technical assistance to new clinic sites. • Selecting positive, energetic staff members to spearhead the new redesign initiative. • Staying within union job descriptions when creating high-performance teams. “We plan to spread the model to two off-site clinics and have developed a two-year phase- in plan. Staffing and keeping providers bought-into the model may be issues in spread absent administrative support.” D. Supporting Sustainability and Spread beyond the Formal Redesign Training Period Most sites informed us that they experienced a “disconnect” at the end of the formal redesign collaborative. They felt disappointed that, after a period of intensely structured activities, they were suddenly on their own and no longer received external support (e.g., Coleman Associates’ coaches) for maintaining or expanding the accomplishments they had achieved. Their experiences suggest that this aspect of the redesign model may benefit from periodic provision of technical assistance to increase the likelihood that their achievements can be sustained and serve as a platform for broader replication. The California HealthCare Foundation and Safety Net Institute are sensitive to factors influencing sustainability and further spreading of their ambulatory redesign initiatives in California safety-net systems. In response to these issues, they plan to launch a follow-on initiative in 2006. The initiative will include a diffusion component designed both to sustain performance improvements and to facilitate spread of the model to additional safety-net clinic sites. Using group learning sessions and a “train the trainer” model, the diffusion collaborative will have two primary goals: • To sustain the gains achieved by the 2005 clinic sites that participated in the first redesign collaborative. • To spread successful redesign models to additional clinics within the safety-net systems that participated in the 2005 redesign collaborative. In this spirit, redesign participants strongly supported bringing teams together periodically to work on identifying and eliminating barriers to sustaining their hard-won performance improvements. Technical assistance topics most frequently recommended by redesign teams include: • Team-building exercises to help strengthen and institutionalize the changing roles called for by the model, such as multitasking. Team-building exercises would also help 22 387610 integrate new clinic staff members and resistant staff members more seamlessly into the model. • Sharing of best practices from other clinics that have adopted the redesign was a technical assistance topic strongly recommended by virtually all sites. This information- sharing would allow the groups to collectively solve problems, such as resource issues, and identify additional areas of improvement. • Identifying or developing staffing models to help address issues associated with rotating residency training programs and nursing shortages in safety-net clinics. This technical assistance topic would target areas of particular concern in safety-net settings. • Sharing the current state of the art in streamlining the patient scheduling and registration systems, from organizing chart rooms to using wireless technology more effectively in patient registration. VI. Recommendations The following recommendations build upon the demonstrated strengths of this redesign model and are designed to further strengthen an already robust program. They are organized to highlight opportunities for the California HealthCare Foundation and Safety Net Institute to refine the current redesign model and facilitate dissemination of successful redesign outcomes. • Conduct on-site assessments of future candidates for participation in the redesign program before finalizing the grantee selection process. Conducting site visits to potential grantees before finalizing funding decisions would allow the CHCF and SNI to more objectively assess clinic readiness for optimizing the collaborative experience. The proposed on-site assessments might include examining such success factors as leadership commitment to redesign and sustainability, adequacy of clinic staffing, space capacity, and IT infrastructure. On-site readiness assessments may increase the likelihood of success in meeting redesign goals and help achieve greater uniformity of outcomes across participating clinic sites. • Instill more transparency into the planning and executive of the redesign journey. Many sites pointed to the need for more communication and time up-front clarifying the training process, timelines, and resources required. This would allow physicians to adjust patient scheduling and administrators to fill in for staff members during learning sessions. It would also allow sites to identify IT infrastructure needs—such as computers on wheels and walkie-talkies—earlier and initiate often-cumbersome public procurement processes. • Allow more flexibility during the redesign training process. A number of clinics noted that the current training approach is a very pre-set process and is not always as flexible as desired for addressing the variety and complexity of issues commonly encountered in safety-net clinic settings. Examples include accommodating staffing limitations associated with resident-based clinics, the requirements of unionized work settings, and time-consuming resource procurement processes. Participating clinic staff members and redesign coaches should consider convening early to identify issues that may require modifying the redesign model to ensure optimal effectiveness. 23 387610 • Refine the objectives of the redesign model to incorporate quality goals and measures. Several participating clinics recommended that the redesign model reach beyond the current focus on improving cycle time and productivity to incorporate a subset of quality goals and measure improvement targets around particular areas of interest. The Institute for Healthcare Improvement through the Idealized Design of Clinical Office Practices (IDCOP) initiative has examined ways to improve chronic care and other health care outcomes in outpatient settings. CHCF may wish to consider this refinement for future redesign collaboratives, for it would fit well with the chronic care needs of many patients presenting in safety-net clinics. For example, Table 4 depicts trends in selected chronic care outcomes from a sample of pilot sites participating in the IDCOP initiative that redesigned their care systems.5 Table 5. Trends in Chronic Care Outcomes at Selected Sites after Redesign Measure Outcomes Clinical site Percent of patients with diabetes receiving 100% (increased from 82% in Bellin Health System, HbA1c test every six months previous year) Green Bay, Wis. Percent of patients with LDL cholesterol 90% (increased from 64% in Buffalo Medical Group, less than 100 previous year) Ocean Park, N.Y. Percent of patients with hypertension 95% (increased from 67% in Tonawanda Medical whose blood pressure is less than or equal previous year) Associates, Buffalo, to 140/86 N.Y. • Explore opportunities to build internal and external synergies. Through the learning sessions, redesign team members reported being very satisfied with networking opportunities available while participating in the program. However, they reported a sudden “disconnect,” as described above, after the formal collaborative experience was over. A lack of ongoing connectivity may limit the full potential of the program to build an infrastructure of leaders who can help create the synergies needed to further lift the profile of ambulatory care redesign. To help address this issue, Lewin recommends that the CHCF consider funding an Ambulatory Care Redesign Collaborative alumni network or association. Its purpose would be to maximize opportunities for ongoing collaboration among graduates and future program participants. CHCF could also consider further facilitating this process by fostering collaboration between grantees and other programs or organizations with compatible interests related to strengthening and spreading ambulatory redesign. The intent would be to help create opportunities to share dialogue, insights, and methods. Doing so could help build a broader base of knowledge and support to help build the field. • Consider incorporating a sustainability module, or component, into the patient visit redesign training program. This would allow redesign teams to focus in a structured way on acquiring tools and techniques to facilitate the process of institutionalizing sustainability earlier. A sustainability component would also help minimize the “disconnect” issue described above by allowing redesign teams and trainers to focus on 24 387610 addressing challenges to sustainability through ongoing interactions with redesign trainers and during redesign training sessions. VII. Conclusion The findings from this evaluation suggest that, overall, the CHCF redesign collaborative has, in a short period of time, established a strong record of achievement in improving the operational performance and the quality of the patient experience across a group of safety-net clinics in California. Participating clinic staff members at all levels with whom we interacted value the program highly and strongly encourage its spread. They agreed that, considering the redesign collaborative’s short tenure and challenging environment, it is a resounding success to date and occupies a unique and valuable niche within the arenas of process improvement and patient- centered care in safety-net settings. Looking to the future, we believe that as the evidence of gains achievable through process redesign in safety-net clinic settings becomes more widespread in California, the experiences there and in a growing number of other public and private clinic and office-based settings nationwide may create a “tipping point.” At that point, growing awareness of the models’ impact and success may favor further rapid spread and continued evolution of ambulatory redesign models. However, to ensure future success, it will be important for all stakeholders involved with redesign to place a greater emphasis on sustainability planning. This is an area that both Lewin’s Environmental Scan of Ambulatory Care Redesign Initiatives and the findings of this evaluation suggest remains underdeveloped. 25 387610 VIII. Forms Used in Evaluation Appendix A: Evaluation of the CHCF Ambulatory Care Redesign Collaborative Key Informant Discussion Guide I. I. Background and Redesign Goals and Objectives 1. We would like to begin by learning about the reason(s) why you chose to plan and implement an ambulatory care process redesign initiative. Probes: Improve patient flow and cycle time Enhance recruitment and retention Improve patient satisfaction Increase number of patients Increase productivity (provider, staff) Improves staff morale and cohesion Improve reputation Improve quality A peer center recommended it Improve market position Outside pressures, e.g., primary care association 2. What were the major goals and objectives of the initiative? What specific aspects of clinic performance were you seeking to improve? 3. Based upon your experiences to date, what outcomes have been achieved? Probes: Improved patient flow and cycle time Improved staff satisfaction Improved patient satisfaction Improved productivity 4. Based upon your experiences to date, which aspects of your redesign initiative appear most effective, and why? Which aspects appear least effective? 5. What were the major start-up challenges faced by your center in implementing the redesign program? How did you address these issues? A-1 387610 Probes: Difficulty getting physician buy-in Lack of incentive for staff Lack of support from clinic leadership We would now like to learn more about the outcomes to date stemming from your re-design initiative(s), such as changes in provider productivity, and patient and staff satisfaction. II. Changes in Health Center Provider Productivity 1. Has total clinic utilization (in terms of numbers of visits and/or users) increased since your redesign initiative? ρ Yes ρ No ρ Unsure a. If yes, what changes in utilization were due to the initiative? 2. Have the number of encounters per provider increased since your initiative was implemented? ρ Yes ρ No b. If yes, what changes in productivity do you believe were due to your clinic redesign efforts? c. If no, why was your productivity not enhanced by the redesign efforts? III. Changes in Levels of Staff Job Satisfaction 1. Overall, how satisfied do you think clinic staff members are with their jobs? ___ Very Satisfied ___ Satisfied ___ Dissatisfied ___ Very Dissatisfied 2. How do you measure staff satisfaction? 3. To what extent has your ambulatory care redesign initiative(s) influenced staff job satisfaction? ρ Increased ρ Decreased ρ No Change ρ Unsure A-2 387610 4. If staff job satisfaction levels have changed due to the redesign initiative, please describe or give examples of redesign-related factors that have influenced increases or decreases in staff job satisfaction. 5. Have clinic staff retention and absenteeism rates risen or fallen since the initiative? If so, in your opinion, did redesign-related changes in staff job satisfaction influence these developments? 6. If the redesign initiative did not appear to change levels of staff job satisfaction, why not? IV. Changes in Patient Satisfaction 1. How satisfied do you think patients are with the quality of care at the health center? ___ Very Satisfied ___ Satisfied ___ Dissatisfied ___ Very Dissatisfied 2. How do you measure patient satisfaction? 3. From your perspective, have patient satisfaction levels improved since the redesign initiative? ρ Yes ρ No ρ Unsure a. If yes, why? Probes: Shorter cycle times More time with provider Fewer patient hand-offs and shorter wait times 4. How satisfied do you think patients are with access to care at the health center? ___ Very Satisfied A-3 387610 ___ Satisfied ___ Dissatisfied ___ Very Dissatisfied 5. From your perspective, has patient satisfaction with access to care improved since your redesign initiative? ρ Yes ρ No a. If so, what factors have had the greatest positive impact? Probes: Convenient and timely appointments More respectful and considerate staff Shorter wait times V. Challenges and Unexpected Outcomes 1. Do you think it is possible to redesign the patient visit to “design out” all unnecessary delays and inefficiencies, or will there always be some level of inefficiency involved? 2. What lessons has your center learned from implementing ambulatory care redesign? What would you do differently if you repeated the process? 3. Have you observed any unexpected/unplanned outcomes stemming from your redesign initiative? VI. Factors Influencing Sustainability of Clinic Redesign Gains 1. Did/Have you developed plans for sustaining the gains you achieved through process redesign? ρ Yes ρ No a. If yes, what are the main features of your plan? 2. Have you encountered any challenges in sustaining the gains accomplished through your redesign initiative? ρ Yes ρ No a. If yes, what types of challenges emerged? A-4 387610 Probes: Continuity of clinic leadership commitment Ability of staff to work together effectively as teams Redesign team staff turnover Lack of investment in ongoing staff training Resource issues 3. Have you been able to address any of these challenges? ρ Yes ρ No a. If yes, which challenges were addressed and how? 4. Are there challenges to sustainability that you have been unable to address to date? ρ Yes ρ No a. If yes, what are they and why have these challenges been difficult to resolve? 5. What areas of ambulatory care process redesign do you see gaining momentum in the future (i.e., emerging trends)? A-5 387610 Appendix B: Evaluation of the CHCF Ambulatory Care Redesign Collaborative Focus Group Facilitation Guide I. I. BACKGROUND AND REDESIGN GOALS AND OBJECTIVES 1. We would like to begin by learning about the reason(s) why you became involved with planning and implementing your redesign initiative. Probes: Improve patient flow and cycle time Enhance recruitment and retention Improve patient satisfaction Increase number of patients Increase productivity (provider, staff) Improve staff morale and cohesion Improve reputation Reduce inappropriate ED use Improve quality A peer center recommended it Improve market position Outside pressures, e.g., primary care association 2. What were the major goals and objectives of the redesign initiative? What specific aspects of clinic performance were you seeking to improve? Did you have any personal goals? 3. What outcomes have been achieved to date? Probes: Improved patient flow and cycle time Improved staff satisfaction Improved patient satisfaction Improved productivity 4. How important is an effective team in maximizing redesign outcomes? What factors made your team successful? How have your individual roles changed since the redesign initiative? B-1 387610 5. What were the major challenges faced by your center in implementing the redesign program? How did you address these issues? Probes: Difficulty getting physician buy-in Lack of incentive for staff Lack of support from clinic/health Resources to fund clinic system leadership renovations and hire needed staff II. CHANGES IN LEVELS OF STAFF JOB SATISFACTION We would now like to discuss in more detail the outcomes to date stemming from your redesign initiative(s), such as changes in patient and staff satisfaction. 1. Does your center measure staff satisfaction? If so, how? 2. To what extent do you think your ambulatory care redesign initiative(s) have influenced your job satisfaction? Probe by role and type of position. ρ Increased ρ Decreased ρ No Change ρ Unsure 3. If you think that staff job satisfaction levels have changed due to the redesign initiative, please describe or give examples of redesign-related factors that have influenced increases or decreases in staff job satisfaction. If no change, why not? B-2 387610 4. Have clinic staff retention rates risen or fallen since the initiative? If so, in your opinion, did redesign-related changes in staff job satisfaction influence these developments? III. CHANGES IN PATIENT SATISFACTION 1. From your perspective, have patient satisfaction levels improved since the redesign initiative? ρ Yes ρ No ρ Unsure (a) If yes, why? Probes: Shorter cycle times More time with provider Fewer patient hand-offs and shorter wait times IV. CHALLENGES AND UNEXPECTED OUTCOMES 1. What lessons have you learned from implementing ambulatory care redesign? What would you do differently if you repeated the process? Probe differences by staff title/responsibility. 2. Have you observed any unexpected/unplanned outcomes stemming from your redesign initiative? B-3 387610 V. FACTORS INFLUENCING SUSTAINABILITY AND SPREAD OF CLINIC REDESIGN GAINS 1. Have you encountered any challenges in sustaining the gains accomplished through your redesign initiative? ρ Yes ρ No If yes, what types of challenges emerged? Probes: Continuity of clinic leadership commitment Ability of staff to work together effectively as teams Redesign team staff turnover Lack of investment in ongoing staff training Resource issues 2. Have you been able to address any of these challenges? ρ Yes ρ No If yes, which challenges were addressed and how? 3. Are there challenges to sustainability that you have been unable to address to date? ρ Yes ρ No If yes, what are they and why have these challenges been difficult to resolve? B-4 387610 4. Did/Have you developed plans for sustaining the gains you achieved through process redesign? ρ Yes ρ No If yes, what are the main features of your plan? If no, do you plan to develop a plan? 5. Would some form of ongoing technical assistance help you sustain and possibly spread the gains you have achieved? If yes, what forms would be most effective? B-5 387610 Appendix C: Dashboard of Key Performance Indicators Template C-1 387610 IX. Endnotes 1. Danzon, P.M., Pauly, M.V. Insurance and new technology: from hospital to drugstore. Health Affairs (Millwood) 2001; 20(5): 86-100. 2. Rogut, L. Beyond the clinic: redefining hospital ambulatory care. Paper Series (United Hospital Fund of New York) 1997: 1-62. 3. National Association of Public Hospitals. How much outpatient care do safety net hospitals provide? National Association of Public Hospitals and Health Systems. (Accessed August 12, 2005, at http://www.naph.org/Template.cfm?Section=Issue_Briefs&template=/ ContentManagement/ContentDisplay.cfm&ContentID=4925.) 4. National Association of Public Hospitals. America's Safety Net Hospitals and Health Systems, 2002: Results of the 2002 Annual NAPH Member Survey. National Association of Public Hospitals and Health Systems. (Accessed August 12, 2005, at http://www.naph.org//Content/ContentGroups/Publications1/MON_2004_9_Character2002.pdf.) 5. White, B. Starting a revolution in office-based care. Journal of Family Practice Management 2001; 8(9): 29-35. C-2