United States Government Accountability Office Testimony Before the Subcommittee on Health, Committee on Veterans Affairs, House of Representatives VA HEALTH CARE For Release on Delivery Expected at 2:00 p.m. ET Thursday, July 13, 2017 Improvements Needed in Data and Monitoring of Clinical Productivity and Efficiency Statement of Randall B. Williamson Director, Health Care GAO-17-722T Letter Letter Chairman Wenstrup, Ranking Member Brownley, and Members of the Subcommittee: I am pleased to be here today to discuss our report on clinical productivity and efficiency at the Department of Veterans Affairs (VA). 1 As you know, VA’s total budgetary resources for its Veterans Health Administration (VHA) have increased substantially over the last decade, rising from $37.8 billion in fiscal year 2006 to $91.2 billion in fiscal year 2016. As VA’s funding levels increase, it is increasingly important that the department spend these funds wisely and ensure that VA attains high levels of productivity among its clinical services and operational efficiency to maximize veterans’ access to care and minimize costs. Beginning in fiscal year 2013, VA began implementing clinical productivity metrics to measure physician providers’ time and effort to deliver various procedures in 32 clinical specialties. 2 In addition, VA developed 12 statistical models to measure clinical efficiency at VA’s medical centers (VAMC). Under the models, VA calculates each VAMC’s utilization and expenditures for different high volume or high expenditure components of health care delivery, such as emergency department and urgent care, and determines the extent to which utilization and expenditures differ from expected levels. The Office of Productivity, Efficiency, and Staffing (OPES), within VA Central Office, is responsible for calculating both the provider productivity metrics and the VAMC efficiency models. My testimony today summarizes the findings from our recent report analyzing VA’s clinical productivity metrics and efficiency models. Accordingly, this testimony addresses (1) whether VA’s clinical productivity metrics and efficiency models provide complete and accurate information on provider productivity and VAMC efficiency and (2) VA’s efforts to monitor and improve clinical productivity and efficiency. In addition, I will highlight four key actions that we recommended in our report that VA can take to improve the completeness and accuracy of 1 GAO, VA Health Care: Improvements Needed in Data and Monitoring of Clinical Productivity and Efficiency, GAO-17-480 (Washington, D.C.: May 24, 2017). 2 In 2012, VA’s Office of Inspector General (OIG) recommended that the department establish clinical productivity metrics for providers at VA’s medical centers. VA OIG, Veterans Health Administration: Audit of Physician Staffing Levels for Specialty Care Services. 11-01827-36. (Washington, D.C.: Dec. 27, 2012). Clinical productivity refers to the workload performed by VA’s clinical providers over a given time period. Page 1 GAO-17-722T VA’s productivity metrics and efficiency models and strengthen the monitoring of clinical productivity and efficiency across VA. To conduct the work for our report, we examined the types of providers and the clinical services captured in the underlying clinical workload and staffing data that inform VA’s metrics and models, as well as the processes used to record these data. We reviewed VA documentation and interviewed officials from VA Central Office and six VAMCs, which we selected based on geographic diversity, differences in facility complexity, and variation in their providers’ performance on VA’s productivity metrics as well as variation in the VAMCs’ performance on VA’s efficiency models for fiscal year 2015. 3 We examined the monitoring and any related improvement efforts of VA Central Office, the six selected VAMCs, and the Veterans Integrated Service Networks (VISN) that are responsible for overseeing the six VAMCs. 4 We reviewed VA documentation and interviewed VA Central Office, VISN, and VAMC officials. As part of our review, we assessed the completeness and accuracy of the information provided by VA’s clinical productivity metrics and efficiency models using federal standards for internal control related to information, and we assessed VA’s monitoring efforts using federal standards for internal control for information and monitoring. 5 Further details on our scope and methodology are included in our report. The work this statement is based on was performed in accordance with generally accepted government auditing standards. 3 The six VAMCs we selected are located in Atlanta, Georgia; Baltimore, Maryland; Harlingen, Texas; Las Vegas, Nevada; Saginaw, Michigan; and Salem, Virginia. 4 VA Central Office is responsible for managing and overseeing the VA health care system and delegates certain responsibilities to its VISNs. 5 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999) and Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: September 2014). Internal control is a process effected by an entity’s oversight body, management, and other personnel that provides reasonable assurance that the objectives of an entity will be achieved. Page 2 GAO-17-722T We found that VA’s productivity metrics and efficiency models may not VA’s Metrics and provide complete and accurate information on provider productivity and Models May Not VAMC efficiency. To the extent that VA’s productivity metrics and efficiency models do not provide complete and accurate information, they Provide Complete may misrepresent the true level of productivity and efficiency across and Accurate VAMCs and limit VA’s ability to determine the extent to which its resources are being used effectively to provide health care services to Information on veterans. Clinical Productivity Specifically, we identified the following limitations with VA’s metrics and and VAMC Efficiency models: • Productivity metrics are not complete because they do not account for all providers or clinical services. Due to systems limitations, the metrics do not capture all types of providers who deliver care at VAMCs, including contract physicians and advanced practice providers, such as nurse practitioners, serving as sole providers. VA Central Office officials explained that VA data system limitations and other factors have made it difficult for VA’s productivity metrics to capture the workload for all types of providers. In addition, the metrics do not capture providers’ workload evaluating and managing hospitalized patients because VA’s data systems are not designed to fully capture providers’ workload delivering inpatient services that do not involve procedures—in particular, evaluating and managing patients who are hospitalized. • Productivity metrics may not accurately reflect the intensity of clinical workload. A 2016 VA audit shows that VA providers do not always accurately code the intensity—that is, the amount of effort needed to perform—of clinical procedures or services. As a result, VA’s productivity metrics may not accurately reflect provider productivity, as differences between providers may represent coding inaccuracies rather than true productivity differences. • Productivity metrics may not accurately reflect providers’ clinical staffing levels. Officials at five of the six selected VAMCs we visited reported that providers do not always accurately record the amount of time they spend performing clinical duties, as distinct from other duties. VA’s productivity metrics are calculated for providers’ clinical duties only. • Efficiency models may also be adversely affected by inaccurate workload and staffing data. To the extent that the intensity and amount of providers’ clinical workload are inaccurately recorded, some of VA’s efficiency models examining VAMC utilization and Page 3 GAO-17-722T expenditures may also be inaccurate. For example, the model that examines administrative efficiency requires accurate data on the amount of time VA providers spend on administrative tasks; if the time providers allocate to clinical, administrative, and other tasks is incorrect, the model may overstate or understate administrative efficiency. To improve the completeness VA’s productivity metrics, we recommended that VA expand existing productivity metrics to track the productivity of all providers of care to veterans by, for example, including contract physicians who are not VA employees as well as advance practice providers acting as sole providers. VA agreed in principle with our recommendation and stated that it plans to establish productivity performance standards for advanced practice providers, using available productivity data, by October 2017. In its response, however, VA did not provide information on whether it plans to expand its productivity metrics to include providers who are not employed by VA, such as contract physicians. In addition, to improve the accuracy of VA’s productivity metrics and efficiency models, we recommended that VA help ensure the accuracy of underlying workload and staffing data by, for example, developing training for all providers on coding clinical procedures. VA agreed in principle with our recommendation and reiterated its existing efforts to improve clinical coding accuracy. It also said that the department would reissue existing policy to VAMCs by June 2017 as well as continue to provide need- based, focused coding training to providers, as appropriate. However, VA did not provide information on how it plans to improve the accuracy of provider staffing data, which inform VA’s productivity metrics and efficiency models. Page 4 GAO-17-722T We found that VA Central Office has taken steps to help VAMCs monitor VA Central Office Has and improve provider clinical productivity but does not systematically Taken Steps to Help monitor VAMCs’ clinical productivity remediation plans and does not require and monitor remediation plans for addressing clinical inefficiency. VAMCs Monitor and As a result, VA cannot ensure that low productivity and inefficiencies are Improve Clinical identified and addressed across VA. Nor can VA systematically identify both the factors VAMCs commonly identify as contributing to low Productivity, but Does productivity and inefficiencies as well as best practices VAMCs have Not Systematically developed for addressing these issues. 6 Oversee Productivity In December 2016, VA Central Office began developing a comprehensive and Efficiency across analytical tool to help VAMCs identify the causes of low productivity at their facilities, a process that would likely occur after VA’s productivity VA metrics have identified low productivity in one or more clinical specialty at the facility. According to VA Central Office officials, the comprehensive analytical tool VA is developing—in the form of a data dashboard—is intended to centralize relevant data sources, including data on clinic utilization, veterans’ access to care, and provider workload, and thereby allow VAMC officials to more easily examine the factors contributing to low productivity. The officials told us that they expect the data dashboard to be developed in stages and rolled out to all VAMCs and VISNs over the course of 2017. While VAMCs are required to monitor VA’s productivity metrics and may take steps to improve clinical productivity, VA Central office does not have an ongoing process to systematically oversee these efforts. VA policy requires VAMCs to develop remediation plans to address any low productivity identified in their clinical specialties and submit these plans to their VISN. Our review found that three of the six selected VAMCs in our study were required to develop remediation plans, and officials from these VAMCs stated that they submitted these plans to their respective VISNs for review. However, we found that VA’s policy does not stipulate that VAMCs or VISNs are to submit approved remediation plans to VA Central Office; nor does the policy stipulate that VISNs or VA Central Office must monitor the implementation of these remediation plans to ensure their success. As a result, for example, officials at one of the VISNs we 6 In its 2012 report, the VA OIG noted that information on productivity can help VA identify best practices and those practices that should be changed or eliminated. See VA OIG, Veterans Health Administration: Audit of Physician Staffing Levels for Specialty Care Services. 11-01827-36. (Washington, D.C.: Dec. 27, 2012). Page 5 GAO-17-722T interviewed told us the VISN does not monitor the implementation of VAMCs’ remediation plans to address low productivity. Regarding VA’s efforts to monitor efficiency, we found that while VA Central Office officials encourage VAMCs to monitor and take steps to improve clinical inefficiency at their facilities, VA policy does not require VAMCs to use VA’s efficiency models and address any inefficiencies identified by them. In particular, VA has not established performance standards based on these models and does not require VAMCs to develop remediation plans to address inefficiencies. 7 According to VA Central Office officials, VA has not required VAMCs to monitor these models and address any inefficiencies because VA officials view the models solely as a tool to guide VAMCs in managing their resources. In the absence of a monitoring requirement, we found that two of the six VAMCs we visited had not taken steps to address inefficiencies identified by VA’s efficiency models. Based on our findings, we recommended that VA develop a policy requiring VAMCs to monitor and improve clinical inefficiency through a standard process, such as establishing performance standards based on VA’s efficiency models, and develop remediation plans for addressing clinical inefficiencies. VA concurred in principle with this recommendation, stating that it would require VAMCs to develop remediation plans. We also recommended that VA establish an ongoing process to systematically review VAMCs’ remediation plans and ensure that VAMCs and VISNs are successfully implementing remediation plans for addressing low clinical productivity and inefficiency. VA concurred with our recommendation and told us it plans to review, twice a year, the progress VAMCs are making in addressing low productivity and inefficiency. Chairman Wenstrup, Ranking Member Brownley, and Members of the Subcommittee, this concludes my statement. I would be pleased to respond to any questions that you may have at this time. 7 VA’s efficiency models are used to track VAMC utilization and expenditures for various health care services and compare these expenditures to expected levels. Page 6 GAO-17-722T If you or your staff members have any questions concerning this GAO Contacts & Staff testimony, please contact me at (202) 512-7114 (williamsonr@gao.gov). Acknowledgments Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Other individuals who made key contributions to this testimony include Rashmi Agarwal, Assistant Director; Michael Zose, Analyst in Charge; Krister Friday; Hannah Grow; and Jacquelyn Hamilton. (102123) Page 7 GAO-17-722T This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. The Government Accountability Office, the audit, evaluation, and investigative GAO’s Mission arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is Obtaining Copies of through GAO’s website (http://www.gao.gov). Each weekday afternoon, GAO GAO Reports and posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to http://www.gao.gov Testimony and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, http://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, LinkedIn, Twitter, and YouTube. Connect with GAO Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. Visit GAO on the web at www.gao.gov and read The Watchblog. Contact: To Report Fraud, Website: http://www.gao.gov/fraudnet/fraudnet.htm Waste, and Abuse in E-mail: fraudnet@gao.gov Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400, Congressional U.S. Government Accountability Office, 441 G Street NW, Room 7125, Relations Washington, DC 20548 Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707 Strategic Planning and U.S. Government Accountability Office, 441 G Street NW, Room 7814, External Liaison Washington, DC 20548 Please Print on Recycled Paper.