GAO United States Government Accountability Office Report to Congressional Requesters February 2022 VA HEALTH CARE Incomplete Information Hinders Usefulness of Market Assessments for VA Facility Realignment GAO-22-104604 GAO Highlights Highlights of GAO-22-104604, a report to congressional requesters Why GAO Did This Study The VA MISSION Act of 2018 required VA to assess its capacity to deliver health care to veterans. This was done in response to challenges facing the department such as an aging veteran population, antiquated health care infrastructure, and limitations with VA's capital planning processes. VA officials said these market assessments- conducted at the VA health care market level-uwill inform the Secretary's planned recommendations on modernizing and realigning VA health care delivery that the VA MISSION Act requires VA to issue by January 31, 2022. GAO was asked to review VA's market assessment methodology. This report (1) describes VA's approach to its market assessments; and (2) identifies limitations in that approach. GAO reviewed VA documents and data used to inform the market assessments and interviewed VA officials responsible for conducting the market assessments. What GAO Recommends GAO is making two recommendations for the VA Secretary: (1) to improve the completeness of VA community care data, and (2) to externally communicate to the Asset and Infrastructure Review Commission information about VA data reliability and any limitations of information used in the market assessments. VA concurred with the recommendations, and identified steps it will take to implement them. For example, VA noted that the department will provide the Commission information outlining the completeness and limitations of VA data used in the market assessments. View GAO-22-104604. For more information, contact Sharon Silas at (202) 512-7114 or SilasS@gao.gov. VA HEALTH CARE Incomplete Information Hinders Usefulness of Market Assessments for VA Facility Realignment What GAO Found The Department of Veterans Affairs (VA) conducted assessments of its capacity within 96 markets to deliver health care to veterans through VA providers and, when the Department cannot provide the care needed, through non-VA providers, known as community care. For these assessments, markets are designated geographic areas made up of a set of contiguous counties that contain one or more VA medical centers and associated clinics. For an overview of VA's approach, see the figure. Overview of Department of Veterans Affairs' Approach to Its Market Assessments Compiled data Flee MUU panied ce Ae tcl to respond to gaps between supply and o(TElale| Identified gaps between supply and demand Issue final proposals including veteran health care supply and demand Source: GAO analysis of Department of Veterans Affairs (VA) documents and interviews with VA officials. | GAO-22-104604 VA officials described the department's process of developing proposals as iterative in that VA continually reviewed and revised draft proposals throughout the market assessments process. The VA Secretary plans to transmit recommendations to the Asset and Infrastructure Review Commission by March 14, 2022-that is, no later than 6 weeks from the statutory deadline of January 31, 2022. The Commission will then review these recommendations, hold public hearings, and prepare and issue their own recommendations to the President. GAO identified gaps in the data VA compiled and certified for the market assessments that were relevant to determining both the supply of and demand for non-VA care. For example, VA lacked complete data on the extent to which its contractors maintain an adequate number of non-VA providers to ensure veterans have timely access to community care. VA officials told GAO that they determined supply and demand based on the most recently available data at the time of data compilation-December 2018 to November 2020. In addition, while VA officials told GAO the end-of-assessment analyses included updated data on community care, these data did not address the gaps GAO identified. Without such information, VA lacks a full understanding of the extent to which community care is able to supplement VA facility care to meet veterans' current and future demand. GAO also found that VA's approach to the market assessments did not include steps to collect information on the quality of VA data compiled from numerous VA data sources or other steps to understand any relevant data limitations. Instead, VA officials leading the market assessments said they relied on VA offices responsible for the databases to ensure the data quality. As a result, VA is unable to communicate to external stakeholders, such as the Asset and Infrastructure Review Commission, all relevant information on the quality of VA data used in market assessments, including any limitations affecting these data and the resulting proposals for realignment. United States Government Accountability Office Contents Letter Background 5 VA's Approach to Its Market Assessments Included Compiling and Certifying Data, Conducting Site Visits, and Developing Proposals lteratively 11 Incomplete Information on Non-VA Care and Data Reliability Limit the Quality of the Information VA Used in Its Approach to the Market Assessments 17 Conclusions 23 Recommendations 23 Agency Comments 23 Appendix | Map of Department of Veterans Affairs (VA) Regional Networks and Markets and Numbers of Veterans in Each Market 25 Appendix II System-Wide Principles and Other Guidance for Department of Veterans Affairs (VA) Market Assessments 31 Appendix III Comments from the Department of Veterans Affairs 38 Appendix IV GAO Contacts and Staff Acknowledgements 43 Tables Table 1: Numbers of Veterans Enrolled in and Eligible for VA Health Care in VA's 96 Health Care Markets Included in Its Market Assessments 27 Table 2: General Questions Considered in the Department of Veterans Affairs (VA) Market Assessments to Analyze Market Data and Identify Gaps Between Supply and Demand of Veteran Health Care, According to VA Officials 32 Table 3: Data Elements Considered in Department of Veterans Affairs (VA) Market Assessments to Analyze Market Data and Identify Gaps Between Supply and Demand, According to VA Officials 32 Page i GAO-22-104604 VA Market Assessments Figures Figure 1: Overview of the Department of Veterans Affairs' Approach to Health Care Market Assessments 9 Figure 2: Timeline for VA MISSION Act of 2018 Requirements for the Department of Veterans Affairs (VA) Market Assessments and the Asset and Infrastructure Review Commission's Review 10 Figure 3: Map of Department of Veterans Affairs' (VA) Regional Networks, Markets, and VA Medical Centers in 50 States and Washington, D.C. 26 Abbreviations MISSION Act VA MISSION Act of 2018 VA Department of Veterans Affairs VHA Veterans Health Administration This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-22-104604 VA Market Assessments i U.S. GOVERNMENT ACCOUNTABILITY OFFICE 441 G St. N.W. Washington, DC 20548 February 2, 2022 The Honorable Jon Tester Chairman Committee on Veterans' Affairs United States Senate The Honorable Mark Takano Chairman Committee on Veterans' Affairs House of Representatives The Department of Veterans Affairs (VA) administers one of the largest health care systems in the nation and offers health care services to about 9 million veterans enrolled in VA health care at 171 VA medical centers and more than 1,100 outpatient facilities organized into regional networks.' In addition, eligible veterans also may obtain services from non-VA providers, known as community care, when VA cannot provide the care needed.? While veterans still receive most of their care in VA medical facilities, according to VA, the number of veterans who received community care increased 64 percent from approximately 1.1 million in 2014 to 1.8 million in 2020. VA provides health care services to various veteran populations, including an aging veteran population and a higher proportion of women veterans than in previous generations. These and other demographic shifts in the population served by VA are expected to drive changes in veterans' needs for care and in their preferences for how their care will be delivered. Such changes have implications for how VA addresses its 1The number of medical centers and outpatient facilities is current as of September 2021. 2The Veterans Community Care Program-established by the VA MISSION Act of 2018 and implemented on June 6, 2019-is the most recent iteration of VA's long-standing practice of allowing veterans to receive care from community providers when they face challenges accessing care at VA medical facilities. VA purchases community care under the Veterans Community Care Program through regional contracts. See Pub. L. No. 115- 182, tit. I, § 101, 132 Stat. 1393, 1395-1404 (2018), codified at 38 U.S.C. § 1703, and implementing regulations at 38 C.F.R. §§ 17.4000 - 17.4040. VA also has the option to use direct agreements, known as Veterans Care Agreements, with community providers for care not included in community care contracts, and the option to refer veterans to other federal health care facilities with whom VA has an agreement, such as Department of Defense military treatment facilities. Page 1 GAO-22-104604 VA Market Assessments aging infrastructure while ensuring veterans' needs are met with timely access to high-quality and cost-effective care.3 Since 1999, VA's efforts to modernize and realign its health care facilities have been the subject of several assessments.* However, in spite of these assessments and their findings, VA faces many of the same challenges with its facilities as it did more than 20 years ago. The VA MISSION Act of 2018 (MISSION Act) required VA to conduct the latest system-wide assessment of its capacity to provide health care services to veterans to be used for making recommendations regarding modernizing and realigning the department's facilities.5 In response to this and other requirements, VA began in December 2018 its Market Area Health System Optimization Assessments, hereafter referred to in this report as market assessments.® These market assessments are to examine the capacity of VA-defined as consisting of both VA facility and community care-to provide quality, accessible, and timely health care 3GAO maintains a high-risk program to focus attention on government operations that it identifies as high risk due to their greater vulnerabilities to fraud, waste, abuse, and mismanagement or the need for transformation to address economy, efficiency, or effectiveness challenges. Aligning VA facilities to improve veteran access to services integrates two of GAO's high risk areas: veterans' health care and federal real property. In 2015, GAO placed veterans' health care on its High Risk List due to persistent weaknesses and systemic problems with timeliness, cost-effectiveness, quality, and safety of the care provided to veterans. GAO, High-Risk Series: An Update, GAO-15-290 (Washington, D.C.: February 2015). In 2003, GAO placed federal real property management-including management of VA real property-on its High Risk List due to long-standing challenges, such as effectively disposing of excess and underutilized federal property. See GAO, High-Risk Series: Federal Real Property, GAO-03-122 (Washington, D.C.: Jan. 1, 2003). 4For example, in 1999, VA initiated a process known as Capital Asset Realignment for Enhanced Services. This process was designed to assess VA's buildings and land ownership in light of expected demand for VA inpatient and outpatient health care services through fiscal year 2022. Through Capital Asset Realignment for Enhanced Services, VA sought to determine what health care services veterans would need in what locations. 5VA MISSION Act of 2018, Pub. L. No. 115-182, tit. Il, § 203, 132 Stat. 1393-1446 (2018). 8V/A officials stated that, during 2016 and 2017, the department conducted a pilot phase for its approach to the market assessments. VA planned to start the market assessments in late 2017 awarding a contract for support services. However, litigation delayed the execution of that contract until December 2018, according to VA officials. Page 2 GAO-22-104604 VA Market Assessments services to veterans within 96 markets." Accordingly, VA officials said the department's objectives for the market assessments included assessing VA and non-VA health care resources available (i.e., supply) to meet the current and future health care needs of veterans (i.e., demand), identifying gaps between supply and demand, and proposing actions to address those gaps, hereafter referred to as proposals.® The MISSION Act also required the Secretary of VA to submit, by January 31, 2022, recommendations for the modernization or realignment of VA facilities to the Asset and Infrastructure Review Commission-a presidentially appointed and Senate confirmed commission that will review and analyze the recommendations made by the Secretary.? As of November 2021, VA continued to review and revise proposals from the market assessment that will form the basis for the Secretary's recommendations. You asked us to review VA's methodology for conducting its market assessments. In this report, we 1. describe what is known about VA's approach for the market assessments; and 2. identify limitations, if any, about the data and information used in the department's approach. To describe what is known about VA's overall approach for the market assessments, we reviewed VA's documentation for its market assessment design (i.e., what the department planned to do). We also reviewed documentation of VA's implementation of the market assessments (i.e., what it actually did) that VA provided to us, which was limited in amount and detail. For example, we reviewed a list of data VA planned to compile in each market, as well as the data VA actually 7For the purposes of these assessments, a "market" is a designated geographic area made up of a set of contiguous counties that usually contains at least one medical center, or in some markets multiple medical centers, and associated clinics. VA divided its regional networks into markets to effectively plan for the provision of services to meet the health care needs of the veteran population that resides within each market's boundaries. According to VA officials, VA has 98 markets as of January 2021, but excluded from the market assessments two markets that cover areas outside of the United States. 8V/A also examined other aspects of supply and demand for VA health care including access, quality, and cost as part of the market assessments. 9Pub. L. No. 115-182, tit. Il, §§ 202(c), 203(b), 132 Stat. at 1443-46. As of January 2022, the Commission members have not been nominated, according to a VA official. Page 3 GAO-22-104604 VA Market Assessments compiled for five of the 96 markets.1° For aspects of VA's approach where implementation was still ongoing during the period of our review, such as how VA developed proposals to respond to identified gaps in veteran health care supply and demand, we reviewed only design documentation. We did not obtain information about the proposals themselves-that is, the results of VA's assessments of supply and demand which offer plans that include modernizing and realigning VA facilities to meet veterans' needs. To identify any limitations in the department's approach for the market assessments, we assessed what is known about VA's approach for the market assessments in the context of federal internal control standards for information and communication.11 For both objectives, we obtained testimonial evidence by interviewing and obtaining written responses on VA's approach from officials in the Veterans Health Administration's (VHA) Chief Strategy Office-the office responsible for conducting the market assessments, according to VA officials. We also interviewed and received written responses from VA officials from other relevant offices who participated in an advisory group that VA officials said provided input to the VHA Chief Strategy Office during the market assessments. These offices included VA's Office of Construction and Facilities Management and Office of Asset Enterprise Management and VHA's VA/DOD Health Affairs. We also interviewed officials from four veteran service organizations-Veterans of Foreign Wars, Disabled American Veterans, Vietnam Veterans of America, and Paralyzed Veterans of America- regarding VA's consultation of such organizations and veterans during the market assessments. We conducted this performance audit from October 2020 to February 2022 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe 10We reviewed VA's slide presentations compilation of data for each of five markets; we selected the markets for variation in number of veterans in the market, rurality, and geographic location. 11Internal control is a process effected by an entity's oversight body, management, and other personnel that provides reasonable assurance that the objectives of an entity will be achieved. In this review, we relied specifically on internal control Principle 13, which states, "Management should use quality information to achieve entity's objectives," and Principle 15, which states, "Management should externally communicate the necessary quality information to achieve the entity's objectives." GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: Sept. 2014). Page 4 GAO-22-104604 VA Market Assessments that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background VA's Health Care System VA reported as of September 2021, that it provides direct health care services to enrolled veterans through its e 171 medical centers that offer inpatient care, such as surgery, critical care, and other advanced care requiring an overnight stay, in addition to outpatient services, such as diagnostic tests and minor surgeries like mole removal, among other services. e 134 nursing homes, known as community living centers, which provide skilled nursing care to veterans with chronic stable conditions, such as dementia; those who require rehabilitation services; and those needing care and comfort at the end of life.'12 e 1,115 outpatient facilities, which generally provide clinic-based services and include primary care, specialty care, and mental health services. In order to meet the needs of the veterans it serves, VA is also authorized to pay for veteran health care services from certain non-VA providers in veterans' communities (i.e., community care). As required by the MISSION Act, VA implemented the Veterans Community Care Program in June 2019, consolidating many of VA's existing community care programs into a permanent program.' To implement the Veterans Community Care Program, VA issued regulations and defined certain eligibility criteria for the new program. '4 For example, veterans may be eligible to receive care under the program when services are not available at a VA facility or when veterans face wait times or drive times to VA 12The number of nursing homes is current as of June 2021. 13Pub. L. No. 115-182, tit. |, § 101, 132 Stat. 1393, 1395 (2018) (codified as amended at 38 U.S.C. § 1703). 14See 38 C.F.R. §§ 17.4000 - 17.4040 (2020). Page 5 GAO-22-104604 VA Market Assessments facilities longer than certain VA standards.'5 VA purchases community care under this program through regional contracts with two third-party administrators, which are responsible for recruiting and building networks of licensed health care community providers and for paying community provider claims.'® VA also has the option to use direct agreements with community providers for care not included in those network contracts, and also may refer veterans to other federal health care facilities with whom VA has an agreement, such as Department of Defense treatment facilities. VA organizes its health care system into 18 regional networks that are each responsible for coordination and oversight of all administrative and clinical activities of the VA medical centers, outpatient clinics, and other health care facilities within its geographic region. For planning purposes, its regional networks are further divided into markets-usually along county lines-in consideration of veteran travel and referral patterns, geographic dispersion of veteran enrollees in VA health care, and locations of medical facilities within the market.17 Each market may have differing numbers of VA medical centers and other VA health care facilities. See appendix | for a map that shows the geographic boundaries of the 96 markets included in VA's market assessments and a table that reports certain demographic information (e.g., the number of veterans 15See 38 C.F.R. § 17.4010 (2020) (veteran's eligibility). VA's designated access standards include when the veteran's average drive time to a VA provider is more than 30 minutes for primary care or more than 60 minutes for specialty care, or the next available appointment with a VA provider is not within 20 days for primary care or 28 days for specialty care of the date of request of care unless a later date has been agreed upon. 38 C.F.R. § 17.4040 (2020) (designated access standards). See also VA, Veterans Health Administration Office of Community Care Field Guidebook (May 21, 2020); Veterans Health Administration: Referral Coordination Initiative Implementation Guidebook (Jan. 10, 2020); and Fact Sheet: Veteran Community Care Eligibility (Aug. 30, 2019). 16V/A's third-party administrators that develop and administer Community Care Networks include Optum Public Sector Solutions and TriWest Healthcare Alliance. 17V/A first designated its markets during an earlier VA assessment of its health care capital-asset priorities that began in 1999. VA also uses these markets in its annual capital planning process. According to VA officials, the department has periodically reviewed and reconsidered the market geographic boundaries over time. According to VA officials, there are currently 98 markets as of January 2021, two of which cover regions located outside of the United States and its territories. Page 6 GAO-22-104604 VA Market Assessments eligible for and the number of veterans enrolled in VA health care) for each market. 18 VA's Health Care Capital Infrastructure Part of the stated goals for the market assessments was to ensure VA modernization efforts helped align veterans' health care demand and needs. Much of VA's health care infrastructure-including more than 5,600 buildings as of October 2021--were built decades ago and designed for an inpatient-driven health care system. Similar to trends in the health care industry overall, VA's model of care has shifted away from providing care in an inpatient setting, to that of an outpatient setting, which VA largely houses in converted inpatient facilities, or in a growing number of outpatient clinics. VA has used the Strategic Capital Investment Planning process annually since 2010, in partnership with VHA's 18 regional networks to manage certain capital assets, such as medical centers and other buildings. 19 There are similarities and some key differences between VA's approach to its market assessments and the Strategic Capital Investment Planning process-VA's main mechanism for planning and prioritizing capital planning: e Like the market assessments, one of the goals of the Strategic Capital Investment Planning process is to identify the full capital need to address VA's service and infrastructure gaps. e Both the VA market assessments approach and the Strategic Capital Investment Planning process use one of VA's models-VA's Enrollee Healthcare Projection Model-to project veteran enrollment, utilization of VA health care, and the associated expenditures VA needs to meet the expected demand for most of the health care services it provides. e One key distinction is that, in general, VA designed the market assessment process to look at its health care system as a whole, while the Strategic Capital Investment Planning process focuses only on capital needs, according to VA officials. e Another key distinction is that VA's market assessment approach considers additional sources of data and information about the supply 18VA excluded from the market assessments the two markets that cover regions outside of the United States and its territories, according to VA officials. 18Capital assets are generally land, structures, equipment, intellectual property, and information technology used by the federal government that have an estimated useful life of 2 years or more. Page 7 GAO-22-104604 VA Market Assessments of non-VA health care resources available in a market, whereas the Strategic Capital Investment Planning process does not. VA Market Assessments and Independent Asset and Infrastructure Review Commission In response to MISSION Act and other requirements, VA conducted market assessments of 96 markets between December 2018 and January 2022. VA officials said the objectives of the market assessments are to help VA e understand the supply of VA and non-VA health care resources available to meet the current and future veteran demand for VA health care services, and e identify and respond to any gaps between supply and demand.2° VA officials said if they identified gaps for a market in any direction-for example, not enough resources to meet higher demand for health care, or outsized resources compared to lower demand-they would then develop a proposal (referred to as an "opportunity" within VA) to align resources with demand for that market or regionally for multiple markets. VA documentation indicated that proposals could include actions such as adding, discontinuing, or relocating services at VA or non-VA sites of care; closing or relocating VA sites of care; or constructing and leasing new sites of care, including the following examples: e Inamarket that is projected to grow in population and enrollees in VA health care, VA leases new space for a VA outpatient clinic in a certain city to increase access to primary care services for veterans. e Inamarket where utilization of VA inpatient services is low and projected to decline further, VA stops providing inpatient medical services at a VA medical center. VA relies on community care facilities to provide the inpatient services to veterans as those facilities have enough availability to cover the needs of veterans affected by the change. If a proposal identified the same need as VA's annual planning processes, VA officials said the proposal was removed from the market assessments. For example, VA officials explained how one market assessment proposal for establishing a new outpatient clinic was removed, because the need for the clinic had been determined during the 20To help achieve these objectives, VA established 10 system-wide principles for the market assessments such as, ensure veterans are provided the opportunity to choose care they trust throughout their lifetime. See appendix II for the full list of system-wide principles. Page 8 GAO-22-104604 VA Market Assessments market assessments through VA's Strategic Capital Investment Planning process. (See Figure 1 for an overview of VA's market assessment process.) Figure 1: Overview of the Department of Veterans Affairs' Approach to Health Care Market Assessments Compiled data Flee MUU Developed proposals to respond to gaps cee Issue final Identified gaps between supply and demand including veteran health care supply and demand between supply and proposals o(TElale| Source: GAO analysis of Department of Veterans Affairs (VA) documents and interviews with VA officials. | GAO-22-104604 According to VA officials, the Secretary will use the proposals developed through the market assessments that meet both the MISSION Act definitions of modernization or realignment and the Secretary's approval as recommendations.2! The MISSION Act also established the independent Asset and Infrastructure Review Commission to assess these recommendations from the VA Secretary, and to issue its own recommendations.22 The MISSION Act requires the VA Secretary to transmit the recommendations to the Commission by January 31, 2022.23 The Commission must then hold public hearings and review the Secretary's recommendations to prepare and issue their own recommendations to the President by January 31, 2023.24 Figure 2 depicts the timelines for reviews that involve the Commission, President, and Congress, as established in the MISSION Act. 21According to the MISSION Act, "modernization" and "realignment" include, among other things, actions such as closures, construction, purchasing, leasing, and sharing of VHA facilities, and actions that change the numbers of or relocate VHA services, functions, and personnel positions. See Pub. L. No. 115-182, tit. Il, § 209, 132 Stat. 1393, 1460 (2018). 22The President was to transmit to the Senate his nominations for appointments to the Commission by May 31, 2021. Pub. L. No. 115-182, tit. Il, § 202(c)(1)(B), 132 Stat. 1393, 1443 (2018). As of January 2022, the Commission members have not been nominated, according to a VA official. 23In January 2022, the VA Secretary announced that the Department would delay the transmission of the recommendations to the Commission to no later than March 14, 2022-that is, no later than 6 weeks from the statutory deadline of January 31, 2022. 24V/A officials noted that the department would not implement those proposals that move forward as recommendations by the Secretary until after the Commission's and other external reviews conclude. Page 9 GAO-22-104604 VA Market Assessments nn -S-CSSCis Figure 2: Timeline for VA MISSION Act of 2018 Requirements for the Department of Veterans Affairs (VA) Market Assessments and the Asset and Infrastructure Review Commission's Review <-__________. Department of Veterans Affairs (VA) conducted ~~ SS > health care market area assessments and submitted findings to the Secretary. VA Secretary transmits report to the Asset and Infrastructure Review Commission with recommendations for modernization and realignment of VA facilities. The Commission reviews the Secretary's report and recommendations, and holds public hearings on the recommendations for modernization and realignment of VA facilities. Deadline for the Commission Deadline for the President Deadline for the President to respond to the President's to communicate reasons to communicate approval reasons for disapproval. for disapproval of the or disapproval of the Commission's report. Commission's report. The Commission transmits report to the Deadline for the President to President with its findings and conclusions report to Congress approval of the on VA's recommendations, together with Commission's recommendations its recommendations. or effort is terminated. To terminate the effort, Congress must respond by introducing a joint resolution within 5 days of the President's approval, and have it enacted by the earlier of: CONGRESSIONAL ACTION AFTER THE PRESIDENT'S APPROVAL 1 - | CONGRESS |_____.1 14) 45 days after the President's approval 1 DISAPPROVES ; -OR- 2) the adjournment of the congressional session in which the President's report was transmitted to Congress. Source: VA MISSION Act of 2018 and GAO interviews with Department of Veterans Affairs officials. | GAO-22-104604 Page 10 GAO-22-104604 VA Market Assessments VA's Approach to Its Market Assessments Included Compiling and Certifying Data, Conducting Site Visits, and Developing Proposals lteratively Note: According to the VA MISSION Act of 2018, "modernization" and "realignment" include, among other things, actions such as closures, construction, purchasing, leasing, and sharing of VHA facilities, and actions that change the numbers of or relocate VHA services, functions, and personnel positions. See Pub. L. No. 115-182, tit. Il, § 209, 132 Stat. 1393, 1460 (2018). ln January 2022, the VA Secretary announced that the department would delay the transmission of the recommendations to the Commission to no later than March 14, 2022-that is, no later than 6 weeks from the statutory deadline of January 31, 2022. According to VA officials and documents, the department's approach to the market assessments included the following: e Compiling data on supply of and demand for veteran health care, among other things, in each of the 96 markets; Certifying the data compiled from VA databases and public sources through reviews by regional network and medical center officials; e Conducting site visits to collect supplemental information in each market; and Developing proposals iteratively to respond to gaps identified between veteran health care supply and demand, based on analyses of the data. Data compilation. VA officials told us that from December 2018 to November 2020, the department compiled data from VA and other sources for each market. VA design documentation indicated the compiled data included, among other things, veteran and non-veteran demographics, the current and future veteran demand for VA health care services, and the supply of VA, community care, and other non-VA health care resources available to meet the current and future veteran demand.25 VA documentation indicated these data included the most current data at the time, as well as 10-year projections for certain data. According to VA design documentation and VA officials, data that the department planned to compile included the following: e Demand for VA health care services: the numbers of outpatient encounters, completed appointments, inpatient discharges, and surgical procedures. e Demand for community care and other non-VA health care services: the number of community and other non-VA care authorizations and amounts disbursed for community and other non-VA care. 25Compiling both veteran and non-veteran data provided information about the market's total population health care needs, according to VA officials. Page 11 GAO-22-104604 VA Market Assessments e Supply of VA health care resources: lists of VA-owned sites and leases and clinical staffing levels by specialty and facility. e Supply of non-VA health care resources: information on commercial hospitals, including the number of operating beds, merger and acquisition activity, key projects for improving capacity or enhancing services, closures or reductions of service in the market, awards and recognitions, patient satisfaction; and locations of facilities run by other federal agencies, such as the Department of Defense. e Cost of providing care: cost analysis for inpatient services comparing cost to provide care in VA facilities to the potential cost to acquire the same care in the market. VA compiled these data from more than 60 sources, most of which were VA enterprise databases, according to VA's documentation of its data definitions.26 VA also compiled data from VA medical centers and affiliated clinics, other federal agencies, and commercial and proprietary sources such as IBM Watson Market Expert. 27 VA compiled data for fiscal years 2018 and earlier from the department's databases, according to VA's design documentation.28 However, this documentation did not include dates for data VA compiled from non-VA data sources. Data certification. From March 2019 to December 2020, VA completed reviews of its compiled demand, supply, and other data through a certification process that involved duplicating samples of the compiled data, and finding and fixing some discrepancies, according to VA documents. Specifically, once VA compiled data for its market assessments, VA documentation indicated the department directed VA medical center and regional network officials to replicate the data compilation that VA conducted by applying the same search parameters 26V/A centrally maintains its enterprise databases used in the management of VA's health care system. The databases include system-wide data on historical actual workload for VA health care services, physician productivity, the physical condition of VA health care buildings, and other administrative data. 27IBM Watson Market Expert is a commercial dataset that combines public and proprietary data to assess demand for health services in a market area, among other things. 28The number of vacant positions that VA documentation indicated the department compiled included fiscal year 2019. Page 12 GAO-22-104604 VA Market Assessments to the same VA-wide databases and public sources.2° From this certification process, VA officials said regional network and VA medical center officials identified 1,132 data discrepancies and other issues across the 96 markets-475 of the issues resulted in changes to the data to make needed fixes. According to VA documentation from some of the markets, examples of data discrepancies included the following:2° e Reviewers identified potentially missing workload data (e.g., data on primary care and telehealth encounters) in at least eight markets. As a result, VA updated data for three of these markets but instructed the other five markets to submit their concerns separately as a supplement to the compiled data, instead of changing these data directly. e Reviewers from at least three different markets disagreed with certain demand projections made for particular markets. In these cases, reviewers thought the original market projections were inaccurate, as they did not account for demand from outside each market's boundaries. VA did not change the compiled data in these cases but asked two of the markets to submit the information on demand outside the market's boundaries separately as a supplement to the compiled data (rather than part of it). Network directors signed certifications after VA resolved all the issues, either by making changes to the data or by VA providing an explanation for not making a change, according to VA officials. All 18 VA regional network directors signed these certifications indicating their agreement that these compiled data could be used for further market assessment analysis for markets in their regions. Site visits. VA officials said the department also supplemented its data compilation by conducting site visits to VA medical centers and other health care facilities in each of the 96 markets from March 2019 through 28Public sources included, for example, public Health Resources and Services Administration data about its health centers. The data that were not included in this review included the data provided to VHA's Chief Strategy Office by VA medical centers and data from proprietary commercial sources, according to VA documents. VA officials said the data provided by VA medical centers included the number of operating beds, facility site plans, and descriptions of architectural and engineering challenges. 300ther issues reviewers raised included suggested formatting edits and corrections to rounding errors. Page 13 GAO-22-104604 VA Market Assessments November 2020.31 These site visits aimed to provide additional information about local health care environments in each market, according to VA officials. VA officials said for site visits that occurred prior to the start of the COVID-19 pandemic in 2020, they conducted in-person tours of VA medical centers and some outpatient clinics that were either the parent facility in the market or particularly critical to the functioning of the market. According to VA officials, the department conducted the site visits that occurred after the pandemic started-for about one third of the markets-virtually and did not include in-person tours. VA officials estimated that the site visits included more than 1,800 interviews with VA medical center and regional network officials.32 Based on our review of VA documentation, the discussion topics during site visit interviews ranged from overarching themes and future vision for the market; demographics and veteran demand; VA supply, access, patient satisfaction and quality of care; and conditions of buildings. According to VA Officials, the site visit interviews also provided insight into the potential to increase the supply of community care within each market. VA documentation indicated that the department planned to summarize what it learned about these topics during each market's interviews, which were to be used in later analyses. VA officials said that while they developed a site visit guide with suggested questions, the guide was not intended to be prescriptive for each interview. Rather, VA officials said they preferred each discussion to be relevant to the interviewee's position and reviews of the compiled data specific to that market or facility, allowing discussion to diverge from the suggested questions. Proposal development. VA officials told us that market teams, comprised of VA and contractor staff responsible for managing each market's assessment, began developing proposals to respond to any identified gaps in supply and demand based on their analysis of the compiled data before the market's site visits. According to VA officials, in making their proposals, which aimed to realign resources and demand, 31VA officials said that site visit teams included representatives from VHA's Chief Strategy Office, VA's capital planning, construction and facility management, and a representative from the Department of Defense in markets where there was a significant Department of Defense presence. 32Officials said that participants in the site visit interviews included, among others, each VA medical center Director and Assistant Director, and the regional Network Director, Deputy Network Director, and network's Chief Medical Officer. Interviews did not include officials from community care providers. Page 14 GAO-22-104604 VA Market Assessments the market teams were expected to apply 10 system-wide principles that VA established for the market assessments. For example, one principle was to "optimize health care services for veterans in each market using a mix of VA care first, supplemented by the Department of Defense, academic affiliates, federally qualified health centers, and community providers." To assist market teams in the application of these principles, VA officials said the department developed guidance that included questions for discussion and key data elements to consider in applying the 10 principles. See appendix II for a complete list of the system-wide principles, questions, and key data elements. VA officials described the process of developing proposals as iterative- that is, VA continually reviewed and revised draft proposals throughout the market assessments; the process was continuing as of November 2021. For example, according to VA officials, various combinations of VA leadership and staff reviewed draft proposals in a series of meetings between May 2019 and March 2021. These individuals included market teams' peer reviews, senior officials within VHA's Chief Strategy Office, regional network directors, the Assistant Under Secretary for Health for Clinical Services, Assistant Deputy Under Secretary for Health for Community Care, and the Under Secretary of Health. VA officials told us that during these reviews, participants were to challenge draft proposals and provide other feedback for market teams to revise their proposals. VA officials told us that in late 2020, VA also began applying a series of five analyses to the more than 1,700 draft proposals that had cleared the prior reviews. VA officials described these analyses as ensuring the proposals had taken into consideration the following: veteran feedback on VA health care; e VA's emergency response to the COVID-19 pandemic; e consistency of VA's management of 10 health care service lines and outpatient facilities; e weighing the financial costs and non-financial benefits of each market's group of proposals; and Page 15 GAO-22-104604 VA Market Assessments e consistency with criteria developed for determining the Secretary's recommendations on modernization and realignment to the Asset and Infrastructure Review Commission. During 2021, VA officials said they updated certain data that they considered critical to use in these analyses to fiscal years 2019 and 2020. Updated data included measures of demand, such as the numbers of veterans enrolled in VA health care and community care outpatient authorizations as of fiscal year 2019. Officials told us that the updated data also included measures of supply, such as VA and commercial hospital capacity, as of fiscal year 2019 and June 2019, respectively. As a result, some of the data projections in future years changed, according to VA officials. In November 2021, VA officials told us the analyses are applied simultaneously and any potential changes to proposals are reviewed again by applying the other analyses before the proposal is revised. VA officials said these analyses of proposals will be ongoing until the Secretary makes his recommendations. On January 12, 2022, the VA Secretary announced that the department would delay the transmission of the recommendations to the Asset and Infrastructure Review Commission to no later than March 14, 2022-that is, no later than 6 weeks after the statutory deadline of January 31, 2022. 33According to VA, service lines help organize health care delivery around broad categories of care to develop consistent care standards and enhance quality of care. Examples of VA health care service lines include inpatient surgery and rehabilitation services for blind patients. The MISSION Act required VA to develop criteria for the Secretary to make recommendations for the modernization and realignment of VA facilities to the Asset and Infrastructure Review Commission. Pub. L. No. 115-182, tit. Il, § 203(a), 132 Stat. at 1446. VA published these criteria in May 2021. See 86 Fed. Reg. 28,932 (May 28, 2021). Page 16 GAO-22-104604 VA Market Assessments Incomplete Information on Non- VA Care and Data Reliability Limit the Quality of the Information VA Used in Its Approach to the Market Assessments VA Lacks Complete Data on the Supply of and Demand for Community Care Although the data the department compiled and certified included some data on community care and other non-VA providers, we identified gaps in the data relevant to determining the supply of, and veteran demand for, such non-VA care.*4 Based on our review of VA documentation, the department planned to compile data on the supply of non-VA resources potentially available to meet the veteran demand for community care such as the number of specialty care physicians in each market. The department also planned to compile data on veteran demand for VA community care, such as the number of times that VA authorized veterans to use such care during fiscal years 2016, 2017, and 2018, and overall (veteran and non-veteran) demand for health care services for each market. 35 VA's planned approach also called for the department to collect additional information on the supply of non-VA care. Specifically, VA documentation indicated that the department planned to supplement the supply data compiled by collecting publically available information on commercial hospitals, including merger and acquisition activity, key projects for improving capacity or enhancing services, closures or reductions of service in the market, awards and recognitions, and patient satisfaction. 34Total supply could include community care providers' ability to care for additional veterans or the extent to which other non-VA providers could begin providing care, for example. 35Except for certain emergency, limited non-emergent, Civilian Health and Medical Program of the Department of Veteran Affairs (CHAMPVA), and pharmacy care, VA must authorize all community care services for veterans before veterans access the care in order for claims to be paid. Based on VA documentation, the overall health care demand reflected 2018 or 2019 depending on when VA compiled data for the market. Page 17 GAO-22-104604 VA Market Assessments According to VA officials, the site visit interviews with officials from VA medical centers and regional networks provided key insights into community care within each market. For example, VA medical center and regional network officials may be familiar with new partnership opportunities with non-VA providers, because they interact regularly with community care partners and, in some cases, have worked for local non- VA health care facilities in the past. In addition, VA officials noted that VA considered the capacity of the community care networks in the end-of- assessment analysis that reviewed the proposals in the context of criteria developed for determining the Secretary's recommendations on modernization and realignment to the Asset and Infrastructure Review Commission. However, we found gaps in the data on community care and other non- VA providers that VA compiled and obtained from site visits. Data on projections of demand for VA health care did not fully account for expanded eligibility to community care. VA implemented its Veterans Community Care Program in June 2019, which expanded certain veterans' eligibility for community care. VA officials told us one of the end-of-assessment analyses included projections of veteran demand for community care based on fiscal year 2019 utilization of community care; these data included the initial 4 months of VA's implementation of expanded eligibility under the Veterans Community Care Program.36 Given that VA developed these projections of veteran demand using 4 months of experience under the Veterans Community Care Program, it is likely these projections provide VA with incomplete information on the effects of the expanded eligibility. VA could have supplemented these projections based on limited experience with the Veterans Community Care Program by, for example, developing and incorporating adjustments to the projections of demand 38According to VA officials, the department considered such data beginning in September 2021 as part of the end-of-assessment analyses applying the criteria to be used for the Secretary's recommendations on modernization and realignment. Specifically, VA used projections produced by the Enrollee Healthcare Projection Model, its actuarial model that informs approximately 90 percent of the department's health care budget estimate as well as its annual capital planning process. VA officials noted that the projections also reflected adjustments to the fiscal year 2019 utilization data intended to estimate the effects of expanded eligibility under the Veterans Community Care Program. VA developed these adjustments based on VA's historical community care experience under the Choice program, a temporary community care program in place prior to the Veterans Community Care Program, as well as on 4 months of experience under the VA MISSION Act. Page 18 GAO-22-104604 VA Market Assessments based on estimates of the cost of expanded access to veterans under the MISSION Act. Estimates could have been based on the Congressional Budget Office examination in December 2018 of VA's past growth in spending and projected VA's spending through 2028 under three scenarios-two of which assumed larger appropriations for VA medical care by incorporating the projected spending required to implement the MISSION Act above VA's past policies.?" Without using estimates of demand for community care to reflect expansion of eligibility under the Veterans Community Care Program, VA may not be fully accounting for the effects that anticipated increases in veterans' use of community care may have on such care as well as VA facility care.38 Incomplete data on community care network adequacy-that is, the extent to which VA's contractors maintain an adequate network of providers to ensure veterans have timely access to community care.39 According to VA officials, the department took steps to consider community care network adequacy, such as compiling survey information regarding veteran experiences under community care networks, which included aspects of access. VA officials also noted that they identified the location of current community care network providers to determine their proximity to veterans. However, these steps did not provide VA complete information about community care network adequacy.4° 37See Congressional Budget Office, Possible Higher Spending Paths for Veterans Benefits, (Washington, D.C., Dec. 2018). 38A Congressional Budget Office report noted that increasing access to community care under the Veterans Community Care Program may reduce utilization of VA facilities that had sufficient capacity, which could lead to higher costs per veteran for VA facility care. See, Congressional Budget Office, The Veterans Community Care Program: Background and Early Effects, (Washington, D.C., Oct. 2021). 38Network adequacy refers to having a sufficient number and variety of providers available to veterans that meet geographic accessibility standards based on drive times and appointment availability within pre-determined time frames. For more information on community care network adequacy see GAO, Veferans Community Care Program: Immediate Actions Needed to Ensure Health Providers Associated with Poor Quality Care Are Excluded, GAO-21-71 (Washington, D.C., Feb. 1, 2021). 404 recent study concluded that challenges reported in a survey of 90 VA medical center directors on their experiences with community care providers raised questions about community care network adequacy, particularly for veterans in rural areas. See K. M. Mattocks, et al., "Understanding VA's Use of and Relationships with Community Care Providers under the MISSION Act," Medical Care, vol. 59, no. 6 (June 2021). Page 19 GAO-22-104604 VA Market Assessments VA could have further supplemented the data the department compiled by, for example, conducting a survey or talking with a sample of community care providers directly about their capacity to provide care to veterans. VA also established standards for monitoring community care network adequacy under its community care program, but it is unknown at this time whether the department is collecting complete information to monitor these standards.*! Without complete information on network adequacy, VA lacks a full understanding of the extent to which community care is able to supplement VA facility care in meeting veterans' current or projected demand. Lack of data that would allow VA to compare costs of providing care to veterans at VA facilities compared with community care. While VA documentation on its approach for the market assessments indicated that the department planned to compile data on the cost per inpatient service for VA and for community care network providers, the documentation was insufficient to determine the extent to which these data provided an accurate comparison of costs. In addition, VA documentation did not include compiling data on cost per outpatient services. 42 VA could have compiled, for example, unit cost data (i.e., the costs to VA of providing a unit of service, such as a 30-day supply of a prescription or a day of care at a medical facility) for both VA facility and community care developed by the department's Enrollee Health Care Projection Model.43 Although these readily available unit cost data have their limitations, they would at least provide VA data for comparing costs of delivering health care services in VA facilities compared to delivering care through community care providers. The lack of such data hinders VA's ability to 41GAO currently has ongoing work examining VA's monitoring of community care network adequacy. 42V/A health care researchers have discussed the need for data that measures cost of care in a way that allows for comparisons of VA facility and community care. For example, see: A. Rosen, M. Vanneman, T. Wagner, "Assessing Cost and Outcomes among Veterans Receiving Community Care," Healifh Economics Seminar, June 19, 2019, accessed November 10, 2021, https:/Awww.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive .cim?Session|D=3645. 43VA's Enrollee Healthcare Projection Model is department's actuarial model that projects: (1) the expected number of veterans who will be enrolled in VA health care, (2) the expected quantity of health care services enrollees will use, and (3) the expected cost of providing one unit of those services, that is, unit costs. Page 20 GAO-22-104604 VA Market Assessments consider cost in determining whether to rely on community care providers in meeting the health care needs of veterans. According to VA officials, they determined the supply and demand for non-VA care based on the most recently available data at the time of data compilation-that is, December 2018 to November 2020. In addition, while VA officials told us the end-of-assessment analyses included updated data on community care these data did not address the gaps we identified. Relying on incomplete information on non-VA care is inconsistent with federal internal controls, which state that agencies should use quality information and communicate quality information externally to achieve its objectives; an aspect of quality information includes completeness.*4 Further, one of VA's stated system-wide principles of the market assessments was to optimize care for veterans in each market using a mix of "VA care first" supplemented by community care or other non-VA providers that do not currently participate in VA's community care networks of providers. This principle would indicate the need for VA to compile information relevant to determining when to rely on community care and other non-VA providers as a cost-effective, accessible, high- quality supplement to VA facility care. As a result, VA did not have complete information about the projected veteran demand, community care network adequacy and comparable costs of community care to use in its market assessments used to develop proposals to inform the Secretary's recommendations for modernization and realignment of VA medical facilities. VA's Approach to the Market Assessments Did Not Include Steps to Determine the Reliability of Data Used and It Is Unclear What Information on Reliability Will Be Reported Documentation on VA's approach to the market assessments indicated that the department relied mostly on data compiled from its own databases; however, VA's approach to market assessments did not include steps to determine the reliability of the department's data for the purposes of the market assessments other than having the Network Directors provide certifications, nor did VA's approach include plans to report externally on what is known regarding the reliability of these data. Specifically, VA officials told us that they did not analyze the reliability of VA data sources as part of the data certification. VA also did not plan to 44GA0-14-704G. Federal internal control standards define quality information as information that is appropriate, current, complete, accurate, accessible, and provided on a timely basis. Page 21 GAO-22-104604 VA Market Assessments include any information related to data reliability wnen documenting its proposals for review by internal stakeholders. When asked what steps were performed to evaluate the quality of the data being used in the market assessments, the VA officials leading the market assessments explained, they relied on data "owners'-that is, the other VA offices that compiled and certified the data for use throughout the department-to ensure the quality of the data.45 According to these officials, they considered themselves data "users." Including steps for determining data reliability, for example, obtaining information from the various data owners with VA regarding reliability of data, including any limitations, could have helped VA to determine the extent to which compiled data were accurate and complete-consistent with federal internal control standards' definition of quality information-and enabled VA to address the limitations. Without such steps, VA is unable to collect and communicate to external stakeholders, such as Congress and the Asset and Infrastructure Review Commission, all relevant information on the quality of VA data used in market assessments, including any limitations affecting these data. For example, such limitations could include VA's inability to evaluate the accuracy and reliability of the department's data on how long veterans wait to receive care at VA facilities-integral to understanding the supply of care in a market.46 Not communicating information on data quality, especially any limitations on the data, is contrary to federal internal control standards, which require agencies to externally communicate the necessary quality information to achieve their objectives. In the case of the market assessments, VA's objective is the modernization and realignment of VA health care facilities.4" If VA does not take steps to disclose such limitations, before any decisions about modernization of realignment of facilities, external stakeholders (including the Asset Infrastructure Review Commission, Congress, and veterans) will not have quality information needed to understand the department's approach to the market assessments and the development of proposals 45V/A officials noted that these data are from the same sources VHA relies on for operational analyses and by internal and external researchers for health care research. 46From November 2017 through August 2018, VHA''s Office of Internal Audit conducted its first performance audit, which assessed the accuracy and reliability of the wait times published on the VA Access and Quality website. VHA issued an internal audit report that is not publically available in February 2019. 47See GAO-14-704G. Page 22 GAO-22-104604 VA Market Assessments Conclusions Recommendations Agency Comments to inform the Secretary's recommendations for modernization and realignment of VA medical facilities. VA's market assessment process is a key step in VA's efforts to modernize and realign its facilities. It is important that VA's approach is comprehensive and that it is clear to all stakeholders-including the Asset and Infrastructure Review Commission and Congress-how VA has arrived at the proposals the department plans to move forward to the Commission. For example, in 2019, VA implemented the Veterans Community Care Program with the intent to improve the timeliness of care by providing veterans expanded access to care in their community. We found that VA's approach includes incomplete information on the demand for and supply of non-VA care, including community care, which is critical to understanding how best to meet veterans' future demand for care. The lack of data on such a key element to VA's delivery of care may erode confidence in any proposals that are ultimately forwarded to the Commission, scheduled to be sent no later than March 14, 2022. Further, information about the quality of data used in the market assessments and about how VA used that information to inform proposals is important to communicate to external stakeholders. Without such information, it may be unclear how VA arrived at its recommendations to the Commission for facility modernization and realignment and increases the risk that the recommendations may not be appropriate. To improve the quality of information used in VA's market assessments, we are making the following two recommendations to VA, to be implemented as soon as possible but no later than January 31, 2023, or before the Commission submits its report to Congress: The Secretary of Veterans Affairs should review the data on community care to identify any gaps and take steps to address data completeness. (Recommendation 1) The Secretary of Veterans Affairs should externally communicate to the Commission information about the completeness and reliability of VA data used to inform the assessments and about how VA considered any data limitations in developing proposals for the modernization and realignment of VA facilities. (Recommendation 2) We provided a draft of this report to VA for review and comment. In its written comments, reproduced in appendix III, VA concurred with our recommendations and identified steps the department will take to Page 23 GAO-22-104604 VA Market Assessments implement them. For example, VA noted that VHA's Chief Strategy Office would provide the Asset and Infrastructure Review Commission with information that specifically outlines the completeness and reliability limitations of VA data used to develop VA's recommendations. VA also provided technical comments, which we incorporated as appropriate. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, and other interested parties. In addition, the report will be available at no charge on GAO's website at http://www.gao.gov/. If you or your staff have any questions about this report, please contact Sharon M. Silas at (202) 512-7114 or silass@gao.gov. Contact points for our Office of Congressional Relations and Office of Public Affairs can be found on the last page of this report. Other major contributors to this report are listed in appendix IV. * Sharon M. Silas Director, Health Care Page 24 GAO-22-104604 VA Market Assessments Appendix |: Map of Department of Veterans Affairs (VA) Regional Networks and Markets and Numbers of Veterans in Each Market VA organizes its health care system into 18 regional networks that are each responsible for coordination and oversight of all administrative and clinical activities of the VA medical centers, outpatient clinics, and other health care facilities within its geographic region. For planning purposes, VA further divides its regional networks into markets, usually along county lines.1 Each market may have differing numbers of VA medical centers and other VA health care facilities. Figure 3 depicts the geographic boundaries as of January 2021 of the regional networks and the markets included in VA's market assessments, and the location of VA medical centers as of February 2020. According to VA officials, VA excluded two of its 98 markets from the market assessments that cover areas outside of the United States. 1According to VA officials, there are 98 markets as of January 2021, two of which cover regions located outside of the United States and its territories. Page 25 GAO-22-104604 VA Market Assessments Appendix |: Map of Department of Veterans Affairs (VA) Regional Networks and Markets and Numbers of Veterans in Each Market Figure 3: Map of Department of Veterans Affairs' (VA) Regional Networks, Markets, and VA Medical Centers in 50 States and Washington, D.C. , 7 Meena. a. eRe [| VA health care markets [ Regional networks A VA medical centers Source: GAO Analysis of Department of Veterans Affairs (VA) data. | GAO-22-104604 Note: Guam, the Commonwealth of the Northern Mariana Islands, and American Samoa are part of network 21, in a pacific islands market that includes Hawaii. Puerto Rico and the US Virgin Islands are in network 8 and comprise their own market. There is one VA medical center in Puerto Rico. Page 26 GAO-22-104604 VA Market Assessments Appendix |: Map of Department of Veterans Affairs (VA) Regional Networks and Markets and Numbers of Veterans in Each Market For each market included in VA's market assessments, Table 1 contains the numbers of veterans enrolled in and eligible for VA health care as of fiscal year 2017 and projected for fiscal year 2027. Table 1: Numbers of Veterans Enrolled in and Eligible for VA Health Care in VA's 96 Health Care Markets Included in Its Market Assessments Fiscal Year 2017 (actual) Fiscal Year 2027 (projected) Percent of Percent of Number of Number of eligible Number of Number of eligible Network enrolled eligible veterans who enrolled eligible veterans who and market veterans veterans are enrolled veterans veterans are enrolled VA New England Healthcare System (01) East 133,809 244,104 54.8 121,617 167,258 72.7 Far North 54,361 85,987 63.2 51,648 70,577 73.2 North 63,139 107,357 58.8 61,492 83,949 73.3 West 92,741 167,076 55.5 81,824 113,978 71.8 New York/New Jersey VA Health Care Network (02) Central 53,348 86,758 61.5 49,807 72,225 69.0 Eastern 49,026 83,322 58.8 42,902 61,044 70.3 Finger Lakes 26,883 43,640 61.6 23,603 31,782 74.3 Southern Tier 14,899 21,786 68.4 12,971 17,135 75.7 VA Long Island 53,496 85,362 62.7 38,541 44,695 86.2 VA Metro New York 119,736 192,565 62.2 95,017 117,768 80.7 VA New Jersey 93,651 173,132 54.1 76,741 103,279 74.3 Western 41,896 65,073 64.4 36,807 48,012 76.7 VA Healthcare - VISN 4 (04) Eastern 258,934 489,226 52.9 242,925 365,745 66.4 Western 150,309 257,065 58.5 136,119 188,212 72.3 VA Capitol Health Care Network (05) Baltimore 80,328 144,083 55.8 79,752 115,560 69.0 Beckley 15,825 22,094 71.6 13,950 18,298 76.2 Clarksburg 22,858 34,588 66.1 20,656 28,820 71.7 Huntington 33,644 49,528 67.9 29,540 39,676 74.5 Martinsburg 43,014 73,356 58.6 43,292 64,606 67.0 Washington 118,318 203,492 58.1 134,047 189,419 70.8 VA Mid-Atlantic Health Care Network (06) Northeast 160,277 294,307 54.5 189,261 308,767 61.3 Northwest 44,019 68,901 63.9 40,798 56,191 72.6 Southeast 178,475 288,546 61.9 222,542 325,599 68.3 Page 27 GAO-22-104604 VA Market Assessments Appendix |: Map of Department of Veterans Affairs (VA) Regional Networks and Markets and Numbers of Veterans in Each Market Fiscal Year 2017 (actual) Fiscal Year 2027 (projected) Percent of Percent of Number of Number of eligible Number of Number of eligible Network enrolled eligible veterans who enrolled eligible veterans who and market veterans veterans are enrolled veterans veterans are enrolled Southwest 142,806 221,055 64.6 148,197 196,620 75.4 VA Southeast Network (07) Alabama 164,509 255,236 64.5 182,963 253,918 72.1 Georgia 248,084 393,576 63.0 280,055 382,703 73.2 South Carolina 200,552 313,946 63.9 234,515 330,395 71.0 VA Sunshine Healthcare Network (08) Atlantic 65,979 102,356 64.5 53,431 64,504 82.8 Central 108,111 159,189 67.9 113,739 143,498 79.3 Gulf 120,781 199,025 60.7 113,974 144,399 78.9 Miami 68,816 110,450 62.3 59,533 76,622 77.7 North 170,160 265,365 64.1 189,747 268,296 70.7 Orlando 133,552 198,263 67.4 137,778 171,568 80.3 Puerto Rico Virgin 69,262 73,630 94.1 45,921 51,965 88.4 Islands VA MidSouth Healthcare Network (09) Central 131,345 218,041 60.2 152,120 218,943 69.5 Eastern 66,353 103,989 63.8 67,962 93,269 72.9 Northern 100,404 157,699 63.7 102,005 143,456 71.1 Western 70,376 113,575 62.0 69,299 99,916 69.4 VA Healthcare System (10) Central Ohio 79,480 129,860 61.2 79,279 107,433 73.8 Easter Michigan 112,552 203,642 55.3 107,717 144,877 74.4 Indiana 137,967 226,148 61.0 138,002 187,490 73.6 MichErie 108,199 198,888 54.4 110,496 155,564 71.0 Northeast Ohio 126,173 201,956 62.5 122,555 150,152 81.6 Western Ohio 104,133 175,107 59.5 104,607 139,452 75.0 VA Great Lakes Health Care System (12) Central 106,932 182,512 58.6 102,008 135,882 75.1 Central Illinois 39,070 71,378 54.7 36,807 55,245 66.6 Northern 53,022 80,399 65.9 48,261 63,783 75.7 Southern 175,546 293,924 59.7 157,376 206,403 76.2 VA Heartland Network (15) East 151,389 255,006 59.4 144,010 208,498 69.1 West 178,197 294,626 60.5 178,685 260,744 68.5 Page 28 GAO-22-104604 VA Market Assessments Appendix |: Map of Department of Veterans Affairs (VA) Regional Networks and Markets and Numbers of Veterans in Each Market Fiscal Year 2017 (actual) Fiscal Year 2027 (projected) Percent of Percent of Number of Number of eligible Number of Number of eligible Network enrolled eligible veterans who enrolled eligible veterans who and market veterans veterans are enrolled veterans veterans are enrolled VA Midwest Health Care Network (23) lowa Central 39,022 61,597 63.4 36,661 48,333 75.9 lowa East 58,919 97,799 60.2 55,328 76,166 72.6 Minnesota Central 39,154 54,571 71.7 38,608 45,196 85.4 Minnesota East 119,435 197,651 60.4 116,842 149,083 78.4 Nebraska 70,755 106,781 66.3 70,381 89,973 78.2 North Dakota 40,268 60,421 66.6 36,967 52,946 69.8 South Dakota East 34,148 51,819 65.9 31,103 42,431 73.3 South Dakota West 21,574 28,491 75.7 20,757 26,813 77.4 South Central VA Health Care Network (16) Central 98,031 154,934 63.3 91,708 136,575 67.1 Northern 122,585 191,455 64.0 119,718 173,885 68.8 Southern 203,508 336,687 60.4 222,601 326,909 68.1 VA Heart of Texas Health Care Network (17) Central 124,995 187,715 66.6 150,927 205,777 73.3 East Texas 149,919 245,402 61.1 164,269 221,927 74.0 North Texas 184,147 299,440 61.5 205,110 282,558 72.6 Northwest Texas 26,849 40,530 66.2 24,992 35,140 71.1 Southern 123,967 184,568 67.2 158,090 209,092 75.6 Southwest Texas 39,861 56,521 70.5 46,608 65,636 71.0 Valley Coastal Bend 45,262 66,393 68.2 47,505 61,149 77.7 West Texas 22,802 38,605 59.1 24,711 38,349 64.4 Rocky Mountain Network (19) Cheyenne 28,667 46,441 61.7 30,937 45,706 67.7 Denver 128,311 222,675 57.6 144,432 208,288 69.3 Eastern Oklahoma 51,391 82,195 62.5 52,548 73,527 71.5 Grand Junction 15,395 21,948 70.1 15,005 18,385 81.6 Montana 46,942 68,456 68.6 46,181 62,263 74.2 Oklahoma City 84,018 141,062 59.6 96,554 145,868 66.2 Salt Lake City 67,400 111,239 60.6 70,532 101,792 69.3 Sheridan 14,631 22,533 64.9 13,562 19,387 70.0 Northwest Network (20) Alaska 33,298 49,445 67.3 37,185 52,493 70.8 Inland North 69,722 109,379 63.7 73,555 99,093 74.2 Page 29 GAO-22-104604 VA Market Assessments Appendix |: Map of Department of Veterans Affairs (VA) Regional Networks and Markets and Numbers of Veterans in Each Market Fiscal Year 2017 (actual) Fiscal Year 2027 (projected) Percent of Percent of Number of Number of eligible Number of Number of eligible Network enrolled eligible veterans who enrolled eligible veterans who and market veterans veterans are enrolled veterans veterans are enrolled Inland South Idaho 37,501 55,625 67.4 40,316 54,123 74.5 South Cascades 153,905 244,665 62.9 153,807 206,088 74.6 Western Washington 143,543 264,049 54.4 163,600 245,242 66.7 Sierra Pacific Network (21) North Coast 58,093 96,980 59.9 50,373 61,717 81.6 North Valley 108,170 174,934 61.8 105,883 136,919 77.3 Pacific Islands 52,842 103,763 50.9 58,403 100,374 58.2 Sierra Nevada 47,470 72,467 65.5 47,063 61,008 77.1 South Coast 75,451 120,843 62.4 64,899 80,848 80.3 South Valley 41,968 62,236 67.4 42,126 53,671 78.5 Southern Nevada 75,563 113,772 66.4 82,104 103,030 79.7 Desert Pacific Healthcare Network (22) Albuquerque 68,387 102,758 66.6 67,121 92,641 72.5 Greater Los Angeles 213,609 363,327 58.8 192,447 243,691 79.0 Loma Linda 102,553 163,180 62.8 105,106 146,690 71.7 Phoenix 128,453 208,176 61.7 137,326 186,246 73.7 Prescott 32,027 45,400 70.5 33,098 39,373 84.1 San Diego 117,341 183,285 64.0 124,598 175,836 70.9 Tucson 64,807 92,962 69.7 68,081 89,432 76.1 Source: VA's Enrollee Healthcare Projection Model Data Provided to GAO. | GAO-22-104604 Page 30 GAO-22-104604 VA Market Assessments Appendix II: System-Wide Principles and Other Guidance for Department of Veterans Affairs (VA) Market Assessments According to VA officials, the department developed system-wide principles and other general guidance to assist its market assessment teams, comprised of VA and contractor staff, in analyzing market data, identifying gaps in supply for and demand of VA health care, and developing proposals to respond to the gaps. The 10 system-wide principles VA established for the market assessments were: 1. Design high-performing networks to better meet the health care needs of veterans in terms of access, quality, outcomes, and satisfaction, in accordance with the requirements outlined in the VA MISSION Act of 2018, and the Market Area Health Systems Optimization (market assessments) initiative. Retain or improve health care services for veterans in all high- performing networks. Ensure VA offers an optimal level of primary services on-site and that VA will be the coordinator of all health care whether provided in VA or in the community. Apply comparative data for performance, quality, patient satisfaction, and health outcomes, where available. Utilize data from the market assessments, adjacent markets and regional Networks, and across the national marketplace to facilitate the enhancement of high-performing networks. Ensure veterans are provided the opportunity to choose care they trust throughout their lifetime. Optimize health care services for Veterans in each market using a mix of VA care first, supplemented by the Department of Defense, academic affiliates, Federally Qualified Health Centers, and community providers. Options should include consideration of innovative alternatives such as "Hospital within a Hospital" ventures and public-private partnerships. Allow veterans to choose long-term care they trust, in the setting most appropriate for their needs, to the greatest extent possible, and when eligible. Maximize productivity, strategically prioritize investments, and leverage virtual care modalities and partnerships rather than build facilities, when possible. Page 31 GAO-22-104604 VA Market Assessments Appendix Il: System-Wide Principles and Other Guidance for Department of Veterans Affairs (VA) Market Assessments 10. Complete market assessments in partnership with local VA healthcare market leaders and a team of consultants who will collectively assess Veterans Health Administration capabilities, community resources, and provide objectivity and external validation in collaborative application of the standardized, data-driven, principles-based methodology. VA Officials said the department developed guidance that included general questions and key data elements to consider in applying these principles. Table 2 lists the general questions, and Table 3 lists key data elements for certain service lines and outpatient facilities. Table 2: General Questions Considered in the Department of Veterans Affairs (VA) Market Assessments to Analyze Market Data and Identify Gaps Between Supply and Demand of Veteran Health Care, According to VA Officials Type of data Question Geography and demographics Where do veterans live now and in the future? Demand What services do veterans need currently and in the future? Can the current medical programs be sustained in the future? Capacity What is the current and potential future supply at VA and other federal and commercial facilities? Access Do veterans have convenient access to high quality care? Are there access gaps? Quality and satisfaction Do internal and external providers meet VA care standards? Staffing Can VA appropriately staff its clinical programs? Facilities Are the current facility conditions and infrastructure capable of providing a safe environment of care that meets current design standards? Mission Would opportunities impact the VA's education, research, or emergency preparedness missions? Source: Department of Veterans Affairs | GAO-22-104604 Table 3: Data Elements Considered in Department of Veterans Affairs (VA) Market Assessments to Analyze Market Data and Identify Gaps Between Supply and Demand, According to VA Officials Inpatient (IP) Medicine/Surgery Demographics « Number of enrollees (FY18) and projected growth in market and/or area (FY17-FY27) « Market reliance on VA and community IP medicine/surgery (med/surg) Demand « Number of med/surg beds; average daily census (ADC);? any relevant shift(s) between fiscal years; case mix index (CMI)? e« Med/surg projections (FY27) « Care purchased in community e Case volume and trends Page 32 GAO-22-104604 VA Market Assessments Appendix Il: System-Wide Principles and Other Guidance for Department of Veterans Affairs (VA) Market Assessments Inpatient (IP) Medicine/Surgery Supply Number of commercial hospital beds; occupancy rate; CMI Local VA medical center (VAMC) / partnerships Future inpatient projections / conceptual excess Access Number and percentage of enrollees within a 60-minute drive time of med/surg point of care (POC)* Proximity to enrollee population density centers / proximity relative to hospital referral region (HRR)4 Physical access to proposed site / access to major highway(s) Quality and satisfaction Quality of Community, partner, and/or VAMC VA Strategic Analytics for Improving and Learning (SAIL) data (readmissions; mortality; admission, continued stay)® Staffing Ability to recruit and retain providers and nurses Facility Condition of facility and infrastructure Facilities Condition Assessment (FCA),' modernization cost, IP med/surg or main facility related engineering challenges Mission Number of residents and advanced fellows Veterans Equitable Resource Allocation (VERA) total funding$ Emergency preparedness designation Cost effectiveness Distance to potential commercial partner or VAMC Cost per IP case / total cost of care for selected services Inpatient Mental Health (MH) Demographics Number of enrollees (FY18) and projected growth in market and/or area (FY17-FY27) Market reliance on VA and community IP MH Demand Number of MH beds; average daily census (ADC); occupancy rate; any relevant shift(s) between FY; CMI IP MH projections (FY27) Care purchased in community Supply Number of Commercial Hospital beds; occupancy rate; CMI Local VAMC / partnerships Future inpatient projections / conceptual excess Access Number and percentage of enrollees within a 60-minute drive time of current facility and/or proposed facility Proximity to enrollee population density centers / proximity relative to HRR Distance to potential commercial partner or VAMC Quality and satisfaction Quality of community, partner, and/or VAMC where MH beds currently are or are proposed to go Staffing Ability to recruit and retain providers Number of dually/multiple appointed providers Facility Condition of facility and infrastructure FCA, modernization cost, IP MH related engineering challenges Page 33 GAO-22-104604 VA Market Assessments Appendix Il: System-Wide Principles and Other Guidance for Department of Veterans Affairs (VA) Market Assessments Inpatient (IP) Medicine/Surgery Mission « Number of residents and advanced fellows e VERA research allocation total funding +» Emergency preparedness designation Cost effectiveness + Distance to potential commercial partner or VAMC Cost per IP case / total cost of care Community Living Centers (CLC) Demographics Number of enrollees (FY18) and projected growth in market and/or area (FY17-FY27) Market age breakdown/average age/growth rates by age group/priority group Demand Number of CLC beds at current and/or proposed VA facility Current ADC (FY18), historic shifts (FY15-FY18), and future projections (FY 17-27) Care purchased (fee-basis long-term care) in community Market projections for CLC/nursing home need Short stay versus long stay bed and demand breakdown Supply Other VA CLCs and partnerships in the area State veteran homes in the area Commercial ADC, number of beds, occupancy rate / number excess beds (FY18) Availability (surplus/deficit) of community providers Access Number and percentage of enrollees within a 60-minute drive time of current facility and/or proposed facility Proximity to nearest VA staffed CLC and potential partners Physical access to proposed site / Access to major highway(s) Proximity to enrollee population density centers / proximity relative to HRR Quality and satisfaction CLC overall quality score VA SAIL data Staffing « Ability to recruit and retain VA physicians and nurses Facility « Condition of facility and infrastructure (age, expansion capabilities, appropriateness of small house model) « Semiprivate and multi-bed patient rooms « Square footage - departmental gross square feet (DGSF) « FCA (overall, by user, per square feet) Mission « NA Cost effectiveness « NA Residential Rehabilitation Treatment Programs (RRTP) Demographics Number of enrollees (FY 18) and projected growth in market and/or area (FY17-FY27) Demand « Number of RRTP beds at current and/or proposed VA facility « VA facility current ADC (FY18), historic shifts (FY15-FY18), and future projections (FY17- 27) « Program specificity of RRTP beds in VA facility Supply e Availability of RRTP or like services in the community Availability (surplus/deficit) of community providers Page 34 GAO-22-104604 VA Market Assessments Appendix Il: System-Wide Principles and Other Guidance for Department of Veterans Affairs (VA) Market Assessments Inpatient (IP) Medicine/Surgery Access Number and percentage of enrollees within a 60-minute drive time of current facility and/or proposed facility Proximity to nearest VA staffed RRTP Physical access to proposed site / access to major highway(s) / proximity to enrollee population density centers / proximity relative to HRR Quality and satisfaction VA SAIL data Staffing Ability to recruit and retain VA physicians and nurses Facility Condition of facility and infrastructure (age, expansion capabilities) Square footage (DGSF) FCA (overall, by user, per square feet) Modernization cost National strategies RRTP/MRRTP Market Analysis conducted by VA Emergency Departments (ED) and Urgent Care Clinics Demographics Number of enrollees (FY 18) and projected growth in market and/or area (FY17-FY27) Demand Number of ED Encounters; percentage change FY15-18 Percentage transfers, percentage admitted Acuity? Number or percentage of encounters between 8am-8pm or on weekdays vs. weekends Outpatient Utilization Projections (RVUs) for emergency medicine' Historical demand purchased in the community (NVCC) Supply Local VAMC / Partnerships Access Number and percentage of enrollees within a 30-minute drive time of current facility and/or proposed facility Diversion hours! Drive time to nearest VA staffed site/partner/community hospitals providing services Quality and satisfaction Quality of community/partner (ED volume and throughput) Number Volume of ED at community provider(s) ED capabilities at community provider(s) (e.g. trauma center level) Staffing ¢ - Ability to recruit and retain providers and nurses Facility e« Square footage allocation e« Condition, flow, and environment of care of facility and infrastructure « FCA Mission « Emergency preparedness designation Cost effectiveness Distance to potential commercial partner or VAMC Cost per IP case / total cost of care Outpatient (OP) Surgery Demographics Number of enrollees (FY18) and projected growth in market and/or area (FY17-FY27 Demand Number of OP surgical cases Historical growth of OP Surgical Cases, by specialty (FY15-18) Projected RVU growth by for surgical services (FY17-FY27) Care purchased in community Page 35 GAO-22-104604 VA Market Assessments Appendix Il: System-Wide Principles and Other Guidance for Department of Veterans Affairs (VA) Market Assessments Inpatient (IP) Medicine/Surgery Supply e« Local VAMC / Partnerships « Commercial hospital ADC, number of beds; occupancy rate / number excess beds; current CMI (FY18) ¢ _ Availability (surplus/deficit) of community providers by specialty Access e Number and percentage of enrollees within a 60-minute drive time of current facility and/or proposed facility « Drive time to nearest VA staffed site/partner/community hospitals providing services « Proximity to enrollee population density centers Quality and satisfaction « Mortality rate, avoidable adverse events, in-hospital complication rate, infection rate Staffing + Availability to recruit and retain VA physicians and nurses Facility e Condition, flow, and environment of care of facility and infrastructure « FCA Mission « Number of residents and advanced fellows Cost effectiveness « Cost per case / total cost of care Community-based Outpatient Clinics, Multi-specialty Community-based Outpatient Clinics, Health Care Clinics, and Other Outpatient Services Demographics « Number of enrollees (FY 18); projected growth of current and/or proposed county (FY17- FY27) e Percentage rural Demand e« Number of core unique patients; number of primary care (PC) unique patients; number of non-overlapping enrollees within 30/60-minutes of current and/or proposed location (FY18) e - Historic growth of core unique patients by service line and PC unique patients (FY15- FY 18) e« Projected RVU growth by service line (FY17-FY27) e _ Referral patterns « NVCC FY18 - outpatient CPT authorizations by county Supply e _ Availability of high-quality community providers and hospitals e Federally qualified health centers (FQHCs), Department of Defense (DOD), and Indian Health Service (IHS) facilities in the area that could help meet demand Access « Number of enrollees within a 30/60-minute drive time of current facility and/or proposed facility « Proximity to nearest VA-staffed site providing these services e - Proximity to enrollee population density centers/proximity relative to HRR « Physical access to proposed site/access to major highway(s) Quality and satisfaction « NA Staffing « _ Ability to recruit and retain primary care, mental health, or specialty providers « _ If decompressing campus: Number of patient-aligned care teams (PACTs) and providers at the current location and the number of PACTs and providers relocating to the new VA site' Facility « Lease expiration date (only if pertinent to opportunity) « Condition of facility and infrastructure (only if pertinent to opportunity) e Space to absorb additional volume « Square footage Page 36 GAO-22-104604 VA Market Assessments Appendix Il: System-Wide Principles and Other Guidance for Department of Veterans Affairs (VA) Market Assessments Inpatient (IP) Medicine/Surgery Mission N/A Cost effectiveness N/A Source: Department of Veterans Affairs | GAO-22-104604 "Average daily census (ADC) is the average number of inpatients per day. 'Case mix index (CMI) is a measure of the relative cost of the patients treated in each hospital or group of hospitals. According to Definitive Healthcare, CMI reflects the diversity, complexity, and severity of patient illnesses treated at a given hospital. 'Point of care (POC) is a site where medical care is received, such as doctors' offices and hospitals. 'Hospital referral regions (HRR) represent regional health care markets for tertiary medical care, according to Dartmouth Atlas Project. Each HRR contains at least one hospital that performs major cardiovascular procedures and neurosurgery. "VA Strategic Analysis for Improving and Learning (SAIL) helps Veterans Health Administration officials assess VA medical center performance and includes 27 quality measures in areas such as acute care mortality and access to care. 'Facility Condition Assessment (FCA) is a multi-disciplinary VA effort that includes architects and engineers to evaluate the condition of each VA medical center. 8Veterans Equitable Resource Allocation (VERA) is a model the Veterans Health Administration uses to allocate general purpose funds to its 18 regional networks. "Acuity indicates the severity of illness or condition among patients. iRelative value units (RVUs) are a consistent measurement basis that can be used for understanding utilization projections in terms of professional staffing requirements. !Diversion refers to a situation in which all patients or a selected group of patients who would normally be treated by the VA medical facility cannot be accepted for admission and evaluation because the appropriate beds are not available, needed services cannot be provided, staffing is inadequate, acceptance of another patient would jeopardize the ability to properly care for those already at the facility, or disaster has disrupted normal operations. 'Patient-aligned care teams (PACT) refers to a team-based model of care in which a team of health professionals, led by a provider, works collaboratively with the patient to provide for all of the patient's healthcare needs-or appropriately coordinates care with other qualified professionals. Page 37 GAO-22-104604 VA Market Assessments Appendix Ill: Comments from the Department of Veterans Affairs DEPARTMENT OF VETERANS AFFAIRS WASHINGTON January 10, 2022 Ms. Sharon M. Silas Director Health Care U.S. Government Accountability Office 441 G Street, NW Washington, DC 20548 Dear Ms. Silas: The Department of Veterans Affairs (VA) has reviewed the Government Accountability Office (GAO) draft report: VA HEALTH CARE: Incomplete Information Hinders Usefulness of Market Assessments for VA Facility Realignment (GAO-22- 104604). The enclosure contains general and technical comments and the actions to be taken to address the draft report recommendations. VA appreciates the opportunity to comment on your draft report. Sincerely, Gyn G) Avacbher Tanya J. Bradsher Chief of Staff Enclosure Page 38 GAO-22-104604 VA Market Assessments Appendix Ill: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Comments to Government Accountability Office (GAO) Draft Report VA HEALTH CARE: Incomplete Information Hinders Usefulness of Market Assessments for VA Facility Realignment (GAOQ-22-104604) Recommendation 1: For the VA Secretary to improve the quality of information used in VA's market assessments by reviewing the data on community care to identify any gaps and take steps to address data completeness. VA Response: Concur. The Veterans Health Administration (VHA) is committed to continuous quality improvement across all decision support and business intelligence systems. VHA is aware of the existing limitations within the Veterans Community Care Program's (VCCP) data systems. Beginning with the Veterans Choice Program and continuing with the subsequent creation of VCCP in the VA MISSION Act of 2018 (MISSION Act), VA has engaged in an enterprise-wide, multi-phase improvement process that fundamentally re-imagines and re-defines VA's interactions with community providers. During this process, VA specifically triaged critical data systems within VCCP to ensure Veteran access to care; timely payment to providers; high-quality care by community providers; and enhanced decision support. Under the oversight of the VA Data Governance Council (DGC) (Co-Chaired by the Office of Enterprise Integration (OEI) and the Office of Information and Technology), VHA's Office of Community Care (OCC) engaged in a data architecture pilot, in collaboration with the Chief Data Officer, which led to process redesign of business architecture and data architecture efforts. OCC is currently identifying data stewards for the data within its purview; assigning responsibilities; and standing up a VHA Data Governance Office to oversee these stewards and the overall management of their data. OCC's data improvements will be briefed regularly to the enterprise VA DGC. As VHA reviews the data on community care to identify any gaps and take steps to address data completeness, VHA's Chief Strategy Office will collaborate with OCC in coordination with the governance processes described above. Target Completion Date: September 2022 Recommendation 2: For the VA Secretary to externally communicate to the Commission about the completeness and reliability of VA data used to inform the assessments and how VA considered any data limitations in developing proposals for the modernization and realignment of VA facilities. VA Response: Concur. VHA is committed to the collection and use of quality information for developing recommendations for the modernization and realignment of VA facilities. While GAO's report is accurate that the Market Assessment process did not include a direct audit of VHA data used within the assessments, proper internal Page 39 GAO-22-104604 VA Market Assessments Appendix Ill: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Comments to Government Accountability Office (GAO) Draft Report VA HEALTH CARE: Incomplete Information Hinders Usefulness of Market Assessments for VA Facility Realignment (GAOQ-22-104604) controls relating to audit and validation for the data used where followed by VHA prior to the Market Assessment process. To enhance clarity of ownership of VHA's data completeness and reliability, VHA has engaged in development of a DGC within VHA's Governing Board structure. Concurrently, OEI's Office of Data Governance and Analytics (DGA) is engaged in the collaborative development of policy and execution of data governance for the enterprise, helping to improve data completeness and reliability across the Agency. Following the issuance of the VA Enterprise Data Strategy in January 2021, VA developed an implementation plan and a roadmap that highlights the Agency's top priority efforts under direction of the VA DGC. Priorities include improving data quality for community care and access data; developing enterprise-wide trusted data objects within a common operating platform that integrates quality, authoritative data across VA; and setting forth guidelines/standards for quality assessments and documentation to accompany all highly influential information, including those used in the market assessment work. VHA's Chief Strategy Office will provide the Asset and Infrastructure Review (AIR) Commission with information that specifically outlines the completeness and reliability limitations of VA data used to develop VA's recommendations. Target Completion Date: July 2022 Page 40 GAO-22-104604 VA Market Assessments Appendix Ill: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Comments to Government Accountability Office (GAO) Draft Report VA HEALTH CARE: Incomplete Information Hinders Usefulness of Market Assessments for VA Facility Realignment (GAOQ-22-104604) General Comments: VA appreciates GAO's recommendations, and we are committed to delivering health care for Veterans when and where they need it based on the highest quality information possible. We are focused on using data that complies with Federal internal control standards relating to information that is appropriate, current, complete, accurate and provided timely. While VA concurs with GAO's recommendations, VA does not believe that what is outlined by GAO "hinders usefulness" of market assessments for VA facility realignment, as stated in the title of the report. The first set of quadrennial market assessments are based on extensive data; interviews with more than 1,800 Veterans Integrated Service Networks (VISN) and VA Medical Center (VAMC) leaders: input from 50 listening sessions conducted with Veterans and other stakeholders across the country; quality assurance analyses; and reviews by senior leaders at the VISN, VHA and Department levels. While all data has its limitations, the market assessments reflect the highest quality set of VA data sources available to date. The VA data used to develop the market assessments came from data sources that are subject to a rigorous testing, evaluation and auditing process conducted by the data owners. These data sources are the same that are relied upon by VHA for operational analyses and health care research. As noted by GAO, the market assessments also included a data validation process by which VISN planners, VAMC planners and subject matter experts (SME) in the data domains were asked to review and validate data. Based on the reviews by their teams, all network directors certified the data as an "acceptable foundation for Market Assessment analysis." The documentation within the market assessments have outlined the data sources for information used for developing facility realignment plans and recommendations. VHA will continue to improve the quality of the data within the market assessment process as VHA data systems continue to evolve. As a learning organization, VHA is constantly improving the data used in decision-making. The data will change and improve over time as VA conducts more detailed planning for recommendations approved by the AIR Commission and as it prepares for subsequent quadrennial assessments. The VA MISSION Act of 2018 established the market assessment process to inform and support the AIR Commission, which will help VA modernize its infrastructure to increase Veteran access to care and improve Veteran outcomes well into the future. The market assessments will design high-performing integrated health care networks to provide coordinated, lifelong, world-class health care and services to Veterans across the country. Page 41 GAO-22-104604 VA Market Assessments Appendix Ill: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Comments to Government Accountability Office (GAO) Draft Report VA HEALTH CARE: Incomplete Information Hinders Usefulness of Market Assessments for VA Facility Realignment (GAOQ-22-104604) VA appreciates the review conducted by GAO in its report about data quality. Data in the health care environment is dynamic. VA's strategy to mitigate this reality has centered on strengthening VA's data governance and engaging with those collaborators that know the data best, the leaders and clinicians in the field. VA interviewed more than 1,800 experts in the field and kept close coordination with the Network Directors throughout the process. Additionally, the Secretary of Veterans Affairs directed that an independent "Red Team' be established to conduct an analysis of VA's market assessment work and to assess potential COVID-19 pandemic impacts. The Secretary also requested that the Red Team, consisting of external health care professionals and former VA Secretaries, assess a sampling of the recommendations currently under consideration. The Red Team findings, as detailed in an October 23, 2021, report, are consistent with GAO's findings. While their review found that the "entire process, successfully completed, will significantly improve VA's health care delivery to Veterans for many years to come," the report also reflects the team's collective opinion that the pandemic has permanently changed health care and that the pre-pandemic market assessment process may not have been expansive enough to identify all emerging health care trends. As a result of national health care's uncertainty of the impact of the pandemic and needed improvement in data quality, VA will continue to refine our data governance and work with the Commission to identify data limitations that will inform and assist the Commission during the review process. Page 42 GAO-22-104604 VA Market Assessments Appendix IV: GAO Contacts and Staff Acknowledgements GAO Contacts Sharon M. Silas, (202) 512-7114 or silass@gao.gov Staff In addition to the contacts named above, Ann Tynan (Assistant Director), Aaron Holling (Analyst-in-Charge), Topher Hoffmann, and Jennie F. Apter Acknowledgements made key coniributions to this report. Also contributing were Jacquelyn Hamilton, Vikki Porter, Ravi Sharma, and Valeria Robayo. (104604) Page 43 GAO-22-104604 VA Market Assessments GAO's Mission Obtaining Copies of GAO Reports and Testimony The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is through our website. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. You can also subscribe to GAO's email updates to receive notification of newly posted products. Order by Phone Connect with GAO To Report Fraud, Waste, and Abuse in Federal Programs Congressional Relations Public Affairs Strategic Planning and External Liaison The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, https:/Awww.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts. Visit GAO on the web at https:/Awww.gao.gov. Contact FraudNet: Website: https:/Avww.gao.gov/about/what-gao-does/fraudnet Automated answering system: (800) 424-5454 or (202) 512-7700 A. Nicole Clowers, Managing Director, ClowersA@gao.gov, (202) 512-4400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548 Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Stephen J. Sanford, Managing Director, spel@gao.gov, (202) 512-4707 U.S. Government Accountability Office, 441 G Street NW, Room 7814, Washington, DC 20548 oy ue Please Print on Recycled Paper.