United States Government Accountability Office Report to Congressional Committees VA HEALTH CARE April 2018 Improved Guidance and Oversight Needed for the Patient Advocacy Program Page i GAO-18-356 GAO-18-356 VHA Patient Advocacy Program GAO-18-356 April 2018 VA HEALTH CARE Improved Guidance and Oversight Needed for the Patient Advocacy Program Highlights of GAO-18-356 a report to congressional committees Why GAO Did This Study What GAO Found VHA has designated patient advocates The Veterans Health Administration (VHA) provided limited guidance to at each VAMC to receive and Department of Veterans Affairs (VA) medical centers (VAMC) on the governance document feedback from veterans or of its patient advocacy program and its guidance, a program handbook, has their representatives, including been outdated since 2010. VAMCs are still expected to follow the outdated requests for information, compliments, handbook, which does not provide needed details on governance, such as and complaints. In recent years, the specifying the VAMC department to which patient advocates should report. importance of a strong patient Officials from most of the VAMCs that GAO reviewed noted that the VAMC advocacy program has taken on new department to which patient advocates report can have a direct effect on the significance given concerns with VHA’s ability of staff to resolve veterans’ complaints. The lack of updated and complete ability to provide veterans timely guidance may impede the patient advocacy program from meeting its access to health care, among other issues. expectations, to receive and address complaints from veterans in a convenient and timely manner. The Comprehensive Addiction and Recovery Act of 2016 included a VHA also has provided limited guidance to VAMCs on staffing the patient provision for GAO to review VHA’s advocacy program. VHA’s handbook states that every VAMC should have at patient advocacy program. This report least one patient advocate and appropriate support staff; however, it did not examines the extent to which VHA has provide guidance on how to determine the number and type of staff needed. (1) provided guidance on the Officials at all but one of the eight VAMCs in GAO’s review stated that their governance of the program; (2) patient advocacy program staff had more work to do than they could accomplish. provided guidance on staffing the This limited guidance on staffing could impede VAMCs’ efforts to ensure that program; (3) assessed the training they have the appropriate number and type of staff to address veterans’ needs of patient advocates and complaints in a timely manner. monitored training completion; and (4) monitored patient advocacy program Further, VHA has recommended training for patient advocates, but it has not data-entry practices and reviewed developed an approach to routinely assess their training needs or monitored program data. GAO reviewed VHA and training completion. VHA officials stated that they relied on VAMC and Veterans VAMC documents, including Integrated Service Network (VISN) staff to conduct these activities. However, summaries of program data. GAO GAO found that for the eight VAMCs in its review, the training needs of patient interviewed VHA officials about the advocates were not routinely assessed, and training completion was not always program, as well as officials from a monitored. Without conducting these activities, VHA increases its risk that staff non-generalizable selection of eight may not be adequately trained to advocate on behalf of veterans. VAMCs and five VISNs selected based on the volume of veteran complaints Finally, VHA has not monitored patient advocacy program data-entry practices or and other factors. GAO also compared reviewed the data to assess program performance. VHA officials stated that they VHA policies and practices to federal relied on VISN and VAMC officials to ensure that all complaints were consistently internal control standards. entered into VHA’s Patient Advocate Tracking System (PATS). However, GAO identified inconsistencies in the extent to which VAMC officials did so. VHA’s lack What GAO Recommends of monitoring may pose a risk that not all complaints are entered into this GAO is making 6 recommendations to tracking system—a goal of the program. Additionally, VHA officials stated they improve guidance for and oversight of did not systemically review data in the system to assess program performance the patient advocacy program, and identify potential system-wide improvements because VHA considered this focusing on governance, staffing, the responsibility of VAMCs. As a result, VHA officials may miss opportunities to training, and PATS data entry and improve veterans’ experiences. assessment. VA concurred with GAO’s VHA is beginning to address many of these governance, staffing, training, and recommendations. data issues, including directing a workgroup to provide recommendations by View GAO-18-356. For more information, spring of 2018. However, because the recommendations will be advisory, and contact Debra A. Draper at (202) 512-7114 or because program deadlines have slipped in the past, the nature and timing of the draperd@gao.gov. actions needed to resolve these issues remain unclear. United States Government Accountability Office Contents Letter 1 Background 6 VHA Has Provided Limited, Outdated Guidance to VAMCs on the Governance of the Patient Advocacy Program 10 VHA Has Provided Limited Guidance to VAMCs on Staffing the Patient Advocacy Program 12 VHA Has Recommended Training for Patient Advocates, but Has Not Developed an Approach to Routinely Assess Their Training Needs or Monitored Training Completion 15 VHA Has Not Monitored Patient Advocacy Data-Entry Practices or Reviewed Patient Advocacy Data to Assess Program Performance and Identify System-Wide Improvements 18 Conclusions 22 Recommendations for Executive Action 23 Agency Comments and Our Evaluation 23 Appendix I Awareness of the Patient Advocacy Program 25 Appendix II Patient Advocate Tracking System (PATS) Data Entry and Management 27 Appendix III Approaches to the Governance and Staffing of the Patient Advocacy Program 36 Appendix IV Comments from the Department of Veterans Affairs 38 Appendix V GAO Contact and Staff Acknowledgments 41 Tables Table 1: Selected Department of Veterans Affairs Medical Centers (VAMC) and Associated Veterans Integrated Service Networks (VISN) Included in Review 4 Page i GAO-18-356 VHA Patient Advocacy Program Table 2: Issue Codes Used to Describe Veterans’ Feedback in the Veterans Health Administration (VHA) Patient Advocate Tracking System (PATS), Fiscal Year (FY) 2017 30 Table 3: The Most Common Issues Entered in the Patient Advocate Tracking System (PATS), according to the Department of Veterans Affairs (VA), Fiscal Year 2017 33 Table 4: Patient Advocate Tracking System (PATS) Data-Entry Practices at Eight Selected Department of Veterans Affairs Medical Centers (VAMC), as of September 2017 34 Table 5: The Number of Positions and Titles Used for Patient Advocates and Service-Level Advocates at Eight Selected Department of Veterans Affairs Medical Centers (VAMC), as of September 2017 36 Figures Figure 1: Process at Department of Veterans Affairs Medical Centers (VAMC) for Entering and Closing Veterans’ Feedback in the Patient Advocate Tracking System 8 Figure 2: Examples of Patient Advocacy Program Signage at Selected Department of Veterans Affairs Medical Centers 26 Figure 3: Example of Information Patient Advocacy Program Staff at VAMCs Enter in the Patient Advocate Tracking System (PATS) to Add a Report of Contact (ROC) 28 Figure 4: Example of Information Patient Advocacy Program Staff at Department of Veterans Affairs Medical Centers (VAMC) Enter in the Patient Advocate Tracking System to Close a Report of Contact (ROC) 29 Page ii GAO-18-356 VHA Patient Advocacy Program Abbreviations CARA Comprehensive Addiction and Recovery Act of 2016 FY fiscal year OIG Office of Inspector General OPA Office of Patient Advocacy OPCC&CT Office of Patient Centered Care and Cultural Transformation PATS Patient Advocate Tracking System ROC report of contact VA Department of Veterans Affairs VAMC VA medical center VHA Veterans Health Administration VISN Veterans Integrated Service Network VSO veterans service organization This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page iii GAO-18-356 VHA Patient Advocacy Program Letter 441 G St. N.W. Washington, DC 20548 April 12, 2018 The Honorable Johnny Isakson Chairman The Honorable Jon Tester Ranking Member Committee on Veterans’ Affairs United States Senate The Honorable Phil Roe, M.D. Chairman The Honorable Tim Walz Ranking Member Committee on Veterans’ Affairs House of Representatives The Veterans Health Administration (VHA), within the Department of Veterans Affairs (VA), operates one of the nation’s largest health care systems. VA has faced a growing demand by veterans for its health care services due, in part, to (1) servicemembers returning from the United States’ military operations in Afghanistan and Iraq and (2) the growing needs of an aging veteran population—trends that are expected to continue. The total number of veteran enrollees in VA’s health care system rose from 7.9 million to almost 9 million from fiscal year (FY) 2006 through FY 2016. As the number of veterans using VA health care services increases, it is important to ensure that feedback about their care is addressed in a convenient and timely manner. To that end, VHA has designated patient advocates at each VA medical center (VAMC) to receive and document feedback from veterans or their representatives, including requests for information, compliments, and complaints. 1 Although VHA’s patient advocacy program has been in place since 1990, it has taken on new significance in recent years given concerns about veterans’ ability to receive timely and quality care, among other issues. Our work, along with that of the VA Office of Inspector General (OIG) and others, has cited longstanding concerns about VA’s oversight of its health 1 In this report, the term “VAMC” refers to an individual VA medical center and any of its associated facilities such as community-based outpatient clinics. Page 1 GAO-18-356 VHA Patient Advocacy Program care system, including ambiguous policies and inconsistent processes. 2 These concerns contributed to the addition of veterans’ health care to GAO’s High-Risk List in 2015, and its continued inclusion in the 2017 update. 3 In 2017, the VA OIG raised concerns about the effectiveness of VHA’s patient advocacy program, including a lack of monitoring of program data, such as feedback documented by advocates, to identify trends across VAMCs. 4 Until recently, VHA’s patient advocacy program was overseen by the Office of Patient Centered Care and Cultural Transformation (OPCC&CT). However, the Comprehensive Addiction and Recovery Act of 2016 (CARA), included a provision for VHA to establish an Office of Patient Advocacy (OPA) by July 2017 to take on oversight responsibilities for the program, such as ensuring that patient advocates at VAMCs receive training. 5 CARA also included a provision for us to review VHA’s patient advocacy program. 6 This report examines the extent to which VHA has 2 For example, see GAO, Veterans Health Care: Additional Actions Could Further Improve Policy Management, GAO-17-748 (Washington, D.C.: Sept. 22, 2017), and VA Health Care: Improved Policies and Oversight Needed for Reviewing and Reporting Providers for Quality and Safety Concerns, GAO-18-63 (Washington, D.C., Nov. 15, 2017). See also, for example, Department of Veterans Affairs, Office of Inspector General, Veterans Health Administration, Review of the Implementation of the Veterans Choice Program, Report No. 15-04673-333 (Washington, D.C.: Jan. 30, 2017), and Veterans Health Administration, Audit of Veteran Wait Time Data, Choice Access, and Consult Management in VISN 6, Report No. 16-02618-424 (Washington, D.C.: Mar. 2, 2017). 3 GAO maintains a high-risk list to focus attention on government agencies and programs that it identifies as high risk due to their greater vulnerabilities to fraud, waste, abuse, and mismanagement or the need for transformation to address economy, efficiency, or effectiveness challenges. See GAO, High-Risk Series: An Update, GAO-15-290 (Washington, D.C.: Feb. 11, 2015). In our 2017 high-risk update, we reported that previously identified issues for policy management continue, in part, because VA lacks the capacity to effectively address the issues we raised. See GAO, High-Risk Series: Progress on Many High-Risk Areas, While Substantial Efforts Needed on Others, GAO-17-317 (Washington D.C.: Feb. 15, 2017). 4 Department of Veterans Affairs, Office of Inspector General, Veterans Health Administration, Audit of the Patient Advocacy Program, Report No. 15-05379-146. (Washington, D.C.: Mar. 31, 2017). 5 Comprehensive Addiction and Recovery Act of 2016, Pub. L. No. 114-198, § 924, 130 Stat. 695, 767-769 (2016). 6 Pub. L. No. 114-198, § 923, 130 Stat. 695, 767 (2016). Page 2 GAO-18-356 VHA Patient Advocacy Program 1. provided guidance to VAMCs on the governance of the program; 2. provided guidance to VAMCs on staffing the program; 3. assessed the training needs of patient advocates and monitored training completion; and 4. monitored patient advocacy program data-entry practices and reviewed program data. For all four objectives, we interviewed officials involved in the patient advocacy program from eight VAMCs and their five associated Veterans Integrated Service Networks (VISN), regional networks of care. We selected six of our eight VAMCs for variation in (1) the number of complaints received and entered in VHA’s Patient Advocate Tracking System (PATS)—an electronic system used to describe and track the resolution of veterans’ feedback across VAMCs—in FY 2016, (2) facility complexity level, (3) geographic location, and (4) the type of staff VAMCs used to administer the program. 7 We selected the remaining two VAMCs based on one’s involvement in piloting a new approach to recording patient advocacy program data, and one’s recent changes to the structure of its program. We also ensured that at least three of the VAMCs we selected received assessments from OPCC&CT on how they had implemented the patient advocacy program to gain perspectives on the office’s involvement with VAMCs. 8 See table 1 for a list of the eight VAMCs we selected and their associated VISNs. Perspectives obtained from the eight VAMCs and five VISNs in our review cannot be generalized. 7 VHA categorizes VAMCs according to complexity level, which is determined on the basis of the characteristics of the patient population, clinical services offered, educational and research missions, and administrative complexity. We obtained the number of complaints received by VAMCs and recorded in PATS in FY 2016 from VISN officials in March 2017. We then ranked VAMCs by the number of complaints and split them into three equal groups. We then selected at least two VAMCs from each of these groups. 8 In 2015 and 2016, OPCC&CT conducted assessments of the implementation of the patient advocacy program at some VAMCs and provided recommendations on how to strengthen program implementation to better align with a proactive approach to patient advocacy. Page 3 GAO-18-356 VHA Patient Advocacy Program Table 1: Selected Department of Veterans Affairs Medical Centers (VAMC) and Associated Veterans Integrated Service Networks (VISN) Included in Review VAMC VISN Edith Nourse Rogers Memorial Veterans Hospital (Bedford, Massachusetts) 1 VA Maryland Health Care System (Baltimore, Maryland) 5 Fayetteville VA Medical Center (Fayetteville, North Carolina) 6 Hampton VA Medical Center (Hampton, Virginia) 6 John J. Pershing VA Medical Center (Poplar Bluff, Missouri) 15 Kansas City VA Medical Center (Kansas City, Missouri) 15 Grand Junction Veterans Health Care System (Grand Junction, Colorado) 19 VA Eastern Colorado Health Care System (Denver, Colorado) 19 Source: GAO analysis of Veterans Health Administration (VHA) information. | GAO-18-356 We also interviewed VHA officials, including those who had overseen the program when it was under OPCC&CT and who transitioned to OPA once the office was established in 2017. In addition, we interviewed officials from five veterans service organizations (VSO) to obtain their perspectives on the patient advocacy program: American Legion, Disabled American Veterans, Iraq and Afghanistan Veterans of America, Veterans of Foreign Wars, and Vietnam Veterans of America. 9 To examine the extent to which VHA has provided guidance to VAMCs on the governance of the patient advocacy program, we reviewed VHA’s handbook for the program to determine, among other things, whether it specified the VAMC department to which patient advocates should report and whether it identifies responsibilities for VHA staff overseeing the program. 10 We also reviewed documentation of VHA’s planned efforts related to improving the governance of the program, such as a draft directive for the program. We evaluated the information we reviewed in VHA’s handbook against federal internal control standards. 11 9 We selected five VSOs based on several criteria including whether the organization published articles or reports about VHA’s patient advocacy program. 10 For example, see Department of Veterans Affairs, Veterans Health Administration, VHA Patient Advocacy Program, VHA Handbook 1003.4 (Sept. 2, 2005). 11 GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: September 2014). Internal control is a process effected by an entity’s oversight body, management, and other personnel that provides reasonable assurance that the objectives of an entity will be achieved. Page 4 GAO-18-356 VHA Patient Advocacy Program To examine the extent to which VHA has provided guidance to VAMCs on staffing the patient advocacy program, we reviewed VHA’s handbook for the program. Specifically, we reviewed the handbook to determine the extent to which it provided guidance on how VAMCs should determine the appropriate number and type of staff needed to administer the program. We also reviewed documentation of VHA’s planned efforts related to staffing the program identified in a workgroup charter. We evaluated VHA’s efforts to provide guidance to VAMCs on staffing against key principles for effective strategic workforce planning and federal internal control standards. 12 To examine the extent to which VHA has assessed the training needs of patient advocates and monitored training completion, we reviewed training materials VHA provided to VAMCs, such as a list of recommended training for patient advocates. We also reviewed documentation of VHA’s planned efforts related to assessing the training needs of patient advocates identified in a workgroup charter. We evaluated the extent to which VHA has monitored training of patient advocates against a guide for assessing strategic training and development efforts and federal internal control standards. 13 To examine the extent to which VHA has monitored data-entry practices and reviewed data from the patient advocacy program, we reviewed VHA’s handbook for the program and summaries of data from PATS. 14 We also reviewed documentation of VHA’s planned efforts related to PATS data-entry practices and reviewing program data identified in a workgroup charter. We evaluated the extent to which VHA has monitored data-entry practices and reviewed PATS data against a guide for assessing the reliability of computer-processed data and federal internal control standards. 15 12 See GAO-14-704G and GAO Human Capital: Key Principles for Effective Strategic Workforce Planning, GAO-04-39 (Washington, D.C.: Dec. 11, 2003). 13 See GAO-14-704G and GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: Mar. 1, 2004). 14 To obtain contextual information related to the patient advocacy program, we reviewed VA analyses of data for fiscal years 2014 through 2017 that had been entered in PATS as of January 4, 2018. 15 See GAO-14-704G and GAO, Assessing the Reliability of Computer-Processed Data, GAO-09-680G (Washington, D.C.: July 2009). Page 5 GAO-18-356 VHA Patient Advocacy Program We conducted this performance audit from February 2017 to April 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. VHA’s patient advocacy program is intended to provide veterans with a Background means to provide feedback about health care services they receive at VAMCs. VHA sets forth minimum expectations for VAMCs’ administration of the program, including that veterans must have easy access to a patient advocate and must have their complaints addressed in a convenient and timely manner. Administration of the The patient advocacy program is administered at the VAMC level. Each Patient Advocacy Program of VA’s 170 VAMCs is responsible for making at least one patient advocate available to respond to veterans’ feedback, and for ensuring that feedback is recorded in PATS. VAMCs may designate other staff to assist patient advocates in responding to feedback, such as lead patient advocates and service-level advocates. Service-level advocates, such as nurses or administrative staff, are designated at some VAMCs to respond to veterans’ feedback before involving a patient advocate. All VAMC staff that have a designated role in the administration of the patient advocacy program are referred to as patient advocacy program staff. In addition to designating program staff, VAMCs may use a variety of methods to make veterans aware of the patient advocacy program, such as displaying signage on site and including information about the program on their websites. (See app. I for more information on the methods selected VAMCs used to make veterans aware of the program.) Patient advocacy program staff enter veterans’ feedback in PATS using a report of contact (ROC) and assign one or more issue codes that generally describe the nature of the feedback, such as coordination of care. (See app. II for additional information on entering veterans’ feedback into PATS.) Each piece of feedback shared is categorized as either a request for information, compliment, or complaint. VHA’s handbook for the program specifies certain goals for data collection and resolution—specifically, that Page 6 GAO-18-356 VHA Patient Advocacy Program • all complaints should be entered in PATS to enable a comprehensive understanding of veterans’ issues and concerns to, in turn, identify potential system-wide improvements; and • responses should occur no later than 7 days after the complaint is made. 16 With this guidance, patient advocacy program staff use a variety of approaches for entering veterans’ feedback in PATS and closing it in the system once addressed. For example, when VAMCs have designated service-level advocates, the process for entering and closing feedback in PATS is generally different than the approach used by VAMCs that have only patient advocates. (See fig. 1.) 16 See VHA Handbook 1003.4. Page 7 GAO-18-356 VHA Patient Advocacy Program Figure 1: Process at Department of Veterans Affairs Medical Centers (VAMC) for Entering and Closing Veterans’ Feedback in the Patient Advocate Tracking System Note: In this figure the term “veteran” includes veterans and their representatives, such as family members or friends, and the term “feedback” includes requests for information, compliments, and complaints. Patient advocates are VHA employees designated at each VAMC to receive feedback from veterans. Service-level advocates, such as nurses or administrative staff, are designated at some VAMCs to receive veterans’ feedback before involving a patient advocate. According to VHA’s handbook for the program, responses to complaints should occur as soon as possible, but no longer than 7 days after the complaint is made. VHA officials told us they interpret this goal to mean that complaints should be closed in PATS within 7 days. Other forms of feedback (requests for information and compliments) are not required to be entered in PATS but may be entered, as this information is useful in determining common areas of confusion that could be addressed in a proactive manner. Page 8 GAO-18-356 VHA Patient Advocacy Program Patient advocacy program staff at each VAMC are assisted by a VISN- level coordinator who acts as a liaison between the VAMCs and VHA and is responsible for ensuring consistency in PATS data collection within the VISN. The VISN director is responsible for designating the coordinator and ensuring that each VAMC within the VISN has at least one patient advocate. Oversight of the Patient The VHA office responsible for overseeing the patient advocacy program Advocacy Program changed as a result of CARA. From January 2011 to July 2017, the program was overseen by OPCC&CT under VHA’s Deputy Under Secretary for Health for Operations & Management. 17 CARA included a provision for VHA to establish OPA to begin overseeing the program and specified that this office would report directly to the Under Secretary for Health, a higher-level office within VHA. 18 Although OPCC&CT is no longer responsible for overseeing the program, it is to continue to play an advisory role to OPA during the initial phases of its work, according to OPCC&CT officials. Many of OPA’s oversight responsibilities are specified in CARA including ensuring that patient advocates advocate on behalf of veterans, manage PATS, and identify trends in the data to determine whether there are opportunities for improving veterans’ health care. Also, OPA’s director is required to ensure that patient advocates receive relevant, consistent training across VAMCs. When establishing the office in July 2017, VHA officials wrote a memo indicating that OPA’s primary objectives were to implement a standardized policy for the patient advocacy program and to resolve any system-wide issues, such as concerns about care across VAMCs identified through veterans’ feedback. In addition, in August 2017, OPA began soliciting feedback from VAMCs on various aspects of the patient advocacy program to identify improvement priorities and best practices. By September 2017, OPA had identified an acting program director, established a workgroup (called the National Strategic Workgroup) to develop recommendations related to program 17 VHA’s patient advocacy program was established in 1990. From the program’s inception to January 2011, the program was overseen by the National Veteran Service and Advocacy Program of the VISN Support Service Center. 18 Pub. L. No. 114-198, § 924, 130 Stat. 767 (2016), codified at 38 U.S.C. §7309A(c)(2)(C). Page 9 GAO-18-356 VHA Patient Advocacy Program administration, and finalized a charter that identifies workgroup deliverables. VHA has provided limited guidance to VAMCs on the governance of the VHA Has Provided patient advocacy program. Specifically, VHA provided limited guidance on Limited, Outdated how to meet the program’s expectations that veterans have easy access to a patient advocate who will hear their complaints and address them in Guidance to VAMCs a timely manner. While VHA’s handbook for the program provides general on the Governance of information on the responsibilities of patient advocacy program staff, it does not specify the VAMC department to which patient advocates should the Patient Advocacy report to help ensure VAMCs meet these expectations. According to VHA Program officials, the lack of specific guidance was intentional and due in part to VHA officials’ view that leadership at each VAMC is in the best position to understand the needs of veterans at their facilities, and therefore should have flexibility to make decisions about governance in response to those needs. In addition to providing limited guidance to VAMCs, VHA’s patient advocacy program handbook is out of date and does not incorporate recent agency-wide changes, such as those made in response to VHA Strategic Plan FY 2013 – 2018 which identifies the goal of providing proactive, patient-driven health care. 19 The handbook for the program was issued in 2005, expired in 2010, and as of January 2018, no updates had been released. In the absence of an updated document, VAMCs are still expected to follow the outdated handbook. 20 However, the handbook does not identify the responsibilities of the current VHA office responsible for overseeing the program. Instead, it identifies the responsibilities of the VHA office that oversaw the program before OPCC&CT began overseeing the program in 2011. In recent years, OPCC&CT reviewed the implementation of the patient advocacy program at some VAMCs and provided specific recommendations on how to change program governance to better reflect a more proactive patient advocacy program model. However, the recommendations from these reviews were provided only to some 19 See Department of Veterans Affairs, Veterans Health Administration, VHA Strategic Plan FY 2013 – 2018. 20 See Department of Veterans Affairs, Veterans Health Administration, Controlled National Policy/Directives Management System, VHA Directive 6330 (Washington, D.C.: June 24, 2016). Page 10 GAO-18-356 VHA Patient Advocacy Program VAMCs; guidance that could be applicable to all VAMCs was not added to the handbook. 21 OPCC&CT officials stated that they did not update the handbook because they decided to instead spend time trying to understand recent feedback they received from VAMC officials and ensure that any updates would reflect system-wide shifts as a result of VHA’s strategic plan. OPCC&CT’s limited and outdated guidance to VAMCs on the governance of the patient advocacy program is inconsistent with federal internal control standards for the control environment, which require agencies to establish an organizational structure, assign responsibility, and delegate authority to achieve agency objectives—key aspects of governance. 22 To do so, an agency may develop an organizational structure that assigns responsibilities to discrete units and defines reporting lines at all levels of the organization. Without providing specific, timely guidance to VAMCs on the governance of the patient advocacy program, the program is at risk of not meeting its minimum expectations. In light of the limited and outdated guidance on the governance of the program, patient advocacy program staff at most of our selected VAMCs noted that the VAMC department to which patient advocates report can have a direct effect on the ability of staff to resolve veterans’ complaints. For example, patient advocates at one VAMC said because of the program’s position within the organization, they did not have the authority to ensure that VAMC officials external to the patient advocacy program, such as physicians, quickly engaged in responding to veterans’ complaints. In these cases, a patient advocate would contact the physician to resolve a complaint, but may not have received a response until the matter was brought to the attention of the physician’s supervisor—a reporting line that is outside of the patient advocacy program at this VAMC. Officials from several of our selected VAMCs and VSOs noted that the position of the patient advocacy program within VAMCs may not give patient advocates the authority to require VAMC 21 According to an assessment OPCC&CT completed for one VAMC on the implementation of the patient advocacy program, for example, in a proactive model of the patient advocacy program, all staff take responsibility for the veteran’s experience and attempt to resolve complaints at the lowest level possible in the organization. To implement a proactive model, OPCC&CT recommended that this VAMC consider developing processes to increase the engagement of VAMC staff external to the patient advocacy program, such as physicians. 22 GAO-14-704G. Page 11 GAO-18-356 VHA Patient Advocacy Program staff to respond to veterans’ complaints. They added that conflict-of- interest concerns could arise when a veteran has a complaint about a VAMC for which the patient advocate works. (See app. III for additional information on the governance of the patient advocacy program at selected VAMCs.) In VA’s written comments on a draft of this report, which are reproduced in Appendix IV, VA stated that it issued its new directive for the patient advocacy program that had been in development as we were conducting our review. 23 While the updated directive specifies that a VAMC’s lead patient advocate should report to the facility director, it does not specify the VAMC department to which other patient advocacy program staff, including patient advocates who are not designated as lead patient advocates and service-level advocates, should report. In addition, OPA’s National Strategic Workgroup recently submitted recommendations to OPA on the governance of the patient advocacy program. OPA officials stated that they plan to prioritize the recommendations and elicit feedback from VISN directors on how to operationalize the recommendations. However, it is unclear whether OPA will provide additional guidance related to the governance of the program based on these recommendations, such as guidance on the VAMC department to which all types of patient advocacy program staff should report. Until actions to address the weaknesses we found are completed, guidance on the governance of the program will continue to be lacking. VHA has provided limited guidance to VAMCs on the number and type of VHA Has Provided patient advocacy program staff needed to ensure that complaints from Limited Guidance to veterans are addressed in a convenient and timely manner. According to VHA’s existing handbook for the program, every VAMC should have at VAMCs on Staffing least one patient advocate and appropriate administrative, technical, and the Patient Advocacy clerical support should be provided to allow for efficient performance of the responsibilities of program staff. OPCC&CT did not provide guidance Program on how VAMCs should determine the appropriate number of administrative, technical, and clerical staff or type of patient advocacy program staff, such as lead patient advocates and service-level advocates. According to officials, this was because no assessment was conducted to identify what staff resources would be needed to meet the 23 See Department of Veterans Affairs, Veterans Health Administration, VHA Patient Advocacy, VHA Directive 1003.04 (Washington, D.C.: Feb. 7, 2018). Page 12 GAO-18-356 VHA Patient Advocacy Program expectations of the program. In the absence of such an assessment, OPCC&CT instead relied on each VAMC to determine what resources would be needed based on the facility’s size and services provided. However, VHA’s handbook for the program does not provide instruction for VAMC or VISN officials on how to determine the number and type of staff needed for the program. OPCC&CT officials added that budget constraints can also affect a VAMC’s ability to hire the appropriate staff for the program. (See app. III for additional information on the number and type of patient advocacy program staff at selected VAMCs.) Officials at all but one of the selected VAMCs stated that program staff at their VAMCs had more work to do than they could handle. For example, VAMC officials cited backlogs in work, such as calls from veterans not being answered, messages not being responded to, voicemail boxes being full, and not all veterans’ feedback being entered into PATS. Officials from one VAMC we spoke with in July 2017 stated that due to workload demands and not enough patient advocacy program staff at their VAMC, they had roughly 300 unanswered phone calls at that time from veterans who want to provide feedback to a patient advocate. Officials from several VSOs we spoke with stated that there is not enough patient advocate staff, adding that veterans reported that their calls to patient advocates were not answered, they were unable to reach an advocate, or their calls were not responded to in a timely manner. The lack of staffing guidance is inconsistent with GAO’s Key Principles for Effective Strategic Workforce Planning, which states that workforce planning is essential to addressing an organization’s critical need to align its human capital program with its current and emerging mission and programmatic goals. 24 Further, federal internal control standards require agencies to design control activities to achieve objectives, a key aspect of effectively staffing a program. 25 Such control activities may include effectively managing the agency’s workforce, such as by continually assessing the knowledge, skills, and abilities of the workforce to achieve organizational goals. 24 See GAO Human Capital: Key Principles for Effective Strategic Workforce Planning, GAO-04-39 (Washington, D.C.: Dec. 11, 2003). As part of the planning process, organizations should determine the skills and competencies that will be needed to achieve current and future programmatic results and develop strategies, such as policies and practices that are tailored to address the gaps in number, deployment, and alignment of human capital approaches. 25 GAO-14-704G. Page 13 GAO-18-356 VHA Patient Advocacy Program The lack of guidance on staffing may impede VAMCs’ efforts to ensure that they have the appropriate number and type of staff to administer the patient advocacy program. The resulting misalignment of staff resources could have negatively affected VAMCs’ ability to achieve the program’s objectives, including addressing veterans’ complaints in a timely manner. For example, if there are not a sufficient number of patient advocates to respond to veterans’ phone calls in a timely manner, VAMCs may not be able to ensure that patient advocates can respond to veterans’ complaints within 7 days, as called for by VHA’s handbook for the program. According to VHA officials, OPA analyzed feedback from VAMCs on the factors that should be considered in developing national guidelines for staffing, such as facility size and complexity level, and directed its National Strategic Workgroup to develop recommendations for determining the extent to which VAMCs have utilized various patient advocacy program staff, such as service-level advocates, by the spring of 2018. However, OPA expects that these efforts will result in recommendations for consideration, and it is unclear what steps, if any, will be taken based on the recommendations. Until actions to address the weaknesses we found are completed, the lack of guidance for VAMCs on determining the appropriate number and types of staff will put the patient advocacy program at risk of being unable to address veterans’ complaints in a convenient and timely manner. Page 14 GAO-18-356 VHA Patient Advocacy Program VHA Has Recommended Training for Patient Advocates, but Has Not Developed an Approach to Routinely Assess Their Training Needs or Monitored Training Completion VHA Has Developed a VHA has recently developed a list of recommended training for patient Recommended Training advocates. In the spring of 2017, OPCC&CT officials updated a recommended training list for patient advocates developed before 2011 List for Patient Advocates, when OPCC&CT began overseeing the patient advocacy program. The but Has Not Developed an training list covers a wide variety of topics, including how to enter and Approach to Assess Their examine trends in PATS data, as well as key responsibilities of patient Training Needs on a advocates outlined in VHA’s handbook for the program. OPCC&CT Routine Basis officials stated that they would like to make the trainings required, but have not pursued this because of the lengthy process within VHA to designate required training for a specific group of staff. To update the list in 2017, OPCC&CT convened a workgroup (which included several patient advocates) to determine whether the old training list was sufficient, and the workgroup shared its suggested updates with VISN- level coordinators for distribution to VAMCs in April 2017. We found that OPCC&CT has not developed an approach to routinely assess the training needs of patient advocates. Rather, OPCC&CT officials stated that they relied on VAMC and VISN staff to conduct these assessments. However, VHA’s handbook for the program does not specify that VAMC or VISN officials are responsible for conducting routine assessments of patient advocates’ training needs. None of our selected VAMCs routinely conducted assessments of the training needs of patient advocates, such as assessing whether advocates were adequately trained to carry out their responsibilities. Officials from two VAMCs said they used ad hoc approaches to assess training needs. For example, one patient advocate supervisor stated that training is offered on an “as Page 15 GAO-18-356 VHA Patient Advocacy Program needed” basis in patient advocate meetings when a training need is identified. The lack of an approach for routinely assessing the training needs of patient advocates is inconsistent with federal standards for internal control related to control activities. 26 Under these standards relating to human capital, management ensures that training is aimed at developing and retaining employee knowledge, skills, and abilities to meet changing organizational needs. Management should also continually assess the knowledge, skills, and ability needs of a program so that the program is able to obtain a workforce that has the required knowledge, skills, and abilities to achieve organizational goals. Without an approach for routinely assessing the training needs of patient advocates, VHA may not be able to clearly identify gaps in the knowledge and skills of these staff over time, which, in turn, could put the program at risk of not meeting its goals. For example, if there is a gap in understanding among patient advocates that all complaints should be entered into PATS, addressing veterans’ complaints may be delayed, if addressed at all, and opportunities to analyze complaint data for the purpose of identifying system-wide improvements may be missed. According to VHA officials, OPA analyzed feedback from VAMCs on the training needs of patient advocates, including how to correctly enter data into PATS, and directed its National Strategic Workgroup to develop recommendations for assessing the training needs of patient advocates by the spring of 2018. OPA expects that these efforts will result in recommendations for OPA to consider, but it is unclear what steps, if any, will be taken based on the recommendations. Until actions to address the weaknesses we found are completed, the lack of routine assessments of training needs will continue to put the program at risk of staff not having the requisite skills and knowledge to carry out their duties. VHA Has Not Monitored VHA has not monitored the completion of training for patient advocates. Training Completion for Specifically, OPCC&CT officials said that they did not monitor the extent to which patient advocates completed the recommended training Patient Advocates distributed in April 2017. Instead, these officials relied on patient advocate supervisors to monitor training completion. However, VHA’s handbook for 26 GAO-14-704G. Page 16 GAO-18-356 VHA Patient Advocacy Program the program does not specify that patient advocate supervisors are responsible for monitoring the completion of training for patient advocates. Half of patient advocate supervisors at our selected VAMCs did not track the completion of patient advocacy training. Patient advocate supervisors said that they are able to track the completion of general VA employee training through VA’s Talent Management System. However, most training specific to patient advocacy were generally not included in this system during the period of our review. Officials from our selected VAMCs who did track patient advocacy training used various methods to record completion, such as keeping attendance lists for the training provided. Taking steps to monitor training completion would be consistent with GAO’s Guide for Assessing Strategic Training and Development Efforts in the Federal Government which identifies components of the training and development process, including having agencies collect and monitor data corresponding to establishing training objectives. 27 Monitoring training completion would also be consistent with federal standards for internal control related to control activities. 28 Under these standards relating to human capital, management ensures that training is aimed at developing and retaining employee knowledge, skills, and abilities to meet changing organizational needs. Management also continually assesses the knowledge, skills, and ability needs of a program so that the program is able to obtain a workforce that has the required knowledge, skills, and abilities to achieve organizational goals—key components for monitoring training completion. If patient advocates are not properly trained in how to use PATS to document and resolve complaints, tracking the status of complaints may be more difficult, which could increase the likelihood that they are not addressed in a timely manner, if at all. Further, CARA specifies that the director of OPA should ensure that patient advocates receive training specific to patient advocacy. 29 27 GAO-04-546G. 28 GAO-14-704G. 29 Pub. L. No. 114-198, § 924, 130 Stat. 767 (2016), codified at 38 U.S.C. § 7309A(c)(2)(C). Page 17 GAO-18-356 VHA Patient Advocacy Program According to VHA officials, OPA did not obtain information on whether patient advocates completed recommended training and did not identify an approach for monitoring training completion moving forward. Without monitoring training completion, there is an increased risk that patient advocates have not received the training they need to effectively fulfill their responsibilities such as advocating on behalf of veterans and consistently using PATS to document and resolve complaints. VHA Has Not Monitored Patient Advocacy Data-Entry Practices or Reviewed Patient Advocacy Data to Assess Program Performance and Identify System-Wide Improvements VHA Has Not Monitored VHA officials have not monitored PATS data-entry practices to ensure Whether Complaints Were complaints were always entered into PATS and issue codes were assigned consistently to ROCs. OPCC&CT officials told us they did not Always Entered into PATS monitor the data-entry practices of patient advocacy program staff to and Issue Codes Assigned ensure that all complaints were entered into PATS, a key goal according Consistently to VHA’s handbook for the program. 30 Rather, they relied on VISN and VAMC officials to ensure that program staff entered all complaints into PATS. Officials from two of the five VISNs we interviewed stated that they did not perform any audits or checks of the data entered into PATS by patient program staff at VAMCs. 30 See VHA Handbook 1003.4. The handbook states that requests for information and compliments may also be entered into PATS, as this information is useful in determining common areas of confusion that could be addressed in a proactive manner. Page 18 GAO-18-356 VHA Patient Advocacy Program We also found inconsistencies in the extent to which VAMC officials entered complaints into PATS, with complaints always entered into PATS at one of our selected VAMCs, while at other VAMCs some complaints were left unrecorded, according to officials. For example, at one VAMC, officials stated that over a third of the complaints received were not entered into PATS due to the competing workload demands of patient advocates. Similarly, at another selected VAMC, almost a quarter of the complaints received were not entered into PATS, according to patient advocates there who explained that they primarily used a document outside of PATS to record veterans’ feedback. In addition, OPCC&CT officials told us they did not monitor whether patient advocates used a consistent practice to assign issue codes to veterans’ feedback recorded into PATS. Using a consistent data-entry practice is important to ensure that PATS data can be compared across VAMCs to better enable an accurate and comprehensive understanding of veterans’ issues and concerns, a goal of the patient advocacy program. OPCC&CT officials stated that they relied on VISN-level coordinators to monitor coding practices because VHA’s handbook for the program states that these coordinators should develop VISN-wide consistent approaches for entering complaints into PATS. VISN-level coordinators from two selected VISNs stated that they created a standard practice for assigning issue codes within a particular VISN; however, the coding practices differed between VISNs, making national level analysis difficult. We also found inconsistencies in how VAMC officials coded specific veterans’ feedback. For example, patient advocates did not use consistent practices to code issues related to the Veterans Choice Program (Choice Program), one of the most common types of issues patient advocates told us they hear about from veterans. 31 Officials from one of our selected VAMCs said they code feedback related to the Choice Program under a specific “request for information” issue code, regardless of whether the feedback was a request for information, compliment, or complaint. In contrast, officials at another VAMC stated that they typically code feedback related to the Choice Program as a complaint related to billing. (See app. II for additional information on data- entry practices at selected VAMCs.) 31 The Veterans Choice Program is a program that allows eligible veterans to receive health care from a community provider when faced with long wait times, lengthy travel distances, or other challenges accessing care at a VAMC. Page 19 GAO-18-356 VHA Patient Advocacy Program OPCC&CT’s lack of monitoring of PATS data-entry practices is inconsistent with GAO’s Assessing the Reliability of Computer-Processed Data which identifies the importance of consistent data-entry practices to ensure that data are reasonably complete and accurate. 32 Further, federal standards for internal control related to information and communications require agencies to use quality information, such as relevant data from reliable sources, to achieve the agency’s objectives. 33 Under internal control standards for control activities, management also is to monitor performance to achieve objectives. Without OPCC&CT monitoring data- entry practices, the patient advocacy program is at risk of not meeting its goal that all complaints are entered into PATS and there is an increased likelihood of VHA not having an accurate understanding of veterans’ complaints across VAMCs. Moving forward, in fall 2017, OPA distributed meeting minutes to all VISN and VAMC directors stating that all veterans’ feedback should be consistently recorded in PATS. OPA officials also updated some of the issue codes in PATS in fall 2017 and added a code specifically for community care issues, such as issues related to the Choice Program. In addition, OPA officials stated that they plan to promote the consistent assignment of issue codes to veterans’ feedback through national training, but have not specified when this training will occur or if OPA staff will monitor patient advocates’ consistent assignment of issue codes or of data-entry practices generally. Until these actions are completed, however, the gaps in monitoring of PATS data-entry practices that we identified will continue to exist, putting the program at risk of incomplete or unreliable data that may not allow an accurate understanding of veterans’ complaints, critical to making system-wide improvements. VHA Has Not VHA officials have not systematically reviewed PATS data to assess Systematically Reviewed program performance and identify potential system-wide improvements, goals of the patient advocacy program. Specifically, OPCC&CT officials PATS Data to Assess stated that they reviewed PATS data in response to inquiries, but did not Program Performance and conduct systematic reviews of the data over time. For example, they did Identify Potential System- not track VAMC performance on responding to complaints in a timely Wide Improvements manner or track the most common complaints across VAMCs to identify potential opportunities for system-wide improvements. 32 GAO-09-680G. 33 GAO-14-704G. Page 20 GAO-18-356 VHA Patient Advocacy Program OPCC&CT officials stated that they did not conduct systematic reviews of PATS data because VISN and VAMC officials were primarily responsible for these analyses. However, according to VHA’s handbook for the patient advocacy program, VHA officials have a responsibility to examine PATS data for trends across VAMCs and identify any areas for system- wide improvement. Officials stated that it was challenging to analyze PATS information included in narrative text, such as descriptions of veterans’ feedback. Not reviewing PATS data is inconsistent with federal standards for internal control for monitoring which require agencies to establish and operate monitoring activities, such as assessing the quality of performance over time, and evaluate the results. 34 Further, not conducting systematic assessments of PATS data made it difficult for OPCC&CT to determine program performance, such as whether the program was meeting its goal that all complaints are entered into PATS and responded to within 7 days. Officials explained that VHA interprets this goal to mean that complaints are closed in PATS within 7 days. According to VA, between FY 2014 and FY 2017 there were more than 53,000 complaints per year open for greater than 7 days. 35 If OPCC&CT officials had conducted systematic reviews of PATS data, they may have been able to identify that there were a significant number of complaints open for longer than 7 days and consider what actions should be taken, such as providing additional guidance to VAMCs on how to address complaints in a timely manner. Furthermore, without systematically reviewing PATS data across VAMCs to identify potential system-wide improvements, OPCC&CT officials may have been unaware of important care issues across VAMCs. For example, patient advocates from several of our selected VAMCs stated that opioid prescription issues are among the most common complaints they received from veterans. If OPCC&CT officials were to have systematically reviewed PATS data across VAMCs to determine the prevalence of these types of complaints, they could have identified the need to address them on a national level and consider system-wide policies or guidance in response. 34 GAO-14-704G. 35 According to VA, between FY 2014 and FY 2017, the total number of reports of contact entered in PATS ranged from about 360,000 to about 414,000 per year. Page 21 GAO-18-356 VHA Patient Advocacy Program According to VHA officials, OPA is in the process of identifying the data it needs to review on a routine basis, and directed its National Strategic Workgroup to identify program data that could be reviewed to assess program performance and identify potential system-wide improvements by the spring of 2018. However, OPA expects that these efforts will result in recommendations for OPA to consider, and it is unclear what steps, if any, will be taken based on the recommendations. Until actions to address the weaknesses we found are completed, the lack of a systematic review of PATS data will persist, putting the program at continued risk of missed opportunities for identifying and addressing weaknesses across VAMCs. As one of the largest health care delivery systems in the nation, it is Conclusions critically important for VHA to ensure that each veteran who receives health care services has easy access to an advocate who listens to that veteran’s feedback and responds in a timely manner. This is especially important given concerns about veterans’ ability to receive timely and quality care. However, VHA’s efforts to ensure that the patient advocacy program is meeting its goals—to identify potential system-wide improvements and respond to complaints within 7 days—have fallen short. OPCC&CT did not provide sufficient oversight to the program in the four key areas of governance, staffing, training, and data-entry practices, which has left the program at risk for not meeting its goals. VHA’s newly established OPA has initiated plans to improve the patient advocacy program in these four areas; however, most of these plans center around a workgroup that will make recommendations for OPA to consider, and it is unclear what specific actions, if any, will be taken based on these recommendations. Further, documentation for several of OPA’s planned efforts has not been finalized. Unless specific actions to address the weaknesses we identified are completed expeditiously, the program is at risk of not meeting its goals, including addressing veterans’ complaints in a convenient and timely manner. Furthermore, without addressing the weaknesses we identified, OPA misses opportunities to review PATS data across VAMCs to identify potential system-wide issues that, if addressed, could significantly improve the experience of veterans. Such reviews are critical to ensuring that VHA is taking steps to both meet its goal in its strategic plan to provide veterans with timely and quality health care, and to address recent issues it has faced, such as veterans’ ability to access care in a timely manner. Page 22 GAO-18-356 VHA Patient Advocacy Program We are making the following six recommendations to the VHA Recommendations for Undersecretary for Health: Executive Action • provide updated guidance to VAMCs on the governance of the patient advocacy program, including clear definitions of reporting lines. (Recommendation 1) • assess and provide guidance to VAMCs on appropriately staffing the patient advocacy program, including guidance on how to determine the appropriate number and type of staff. (Recommendation 2) • develop an approach to routinely assess the training needs of patient advocates. (Recommendation 3) • monitor the completion of training for patient advocates. (Recommendation 4) • monitor PATS data-entry practices to ensure all complaints are entered into PATS and that veterans’ feedback is coded consistently. (Recommendation 5) • systematically review PATS data to assess program performance and identify potential system-wide improvements. (Recommendation 6) We provided a draft of this report to VA for comment. In its written Agency Comments comments, which are reproduced in Appendix IV, VA concurred with our and Our Evaluation recommendations and noted that it recently issued the new directive for patient advocacy that had been in development as we were conducting our review. The directive supersedes the outdated handbook for the patient advocacy program and describes certain aspects of program governance, including certain reporting lines, roles, and responsibilities. Accordingly, VA requested that we close our first recommendation related to governance. We revised our report to reflect the issuance of the new directive. However, we do not believe the directive fully implements our recommendation. While the updated directive specifies that a VAMC’s lead patient advocate should report to the facility director, it does not specify the VAMC department to which other patient advocacy program staff, including patient advocates who are not designated as lead patient advocates and service-level advocates, should report. Until VA specifies the reporting lines for these other patient advocacy program staff, our recommendation will remain open. In addition, VA stated in its written comments that OPA has efforts underway related to staffing, training, and PATS data entry and assessment and provided estimated completion Page 23 GAO-18-356 VHA Patient Advocacy Program dates for these efforts. We will monitor VA’s efforts to address our recommendations. VA did not provide technical comments. We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, the Under Secretary for Health, and other interested parties. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or at draperd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in Appendix V. Debra A. Draper Director, Health Care Page 24 GAO-18-356 VHA Patient Advocacy Program Appendix I: Awareness of the Patient Appendix I: Awareness of the Patient Advocacy Program Advocacy Program Our eight selected Department of Veterans Affairs (VA) medical centers (VAMC) use a variety of methods to make veterans or their representatives aware of the patient advocacy program, including providing brochures on the program, displaying signage, and providing program information on the VAMC’s website. 1 (See fig. 2 for examples of patient advocacy program signage at some of the VAMCs we visited.) 1 The term “VAMC” refers to an individual VAMC and any of its associated facilities, such as community-based outpatient clinics. Page 25 GAO-18-356 VHA Patient Advocacy Program Appendix I: Awareness of the Patient Advocacy Program Figure 2: Examples of Patient Advocacy Program Signage at Selected Department of Veterans Affairs Medical Centers Page 26 GAO-18-356 VHA Patient Advocacy Program Appendix II: Patient Advocate Tracking Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management System (PATS) Data Entry and Management Patient advocacy program staff, such as patient advocates or service- level advocates who are designated to respond to veterans’ feedback, enter feedback from veterans or their representatives in the Veterans Health Administration (VHA) Patient Advocate Tracking System (PATS) by creating a report of contact (ROC). Each ROC includes basic information regarding the individuals involved, a description of the feedback provided by the veteran, and a description of the steps taken to resolve the issue. Patient advocacy program staff assign one or more issue codes that generally describe the nature of the feedback, such as “coordination of care.” (See figures 3 and 4.) Page 27 GAO-18-356 VHA Patient Advocacy Program Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management Figure 3: Example of Information Patient Advocacy Program Staff at VAMCs Enter in the Patient Advocate Tracking System (PATS) to Add a Report of Contact (ROC) Note: In addition to contacting a patient advocate, veterans may use VA’s Inquiry Routing and Information System, an Internet-based public messaging system, to express feedback. Page 28 GAO-18-356 VHA Patient Advocacy Program Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management Figure 4: Example of Information Patient Advocacy Program Staff at Department of Veterans Affairs Medical Centers (VAMC) Enter in the Patient Advocate Tracking System to Close a Report of Contact (ROC) Page 29 GAO-18-356 VHA Patient Advocacy Program Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management In order to organize veterans’ feedback, VHA categorizes feedback as either requests for information, compliments, or complaints. Within each of these categories VHA defines specific issue codes for program staff to select from based on the description of the veteran’s feedback. (See table 2.) Table 2: Issue Codes Used to Describe Veterans’ Feedback in the Veterans Health Administration (VHA) Patient Advocate Tracking System (PATS), Fiscal Year (FY) 2017 Category Issue description Request for information Application for care/eligibility for medical benefits Request for information Department of Veterans Affairs (VA) billing for service Request for information Advance directives Request for information Referral issues (internal/community) Request for information Medical center regulations Request for information Obtaining copies of medical records/completion of forms Request for information VA regional office questions on compensation and pension Request for information Legal issues Request for information Patient rights and responsibilities Request for information Other Request for information Social services Compliment Compliment received from patient/family member/staff Complaint Excessive wait at facility for a scheduled appointment Complaint Excessive wait at facility for an unscheduled appointment Complaint Excessive delay in scheduling or rescheduling appointment Complaint Delay/postponement in scheduled test/procedures or surgery Complaint Delay in receiving test results Complaint Excessive wait for care (inpatient) Complaint Excessive wait for equipment/wrong equipment sent Complaint Delay ordering/renewing pain medications Complaint Delay ordering/renewing other medications Complaint Excessive wait in pharmacy Complaint Excessive wait for pharmacy mailings Complaint Phone calls/letters/secure messages not returned/or timely Complaint Delay in receiving travel pay Complaint Delay in receiving completed forms from provider Complaint Appeal - denial of application Complaint Appeal - tier level hierarchy Page 30 GAO-18-356 VHA Patient Advocacy Program Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management Category Issue description Complaint Appeal - revocation/removal from program Complaint All other clinical appeals (not caregiver related) Complaint Dissatisfied with referral outcome Complaint Patient perceives care is not coordinated/expectations not met Complaint Inconsistency in information given to patient/family Complaint Appointment date/time misunderstood/not communicated/wrong Complaint Diagnosis/care/prevention Complaint Purpose/side effects of medication Complaint Emotional needs not met Complaint Provider is difficult to understand Complaint Lack of privacy Complaint Complaints concerning canteen cafeteria/store/vending areas Complaint Difficulty finding parking Complaint Cleanliness/temperature/sensory concerns Complaint Signage not clear (directional maps etc.) Complaint Family not involved in patient’s care Complaint Application for care/eligibility for medical benefits Complaint Eligibility for health care/follow up/hospital/extended/community living center Complaint Dental eligibility Complaint Ambulance/private hospital/private care payment eligibility Complaint VA billing for service/pharmacy co-payment Complaint Prosthetics eligibility Complaint Travel eligibility Complaint Complementary and integrative health Complaint Patient does not have one provider Complaint Patient does not know who is their provider Complaint Hygiene, diet, feeding, therapy, ambulation needs (inpatient/health care) Complaint Problems with pain management Complaint Patient/family not included in planning care Complaint Patient/family disagrees about decisions on care Complaint Lack of confidence or trust in caregiver Complaint Request for non-formulary medication Complaint Complaints concerning medical records Complaint Concerns with confidentiality/privacy in medical record Complaint Medical center regulations Complaint VA regional office and/or compensation and pension issues Complaint Fiduciary issues Page 31 GAO-18-356 VHA Patient Advocacy Program Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management Category Issue description Complaint Missing and/or damaged personal property Complaint Allegations of negligence/malpractice Complaint Allegations of abuse Complaint Medication error Complaint Issue related to safety Complaint Patient not treated with dignity and respect/perceived rudeness Complaint Perceived retaliation for expressing concerns Complaint Lack of coordination between inpatient and outpatient care Complaint Deceased patient issues by a family member Complaint Excessive delay of discharge/discharge too soon Source: GAO analysis of VHA documents. | GAO-18-356 Note: This table includes issue codes used by VHA patient advocacy program staff to document feedback recorded in PATS in fiscal year (FY) 2017. In October 2017, the Office of Patient Advocacy updated the list of issue codes in PATS for use during FY 2018. The updated list deletes some existing codes, adds new codes, and updates definitions for some existing codes. The Comprehensive Addiction and Recovery Act of 2016 (CARA), includes a provision for every VAMC to display the purpose of the program, along with the contact information of a patient advocate at the facility, in as many prominent locations as deemed appropriate to be seen by the largest percentage of veterans. 1 In September 2016, VHA Central Office sent a memo to Veterans Integrated Service Network (VISN) directors explaining this requirement and an Office of Patient Centered Care and Cultural Transformation (OPCC&CT) official obtained confirmation from all VHA facilities that this requirement was met in October 2016. Nevertheless, officials from two veterans service organizations (VSO) we interviewed stated they often encounter veterans who are not aware of the patient advocacy program. According to VA, in fiscal year (FY) 2017, there were 268,114 veterans associated with ROCs entered in PATS. 2 VA also reported that, in the same year, patient advocacy program staff entered 414,256 unique reports of contact in PATS. According to VA, from the unique reports of contact in PATS, program staff documented 473,564 issues, which 1 Comprehensive Addiction and Recovery Act of 2016, Pub. L. No. 114-198, § 922, 130 Stat. 695, 766-767 (2016). 2 VHA officials stated that veterans’ feedback is not always entered in PATS, so the number of unique veterans associated with reports entered in PATS may not represent the total percentage of veterans who provide feedback to patient advocacy program staff. Page 32 GAO-18-356 VHA Patient Advocacy Program Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management included (but were not limited to) 112,722 requests for information, 35,839 compliments, and 325,003 complaints. 3 See table 3 for the top five issues that patient advocacy program staff across VAMCs entered in PATS for FY 2017. According to VA, in FY 2017, a total of 1,391 program staff system-wide entered data in PATS. In the same year, according to PATS, veterans, rather than family members or friends, most often provided feedback to patient advocacy program staff. Table 3: The Most Common Issues Entered in the Patient Advocate Tracking System (PATS), according to the Department of Veterans Affairs (VA), Fiscal Year 2017 Category Issue description Number of issues entered in PATSa Request for Information Referral issues (internal/community) 54,903 Compliment Compliment received from patient, family member, and/or staff 35,839 member Complaint Patient/family disagrees about decisions on care 31,718 Complaint Phone calls/letters/secure messages not returned/answered in 24,949 a timely manner Complaint Patient perceives care is not coordinated/expectations not met 23,617 Source: VA. | GAO-18-356 Notes: Data presented in the table were provided by VA officials in January 2018 based on the number of issues entered in PATS in fiscal year 2017. According to Veterans Health Administration (VHA) officials involved in overseeing the patient advocacy program, patient advocates may interpret an issue description (designated by a specific issue code) differently. These differing interpretations may have contributed to the data shown in this table. In addition, VHA officials stated that veterans’ feedback is not always entered in PATS, so the number of issues shown in the table may not represent all feedback that patient advocates received from veterans. a Patient advocates or other VA officials can assign more than one issue code to each report of contact (ROC) entered in PATS. According to VAMC officials, multiple issue codes may be assigned to one ROC when a veteran describes multiple issues at the same time, or when there are multiple issue codes that describe a veterans’ complaint. Therefore, the number of issues may not be equivalent to the number of unique ROCs for a given year. Our eight selected VAMCs varied in the number of patient level advocates and service-level advocates who had access to PATS, whether veterans’ feedback was recorded outside of PATS, and which issue code or codes were used to record feedback related to the Veterans Choice Program. 4 (See table 4.) 3 According to VA officials, these numbers are likely an underestimate of feedback provided by veterans, as not all feedback is entered in PATS. 4 The Veterans Choice Program is a program that allows eligible veterans to receive health care from a community provider when faced with long wait times, lengthy travel distances, or other challenges accessing care at a VAMC. Page 33 GAO-18-356 VHA Patient Advocacy Program Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management Table 4: Patient Advocate Tracking System (PATS) Data-Entry Practices at Eight Selected Department of Veterans Affairs Medical Centers (VAMC), as of September 2017 Number of Number of patient service-level Recorded advocates who advocates who feedback entered data in entered data in outside of Description of issue code(s) used to record feedback VAMC PATS PATS PATS? related to the Veterans Choice Programc A 2 0 Yes Ambulance, private hospital, private care, payment eligibilityd B 6 3 Yes —e C 2 n/ab Yes Ambulance, private hospital, private care, payment eligibility a D 1 2 No VA billing for service;d VA billing for service, pharmacy co-payment E 4 0 Yes —f F 3 13 Yes Dissatisfied with referral outcome b G 5 n/a Yes Referral issues (internal, community)g H 1 10 No —h Legend: n/a = not applicable, em dash (—) = a single issue code was not identified Source: GAO summary of information from selected VAMCs. | GAO-18-356 Notes: The PATS is VA’s electronic system for documenting and tracking veterans’ complaints and other feedback. The term “VAMC” refers to an individual VAMC and any of its associated facilities, such as community-based outpatient clinics. Service-level advocates, such as nurses or administrative staff, are designated at some VAMCs to respond to veterans’ feedback before involving a patient advocate. a VAMC officials stated that in addition to one patient advocate, there are six quality management staff that serve as back up to the patient advocate and enter information into PATS. b This VAMC did not utilize designated service-level advocates as part of the patient advocacy program. c The Veterans Choice Program is a program that allows eligible veterans to receive health care from a community provider when faced with long wait times, lengthy travel distances, or other challenges accessing care at a VAMC. d In addition to using the issue codes listed, VAMC officials stated they created a special entry in the PATS facility drop-down menu to code Veterans Choice Program feedback. When officials enter a report of contact (ROC) in PATS, each VAMC has a drop-down menu of locations within their VAMC that they populate for each ROC. According to VA officials, this is one of the only categories within PATS that VAMC employees are able to change. e VAMC officials stated that there was no single code or codes used to categorize Veterans Choice Program feedback, but added the Veterans Choice Program to the medical service drop-down menu within PATS. f VAMC officials stated that there was no single code or codes used to categorize Veterans Choice Program feedback. g This issue code is used for requests for information with no associated complaints. h VAMC officials stated they created a special entry in the PATS facility drop-down menu to code Veterans Choice Program feedback. Page 34 GAO-18-356 VHA Patient Advocacy Program Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management Examples of methods that patient advocates and service-level advocates used at selected VAMCs to record veterans’ feedback outside of PATS included call logs and tracking spreadsheets. VAMC officials indicated that recording information outside of PATS helped them track their responses to veterans’ feedback. Some of the information recorded outside of PATS was additional information that is not required to be entered into PATS, such as requests for information. 5 5 VHA’s handbook sets a goal requiring that complaints are entered into PATS, but does not require that requests for information and compliments are entered. The handbook states that requests for information and compliments can be entered into PATS, as this information is useful in determining common areas of confusion that could be addressed in a proactive manner. See VA, VHA Patient Advocacy Program, VHA Handbook 1003.4 (Sept. 2, 2005). Page 35 GAO-18-356 VHA Patient Advocacy Program Appendix III: Approaches to the Governance Appendix III: Approaches to the Governance and Staffing of the Patient Advocacy Program and Staffing of the Patient Advocacy Program The eight Department of Veterans Affairs (VA) medical centers (VAMC) selected for our review used a variety of approaches to govern the patient advocacy program, resulting in differences in the number of positions for patient advocates and service-level advocates and the title of the positions. Service-level advocates, such as nurses or administrative staff, are designated at some VAMCs to respond to veterans’ feedback before involving a patient advocate. (See table 5.) Table 5: The Number of Positions and Titles Used for Patient Advocates and Service-Level Advocates at Eight Selected Department of Veterans Affairs Medical Centers (VAMC), as of September 2017 Patient advocates Service-level advocates VAMC Number of positions Title Number of positions Title A 2 Patient advocate 28 Service-level liaison B 6 Patient experience liaison 29 Service-level advocate C 2 Patient representative 0 Service line champion D 1 Patient representative 16 Patient advocate liaison E 4a Patient advocate 57 Patient advocate liaison F 3 Patient representative 86 Service-level advocate b G 9 Patient advocate 0 Service-level advocate H 1 Patient representative 10 Service line patient experience liaison Source: GAO summary of information from selected VAMCs. | GAO-18-356 Notes: The term “VAMC” refers to an individual VAMC and any of its associated facilities such as community-based outpatient clinics. Service-level advocates, such as nurses or administrative staff, are designated at some VAMCs to respond to veterans’ feedback before involving a patient advocate. a This VAMC also has four patient advocates who only handle complaints related to non-VA care. Since these patient advocates report outside of the VAMC’s patient advocacy program, we did not include them in the count of patient advocates. According to an official from this VAMC, these advocates do not enter veterans’ feedback in PATS. b This VAMC also has two assistant patient advocates who help the patient advocates respond to veterans’ feedback. These assistants are not included in the number of patient advocates. Patient advocates reported to a variety of departments among our selected VAMCs. At two of the VAMCs, patient advocates reported to the customer or consumer relations department, while at three, patient advocates reported to the quality management department. In addition, the placement of the department that patient advocates reported to within the VAMC differed. For example, the patient advocate supervisor at one of the selected VAMCs said that patient advocates reported to the customer service manager, who did not report directly to the VAMC director. At another VAMC, the patient advocate reported directly to the VAMC director. Page 36 GAO-18-356 VHA Patient Advocacy Program Appendix III: Approaches to the Governance and Staffing of the Patient Advocacy Program In addition to the Veterans Health Administration (VHA) handbook for the patient advocacy program, all eight of our selected VAMCs developed their own policies for the administration of the program, and these policies varied. For example, while almost all of the policies specified the responsibilities with respect to the patient advocacy program of the service chiefs—officials who oversee the administration and operation of service lines such as primary care, these responsibilities varied. For example, two of the policies required service chiefs to incorporate veterans’ feedback into performance measures used for VAMC staff external to the patient advocacy program, such as physicians, while the other policies did not. We also found variation between our selected VAMCs with respect to whether they had written descriptions of the service-level advocates’ roles. Of the six VAMCs that designated service-level advocates, three had written descriptions of their roles, while three did not. Further, among the VAMCs that had a written description of the role of a service-level advocate, the expectations for these advocates varied. For example, one VAMC’s written description specified that service-level advocates are expected to enter veterans’ feedback into PATS within 7 days of receiving the feedback. The written descriptions at the other two VAMCs did not specify this expectation. Page 37 GAO-18-356 VHA Patient Advocacy Program Appendix IV: Comments from the Appendix IV: Comments from the Department of Veterans Affairs Department of Veterans Affairs Page 38 GAO-18-356 VHA Patient Advocacy Program Appendix IV: Comments from the Department of Veterans Affairs Page 39 GAO-18-356 VHA Patient Advocacy Program Appendix IV: Comments from the Department of Veterans Affairs Page 40 GAO-18-356 VHA Patient Advocacy Program Appendix V: GAO Contact and Staff Appendix V: GAO Contact and Staff Acknowledgments Acknowledgments Debra A. Draper, (202) 512-7114 or draperd@gao.gov GAO Contact In addition to the contact named above, Hernán Bozzolo (Assistant Staff Director), Rebecca Rust Williamson (Analyst-in-Charge), Jennie F. Apter, Acknowledgments Q. Akbar Husain, and Emily Loriso made key contributions to this report. Also contributing were Julie Flowers, Jacquelyn Hamilton, and Vikki Porter. (101674) Page 41 GAO-18-356 VHA Patient Advocacy Program The Government Accountability Office, the audit, evaluation, and investigative GAO’s Mission arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is Obtaining Copies of through GAO’s website (https://www.gao.gov). Each weekday afternoon, GAO GAO Reports and posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to https://www.gao.gov Testimony and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, https://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Connect with GAO Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. Visit GAO on the web at https://www.gao.gov. Contact: To Report Fraud, Website: https://www.gao.gov/fraudnet/fraudnet.htm Waste, and Abuse in E-mail Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, Congressional U.S. Government Accountability Office, 441 G Street NW, Room 7125, Relations Washington, DC 20548 Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707 Strategic Planning and U.S. Government Accountability Office, 441 G Street NW, Room 7814, External Liaison Washington, DC 20548 Please Print on Recycled Paper.