United States Government Accountability Office Report to Congressional Addressees COVID-19 January 2022 HHS and DOD Transitioned Vaccine Responsibilities to HHS, but Need to Address Outstanding Issues GAO-22-104453 January 2022 COVID-19 HHS and DOD Transitioned Vaccine Responsibilities to HHS, but Need to Address Outstanding Issues Highlights of GAO-22-104453, a report to congressional addressees Why GAO Did This Study What GAO Found Vaccines have played a crucial role in Starting in May 2020, federal efforts to accelerate the development, battling the COVID-19 pandemic. The manufacturing, and distribution of COVID-19 vaccines had been led by a CAG worked with vaccine companies partnership between the Department of Health and Human Services (HHS) and to develop COVID-19 vaccines, and the Department of Defense (DOD). Formerly known as Operation Warp Speed, made available a sufficient supply for the partnership was renamed the HHS-DOD COVID-19 Countermeasures all eligible people in the nation. An Acceleration Group (CAG). According to HHS and DOD officials, the CAG April 2021 memorandum of dissolved and transitioned its responsibilities-including DOD-led vaccine understanding between HHS and DOD activities-to HHS by December 31, 2021, as required by an April 2021 called for the transfer of remaining memorandum of understanding between the two departments. CAG responsibilities to HHS and for identification of lessons learned. Manufacturing of COVID-19 Vaccines The CARES Act includes a provision for GAO to report on its ongoing monitoring and oversight efforts related to the COVID-19 pandemic. This report examines, among other things, the CAG's progress on (1) transitioning its responsibilities to HHS, and (2) developing a process for a joint interagency lessons learned review. GAO reviewed CAG transition and contracting documents and interviewed or received written responses from CAG officials, federal agencies, and representatives from the six vaccine companies that worked with the CAG. While HHS and DOD officials said they achieved transition milestones indicating that HHS is ready to assume responsibilities formerly led by DOD, it is unclear What GAO Recommends how HHS will address its workforce needs now that the CAG has dissolved. Specifically, GAO found that HHS has assessed its workforce capabilities, but GAO is making five recommendations lacks strategies for addressing these workforce needs. By formally providing its related to workforce needs, scheduling support until HHS develops and implements these strategies, DOD can help best practices for vaccine-related ensure that HHS can continue these responsibilities uninterrupted, including activities; and lessons learned from key stakeholders. responsibilities for addressing ongoing vaccine needs for boosters or for any emerging COVID-19 variants. Moreover, HHS does not have a schedule that is HHS did not concur with GAO's consistent with best practices to help it manage remaining vaccine-related recommendation on workforce needs. activities. Such a schedule could help HHS better plan actions and mitigate GAO revised this recommendation delays, and be a source for identifying lessons learned for any future pandemics. based on updated information, but maintains that it continues to be valid, The CAG developed a plan for conducting a joint, interagency lessons-learned as discussed in the report. review. This plan outlines an approach for collecting information-such as perspectives on challenges-from CAG staff, and for sharing the plan with HHS. However, the plan misses an opportunity to gather perspectives from key external stakeholders, including vaccine companies, critical to developing vaccines. Obtaining these perspectives could provide a more comprehensive understanding of areas where the CAG was successful and opportunities for View GAO-22-104453. For more information, improvement, which could help inform HHS's ongoing and future vaccine work. contact Alyssa M. Hundrup at (202) 512-7114 or hundrupa@gao.gov United States Government Accountability Office Contents Letter 1 Background 6 CAG Transitioned to HHS, but HHS Readiness to Assume All Responsibilities Unclear 9 The CAG Began a Lessons-Learned Review Process, but Its Plans Do Not Include Collecting Information from Stakeholders 19 Preliminary Lessons Learned from the CAG, Federal Agencies, and Vaccine Company Representatives 22 Conclusions 27 Recommendations for Executive Action 28 Agency Comments and Our Evaluation 29 Appendix I Status of U.S. COVID-19 Vaccine Candidates and Related Obligations and Doses 34 Appendix II Contract Techniques Used to Obtain COVID-19 Vaccines for the U.S. 38 Appendix III Abridged Assessment of the Countermeasures Acceleration Group's Schedule 42 Appendix IV Comments from the Department of Health & Human Services 45 Appendix V Comments from the Department of Defense 48 Appendix VI GAO Contact and Staff Acknowledgments 51 Related GAO Products 52 Page i GAO-22-104453 HHS-DOD CAG Partnership Tables Table 1: Nine Lines of Effort for Transitioning DOD Responsibilities in the CAG to HHS 11 Table 2: Status of the CAG's Six Vaccine Candidates, as of January 11, 2022 34 Table 3: Vaccine Doses Contracted for Purchase by the Federal Government for Domestic Use, as of October 22, 2021 37 Table 4: Assessment of the CAG's Integrated Master Schedule, June 2021 43 Figures Figure 1: Leading Practices of a Lessons-Learned Review Process 20 Figure 2: HHS and DOD Obligations for COVID-19 Vaccine Candidates and Others under the CAG, as of September 30, 2021 36 Page ii GAO-22-104453 HHS-DOD CAG Partnership Abbreviations ASPR Office of the Assistant Secretary for Preparedness and Response BARDA Biomedical Advanced Research and Development Authority CAG HHS-DOD COVID-19 Countermeasures Acceleration Group CAG Continuity Book DOD's Countermeasures Acceleration Group Continuity Book CDC Centers for Disease Control and Prevention COVID-19 Coronavirus Disease 2019 DOD Department of Defense DPA Defense Production Act EUA emergency use authorization FAR Federal Acquisition Regulation FDA Food and Drug Administration HHS Department of Health and Human Services JPEO-CBRND Joint Program Executive Office for Chemical, Biological, Radiological, and Nuclear Defense NIH National Institutes of Health OTA other transaction agreement This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page iii GAO-22-104453 HHS-DOD CAG Partnership Letter 441 G St. N.W. Washington, DC 20548 January 19, 2022 Congressional Addressees Since the President declared the Coronavirus Disease 2019 (COVID-19) pandemic a national emergency on March 13, 2020, the country has reported more than 56 million confirmed cases and more than 830,000 deaths as of January 3, 2022, including a sharp increase in cases at the end of December 2021 due largely to the Omicron variant. 1 In the second year of the pandemic's catastrophic effects, the federal government's efforts to help develop and make available an adequate supply of safe and effective vaccines have been crucial to the nation's ongoing recovery. It is also critical that the federal government learn from its experiences with accelerating vaccine production so that it can be prepared to meet future vaccine needs for this pandemic-such as to counteract the emergence of new variants-or for future pandemics. 2 To help make safe and effective vaccines available as quickly as possible, in April 2020, the federal government announced the creation of Operation Warp Speed, a partnership between the Department of Health and Human Services (HHS) and the Department of Defense (DOD). In April 2021, Operation Warp Speed was renamed the HHS-DOD COVID- 19 Countermeasures Acceleration Group (CAG), and for the purposes of this report, we refer to both iterations of this partnership as the CAG. The CAG was set up to support the acceleration of vaccine development, manufacturing, and distribution to states, other jurisdictions, and federal agencies and programs for vaccine administration. 3 As a part of those 1Inaddition to declaring a national emergency under the National Emergencies Act, the President also declared a nationwide emergency under section 501(b) of the Robert T. Stafford Disaster Relief and Emergency Assistance Act and approved major disaster declarations for all 50 states, the District of Columbia, five territories, and three federally recognized Indian tribes. On January 31, 2020, the Secretary of Health and Human Services declared a public health emergency for the United States, retroactive to January 27, 2020. 2As of January 2022, the Centers for Disease Control and Prevention (CDC) had listed the Delta and Omicron variants as variants of concern in the U.S. CDC had previously characterized three other variants (Alpha, Beta, and Gamma) as variants of concern but later downgraded them. 3Jurisdictions include all 50 states, the District of Columbia, eight U.S. territories, and a small number of major cities. Page 1 GAO-22-104453 HHS-DOD CAG Partnership efforts, HHS and DOD awarded contracts and other transaction agreements (OTA) to six vaccine companies and others for the development and manufacturing of vaccines, the purchase of vaccine doses, and the acquisition of other critical supplies and services. 4 The CAG also coordinated with other external stakeholders, including federal agencies, such as the Centers for Disease Control and Prevention (CDC), on vaccine distribution planning, according to officials. 5 In December 2020, the first two COVID-19 vaccines-sponsored by Moderna and Pfizer-were authorized by the Food and Drug Administration (FDA) for emergency use. 6 The CAG began distributing doses of these vaccines to states and others immediately upon their authorization. As of January 2022, three COVID-19 vaccines (with the third sponsored by Janssen) were available in the United States. 7 • Pfizer's vaccine was licensed for individuals ages 16 and older and was also available under an emergency use authorization (EUA) for individuals ages 12 to 15 years, as a lower dose for individuals ages 5 to 11 years, as a third dose for certain immunocompromised individuals ages 5 years and older, and as a booster for individuals ages 12 years and older. 4OTAs are flexible agreements that allow the parties to negotiate terms and conditions without requiring parties to comply with certain federal procurement laws and regulations. See 10 U.S.C. § 2371b. For more information on the accelerated COVID-19 vaccine development process, see GAO, Operation Warp Speed: Accelerated COVID-19 Vaccine Development Status and Efforts to Address Manufacturing Challenges. GAO-21-319, (Washington, D.C.: Feb. 11, 2021). 5Formore information about the CAG, see our April 2021 report on the federal government's vaccine efforts. GAO, COVID-19: Efforts to Increase Vaccine Availability and Perspectives on Initial Implementation. GAO-21-443 (Washington, D.C.: Apr. 14, 2021). 6The Secretary of Health and Human Services may declare that circumstances, prescribed by statute, exist justifying the emergency use of certain medical products, such as vaccines. Once a declaration has been made, FDA may temporarily allow use of unlicensed vaccines through an emergency use authorization (EUA). For FDA to issue an EUA for a vaccine, it must be reasonable to believe that the vaccine may be effective and that the known and potential benefits of the vaccine outweigh the known and potential risks, among other statutory criteria. See 21 U.S.C. § 360bbb-3. Pfizer developed its COVID-19 vaccine in collaboration with BioNTech. 7Janssen Pharmaceutical Companies are a part of Johnson & Johnson. Page 2 GAO-22-104453 HHS-DOD CAG Partnership • Moderna's vaccine was authorized for individuals ages 18 and older, as a third dose for certain immunocompromised individuals ages 18 an older, and as a booster for individuals ages 18 and older. • Janssen's vaccine was authorized for individuals ages 18 and older and as a booster for the same population. 8 By September 2021, the federal government had acquired over 673 million doses of these three vaccines, sufficient to fully vaccinate 373 million people, in line with the CAG's overall goal of having sufficient adult vaccines for the American public. 9 An April 2021 memorandum of understanding between HHS and DOD called for the two departments to coordinate plans to transition the CAG's responsibilities to HHS, including activities that had been led by DOD. The memorandum also called for HHS and DOD to develop a joint interagency process for incorporating lessons learned from the CAG's work into HHS's continued operations, and to dissolve the CAG by December 31, 2021. The CARES Act includes a provision for us to report on the federal response to the COVID-19 pandemic. Specifically, the act requires us to monitor and oversee the federal government's efforts to prepare for, respond to, and recover from the pandemic. 10 This report is part of our body of work in response to the CARES Act and focuses on the activities of the CAG. 11 8Allthree vaccines (Janssen, Moderna, and Pfizer) were also authorized for use as a heterologous (or "mix and match") booster for individuals ages 18 years and older, as of January 2022. 9According to CDC, as of October 2021, people were considered "fully vaccinated" 2 weeks after their second dose in a two-dose series, such as the Pfizer or Moderna vaccine, or 2 weeks after a single-dose vaccine, such as the Janssen vaccine. As of January 11, 2022, the term "fully vaccinated" did not include boosters. 10Pub. L. No. 116-136, § 19010, 134 Stat. 281, 579-81 (2020). This report also responds, in part, to a bipartisan request from the House Select Subcommittee on the Coronavirus Crisis for GAO to examine Operation Warp Speed (renamed the CAG in April 2021). 11We have regularly issued government-wide reports on the federal response to COVID- 19. For the latest report, see GAO, COVID-19: Additional Actions Needed to Improve Accountability and Program Effectiveness of Federal Response, GAO-22-105051 (Washington, D.C.: Oct. 27, 2021). Our next government-wide report will be issued in January 2022 and will be available on GAO's website at https://www.gao.gov/coronavirus. Also, see the GAO Related Products section at the end of this report for additional work we have done on COVID-19 vaccines. Page 3 GAO-22-104453 HHS-DOD CAG Partnership In this report, we 1. examine the CAG's progress on transitioning responsibilities to HHS for developing, manufacturing, and distributing COVID-19 vaccines; 2. examine the CAG's progress on developing a joint interagency review process to identify lessons learned from its COVID-19 vaccine development, manufacturing, and distribution efforts; and 3. describe preliminary lessons learned we obtained from the CAG, selected federal departments, and vaccine company representatives who coordinated with the CAG on COVID-19 vaccine development, manufacturing, and distribution. To address the first objective, we reviewed documents pertaining to the CAG's transition efforts, including the April 2021 memorandum between HHS and DOD, and the CAG's July 2021 transition plan. Specifically, we reviewed the CAG's transition readiness, including its efforts to establish milestones to gauge its progress towards transition readiness, as required by the April 2021 memorandum, and compared these efforts to selected leading practices for agency reforms, including transitions, which we identified in prior work. 12 We also reviewed the CAG's efforts to assess HHS's workforce capacity to assume the CAG's responsibilities and to develop workforce strategies to address any workforce capacity gaps. We also then compared these efforts to selected leading practices for both agency reforms and strategic workforce planning. 13 We also obtained information on the CAG's plans to assess the need for continued use of certain tools for tracking and managing remaining vaccine development, 12Specifically, we assessed HHS's transition readiness efforts against the following selected agency reform leading practices: (1) developing an implementation plan with key milestones and deliverables to track implementation progress; (2) establishing a dedicated implementation team to manage the reform process; and (3) designating leaders to be responsible for the implementation of the proposed reforms. See GAO, Government Reorganization: Key Questions to Assess Agency Reform Efforts, GAO-18-427 (Washington, D.C., June 13, 2018). 13Specifically, we assessed HHS's efforts to assess its workforce capacity against the leading practice for agency reform that agencies conduct strategic workforce planning to determine whether they will have the needed resources and capacity, including the skills and competencies, in place for the proposed reforms or reorganization. See GAO-18-427. In addition, we also assessed these efforts against selected leading practices for strategic workforce planning, including that agencies should determine the critical skills and competencies needed to achieve programmatic results and develop strategies to address gaps in critical skills and competencies. See GAO, Human Capital: Key Principles for Effective Strategic Workforce Planning, GAO-04-39 (Washington, D.C., Dec. 11, 2003). Page 4 GAO-22-104453 HHS-DOD CAG Partnership manufacturing, and distribution activities. We compared HHS's plans to manage the scheduling of vaccine development, manufacturing, and distribution activities to best practices for scheduling, as outlined in the GAO Schedule Assessment Guide. 14 In addition, we conducted an abridged assessment of the CAG's schedule as of May 11, 2021. 15 To address the second objective, we reviewed documentation on the progress the CAG had made as of October 2021 to develop a joint interagency review process to identify lessons learned from its COVID-19 vaccine efforts. We interviewed CAG officials about its joint, interagency plan to develop a lessons-learned process. We compared the CAG's lessons-learned plan to leading practices we and others have previously identified for conducting a lessons-learned process, such as collecting and disseminating lessons learned. 16 We also compared the departments' plans with the Project Management Institute's program management standards, which call for program managers to engage with key stakeholders. 17 To address the third objective, we reviewed DOD's Countermeasures Acceleration Group Continuity Book (CAG Continuity Book), which 14SeeGAO, Schedule Assessment Guide: Best Practices for Project Schedules, GAO-16 89G (Washington, D.C.: Dec. 22, 2015). 15Our abridged assessment of the CAG's schedule focused on the extent to which it was "well-constructed," as defined in the GAO Schedule Assessment Guide. The "well- constructed" characteristic is one of four general characteristics associated with high- quality schedules, according to the guide, and we focused on it for this abridged assessment because it reflects basic quality measures of a schedule. 16See GAO, Federal Real Property Security: Interagency Security Committee Should Implement a Lessons-Learned Process, GAO-12-901 (Washington, D.C.: Sept. 10, 2012); and Project Management: DOE and NNSA Should Improve Their Lessons Learned Process for Capital Asset Projects, GAO-19-25 (Washington, D.C.: Dec. 21, 2018). Department of the Army, Combined Arms Center, Center for Army Lessons Learned, Establishing a Lessons Learned Program: Observations, Insights, and Lessons (Fort Leavenworth, KS: June 2011). 17Project Management Institute, Inc., A Guide to the Project Management Body of Knowledge (PMBOK® Guide), Sixth Edition, 2017; Project Management Institute, Inc., Implementing Organizational Project Management: A Practice Guide, First Edition, 2014. PMBOK is a trademark of Project Management Institute, Inc. The PMBOK® Guide. provides guidelines for managing individual projects, including collecting requirements and defining the project's scope. The Project Management Institute is a not-for-profit association that provides global standards for, among other things, project and program management. These standards are utilized worldwide and provide guidance on how to manage various aspects of projects, programs, and portfolios. Page 5 GAO-22-104453 HHS-DOD CAG Partnership highlighted information about lessons learned that DOD officials within the CAG documented over the course of the CAG's COVID-19 response efforts as of May 2021. 18 In June and July 2021, we also obtained written responses or interviewed representatives from the six vaccine companies that were awarded contracts or OTAs under the CAG-AstraZeneca, Janssen, Moderna, Novavax, Pfizer, and Sanofi-about their experiences working with the CAG on vaccine development, manufacturing, and distribution. We also interviewed officials from the CAG and component offices and agencies of HHS and DOD about their perspectives on activities that worked well and any challenges they identified with the CAG's COVID-19 vaccine response efforts in June and July 2021. Within HHS, we interviewed or obtained written responses from officials from the Office of the Assistant Secretary for Preparedness and Response (ASPR), Biomedical Advanced Research and Development Authority (BARDA), CDC, FDA, and the National Institutes of Health (NIH). Within DOD, our interviews included officials from the Joint Program Executive Office for Chemical, Biological, Radiological, and Nuclear Defense (JPEO-CBRND) and the Army Contracting Command. We selected these offices and agencies because the CAG indicated that they were integral to the CAG's vaccine efforts. We conducted this performance audit from July 2020 to January 2022 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. A June 2020 memorandum of understanding between HHS and DOD Background established the CAG, then called Operation Warp Speed, with the primary purpose of accelerating the development, production, and distribution of COVID-19 vaccines. (See appendix I for information on the status of vaccine candidates and obligations made under the CAG.) The CAG was co-chaired by the Secretaries of Health and Human Services and Defense, who appointed HHS and DOD officials to lead five key initiatives: (1) vaccine development; (2) therapeutics; (3) supply, 18CAG officials from DOD said they provided the CAG Continuity Book to incoming CAG personnel when they joined the partnership. DOD officials developed it to capture key personnel's mission, work streams, and lessons learned. Page 6 GAO-22-104453 HHS-DOD CAG Partnership production, and distribution; (4) security and assurance; and (5) research, development, acquisition, and contracting. 19 In addition, the CAG had an HHS Chief Advisor, an HHS Chief Operating Officer, and a DOD Chief of Staff. 20 As of September 2021, DOD had assigned 76 officials from the Army, the Navy, the Air Force, and the Marine Corps to work on the CAG's five key initiatives. HHS officials told us that the department generally did not assign a specific number of staff to work directly on the CAG, but stated that hundreds of officials from various HHS agencies, such as ASPR, CDC, and NIH, have worked on CAG-related efforts. DOD Designated Lead for The April 2021 memorandum of understanding between HHS and DOD Most Vaccine-Related designated DOD as the lead for implementing most vaccine-related initiatives within the CAG, with HHS to provide support. In particular, the CAG Activities memorandum tasked DOD with designating DOD officials to serve in the following roles, leading three of the five key initiatives: • Director of vaccine development-directed and oversaw development and testing of vaccines in coordination with the six vaccine companies that were part of the CAG; • Director of supply, production, and distribution-implemented and oversaw acquisition of supplies and vaccines, including having DOD officials embedded in vaccine production factories to assist with supply chain management and development of a federal governmental plan to distribute these items to the jurisdictions and other federal agencies and programs; and • Director of security and assurance-developed security measures for the CAG, and a security plan to support production and distribution of vaccines and supplies from the development phase to distribution. The April 2021 memorandum also identified lead roles for DOD and HHS on the two other key initiatives. The memorandum indicated that officials from both departments shared responsibility for overseeing and 19This report focuses primarily on the CAG's vaccine-related responsibilities. 20The Chief Advisor was responsible for providing technical advice regarding vaccine development and manufacturing and coordinated CAG activities with other federal departments. The Chief Operating Officer was responsible for coordinating the logistics, supply chain, development, production, and delivery of vaccines, as well as supporting HHS efforts under the CAG. The Chief of Staff supported the Chief Operating Officer in leading the CAG. Page 7 GAO-22-104453 HHS-DOD CAG Partnership supporting the research, development, acquisition, and contracting initiative, and it tasked HHS with designating a director to lead the therapeutics initiative. Plans to Transition CAG The April 2021 memorandum between HHS and DOD tasked HHS and Responsibilities to HHS DOD to jointly develop a plan to transition all CAG responsibilities to HHS and fully dissolve the CAG by December 31, 2021, with transition activities to begin in the fall of 2021. 21 The April 2021 memorandum stated that the transition plan was to include • a cross-walk plan that transfers DOD activities to HHS, • key milestones that must be achieved to ensure successful transition, • synchronization of transition planning to ensure uninterrupted support, communications, and decision-making related to ongoing vaccine- related activities, and • full incorporation of current efforts and lessons learned into the U.S. National Vaccine Program. 22 Leading Practices for Reforming and reorganizing the federal government is a major endeavor Agency Reforms that can include refocusing, realigning, or enhancing agency missions, including transitions. We have previously identified leading practices for federal agencies to follow when planning and implementing agency reforms. 23 Examples of these leading reform practices include • identifying leaders to be responsible for the implementation of the proposed reform; • establishing a dedicated implementation team that has the capacity, including staffing, resources, and change management, to manage the reform process; 21The original June 2020 memorandum between HHS and DOD specified that their partnership would be in place until January 31, 2021. However, on January 14, 2021, HHS and DOD extended their partnership until May 2021. Then, through their April 2021 memorandum, HHS and DOD further extended their partnership until the end of December 2021, and renamed it the CAG. 22The U.S. National Vaccine Program is located in the Office of the Assistant Secretary for Health within HHS. The program was established in 1986 and is responsible for the research, development, and testing of all types of vaccines, such as for the influenza and other infectious diseases, as well as the production, procurement, and distribution of vaccines to the public, among others. 23See GAO-18-427. Page 8 GAO-22-104453 HHS-DOD CAG Partnership • developing an implementation plan with key milestones and deliverables to track implementation progress; and • conducting strategic workforce planning to determine whether the new agency will have the needed resources and capacity, including the skills and competencies, in place for the proposed reform. CAG Transitioned to HHS, but HHS Readiness to Assume All Responsibilities Unclear The CAG Completed its In July 2021, the CAG established a team led by co-chairs from DOD and Transition to HHS in HHS for transitioning all of the CAG's responsibilities-particularly those led by DOD-to HHS. The CAG also developed a transition plan, as December 2021 called for in the April 2021 memorandum of understanding between HHS and DOD. These steps are consistent with selected leading reform practices identified in our prior work, which highlight the importance of establishing a dedicated implementation team and plan when an agency undergoes an agency reform, such as a transition. As part of the transition plan, CAG officials developed a crosswalk-as required by the April 2021 memorandum-to guide the incremental transfer of DOD-led responsibilities to HHS. Specifically, the crosswalk organized the transition of responsibilities from DOD to HHS into nine lines of effort. 24 These lines of effort identified DOD-led responsibilities across the CAG's five key initiatives that HHS was to assume as part of the transition (see table 1). According to CAG officials, ASPR was the HHS office in charge of leading the transition of CAG responsibilities, including those led by DOD, to HHS. 25 In addition, CAG officials told us that a new office within ASPR, called the HHS Coordination Operations and Response Element, would be responsible for continuing all of the 24CAG officials told us in October 2021 that DOD officials led all nine lines of effort. However, HHS officials told us in January 2022 that the Legal line of effort and the Research and Clinical Trials for Vaccines and Therapeutics line of effort were already supported by HHS officials and therefore were not transitioned from DOD to HHS. 25ASPR serves as the principal advisor to the Secretary of Health and Human Services on all matters related to federal public health and medical preparedness and responses for public health emergencies, among other things. 42 U.S.C. § 300hh-10(b)(1). Page 9 GAO-22-104453 HHS-DOD CAG Partnership CAG's responsibilities. CAG officials told us in July 2021 that this new office would remain in place at least during the COVID-19 pandemic, but did not indicate whether it would be permanent. Page 10 GAO-22-104453 HHS-DOD CAG Partnership Table 1: Nine Lines of Effort for Transitioning DOD Responsibilities in the CAG to HHS Line of Effort Description Vaccine and Therapeutics Coordinates the management of private-sector vaccine and therapeutic manufacturing Development and Manufacturing capacity, assesses potential areas of vaccine and therapeutics supply risk and recommends mitigation options, and supports the acquisition of equipment and materials for delivery of vaccine doses. Distribution and Administration of Supports HHS's efforts to distribute COVID-19 therapeutics. Therapeutics Analytics and Information Technology Coordinates the requirements, design, and implementation of information networks. Additionally, it is responsible for leading collaborative analysis of production forecast models used by CAG leadership for decision-making. Research and Clinical Trials for Provides advice and alignment of all aspects of research and clinical trials for COVID-19 Vaccines and Therapeutics vaccine and therapeutics, including supporting project coordination team efforts-such as reviewing clinical trial protocol documents. Comptroller Oversees budget efforts and provides executive budget analysis to senior CAG officials. It also acts as the financial liaison between HHS and DOD. External Affairs Provides public and legislative affairs support to CAG leadership and executive staff. Specifically, it works to maintain an open and transparent dialogue with federal, state, and local government, as well as interagency partners such as the Centers for Disease Control and Prevention (CDC) and Biomedical Advanced Research and Development Authority (BARDA). Security and Assurance Coordinates the programs designed to safeguard the development, manufacture, and distribution of vaccines and therapeutics, and to prevent disruption across the vaccine supply chain. Additionally, it provides industrial security for companies' supply chains to identify and mitigate foreign influence or threats. Legal Provides legal guidance, and serves as the designated ethics counsel for personnel that are part of the CAG. Supply, Production, and Distribution Coordinates supply chain management, distribution, operations, and administration: • Supply chain management recommends the use and administration of Defense Production Act ratings. • Distribution is responsible for monitoring and enabling the movement of vaccine products from the distributor's location to the administration site. • Operations is responsible overseeing vaccine production capacity, national and international distribution, and administration of COVID-19 vaccines. • Administration is responsible for coordination activities and disseminating guidance to states and federal partners to ensure timely and efficient distribution and administration of COVID-19 vaccines. Source: GAO analysis of information from the Department of Health and Human Services (HHS)-Department of Defense (DOD) COVID-19 Countermeasures Acceleration Group (CAG). | GAO-22-104453 Notes: The lines of effort outlined in this table represent all of the CAG's responsibilities, according to documentation we reviewed and interviews with CAG officials; however, our review was primarily focused on the CAG's vaccine-related responsibilities. CAG officials told us in October 2021 that DOD officials led all nine lines of effort. However, HHS officials told us in January 2022 that the Legal line of effort and the Research and Clinical Trials for Vaccines and Therapeutics line of effort were already supported by HHS officials and therefore were not transitioned from DOD to HHS. Page 11 GAO-22-104453 HHS-DOD CAG Partnership Transition activities for each of the nine lines of effort began by October 15, 2021, according to CAG officials. For example, for each line of effort, CAG officials said HHS and DOD started what they refer to as "left-seat, right-seat" training, in which DOD officials trained their HHS counterparts on activities they led for about 1 month, followed by HHS officials leading activities with DOD officials assisting. CAG officials told us that the goal of this "left-seat, right seat" training is to ensure that HHS is ready to continue all of the CAG's responsibilities using HHS's own capabilities. According to DOD and HHS officials, HHS completed its planned transition activities and the CAG dissolved on December 31, 2021, as called for in the April 2021 memorandum. Additionally, CAG officials said HHS and DOD determined that DOD would provide acquisition support to HHS through at least September 2023, based on a May 2021 memorandum regarding the need for ongoing acquisition services related to vaccines, such as continued purchases of vaccine doses and other supplies. DOD's acquisition support has played a significant role in fulfilling the CAG's vaccine-related responsibilities, such as by helping to enable increases in manufacturing capacity and the purchase of vaccine doses and related supplies. DOD's contracting techniques, including solicitation approaches like broad agency announcements and commercial solutions openings, allowed DOD to obtain company input related to vaccine requirements, facility expansion, and related supplies. 26 In addition, DOD used prototype OTA and technology investment agreements to attract companies the government does not normally do business with, including vaccine companies, and to obtain vaccine-related items such as vials. 27 For more information on these and other contracting techniques used for vaccine-related acquisitions under the CAG, see appendix II. DOD acquisition officials told us they used some of these techniques for the 26A broad agency announcement is a notice from the government that requests scientific or research and development proposals from private firms concerning certain areas of interest to the government. See Federal Acquisition Regulation (FAR) § 35.016. The commercial solutions opening pilot program allows DOD to mirror the contracting practices that commercial companies normally use, enabling DOD to design projects, and negotiate payment milestones, intellectual property rights, and other terms and conditions for a desired completion period of within 60 days. See National Defense Authorization Act for Fiscal Year 2017, Pub. L. No. 114-328, § 879, 130 Stat. 2000, 2312-13 (2016) (codified at 10 U.S.C. § 2303 note). 27Technology investment agreements are used to stimulate or support research to foster the best technologies for future defense needs. See 32 C.F.R. Part 37 (2020). Page 12 GAO-22-104453 HHS-DOD CAG Partnership first time to address the COVID-19 pandemic, and doing so required a high level of acquisition expertise and staff investment. According to DOD officials, HHS and DOD are developing plans to incrementally reduce DOD acquisition responsibilities and transition them to HHS after September 2023. DOD officials told us that as of September 15, 2021, they did not have an estimated date for when they would complete their acquisition transition plans, but the extension of DOD support through September 2023 would allow sufficient time to complete them. It Is Unclear If HHS Is Neither the CAG nor HHS have completed all the tasks necessary to Ready to Fully Assume the ensure that HHS is in a position to fully assume all of the CAG's responsibilities beginning January 1, 2022. In particular, HHS and DOD CAG's Responsibilities officials indicated that they developed and completed transition milestones showing HHS's readiness to assume the CAG's responsibilities without a loss of capabilities. However, the documentation HHS provided to us in January 2022 does not identify or describe such milestones for all lines of effort or show how the milestones were achieved. Similarly, although HHS conducted workforce assessments to evaluate its workforce capacity for leading the lines of effort, it has not developed and implemented strategies to address identified workforce needs for some lines of effort. Further, HHS has not developed a schedule to manage the remaining vaccine development, manufacturing, and distribution activities. Documentation Does Not The April 2021 memorandum between HHS and DOD and the CAG's July Identify Transition Milestones 2021 transition plan specify that the CAG would develop milestones, by or How They Were Achieved August 2021, to gauge the progress of the transition and indicate when it is ready to transfer responsibilities for each of the nine lines of effort to HHS. 28 Developing such milestones is consistent with leading practices we identified for agency reform practices. 29 In documentation HHS and DOD provided to us in January 2022, the departments indicated that they 28The transition included nine lines of effort. In providing comments on a draft of this report, HHS stated that two lines of effort-the Legal line of effort and the Research and Clinical Trials for Vaccines and Therapeutics line of effort-were already supported by existing staff within HHS and therefore did not require transition milestones. 29See GAO-18-427. One leading agency reform practice is that agencies should develop and implement a plan with key milestones and deliverables to track progress for the reform, such as a transition. Page 13 GAO-22-104453 HHS-DOD CAG Partnership developed and achieved transition milestones for the lines of effort by December 2021. While the transition is now complete, and responsibilities have been transferred to HHS, the documentation the departments provided to us in January 2022 did not identify or describe such milestones for all lines of effort, or show how they were achieved. Specifically, CAG officials provided finalized milestones for two lines of effort-the External Affairs and the Comptroller lines of effort. For these lines of effort, the CAG developed specific milestones to be achieved-such as the reconciliation and handoff of CAG financial documents to HHS for the Comptroller line of effort-in its transition progress assessment documentation. For five other lines of effort, HHS provided documentation that a senior official reviewed transition milestones and associated tasks and approved HHS to assume responsibilities, but the documentation does not identify specific milestones or show how they were achieved. For example, documentation for the Supply, Production, and Distribution and Analytics and Information Technology lines of effort identified transition tasks that had not yet been completed, without an indication of when or by whom those tasks would be finalized. According to HHS officials, milestones were continuously adjusted based on a variety of factors as the transition progressed. Without documentation of specific milestones and how they were achieved, it is unclear how HHS determined its readiness to assume the CAG's responsibilities. HHS Has Not Finalized As of January 2022 HHS had conducted assessments of HHS's Strategic Workforce Planning workforce capacity for seven lines of effort. 30 For two of these- Comptroller and Vaccine and Therapeutics Development and Manufacturing-HHS determined that it had a sufficient amount of personnel in place with the required skills to continue the lines of effort's responsibilities. For another line of effort-External Affairs-HHS completed a workforce assessment that showed it identified and filled a position for a communication specialist. This action resulted in sufficient capacity for that line of effort, according to HHS. However, for the remaining four lines of effort, the transition documentation we reviewed did not indicate that HHS had developed and implemented a workforce strategy to resolve identified personnel gaps resulting from DOD's 30The transition included nine lines of effort. In providing comments on a draft of this report, HHS stated that two lines of effort-the Legal line of effort and the Research and Clinical Trials for Vaccines and Therapeutics line of effort-were already supported by existing staff within HHS and therefore did not require workforce assessments as part of the transition from CAG to HHS. Page 14 GAO-22-104453 HHS-DOD CAG Partnership departure. For instance, activities under the Security and Assurance line of effort include coordinating programs designed to safeguard the development, manufacture, and distribution of vaccines, and to provide industrial security for vaccine companies' supply chains. The CAG's transition document for this line of effort indicated that the workforce assessment found that HHS did not have the necessary staff to continue the CAG's responsibilities, and identified a need for multiple additional contractors to maintain the line of effort's capabilities. However, the documentation did not identify a strategy for addressing this need. In another example, activities under the Supply, Production, and Distribution line of effort included providing assistance to vaccine companies, such as through detailing DOD personnel to serve as temporary quality control staff at vaccine manufacturing sites. According to CAG officials, additional activities for this Supply, Production, and Distribution line of effort included managing the supply of manufacturing items and the allocation and distribution of vaccine doses to jurisdictions and other entities. CAG officials' assessment of this line of effort found that HHS did not have the staff with the necessary specialized skills and competencies-including communication and logistic expertise-to continue the CAG's responsibilities, according to officials. As a result, officials stated that HHS awarded a contract on October 15, 2021 to bring in personnel to help resolve those skill gaps. However, the transition document we reviewed noted that as of December 1, 2021, HHS was awaiting contractor personnel for the Supply, Production, and Distribution line of effort while managing the limited availability of existing contracting support, and that none of that particular line of effort's responsibilities had transitioned to HHS, raising questions about HHS's readiness to take on such work. 31 Our prior work shows that delays in developing and implementing workforce strategies to address identified personnel gaps may be problematic. For example, in our September 2020 report describing, in part, medical supply management responsibility shifting to HHS related to the COVID-19 pandemic, we found that it could be hard to hire the acquisition staff with expertise in Defense Production Act (DPA) contracting that ASPR had determined they would need to further support 31Our review of the transition document for the Supply, Production, and Distribution line of effort indicated that DOD personnel would continue their responsibilities for this line of effort, beyond December 31, 2021. Page 15 GAO-22-104453 HHS-DOD CAG Partnership COVID-19 vaccine and related supply acquisitions. 32 This misalignment of staff needs with available resources could lead to skill gaps within HHS's workforce, and put HHS in jeopardy of successfully assuming all of the CAG's responsibilities. Determining critical workforce skills and capabilities-and developing strategies to address any needs-also follows selected leading practices identified by our prior work. 33 Given that DOD has specialized skills and competencies and had been leading many of the CAG's vaccine-related responsibilities before the CAG dissolved, addressing workforce capacity needs is particularly important, so that HHS can continue these responsibilities uninterrupted. Without doing so, there is a risk that HHS may not have some of the capabilities necessary to continue critical vaccine-related activities going forward. HHS Indicated It Will Continue As HHS was assuming the CAG's responsibilities, HHS officials indicated Using a Key Information they planned to continue utilizing a key information system-Tiberius-for System, but Has Not tracking their ongoing COVID-19 vaccine manufacturing and distribution Developed a Schedule to activities. However, HHS has not developed a schedule to manage Manage Activities remaining vaccine development, manufacturing, and distribution activities. Tiberius. The CAG used Tiberius-an integrated information technology database that incorporates information from outside sources, such as vaccine companies and jurisdictions-to help manage and track its COVID-19 vaccine manufacturing, distribution, and administration activities. 34 In particular, according to CAG officials, Tiberius was the principal information system used by jurisdictions to review weekly vaccine allocations. Specifically, Tiberius received a data feed from the electronic system the jurisdictions used to order their vaccines, and then information from that data feed displayed various metrics for each 32See GAO, COVID-19: Federal Efforts Could be Strengthened by Timely and Concerted Actions, GAO-20-701 (Washington, D.C., Sept. 21, 2020). The DPA facilitates the supply and timely delivery of products, materials, and services to military and civilian agencies in support of the national defense, including in response to emergency preparedness activities. See Pub. L. No. 81-774, 64 Stat. 798 (1950) (codified, as amended, at 50 U.S.C. §§ 4501 et seq.). 33See GAO-18-427 and GAO-04-39. 34According to HHS and DOD officials, the federal government owns the Tiberius software, but licenses the right to use its hosting platform from the private company that established and maintains the platform. Page 16 GAO-22-104453 HHS-DOD CAG Partnership jurisdiction, such as the number of doses available to order and ordered by a jurisdiction. As part of the transition, HHS officials said they extended a key Tiberius contract for software development through at least July 2022. 35 Additionally, HHS officials told us that the HHS Office of the Chief Information Officer coordinated with officials from the CAG, CDC, and ASPR to transition management of Tiberius to CDC. According to HHS officials, CDC plans to integrate Tiberius into its existing systems, which would allow CDC to continue using the system, as well as determine any long-term plans for using Tiberius after July 2022. Schedule. HHS has not developed a specific schedule to help manage the remaining COVID-19 vaccine-related responsibilities once the CAG dissolved-such as those related to managing the distribution of vaccine doses for boosters or children, or to address any emerging COVID-19 variants. 36 According to the GAO Schedule Assessment Guide, a schedule-as a normal part of project management-assists project managers by providing a road map for systematic project execution, defining when and how long work will occur, and indicating how each activity within a project is related to others, among other benefits. 37 For example, a schedule could include activities for distribution of sufficient vaccine doses across the states and other jurisdictions for boosters, for children, or for addressing variants. As such, a schedule-particularly one that follows best practices such as including all activities, placing them in sequence, assigning resources to them, and establishing their duration- can help managers to identify and mitigate scheduling risks, such as delays. According to the guide, a schedule for a completed project can also be a valuable source of lessons learned-showing what actually happened compared to expectations-when planning similar future projects. 35CAG officials estimate that it will cost approximately $32 million per year to maintain and continue to use Tiberius. 36Additionally, three COVID-19 vaccines that were part of the CAG remained in development; as of January 2022, the AstraZeneca, Novavax, and Sanofi vaccines were still in development and had not been authorized or licensed for use in the United States. 37See GAO-16-89G. Scheduling allows officials to decide between possible sequences of activities, determine the flexibility of the schedule according to available resources, predict the consequences of managerial action or inaction on events, and develop contingency plans to mitigate risks. Page 17 GAO-22-104453 HHS-DOD CAG Partnership HHS officials told us that the department will no longer use an existing schedule developed by the CAG, as the contract that supported the CAG's schedule has ended. 38 In addition, rather than developing a new schedule, HHS officials told us that the department will instead rely on its project coordination teams to manage the schedules for these activities, by working with the Biomedical Advanced Research and Development Authority (BARDA) and the vaccine companies. 39 However, officials have not provided information on how this coordination would result in a reliable schedule that follows best practices. HHS officials also noted that the information once provided by the CAG's schedule will continue to be made available via the project coordination teams and BARDA officials. However, our separate review of the CAG's schedule indicated that its information was not reliable. Specifically, we found that the CAG's schedule minimally or partially met best practices associated with a "well-constructed" schedule, as defined in the GAO Schedule Assessment Guide. 40 In particular, we found a number of missing logical dependencies, which help show how activities that fall behind in the schedule will affect succeeding activities that depend on them, or how these delays would affect the overall project schedule. We also could not identify a valid critical path, and determined the schedule exhibited an unreasonable amount of total float. For more information 38In June 2020, the CAG developed a schedule to help manage the alignment of the hundreds of individual tasks needed to support its efforts to develop, manufacture, and distribute the various COVID-19 vaccines. Examples of these tasks include monitoring the timing of clinical trials conducted by vaccine companies; vaccine quality control testing; and the shipment of vaccine doses to specific sites across various jurisdictions for administration. 39According to CAG officials, project coordination teams consisted of federal employees and contractors from HHS and DOD who were responsible for coordinating vaccine development and manufacturing efforts. Additionally, CAG officials told us these teams also served as contact points between the federal government and each of the six vaccine companies that were part of the CAG. Furthermore, CAG officials told us that collectively the project coordination teams fell under the DOD-appointed Director of Vaccines, but each of the six teams were led by HHS officials. 40A schedule is well-constructed if all its activities are logically sequenced with the most straightforward logic possible. The best practices of a well-constructed schedule include (1) logical sequencing of all activities-that is, listing the activities in the order in which they are to be logically carried out; (2) confirming that the schedule has a valid critical path-that is, the longest continuous sequence of activities in a schedule and the path that defines the program's earliest completion date or minimum duration; and (3) identifying reasonable total float or slack-the amount of time an activity could be delayed before that delay affects the program's overall estimated finish date. Page 18 GAO-22-104453 HHS-DOD CAG Partnership about the CAG's schedule, including our abridged assessment of it, see appendix III. Having a schedule that is consistent with identified best practices can help officials have confidence in their ability to manage vaccine development, manufacturing, and distribution efforts. Specifically, developing a schedule that is consistent with best practices could help HHS determine the amount of scheduling flexibility it has, predict the consequences of managerial action or inaction in events, and develop contingency plans to mitigate delays or other risks while completing activities related to vaccines for boosters, for children, or for addressing emerging variants. In addition, developing such a schedule would help HHS to better archive the actual sequence of events for completing its remaining activities, which could be a source of lessons learned to analyze during its planning for any future pandemics. The CAG began a joint interagency lessons-learned review process in The CAG Began a August 2021 to assess its COVID-19 vaccine development, Lessons-Learned manufacturing, and distribution efforts, as called for by the April 2021 memorandum. Specifically, the CAG established an interagency Review Process, but workgroup and a review process the CAG would use for collecting Its Plans Do Not lessons learned across each of the nine lines of effort, according to officials. As of September 2021, CAG officials expected they would Include Collecting complete their process of collecting lessons learned no later than Information from December 31, 2021, and that these lessons learned would be available to help guide the transition of CAG responsibilities to HHS. However, we did Stakeholders not receive updated information about the process's completion. Officials said they planned to compile the lessons learned collected across all nine lines of effort into a single report, issued no later than March 31, 2022. We found that the CAG's lessons-learned plan-finalized in September 2021-aligns with some leading practices for conducting lessons-learned reviews that we and others have identified, including collecting and sharing information on any positive and negative experiences. 41 Figure 1 shows these leading practices. 41See GAO-21-8, GAO-20-104, GAO-19-25, and GAO-12-901. See also Center for Army Lessons Learned, Establishing a Lessons Learned Program. Page 19 GAO-22-104453 HHS-DOD CAG Partnership Figure 1: Leading Practices of a Lessons-Learned Review Process For example, the CAG's lessons-learned plan delineates responsibilities for collecting information on lessons learned by completing after-action reports and exit interviews with CAG officials. 42 It states that prior to departure from the CAG, every official assigned to the CAG should provide input on lessons learned-such as their perspectives on significant challenges and best practices-to their line of effort's point of contact for inclusion in an after-action report. 43 The lessons-learned plan also includes an overview of how CAG officials would transfer or share relevant records with HHS, so that HHS has the necessary information to continue the work. 42According to the lessons-learned plan, the objective of the after-action reports and associated exit interviews is to provide context to HHS regarding CAG operations and decisions in order to streamline decision-making for efforts to respond to potential future pandemics. The after-action reports will serve as a historic record of accomplishments facilitated by the CAG. 43According to CAG officials, the Supply, Production, and Distribution line of effort was subdivided into five separate sections. They said that while one official will continue to serve as the primary point of contact, the official will coordinate with five other individuals who are responsible for completing the after-action reviews for the five subdivided areas under this specific line of effort. Page 20 GAO-22-104453 HHS-DOD CAG Partnership However, based on our review, the CAG's lessons-learned plan did not meet other leading practices. Specifically, the CAG's planned interagency review process did not include efforts to engage stakeholders in identifying lessons learned. Leading practices emphasize the importance of engaging with key stakeholders-that is, those entities who played significant roles in supporting the CAG's activities. As an example of these leading practices, The Standard for Program Management, produced by the Project Management Institute, states that program managers should actively engage key stakeholders throughout the life cycle of a program, which would include lessons-learned processes and evaluation activities, such as completing after-action reviews. 44 According to CAG officials, the lessons-learned review process is internally focused and does not outline an approach to collect information from key stakeholders outside of the CAG, such as the vaccine companies or other federal agencies that coordinated with the CAG. The six vaccine companies, as well as several HHS and DOD component offices and agencies, were key stakeholders that played significant roles in the CAG's COVID-19 vaccine development, manufacturing, and distribution efforts. As such, they may have valuable contributions to make to the lessons-learned reviews. CAG officials said their lessons-learned review process was intended to collect information internally from staff assigned to the CAG. Officials added that CAG officials may have chosen to gather information from key stakeholders outside the CAG-at their discretion during the lessons- learned process-but that there was no plan or requirement for them to do so. By expanding the lessons learned review to also obtain and incorporate the perspectives of key external stakeholders, HHS and DOD could better ensure that HHS has the full benefit of capturing perspectives from all of the significant players involved in the CAG's vaccine-related efforts. Moreover, these perspectives could help provide a more comprehensive understanding of what worked well and of areas for potential 44See Project Management Institute, Inc., A Guide to the Project Management Body of Knowledge (PMBOK® Guide), Sixth Edition, 2017. GAO, Disaster Response: HHS Should Address Deficiencies Highlighted by Recent Hurricanes in the U.S. Virgin Islands and Puerto Rico, GAO-19-592 (Washington, D.C.: Sept. 20, 2019). The Project Management Institute is a not-for-profit association that provides global standards for, among other things, project and program management. These standards are utilized worldwide and provide guidance on how to manage various aspects of projects, programs, and portfolios. Page 21 GAO-22-104453 HHS-DOD CAG Partnership improvement to help inform ongoing and any future vaccine development, manufacturing, and distribution efforts. Based on our review of information from and interviews with CAG Preliminary Lessons officials, federal agency officials, and vaccine company representatives in Learned from the June 2021 and July 2021, we identified some examples of preliminary lessons learned. The preliminary lessons learned we identified covered CAG, Federal several areas, as described below. Agencies, and Selecting multiple vaccine companies and platforms. According to Vaccine Company CAG officials, the CAG's strategy to build a diverse portfolio of vaccine Representatives candidates from multiple companies that use different platform technologies worked well. 45 CAG officials highlighted the federal government's strategy to award contracts and OTAs to multiple vaccine companies developing COVID-19 vaccines. As we have previously reported, the federal government took on financial risk to support or purchase doses of the six vaccine candidates by enabling large-scale manufacturing to start while clinical trials were ongoing, before their safety and effectiveness of the candidates was fully known. 46 We noted that this approach helped to enable distribution of vaccines as soon as possible upon receiving FDA authorization or licensure. CAG officials noted that including vaccine candidates from different platforms-before knowing whether any would be successful-involved significant costs but helped to allow multiple candidates to receive EUAs within 9 months of establishing the CAG, and to have significant numbers of manufactured doses available for distribution at the time the EUAs were issued. We previously reported that because these platforms use different mechanisms to stimulate an immune response, having a variety of platforms decreases the risk of failure due to safety, effectiveness, or manufacturing factors that may affect some platforms but not others. 45A vaccine platform is the mechanism used to stimulate an immune response in a recipient. 46See GAO-21-319 and GAO-21-443. Pfizer funded the research and development of its COVID-19 vaccine. The federal government agreed to pay for doses of Pfizer's vaccine upon FDA authorization or licensure and as the doses were delivered. HHS officials stated that this agreement required the government to buy Pfizer vaccine doses upon authorization or licensure, even if other vaccines were found to be more cost effective or efficacious. Page 22 GAO-22-104453 HHS-DOD CAG Partnership Having multiple candidates also increased the chances that one or more candidate would be successful. 47 Sharing expertise and coordinating across DOD, HHS, and vaccine companies. Officials from the CAG, DOD, HHS, and NIH told us that HHS's expertise in the sciences and DOD's expertise in planning, logistics, programming, and contract management complemented each other, and both were necessary to make the CAG's COVID-19 vaccine development, manufacturing, and distribution efforts successful. Specifically, according to CAG officials, HHS worked with several vaccine companies as they conducted clinical trials, focusing on the safety and effectiveness of the vaccine candidates throughout the clinical trial process. For example, NIH's National Institute of Allergy and Infectious Diseases made research staff available to several vaccine companies to help collaborate on the companies' clinical trials. NIH staff also helped vaccine companies find sites to conduct clinical trials and solicit enrollees for the trials. In addition, the CAG DOD team provided operational and logistics expertise, such as delivering personal protective equipment for the clinical trial sites when needed, and installing mobile trailers if additional capacity was required. HHS and DOD's coordination worked well in this instance and both agencies learned from each other, according to CAG officials. In contrast, representatives from two vaccine companies told us that there were initial coordination challenges in their interactions with the CAG, which improved over time. For example, the representatives said that they sometimes needed to report the same information multiple times to officials from different federal agencies, or to different groups within the same agency. They said that the multiple reporting requirements became time consuming and distracting at times, and led to confusion about who within the government was responsible for making decisions. According 47See GAO-21-319. It is also possible that a diverse portfolio of vaccine platforms could offer additional benefits over time. Different vaccine platforms may also be more effective in different individuals-for example, older adults or pediatric populations. Further, a diverse portfolio of vaccine platforms may allow for the use of mixed vaccine platform boosters. Scientific studies have shown that mixed boosters might help better address the emergence of SARS-CoV-2 variants and decreasing immunity over time. For example, mixed vaccine platform boosters may help stimulate different parts of the immune system, resulting in better and longer-lasting immune responses, or decrease the risk of adverse events from the original vaccine from developing or happening again. Page 23 GAO-22-104453 HHS-DOD CAG Partnership to these representatives, coordination and communication with the CAG improved as they continued working with the officials over time. Using project coordination teams. BARDA officials noted the importance of forming project coordination teams to support vaccine companies' efforts, and including a leader (from BARDA, NIH, or DOD) and a team of program managers and subject matter experts in all critical areas of vaccine development and manufacturing. BARDA officials said that in the fall of 2020, project coordination teams would meet daily with each vaccine company to discuss the resources necessary to accelerate vaccine development and to help identify scientific, technical, and strategic risks and any plans for mitigating those risks, among other things. According to CAG officials, the teams continued to conduct meetings 4 times weekly to provide progress updates to the CAG's leadership and other stakeholders about any vaccine-related issues or resource constraints. Using supply chain managers and the DPA for vaccine manufacturing. CAG officials said that they embedded federal government supply chain managers to work at vaccine manufacturing sites. According to CAG officials, this approach allowed for daily progress updates to its leadership during the fall of 2020, and close coordination between the federal government and manufacturing companies regarding any supply chain issues that could affect manufacturing. Representatives from two vaccine companies commented on the CAG's efforts to support manufacturing efforts through use of the Defense Production Act (DPA), which has allowed companies to have priority access to necessary materials. 48 Federal agencies used the DPA and other actions more than 100 times to respond to the COVID-19 pandemic and stabilize the medical supply chain through September 2021. 49 Use of the DPA enabled one vaccine company, for example, to have priority 48Contracts with a priority rating under the DPA require a contractor to give preference to these contracts over any other unrated contracts if the contractor cannot meet all required delivery date needs for all contracts. See 50 U.S.C. § 4511. 49"Other actions" refers to industrial base expansion projects for medical supplies that have similar goals but were not executed under the DPA Title III authority. See GAO, COVID-19: Agencies Are Taking Steps to Improve Future Use of Defense Production Act Authorities. GAO-22-105380. (Washington, D.C.: Dec. 16, 2021). Page 24 GAO-22-104453 HHS-DOD CAG Partnership access to lipids, bags, and filters to help manufacture its COVID-19 vaccine. Representatives from one vaccine company said the use of the DPA in limited circumstances could help secure supply-constrained raw materials needed to produce COVID-19 vaccines. However, they added that the company's suppliers have noted that exercising the DPA can create production challenges and disruptions, because it can restrict suppliers' flexibility and ability to maximize capacity. 50 According to DOD's CAG Continuity Book, use of the DPA requires close coordination of priorities and deliveries to ensure production stability, and is "not a miracle solution." For example, the CAG Continuity Book notes that domestic fill-finish capacity was limited even prior to the COVID-19 pandemic. It states that the need to use existing capacity to address needs during the pandemic meant that less capacity was available for other life-saving medicines. The CAG Continuity Book also emphasizes the importance of expanding manufacturing capacity for future public health emergencies, to ensure sufficient capacity for both routine commercial business and the surge of increased capacity needs due to the emergency. Contracting for vaccine development and manufacturing. According to CAG officials, constant communication between contracting, legal, and scientific experts was key to obtaining the vaccines. Specifically, DOD's Joint Program Executive Office for Chemical, Biological, Radiological, and Nuclear Defense (JPEO-CBRND) and Army Contracting Command used their acquisition workforce expertise and prototype OTA authority-which DOD officials say enabled command officials to negotiate terms and conditions in agreements with five of the six vaccine companies. 51 50We previously reported on the unintended consequences that could result from use of the DPA to expedite the receipt of supply-constrained materials for vaccine manufacturing (i.e., creating constrained supplies for other life-saving medicines). See GAO-21-443. 51As we previously reported, OTAs can help agencies contract with entities that have not previously done business with the federal government due to concerns about standard contracting requirements. However, there may be challenges associated with their use, including a risk of reduced accountability and transparency, which we addressed in prior reports. See GAO, COVID-19 Contracting: Actions Needed to Enhance Transparency and Oversight of Selected Awards, GAO-21-501 (Washington, D.C.: July 26, 2021) and GAO, COVID-19 Contracting: Observations on Federal Contracting in Response to the Pandemic, GAO-20-632 (Washington, D.C.: July 29, 2020). Page 25 GAO-22-104453 HHS-DOD CAG Partnership In addition, JPEO-CBRND officials said that using a flat organizational structure-with direct contact between acquisition senior leaders and staff-allowed for efficient communication, which enabled quick decision- making when it was necessary. For example, these officials told us that having access to senior leadership on a regular basis, through the leaders' participation in work group meetings and team discussions, helped to reduce the normal acquisition processing times for vaccines by months. Officials added that senior leaders were prepared to review contract documents quickly when they came in for approval, thereby minimizing the contract award time. CAG officials noted that a challenge they experienced was the initial lack of enough acquisition staff to support the sustained high-volume, high- tempo workload. JPEO-CBRND provided 10-12 employees in direct support of CAG operations, embedded with the project coordination teams, and asked them to work longer hours. According to CAG officials, in approximately May 2021, JPEO-CBRND was authorized to acquire about 75 additional employees and contractors, which helped to relieve the burden on JPEO-CBRND staff. Communicating COVID-19 vaccine-related activities to the public and vaccine companies. CAG officials told us that the CAG helped to support the federal government's frequent public updates on vaccine development, starting around November 2020. However, the officials noted that the federal government could have better communicated with the public to help people who were unfamiliar with vaccine development understand the inherent unpredictability of the process. Specifically, CAG officials explained that development of biologics, including vaccines, is more unpredictable than other types of drug development and can lead to unforeseen production failures. For example, actual production amounts could be lower than estimated amounts due to the inherent fragility in the process. CAG officials noted that the federal government could work to better communicate expectations for some fluctuation in estimated OTAs are generally exempt from federal procurement laws and regulations, allowing intellectual property rights under each OTA to be tailored to suit the goals of the project. Under an OTA, the parties can tailor provisions to address concerns about intellectual property and unique government requirements and regulations. The parties are not necessarily bound by FAR-based contract requirements. For example the Bayh-Dole Act governs intellectual property rights in FAR-based contracts but not in OTAs. 35 U.S.C. §§ 200-212. The FAR is the primary regulation for use by all executive agencies in their acquisition of supplies and services with appropriated funds. Page 26 GAO-22-104453 HHS-DOD CAG Partnership amounts of available vaccine doses given the nature of vaccine production. Representatives from one vaccine company noted that ongoing communication with the involved federal agencies allowed them to continuously improve their plans for vaccine rollout. Another vaccine company's representatives stated that the federal agencies provided guidance, feedback, and resources needed through each stage of their clinical trial programs. According to these representatives, direct and frequent communication with federal agencies helped to ensure the vaccine company aligned with safety and effectiveness evaluation practices expected of all federally-funded studies. The accelerated development, manufacturing, and distribution of vaccines Conclusions has been a critical part of the U.S. response to the COVID-19 pandemic. The federal government has shown it can rapidly and effectively mobilize partnerships to produce safe and effective vaccines to help the nation respond to and recover from the pandemic's catastrophic effects. The CAG partnership between HHS and DOD is an example of a quickly built and effective partnership, as are related agreements between the federal government and private vaccine companies. As HHS assumes all responsibilities of the CAG, it is vital that the department continues these activities without interruptions. This is especially critical as the pandemic continues well into its second year and the federal government looks to provide greater access to vaccines, such as through the recent availability of boosters and vaccines for children, as well as to prepare for emerging COVID-19 variants and for future pandemics. HHS and DOD stated that they developed and achieved milestones indicating HHS's readiness to assume the CAG's responsibilities-a critical step to ensuring the successful continuation of the CAG's vaccine work. However, available documentation does not identify most of these milestones, nor does the documentation demonstrate how the milestones were achieved. In addition, while HHS has assessed its workforce's ability to take on tasks that DOD had previously led within the CAG, HHS has yet to address some of the workforce needs that it identified. By developing and implementing a workforce strategy to address HHS's workforce needs, HHS and DOD can mitigate any risks and help ensure that HHS is ready to continue the CAG's work uninterrupted. Page 27 GAO-22-104453 HHS-DOD CAG Partnership Furthermore, HHS has not developed a schedule to help it manage remaining vaccine development, manufacturing, and distribution activities. Developing a schedule that is consistent with best practices would provide a road map for systematic execution of vaccine-related activities, as well as a means by which to help better plan actions, gauge progress, and identify and resolve potential problems. In addition, such a schedule could help HHS better archive the actual sequence of events for completing its remaining activities, which could in turn provide a source of lessons learned for any future pandemics. Additionally, although HHS and DOD have begun their joint interagency review to help inform HHS's future vaccine activities, the departments do not have plans to gather perspectives from stakeholders outside of the CAG. By also obtaining and incorporating the perspectives of key external stakeholders-such as from vaccine companies or other federal agency officials who worked with the CAG-HHS and DOD have an opportunity to obtain a more comprehensive understanding of what worked well and of areas for potential improvement to inform ongoing and any future vaccine development, manufacturing, and distribution efforts. We are making a total of five recommendations, including three to the Recommendations for Secretary of Health and Human Services, and two to the Secretary of Executive Action Defense: The Secretary of Health and Human Services, in coordination with DOD, should develop and implement workforce strategies to address the workforce needs it identified as part of the CAG's transition to HHS. (Recommendation 1) The Secretary of Defense, in coordination with HHS, should establish a mechanism-such as through an interagency agreement-to provide support to HHS until it develops and implements workforce strategies to address the workforce needs it identified as part of the CAG's transition to HHS. (Recommendation 2) The Secretary of Health and Human Services should develop a schedule that is consistent with the best practices established in the GAO Schedule Assessment Guide to manage remaining vaccine-related responsibilities. (Recommendation 3) The Secretary of Health and Human Services, in coordination with DOD, should expand the CAG's lessons-learned review to also obtain and incorporate input from key external stakeholders, such as vaccine Page 28 GAO-22-104453 HHS-DOD CAG Partnership companies and other federal agencies that coordinated with the CAG on its vaccine-related responsibilities. (Recommendation 4) The Secretary of Defense, in coordination with HHS, should expand the CAG's lessons-learned review to obtain and incorporate input from key external stakeholders, such as vaccine companies and other federal agencies that coordinated with the CAG on its vaccine-related responsibilities. (Recommendation 5) We provided a draft of this report to HHS and DOD for review and Agency Comments comment. Both departments provided written comments, which are and Our Evaluation reproduced in appendix IV and V, respectively. HHS also provided technical comments, which we incorporated as appropriate. In written comments from HHS, the department concurred with two recommendations, and did not concur with two other recommendations. In the draft report, provided to the departments in early December 2021 before the CAG dissolved, we recommended that HHS, in coordination with DOD, should finalize and achieve transition milestones, to help demonstrate HHS's readiness to assume the CAG's responsibilities before DOD formally ended its support. At that time, the CAG had not finalized transition milestones for seven of the nine lines of effort. HHS did not concur with this recommendation because, according to HHS in its comments on our draft report, the department had successfully completed the planned transition of responsibilities to the newly established HHS Coordination Operations and Response Element by its deadline of January 1, 2022. HHS provided memos stating that milestones had been achieved and detailing transition activities. While the memos did not identify or describe such milestones or show how they were achieved, we removed this draft recommendation because the CAG's responsibilities have transferred to HHS. Nonetheless, without a clear understanding of what information the transition was based on-including the specific milestones developed and how those were achieved-we remain concerned that HHS has not demonstrated its readiness to fully assume all responsibilities formerly undertaken by the CAG. HHS also did not concur with the draft report's second recommendation to complete assessments of its workforce capacity and develop corresponding workforce strategies. In its written comments on our draft report, HHS stated this was because the department completed workforce assessments by January 1, 2022, after we sent the draft report for review. HHS also provided additional documents describing these workforce assessments, and we revised our report to indicate that HHS Page 29 GAO-22-104453 HHS-DOD CAG Partnership had completed these assessments. However, as our report indicates, the assessments also identified specific workforce needs, but do not include strategies for addressing those needs. As a result, our report recommends that HHS develop such strategies so that it can mitigate any workforce-related risks and help ensure that it successfully continues its work uninterrupted. HHS concurred with our recommendation to develop a schedule to manage remaining vaccine-related responsibilities, consistent with best practices. In its written comments, HHS stated that its new office, the HHS Coordination Operations and Response Element, will make assessments regarding the future use of schedules, taking into consideration the best practices established in the GAO Schedule Assessment Guide. HHS also concurred with our recommendation to expand the CAG's lessons-learned review to obtain and incorporate input from key external stakeholders. In written comments from DOD, the department concurred with the two recommendations directed to it. In the draft report, we recommended that DOD establish a mechanism to formally provide support to HHS until CAG transition milestones are finalized and achieved, and until HHS completes its planned assessments of its workforce capacity and develops corresponding workforce strategies. DOD stated that it concurred with the recommendation, although transition milestones were finalized and achieved by December 31, 2021. As noted above, we also received documentation from HHS that it had finalized milestones and developed workforce assessments. In light of this new information, we modified our recommendation to DOD to provide support to HHS in developing workforce strategies to address the workforce needs it identified as part of the CAG's transition to HHS. DOD also concurred with our recommendation to expand the CAG's lessons-learned review to obtain and incorporate input from key external stakeholders. We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, the Secretary of Health and Human Services, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. Page 30 GAO-22-104453 HHS-DOD CAG Partnership If you or your staff have any questions about this report, please contact Alyssa M. Hundrup at (202) 512-7114 or hundrupa@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VI. Alyssa M. Hundrup Director, Health Care Page 31 GAO-22-104453 HHS-DOD CAG Partnership List of Addressees The Honorable Patrick Leahy Chairman The Honorable Richard Shelby Vice Chairman Committee on Appropriations United States Senate The Honorable Ron Wyden Chairman The Honorable Mike Crapo Ranking Member Committee on Finance United States Senate The Honorable Patty Murray Chair The Honorable Richard Burr Ranking Member Committee on Health, Education, Labor, and Pensions United States Senate The Honorable Gary C. Peters Chairman The Honorable Rob Portman Ranking Member Committee on Homeland Security and Governmental Affairs United States Senate The Honorable Rosa L. DeLauro Chair The Honorable Kay Granger Ranking Member Committee on Appropriations House of Representatives The Honorable Frank Pallone, Jr. Chairman The Honorable Cathy McMorris Rodgers Republican Leader Committee on Energy and Commerce House of Representatives Page 32 GAO-22-104453 HHS-DOD CAG Partnership The Honorable Bennie G. Thompson Chairman The Honorable John Katko Ranking Member Committee on Homeland Security House of Representatives The Honorable Carolyn B. Maloney Chairwoman The Honorable James Comer Ranking Member Committee on Oversight and Reform House of Representatives The Honorable Richard Neal Chairman The Honorable Kevin Brady Republican Leader Committee on Ways and Means House of Representatives The Honorable James E. Clyburn Chair Select Subcommittee on the Coronavirus Crisis Committee on Oversight and Reform House of Representatives The Honorable Bill Foster House of Representatives The Honorable Mark E. Green, MD House of Representatives Page 33 GAO-22-104453 HHS-DOD CAG Partnership Appendix I: Status of U.S. COVID-19 Appendix I: Status of U.S. COVID-19 Vaccine Candidates and Related Obligations and Doses Vaccine Candidates and Related Obligations and Doses To support the Department of Health and Human Services (HHS)- Department of Defense (DOD) COVID-19 Countermeasures Acceleration Group (CAG) (formerly known as Operation Warp Speed), HHS and DOD awarded contracts and other transaction agreements (OTA) to six vaccine companies and others. 1 We reviewed the related contract documentation and found that these contracts and OTAs were awarded for the development and manufacturing of vaccines, the purchase of vaccine doses, and the acquisition of other critical supplies and services. See table 2 for the status of each of the six vaccine candidates as of January 11, 2022. Table 2: Status of the CAG's Six Vaccine Candidates, as of January 11, 2022 Findings from phase Biologics license Vaccine company 3 clinical trials announced EUA in effecta application (BLA) approvedb AstraZeneca ● - - Janssenc 18 and older; booster ● - for 18 and olderd Moderna 18 and older; booster ● - for 18 and oldere Novavax ● - - Pfizerf 5-15 years; booster 16 and olderh ● for 12 and olderg Sanofii - - - Source: GAO analysis of information provided by vaccine companies, the Food and Drug Administration (FDA), and the Department of Health and Human Services (HHS)-Department of Defense (DOD) COVID-19 Countermeasures Acceleration Group (CAG). | GAO-22-104453 Note: The CAG was formerly known as Operation Warp Speed. The columns provide information on specific groups for which the vaccines are authorized or licensed. a The Secretary of Health and Human Services may declare that circumstances, prescribed by statute, exist justifying the emergency use of certain medical products, such as vaccines. Once a declaration of an emergency has been made, FDA may temporarily allow use of unlicensed vaccines through an emergency use authorization (EUA), provided certain statutory criteria are met. For FDA to issue an EUA for a vaccine, it must be reasonable to believe that the vaccine may be effective and that the known and potential benefits of the vaccine outweigh the known and potential risks, among other statutory criteria. See 21 U.S.C. § 360bbb-3. b FDA licenses biologics, such as vaccines, through review and approval of BLAs. FDA guidance indicates that licensure is the goal for COVID-19 vaccine candidates, including those that first receive an EUA. c Janssen Pharmaceutical Companies are a part of Johnson & Johnson. 1The six vaccine companies are AstraZeneca, Janssen, Moderna, Novavax, Pfizer, and Sanofi. Pfizer developed its COVID-19 vaccine in collaboration with BioNTech, and Sanofi is developing its COVID-19 vaccine candidate in collaboration with GSK. Janssen Pharmaceutical Companies are a part of Johnson & Johnson. OTAs are flexible agreements that allow the parties to negotiate terms and conditions without requiring parties to comply with certain federal procurement laws and regulations. See 10 U.S.C. § 2371b. Page 34 GAO-22-104453 HHS-DOD CAG Partnership Appendix I: Status of U.S. COVID-19 Vaccine Candidates and Related Obligations and Doses d Janssen's one-dose COVID-19 vaccine was first authorized for emergency use on February 27, 2021 for those 18 years of age and older. FDA amended the authorization for Janssen's vaccine on October 20, 2021 to allow a single booster dose to be administered to individuals 18 years and older and who were vaccinated 2 or more months ago. e Moderna's two-dose COVID-19 vaccine was first authorized for emergency use on December 18, 2020 for those 18 years of age and older. FDA has amended the authorization for Moderna's vaccine numerous times, including on August 12, 2021 to allow a third dose to be administered to certain immunocompromised individuals; on October 20, 2021 to allow a single booster dose to be administered 6 months or more after the initial two-dose series to certain groups of individuals; on November 19, 2021 to allow a single booster dose to be administered 6 months or more after the initial two-dose series to individuals ages 18 and older; and on January 7, 2022 to reduce the dosing interval between the initial two-dose series and the booster from 6 months to 5 months. f Pfizer developed its COVID-19 vaccine in collaboration with BioNTech. g Pfizer's two-dose COVID-19 vaccine was first authorized for emergency use on December 11, 2020 for those 16 years of age and older. FDA has amended the authorization for Pfizer's vaccine numerous times, including on May 10, 2021 to include individuals ages 12 to 15; on August 12, 2021 to allow a third dose to be administered to certain immunocompromised individuals; on September 22, 2021 to allow a single booster dose to be administered 6 months or more after the initial two-dose series to certain groups of individuals; on October 29, 2021, FDA amended the authorization again to allow for a lower dose of Pfizer's vaccine to be administered to individuals ages 5 to 11; on November 19, 2021 to allow a single booster dose to be administered 6 months or more after the initial two-dose series to individuals ages 18 and older; and on January 3, 2022 to allow a single booster dose to be administered 5 months or more after the initial two-dose series to individuals 12 and older and to allow a third dose to be administered to certain immunocompromised individuals ages 5 to 11. h On August 23, 2021, FDA licensed Pfizer's COVID-19 vaccine for individuals 16 years and older. Sanofi is developing its COVID-19 vaccine candidate in collaboration with GSK. i Page 35 GAO-22-104453 HHS-DOD CAG Partnership Appendix I: Status of U.S. COVID-19 Vaccine Candidates and Related Obligations and Doses HHS and DOD Vaccine- Our review found that, to support these vaccine candidates, HHS and related Obligations DOD had obligated at least $29.8 billion as of September 30, 2021, as shown in figure 2. Figure 2: HHS and DOD Obligations for COVID-19 Vaccine Candidates and Others under the CAG, as of September 30, 2021 Note: The Department of Health and Human Services (HHS)-Department of Defense (DOD) COVID- 19 Countermeasures Acceleration Group (CAG) was formerly known as Operation Warp Speed. We used the HHS Operation Warp Speed website and HHS press releases to determine which contract obligations to include in our analysis. HHS and DOD awarded contracts and other transaction agreements to six vaccine companies and others for the development and manufacturing of vaccines, the purchase of vaccine doses, and the acquisition of other critical supplies and services. HHS announced one award related to distribution for which we could not identify obligations in the Federal Procurement Data System-Next Generation; that award is not included in the chart above. These obligations were, in part, for contracting to purchase vaccine doses for use in the U.S. As of October 22, 2021, 1.2 billion of these 1.7 billion doses were for vaccines that had been authorized or licensed by the Food and Drug Administration (FDA). (See table 3.) Page 36 GAO-22-104453 HHS-DOD CAG Partnership Appendix I: Status of U.S. COVID-19 Vaccine Candidates and Related Obligations and Doses Table 3: Vaccine Doses Contracted for Purchase by the Federal Government for Domestic Use, as of October 22, 2021 Vaccine company Contracted amount (millions of doses) AstraZeneca 300 Janssena 100 Moderna 500 Novavax 100b Pfizerc 600 Sanofid 100 Total 1,700e Source: GAO analysis of award and other acquisition related documents and information from the Department of Health and Human Services (HHS), the Department of Defense (DOD), Advanced Technology International, and vaccine companies. | GAO-22-104453 Note: The contracted amount includes base and exercised options. According to officials from the HHS-DOD COVID-19 Countermeasures Acceleration Group (CAG), these doses were intended for domestic use. However, according to these officials, as of September 30, 2021, 122.7 million doses were donated by the federal government for international use. a Janssen Pharmaceutical Companies are a part of Johnson & Johnson. b According to a Novavax representative, the company was also awarded a DOD contract in June 2020 that includes the delivery of 10 million doses. We do not include that amount in this table. c Pfizer developed its COVID-19 vaccine in collaboration with BioNTech. d Sanofi is developing its COVID-19 vaccine candidate in collaboration with GSK. e As of October 22, 2021, 1.2 billion of the 1.7 billion vaccine doses contracted for purchase were authorized or licensed (the Janssen, Moderna, and Pfizer vaccines had received authorization or licensure). COVID-19 vaccine candidates developed by AstraZeneca, Novavax, and Sanofi had not been authorized or licensed as of this date. Page 37 GAO-22-104453 HHS-DOD CAG Partnership Appendix II: Contract Techniques Used to Appendix II: Contract Techniques Used to Obtain COVID-19 Vaccines for the U.S. Obtain COVID-19 Vaccines for the U.S. To support the Department of Health and Human Services (HHS)- Department of Defense (DOD) COVID-19 Countermeasures Acceleration Group (CAG) (formerly known as Operation Warp Speed), DOD and HHS officials utilized multiple contracting techniques. 1 Specifically, DOD's Joint Program Executive Office for Chemical, Biological, Radiological and Nuclear Defense (JPEO-CBRND) and Army Contracting Command leveraged these techniques to obtain vaccine doses from six companies in response to the COVID-19 pandemic. 2 Contracting techniques employed to obtain COVID-19 vaccines included the following: Broad agency announcements. DOD and HHS used broad agency announcements to solicit vaccine rapid advanced research and development and large scale manufacturing. A broad agency announcement is a notice from the government that requests scientific or research and development proposals from private firms concerning certain areas of interest to the government. 3 Compared to a normal request for proposals, a broad agency announcement does not provide a standard-more specific-statement of work. Rather, it details a problem statement and challenges, and it solicits a solution. Proposals submitted by private firms in response to the announcement may lead to contracts. According to officials from the JPEO-CBRND and Army Contracting Command, vaccine-related broad agency announcements allowed them to obtain input from private companies on how the federal government could meet broad requirements on vaccine development and manufacturing. Other transaction agreements. DOD awarded prototype other transaction agreements (OTA) to five of the six companies. 4 OTAs are flexible agreements that allow the parties to negotiate terms and conditions without requiring parties to comply with certain federal 1We reviewed contract documentation and conducted agency interviews to determine which contract techniques HHS and DOD used. 2The six vaccine companies are AstraZeneca, Janssen, Moderna, Novavax, Pfizer, and Sanofi. Pfizer developed its COVID-19 vaccine in collaboration with BioNTech, and Sanofi is developing its COVID-19 vaccine candidate in collaboration with GSK. Janssen Pharmaceutical Companies are a part of Johnson & Johnson. 3See Federal Acquisition Regulation (FAR) § 35.016. The Federal Acquisition Regulation (FAR) is the primary regulation for use by all executive agencies in their acquisition of supplies and services with appropriated funds. 4Thecontract with the sixth company was awarded in accordance with the FAR and Defense FAR Supplement. Page 38 GAO-22-104453 HHS-DOD CAG Partnership Appendix II: Contract Techniques Used to Obtain COVID-19 Vaccines for the U.S. procurement laws and regulations. 5 This flexibility can help agencies attract and contract with entities that have not previously done business with the federal government due to concerns about standard contracting requirements. 6 For example, OTAs are generally exempt from federal procurement laws and regulations, allowing intellectual property rights under each OTA to be tailored to suit the goals of the project. 7 However, our recent work to review specific aspects of COVID-related OTAs found several challenges associated with their use, including a risk of reduced accountability and transparency. 8 For example, agencies did not accurately reflect all dollars obligated on COVID-19 OTAs in the federal procurement database and, in cases where OTAs were awarded through industry consortia, the agencies did not publicly report which consortium members received the OTA awards because of limitations with the federal procurement database. 9 Defense Production Act (DPA) awards and priority-rated contracts. DOD and HHS provided priority ratings under the DPA to the contracts with the six vaccine companies to expedite production. The DPA facilitates the supply and timely delivery of products, materials, and services to military and civilian agencies in support of the national defense, including in response to emergency preparedness activities. 10 For example, agencies can require private companies to prioritize fulfilling federal government contracts or orders before fulfilling contracts or orders from other customers. DOD and HHS have also used the DPA to award 5See 10 U.S.C. § 2371b. 6See GAO, COVID-19 Contracting: Observations on Federal Contracting in Response to the Pandemic, GAO-20-632 (Washington, D.C.: July 29, 2020). 7Under an OTA, the parties can tailor provisions to address concerns about intellectual property and unique government requirements and regulations. The parties are not necessarily bound by FAR-based contract requirements. For example, the Bayh-Dole Act governs intellectual property rights in FAR-based contracts but not in OTAs. 35 U.S.C. §§ 200-212. 8See GAO, COVID-19 Contracting: Actions Needed to Enhance Transparency and Oversight of Selected Awards, GAO-21-501 (Washington, D.C.: July 26, 2021). 9A consortium is comprised of members which can include traditional contractors, nontraditional companies, nonprofit organizations, and academic institutions interested in a specific topic area. 10See Pub. L. No. 81-774, 64 Stat. 798 (1950) (codified, as amended, at 50 U.S.C. §§ 4501 et seq.). Page 39 GAO-22-104453 HHS-DOD CAG Partnership Appendix II: Contract Techniques Used to Obtain COVID-19 Vaccines for the U.S. projects to expand domestic production of health and medical resources. According to officials from the JPEO-CBRND and Army Contracting Command, DPA authorities allowed them to ramp up vaccine production whether they used an OTA or Federal Acquisition Regulation (FAR)- based contract; they believe the DPA use has been instrumental in their ability to quickly obtain vaccine doses. For other efforts related to securing COVID-19 vaccines, such as increasing manufacturing capacity and obtaining necessary supplies like vials and syringes, DOD used the following contracting techniques: Commercial solutions openings. The commercial solutions opening pilot program allows DOD to mirror the contracting practices that commercial companies normally use, enabling DOD to solicit company input to design projects, and negotiate payment milestones, intellectual property rights, and other terms and conditions for a desired completion period of within 60 days. 11 According to officials from the JPEO-CBRND and Army Contracting Command, they leveraged this authority to use commercial solutions openings for the first time during the COVID-19 pandemic to solicit companies for manufacturing facility expansion, including for production of vaccine vials, and to acquire therapeutics. While this authority is set to expire in September 2022, legislation may be enacted to extend the authority. Technology investment agreements. DOD used technology investment agreements to obtain needed items, such as vials and syringes, as well as for fill-finish work-the process of sealing bulk quantities of vaccines into sterile containers. 12 A technology investment agreement is used to stimulate or support research to foster the best technologies for future defense needs. 13 According to DOD, technology investment agreements 11See National Defense Authorization Act for Fiscal Year 2017, Pub. L. No. 114-328, § 879, 130 Stat. 2000, 2312-13 (2016) (codified at 10 U.S.C. § 2303 note). The authority for this pilot program will expire on September 30, 2022. 12Experts from three pharmaceutical industry groups we interviewed for a previous report said there was a shortage of facilities with capacity to handle fill-finish manufacturing, which could lead to production bottlenecks. See GAO COVID-19: Federal Efforts Accelerate Vaccine and Therapeutic Development, but More Transparency Needed on Emergency Use Authorizations, GAO-21-207 (Washington, D.C.: Nov.17, 2020). 13See 32 C.F.R. Part 37 (2020). Page 40 GAO-22-104453 HHS-DOD CAG Partnership Appendix II: Contract Techniques Used to Obtain COVID-19 Vaccines for the U.S. are appropriate when research objectives are unlikely to be achieved using other types of contract vehicles. Page 41 GAO-22-104453 HHS-DOD CAG Partnership Appendix III: Abridged Assessment of the Appendix III: Abridged Assessment of the Countermeasures Acceleration Group's Countermeasures Acceleration Group's Schedule Schedule Officials from the Department of Health and Human Services (HHS)- Department of Defense (DOD) COVID-19 Countermeasures Acceleration Group (CAG) (formerly known as Operation Warp Speed) created a schedule to help manage, among other things, tasks needed to support the CAG's efforts to develop, manufacture, and distribute COVID-19 vaccines. 1 The GAO Schedule Assessment Guide has identified 10 best practices associated with effective schedule estimating, collapsed into four general characteristics that sound schedules should be: well-constructed, comprehensive, credible, and controlled. 2 The best practices of a well- constructed schedule include: • logical sequencing of all activities-that is, listing the activities in the order in which they are to be logically carried out; • confirming that the schedule has a valid critical path- that is, the longest continuous sequence of activities in a schedule and the path that defines the program's earliest completion date or minimum duration; and • identifying reasonable total float or slack-the amount of time an activity could be delayed before that delay affects the program's overall estimated finish date. Our abridged analysis indicates that as of June 2021, the CAG's schedule partially met one best practice for a well-constructed schedule and minimally met the remaining two, as shown in the table 4 below. For example, we found the schedule was missing a significant number of dependent logical links between related activities. As of June 2021, around 41 percent of remaining incomplete activities in the schedule- 1Schedules typically define when and how long work will occur and how each activity is related to the others. Scheduling allows program management to decide between possible sequences of activities, determine the flexibility of the schedule according to available resources, predict the consequences of managerial action or inaction on events, and develop contingency plans to mitigate risks. 2A schedule is well-constructed if all its activities are logically sequenced with the most straightforward logic possible. A comprehensive schedule includes all activities for both the government and its contractors necessary to accomplish a program's objectives as defined in the program's work breakdown structure. A schedule is credible if it is horizontally traceable-that is, it reflects the order of events necessary to achieve aggregated products or outcomes. Finally, a schedule is controlled if trained schedulers update it regularly using actual progress and logic-based on information provided by activity owners-to realistically forecast dates for program activities. See GAO, Schedule Assessment Guide: Best Practices for Project Schedules, GAO-16-89G (Washington, D.C.: Dec. 22, 2015). Page 42 GAO-22-104453 HHS-DOD CAG Partnership Appendix III: Abridged Assessment of the Countermeasures Acceleration Group's Schedule including manufacturing potential vaccine booster doses and developing pediatric vaccine doses-were missing a logical link with other activities. According to the GAO Schedule Assessment Guide, unless a schedule identifies all logical dependencies, it will not indicate how activities that fall behind in the schedule will affect succeeding activities that depend on them, or how these delays would affect the overall project schedule. Table 4: Assessment of the CAG's Integrated Master Schedule, June 2021 GAO Schedule Assessment Extent Guide best practicesa incorporated Description Sequencing. Schedules should be planned so that important dates can ◑ b Our analysis indicated that the CAG's integrated master schedule missed a significant number of logical links. In some cases, the schedule did not be met. To do this, activities must be transmit any adjustments for delays in activities deemed necessary to logically sequenced and linked-that complete other needed activities. is, listed in the order in which they For example, the schedule called for a large-scale comparability study for are to be carried out and logically drug substances and products. Though the study was estimated to take 72 linked so that activities that depend days, the schedule showed no effect on expected vaccine completion date on each other can all be completed. if the time frames for this study were extended to as long as 300 days. The purpose of a logical relationship, or dependency, is to depict the Without the appropriate predecessor and successor links, the schedule did sequence in which activities occur. not indicate how activities that fell behind early in the schedule would have affected later activities that depended on them or the overall project schedule. Additionally, in these instances, the schedule would not have provided confidence to leadership regarding the schedule's dates and indications of key activities that had to be completed. Critical path. Schedules should identify a critical path-that is, the ◔ c Our analysis found that the sequence of activities through the schedule did not result in a valid critical path because, among other issues, it was not sequence of activities that continuous and was missing logic links to other activities. That is, we could determines the program's earliest not validate the sequence of critical activities necessary for the CAG to completion date. The critical path complete a specified goal within its stated timelines, including the vaccine focuses the team's energy and milestones. management's attention on the For example, a critical activity regarding a vaccine company's clinical work activities that will lead to the did not have a predecessor activity and therefore the schedule did not project's success. show what other activities would have needed to be completed before this work have begun. Without a valid critical path, leadership would not have been able to use the schedule to provide realistic timeline estimates, or identify the downstream effects of any changes that occur. Total float. Schedules should identify reasonable total float-that ◔ d Our analysis identified an unreasonable amount of flexibility in the schedule, so that some activities could have been delayed weeks, months, is, the amount of time a predecessor or years before the schedule would have indicated to leadership an overall task can miss its finish date before delay in milestones. For example, a task which began phase 3 clinical trials the delay affects the estimated finish for a vaccine candidate could have been delayed 667 days before date. transferring the delay to the vaccine milestone. Incorrect float estimates may result in an invalid critical path, such that the schedule would not have given reliable indicators of how leadership could have shifted resources to support other critical activities, which could have ultimately delayed completion of those activities. Legend: ● = fully met, ◕ = substantially met, ◑ = partially met, ◔ = minimally met, ○ = not met Source: GAO analysis of the integrated master schedule created by the Department of Health and Human Services (HHS)-Department of Defense (DOD) COVID-19 Countermeasures Acceleration Group (CAG). | GAO-21-104453. Page 43 GAO-22-104453 HHS-DOD CAG Partnership Appendix III: Abridged Assessment of the Countermeasures Acceleration Group's Schedule Note: For the best practice assessment described here, we defined the five levels as follows: (1) fully met – CAG officials provided complete evidence that satisfied the entire criterion; (2) substantially met – CAG officials provided evidence that satisfied a large portion of the criterion; (3) partially met – CAG officials provided evidence that satisfied about half of the criterion; (4) minimally met – CAG officials provided evidence that satisfied a small portion of the criterion; and (5) not met – CAG officials provided no evidence that satisfied any of the criterion. a See GAO, Schedule Assessment Guide: Best Practices for Project Schedules, GAO-16-89G (Washington, D.C.: Dec. 22, 2015). b We evaluated the schedule against sequencing best practice criterion, such as measuring the number of missing logic links. c We evaluated the schedule against critical path best practice criterion, such as assessing its sequencing and comparing it to the longest path in the schedule. d We evaluated the schedule against total float best practice criterion, such as comparing the average and median values of total float in the schedule and comparing it to the remaining duration of the project schedule. When we told CAG officials about our analysis, they explained that their schedule was never intended to be fully completed due to time constraints, and because the CAG's work was already underway. For example, CAG officials explained that the limitations in their schedule were due to their work's constricted timeline and the limited amount of information they had regarding certain activities. Specifically, due to the nature of the COVID-19 pandemic, officials said that they had to set overall time frames within the schedule to finish the work as quickly as possible. They also said that they did not consistently have reliable information about the expected duration of some activities conducted by vaccine companies or by different federal agencies, which in turn affected their ability to develop a more specific schedule. Nonetheless, they said that the schedule-although imperfect-allowed them to better communicate the assumed time frames for activities across the multiple organizations contributing to the CAG's vaccine-related efforts. The integrated master schedule could serve as a critical tool to help HHS officials as they manage remaining vaccine development, manufacturing, and distribution activities following the CAG's dissolution; however the limitations described above could make it less useful for project execution, gauging progress, identifying and resolving potential problems, and promoting accountability. Page 44 GAO-22-104453 HHS-DOD CAG Partnership Appendix IV: Comments from the Appendix IV: Comments from the Department of Health & Human Services Department of Health & Human Services Page 45 GAO-22-104453 HHS-DOD CAG Partnership Appendix IV: Comments from the Department of Health & Human Services Page 46 GAO-22-104453 HHS-DOD CAG Partnership Appendix IV: Comments from the Department of Health & Human Services Page 47 GAO-22-104453 HHS-DOD CAG Partnership Appendix V: Comments from the Department Appendix V: Comments from the Department of Defense of Defense Page 48 GAO-22-104453 HHS-DOD CAG Partnership Appendix V: Comments from the Department of Defense Page 49 GAO-22-104453 HHS-DOD CAG Partnership Appendix V: Comments from the Department of Defense Page 50 GAO-22-104453 HHS-DOD CAG Partnership Appendix VI: GAO Contact and Staff Appendix VI: GAO Contact and Staff Acknowledgments Acknowledgments Alyssa M. Hundrup, (202) 512-7114, hundrupa@gao.gov GAO Contact In addition to the contacts named above Will Simerl (Assistant Director), Staff Jeffrey Mayhew (Analyst in Charge), Darnita Akers, Amy Andresen, Brian Acknowledgments Bothwell, La Sherri Bush, Michael Dickens, Anna Irvine, Katheryn Summers Hubbell, Jason Lee, and Patrick Netherclift made key contributions to this report. Other contributors to this report were Nora Adkins, Jennie Apter, Kaitlin Farquharson, Lori Fields, Miranda Riemer, Ethiene Salgado-Rodriguez, and Sarah Veale. Page 51 GAO-22-104453 HHS-DOD CAG Partnership Related GAO Products Related GAO Products COVID-19: Agencies Are Taking Steps to Improve Future Use of Defense Production Act Authorities. GAO-22-105380. Washington, D.C.: December 16, 2021. COVID-19: HHS Agencies' Planned Reviews Of Vaccine Distribution and Communication Efforts Should Include Stakeholder Perspectives. GAO-22-104457. Washington, D.C.: November 4, 2021. COVID-19: Additional Actions Needed to Improve Accountability and Program Effectiveness of Federal Response. GAO-22-105051. Washington, D.C.: October 27, 2021. COVID-19: Continued Attention Needed to Improve Federal Preparedness, Response, and Service Delivery and Enhance Program Integrity. GAO-21-551. Washington, D.C.: July 19, 2021. COVID-19: Efforts to Increase Vaccine Availability and Perspectives on Initial Implementation. GAO-21-443. Washington, D.C.: April 14, 2021. COVID-19: Sustained Federal Action Is Crucial as Pandemic Enters Its Second Year. GAO-21-387. Washington, D.C.: March 31, 2021. Operation Warp Speed: Accelerated COVID-19 Vaccine Development Status and Manufacturing Challenges. GAO-21-319. Washington, D.C.: February 11, 2021. COVID-19: Critical Vaccine Distribution, Supply Chain, Program Integrity, and Other Challenges Require Focused Federal Attention. GAO-21-265. Washington, D.C.: January 28, 2021. COVID-19: Urgent Actions Needed to Better Ensure an Effective Federal Response. GAO-21-191. Washington, D.C.: November 30, 2020. COVID-19: Federal Efforts Accelerate Vaccine and Therapeutic Development, but More Transparency Needed on Emergency Use Authorization. GAO-21-207. Washington, D.C.: November 17, 2020. COVID-19: Federal Efforts Could Be Strengthened by Timely and Concerted Actions. GAO-20-701. Washington, D.C.: September 21, 2020. COVID-19: Opportunities to Improve Federal Response and Recovery Efforts. GAO-20-625. Washington, D.C.: June 25, 2020. Page 52 GAO-22-104453 HHS-DOD CAG Partnership Related GAO Products Influenza Pandemic: Lessons from the H1N1 Pandemic Should Be Incorporated into Future Planning, GAO-11-632. Washington, D.C.: June 27, 2011. (104453) Page 53 GAO-22-104453 HHS-DOD CAG Partnership The Government Accountability Office, the audit, evaluation, and investigative GAO's Mission arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is Obtaining Copies of through our website. Each weekday afternoon, GAO posts on its website newly GAO Reports and released reports, testimony, and correspondence. You can also subscribe to GAO's email updates to receive notification of newly posted products. Testimony Order by Phone The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, https://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Connect with GAO Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts. Visit GAO on the web at https://www.gao.gov. Contact FraudNet: To Report Fraud, Website: https://www.gao.gov/about/what-gao-does/fraudnet Waste, and Abuse in Automated answering system: (800) 424-5454 or (202) 512-7700 Federal Programs A. Nicole Clowers, Managing Director, ClowersA@gao.gov, (202) 512-4400, U.S. Congressional Government Accountability Office, 441 G Street NW, Room 7125, Washington, Relations DC 20548 Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Stephen J. Sanford, Managing Director, spel@gao.gov, (202) 512-4707 Strategic Planning and U.S. Government Accountability Office, 441 G Street NW, Room 7814, External Liaison Washington, DC 20548 Please Print on Recycled Paper.