Department of Health and Human Services OFFICE OF INSPECTOR GENERAL CDC'S INTERNAL CONTROL WEAKNESSES LED TO ITS INITIAL COVID-19 TEST KIT FAILURE, BUT CDC ULTIMATELY CREATED A WORKING TEST KIT Inquiries about this report may be addressed to the Office of Public Affairs at Public.Affairs@oig.hhs.gov. Christi A. Grimm Inspector General October 2023 A-04-20-02027 Office of Inspector General https://oig.hhs.gov The mission of the Office of Inspector General (OIG) is to provide objective oversight to promote the economy, efficiency, effectiveness, and integrity of the Department of Health and Human Services (HHS) programs, as well as the health and welfare of the people they serve. Established by Public Law No. 95-452, as amended, OIG carries out its mission through audits, investigations, and evaluations conducted by the following operating components: Office of Audit Services. OAS provides auditing services for HHS, either by conducting audits with its own audit resources or by overseeing audit work done by others. The audits examine the performance of HHS programs, funding recipients, and contractors in carrying out their respective responsibilities and provide independent assessments of HHS programs and operations to reduce waste, abuse, and mismanagement. Office of Evaluation and Inspections. OEI's national evaluations provide HHS, Congress, and the public with timely, useful, and reliable information on significant issues. To promote impact, OEI reports also provide practical recommendations for improving program operations. Office of Investigations. OI's criminal, civil, and administrative investigations of fraud and misconduct related to HHS programs and operations often lead to criminal convictions, administrative sanctions, and civil monetary penalties. OI's nationwide network of investigators collaborates with the Department of Justice and other Federal, State, and local law enforcement authorities. OI works with public health entities to minimize adverse patient impacts following enforcement operations. OI also provides security and protection for the Secretary and other senior HHS officials. Office of Counsel to the Inspector General. OCIG provides legal advice to OIG on HHS programs and OIG's internal operations. The law office also imposes exclusions and civil monetary penalties, monitors Corporate Integrity Agreements, and represents HHS's interests in False Claims Act cases. In addition, OCIG publishes advisory opinions, compliance program guidance documents, fraud alerts, and other resources regarding compliance considerations, the anti-kickback statute, and other OIG enforcement authorities. Notices THIS REPORT IS AVAILABLE TO THE PUBLIC at https://oig.hhs.gov Section 8M of the Inspector General Act, 5 U.S.C. App., requires that OIG post its publicly available reports on the OIG website. OFFICE OF AUDIT SERVICES FINDINGS AND OPINIONS The designation of financial or management practices as questionable, a recommendation for the disallowance of costs incurred or claimed, and any other conclusions and recommendations in this report represent the findings and opinions of OAS. Authorized officials of the HHS operating divisions will make final determination on these matters. Report in Brief Date: October 2023 Report No. A-04-20-02027 Why OIG Did This Audit CDC's Internal Control Weaknesses Led to Its Initial HHS is the U.S. Government's principal agency for protecting the COVID-19 Test Kit Failure, but CDC Ultimately Created health of all Americans. Included in a Working Test Kit this role is the responsibility to respond to pandemics. The first cases of COVID-19 were identified in What OIG Found China and reported to the Centers for Ultimately, CDC developed a viable COVID-19 test kit within 2 months of China Disease Control and Prevention (CDC) publishing the genome sequence of the novel virus that caused the COVID-19 in December 2019, and the first case outbreak. However, some of the initial COVID-19 test kits that CDC developed in the United States was officially and distributed to public health laboratories could not be verified by the public diagnosed on January 20, 2020. In health laboratories, and CDC initially identified multiple potential causes of this mid-January 2020, CDC began failure. We identified weaknesses in CDC's COVID-19 test kit development developing a diagnostic test kit to processes and the agencywide laboratory quality processes that may have detect COVID-19. contributed to the failure of the initial COVID-19 test kits. Our objective was to review CDC's Without effective internal controls, CDC may: (1) experience delays in the process for developing the COVID-19 development of test kits when responding to future public health emergencies; test kits and determine factors that (2) not identify problems in a timely manner when developing test kits; and (3) contributed to the initial COVID-19 risk damaging public trust, which could undermine its ability to accomplish its test kit failure. mission. How OIG Did This Audit What OIG Recommends and CDC Comments Using Federal internal control We made several recommendations to CDC, including that CDC: (1) create standards, we assessed the policies and procedures for developing test kits, (2) ensure that the recently effectiveness of CDC's controls finalized Graduated Response Framework addresses our report findings, (3) throughout the test kit development develop and implement documented processes to ensure that adequate staffing process. We interviewed HHS and laboratory space can be obtained for future responses, (4) re-evaluate the personnel who participated in the Incident Management System structure at all levels of CDC's response development of the COVID-19 test framework and integrate positions or roles and responsibilities that provide kits, focusing mainly on the CDC effective oversight of a laboratory-based response effort, (5) implement a CDC- laboratories that developed the test wide laboratory document control system, and (6) ensure that all infectious kits. We requested and reviewed disease laboratories implement and periodically evaluate a laboratory quality available documentation such as management system. white papers, final and draft policies In response to our draft report, CDC neither concurred nor nonconcurred with and procedures, strategies, and our recommendations. Instead, CDC discussed actions it has taken or plans to emails that were produced between take to implement our recommendations. CDC stated that it developed a January and March 2020. We also Laboratory Quality Plan to address issues of quality and oversight. CDC also reviewed the results of CDC's Root stated that it published documentation outlining laboratory functions during an Cause Analysis, in addition to HHS's emergency response, evaluated the operating effectiveness of its internal Office of the General Counsel's controls, and elevated oversight of emergency response efforts to ensure report of its investigation into the accountability. Finally, CDC stated that it is continuing to work on the initial test kit failure. implementation of the electronic quality management system, which facilitates laboratory quality activities. We commend CDC on the actions it has taken or is taking to address our recommendations. The full report can be found at https://oig.hhs.gov/oas/reports/region4/42002027.asp. TABLE OF CONTENTS INTRODUCTION ............................................................................................................................... 1 Why We Did This Audit ....................................................................................................... 1 Objective ............................................................................................................................. 1 Background ......................................................................................................................... 1 CDC's Mission, Authority, and Organizational Structure ........................................ 1 CDC's Processes for Responding to Public Health Threats and Developing Test Kits ...................................................................................... 3 CDC's Timeline for COVID-19 Test Kit Development .............................................. 6 How We Conducted This Audit ........................................................................................... 7 FINDINGS ......................................................................................................................................... 8 CDC Ultimately Developed a Viable COVID-19 Test Kit ...................................................... 9 CDC's Test Kit Development Process Had Control Weaknesses......................................... 9 CDC Lacked Guidance and RVD Lab Lacked Policies and Procedures for Developing a Test Kit ................................................................... 9 CDC Lacked an Established Process, Which Led to Minimal Resources Allocated to the Lead Laboratory ....................................................................................... 11 Escalation of COVID-19 Response Efforts Exposed a Gap in Oversight of Test Kit Development..................................................................................... 14 Agencywide Control Weaknesses Impacted CDC's Test Kit Development Process ......... 16 CDC Did Not Have an Agencywide Laboratory Document Control System.......... 16 CDC Lacked an Agencywide Laboratory Quality Management System................ 17 CONCLUSION ................................................................................................................................. 19 RECOMMENDATIONS ................................................................................................................... 21 CENTERS FOR DISEASE CONTROL AND PREVENTION COMMENTS AND OIG RESPONSE ............. 21 APPENDICES A: Scope and Methodology .............................................................................................. 23 B: Flow Chart of CDC's Process for Developing an Emergency Test Kit ........................... 26 CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) C: Centers for Disease Control and Prevention Comments.............................................. 27 CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) INTRODUCTION WHY WE DID THIS AUDIT The Department of Health and Human Services (HHS) is the U.S. Government's principal agency for protecting the health of all Americans. Included in this role is a charge to respond to pandemics. COVID-19 has created extraordinary challenges for the delivery of health care and human services to the American people. As the oversight agency for HHS, the Office of Inspector General (OIG) oversees HHS's COVID-19 response and recovery efforts. Work that is focused on CDC's experience with developing diagnostic tests for COVID-19 is of interest to stakeholders and aligns with OIG's COVID-19 oversight priorities as outlined in our COVID-19 response strategic plan.1, 2 The first cases of COVID-19 were identified in China and reported to the Centers for Disease Control and Prevention (CDC) in December 2019, and the first case in the United States was officially diagnosed on January 20, 2020. In mid-January 2020, CDC began developing a diagnostic test kit to detect COVID-19. In February 2020, CDC began sending test kits to public health laboratories (PHLs). However, within days of receiving these test kits, some PHLs were unable to validate some of the COVID-19 test kits. CDC began to troubleshoot the failed test kit and was able to release a viable test kit in late February 2020. OBJECTIVE Our objective was to review CDC's process for developing the COVID-19 test kits and determine factors that contributed to the initial COVID-19 test kit failure. BACKGROUND CDC's Mission, Authority, and Organizational Structure CDC's mission is to protect the Nation from health, safety, and security threats, both foreign and domestic. CDC fights disease and supports communities and citizens to do the same. To accomplish its mission, CDC conducts critical scientific research, provides health information that protects our Nation against expensive and dangerous health threats, and responds when those threats arise. 1 OIG's COVID-19 response strategic plan and information on OIG's oversight activities can be accessed at https://oig.hhs.gov/coronavirus/index.asp. Other OIG work looking at various aspects of the COVID-19 response includes an evaluation of FDA's EUA process. See FDA Repeatedly Adapted Emergency Use Authorization Policies To Address the Need for COVID-19 Testing (OEI-01-20-00380) September 2022. 2 Letter dated Mar. 19, 2020. Available online at https://www.help.senate.gov/imo/media/doc/03192020%20HHS%20IG%20Testing%20Request%20FINAL.pdf. Accessed on May 9, 2023. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 1 The Public Health Service Act authorizes HHS to identify and respond to emerging infectious diseases and public health emergencies. It authorizes the Secretary of Health and Human Services to determine that a public health emergency exists, lead the Federal public health and medical response to a public health emergency, and assist States and PHLs in responding to a public health emergency. CDC plays a key role in carrying out these activities, including the development and distribution of test kits to PHLs to enable PHLs to independently test for a pathogen. CDC is made up of Centers, Institutes, and Offices (CIOs) that carry out its mission. Some of the CIOs contain laboratories that specialize in researching, monitoring, and responding to infectious diseases or pathogens. When a disease outbreak involving an infectious disease or pathogen is detected, these laboratories may develop a test kit, especially when a test kit does not exist or when existing test kits are inadequate. 3 Two Centers, the National Center for Immunization and Respiratory Diseases (NCIRD) and the National Center for Emerging and Zoonotic Infectious Diseases, were primarily involved in the development of the COVID-19 test kit. (See Figure 1.) Figure 1: CDC Centers Primarily Involved in COVID-19 Test Kit Development 3 CDC's COVID-19 test kit was designed to identify infection in individuals and was performed when a person had signs or symptoms consistent with COVID-19 or was asymptomatic but had a recent known or suspected exposure to someone with suspected or confirmed SARS-CoV-2 infection. CDC works to make test kits available to PHLs when it is anticipated that there will be a need for widespread testing of a novel or emerging pathogen for which there are no FDA-approved or cleared diagnostic tests. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 2 The following three laboratories within the two Centers were involved in developing the COVID- 19 test kit: • Respiratory Virus Diagnostic (RVD) Lab-The lead laboratory that largely developed the test kit components and the strategy for producing and distributing the test kit. RVD Lab also developed internal policies and procedures related to the development of the test kit, conducted quality control on the test kit, and created documentation for regulatory approval, as well as instructions to PHLs. • Biotechnology Core Facility Branch (CORE) Lab-Manufactured the primers 4 and probes 5 (together referred to as reagents) for the test kit, as well as the small pieces of deoxyribonucleic acid (DNA) material used by RVD Lab to replicate the SARS-CoV-2 virus, which causes COVID-19. • Reagent Diagnostic Services Branch (RDSB) Lab-Labeled and packaged the test kit components it received from RVD Lab. Once RDSB Lab prepared the test kits, it sent the test kits to RVD Lab for quality control before CDC distributed them to the PHLs. RDSB Lab was not involved in the creation of test kit material. CDC's Processes for Responding to Public Health Threats and Developing Test Kits The process for developing a test kit in response to an emerging infectious disease is part of CDC's process for responding to a public health threat. The following sections include brief descriptions of these processes. CDC's Process for Responding to a Public Health Threat CDC relies on a worldwide network of offices and other international organizations to identify public health threats. CDC created the draft Graduated Response Framework (GRF) to aid CIOs when responding to public health threats.6 Although in draft form, CDC used this framework to focus emergency response efforts at the different levels of agency engagement. The GRF is a 4 Primers are short single-stranded pieces of DNA that bind with specific regions of the viral genome and thus define the specific region to be amplified. Primers serve as the starting point of DNA synthesis by initiating DNA replication once they are inserted into a cell. 5 Probes are single-stranded pieces of DNA that are chemically or radioactively labeled and bind to and thus detect the amplified target of the viral genetic material. The probe is used to detect the presence of specific DNA within a sample. 6 The GRF was in draft form before and during our audit. The intent is for the GRF to be used across CDC enabling better response to future public health threats. As of October 2021, when we discussed our findings with CDC, the GRF was still in draft. The draft GRF was used as an operational document at CDC and was followed during response efforts. CDC provided documentation to show how the GRF was evaluated when implemented during its response to cases of Acute Flaccid Myelitis in 2018. In addition, processes within the draft GRF were followed during the COVID-19 response. Since our audit work was completed, CDC finalized the GRF effective April 2022, as well as an Annex to the GRF, effective July 2022, related to managing laboratory functions during an emergency response. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 3 tiered response process that CDC can follow during a public health emergency and includes the following three levels: • Program-Led Response: Used for responding to a routine health threat as part of CDC's day-to-day operations. For example, a limited event, such as a foodborne outbreak, would be handled through a Program-Led Response. • Center-Led Response: Used when a public health threat requires increased operations or when normal program resources will be exceeded for a sustained length of time (such as during CDC's response to the 2019 measles outbreak). A center-led Incident Management System (IMS) is used to lead and conduct CDC response activities under a Center-Led Response. 7 • Agency-Wide Response: Used when a domestic or international public health response calls for a unified agencywide IMS response (as was the situation during the H1N1 and Zika responses). During an Agency-Wide Response, CDC activates its Emergency Operations Center (EOC). 8 Included in the GRF is the Operations Handbook. The Operations Handbook contains policies and procedures for a variety of response activities including, for example, protocols for emergency response clearance and discussion of increased operations and the impact that has on aspects such as planning and logistics. 9 When a response is agencywide, CDC follows its All-Hazards Plan (AHP) for responding to a public health threat. The AHP describes CDC's basic principles, organizational structure, and responsibilities during a public health emergency response. CDC's Process for Developing a Test Kit Historically, CDC has led the development of diagnostic tests for new diseases and has led the 7 An IMS outlines responders' specific roles and responsibilities during an event, providing a common framework for organizations to work together. In an IMS, each person is assigned a role and follows the designated command structure. An incident's level of complexity dictates which roles are activated. 8 When activated, the EOC operates continuously and enables CDC to quickly identify, respond to, and address public health questions. The EOC also provides a facility from which CDC can respond to any public health threat, whether domestic or global. 9 Emergency response clearance is a customized process of rapid review and approval of prevention messages and publications by the appropriate CDC staff members during a CDC EOC response. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 4 distribution of the tests to its network of PHLs. 10, 11 CDC's process for responding to emerging infectious diseases begins with test kit development with the lead laboratory developing the components. The lead laboratory is identified as the laboratory with a portfolio of diseases that includes the emerging infectious disease. The lead laboratory develops a test kit that returns the most accurate results. Before CDC can use the test kit to test human samples for medical diagnosis, the lead laboratory coordinates the verification of performance specifications so that the test kit can be used in CDC's internal Clinical Laboratory Improvement Amendments (CLIA)- certified laboratory. 12 Simultaneously, a small lot of the test kit is manufactured and validated using manufacturing and quality control requirements. If the small lot is validated, CDC submits an application package to the Food and Drug Administration (FDA) for emergency use authorization (EUA) for the test kit. The lead laboratory coordinates the preparation of the EUA application package. However, if the small lot is not validated, the lead laboratory repeats this cycle until it successfully validates the small lot. Once a small lot is validated, either a CDC laboratory or a contractor produces a large lot of test kits following the same manufacturing and quality control requirements under which the small lot was produced. The test kit components are sent to the International Reagent Resource (IRR) for distribution to PHLs. 13 The IRR may not distribute the test kits until FDA authorizes CDC's application for an EUA. 10 At the Federal level, CDC, the National Institutes of Health, and the Department of Defense all have the capability to develop test kits to identify an emerging infectious disease. A non-Federal laboratory (e.g., academic, public health, or commercial laboratories) may also create its own test to identify an emerging infectious disease. 11 Formally called in vitro diagnostic devices, diagnostic tests may be commercially developed and distributed as kits or developed, validated, and used within a single laboratory. See the Congressional Research Service Report, Development and Regulation of Domestic Diagnostic Testing for Novel Coronavirus (COVID-19): Frequently Asked Questions, issued Mar. 9, 2020. 12 All clinical laboratories that conduct testing on humans in the United States must be properly certified by the Centers for Medicare & Medicaid Services for adherence to the CLIA of 1988. The CLIA-certified laboratory at CDC had to approve the COVID-19 test kit package before the CDC laboratory could use it on humans. Similarly, PHLs had to verify the COVID-19 test at each lab before using it on human samples. Until PHLs could verify the CDC COVID-19 test kit, the internal CDC CLIA laboratory was the only laboratory in the United States able to test human samples for COVID-19. 13 CDC established the IRR to provide registered users with reagents, tools, and information for studying and detecting the influenza virus and other pathogens. The IRR is primarily a resource used for procuring pathogen test kit components and for assembling and distributing the test kits for use in public health activities. The IRR's role in the COVID-19 test kit process was to house and distribute the test kits when requested by outside parties. Visit IRR's website online at https://www.internationalreagentresource.org/About/IRR.aspx for more information. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 5 PHLs must verify the performance specifications for the test kit using the instructions provided by CDC before they can use the test kit on clinical specimens. 14, 15 If PHLs cannot verify the test kit, they must notify CDC, and CDC must notify FDA. CDC will then attempt to determine the issue with the test kit and what corrective action needs to be taken. Corrective action may include redeveloping the test kit components, seeking FDA authorization for an updated submission package, or re-evaluating the large lot manufacturing and quality control requirements. CDC repeats this process until the PHLs can verify the test kit. Once the PHLs verify and start using the test kit, CDC or a contractor monitors the performance of the test kit and addresses any issues that may arise. CDC conducts many of these steps concurrently rather than sequentially to expedite the process. 16 (See Appendix B for a flowchart of CDC's process for developing a test kit.) CDC's Timeline for COVID-19 Test Kit Development CDC did not have access to human samples of the COVID-19 virus that it could use to create and verify a COVID-19 test kit. Once China posted the genome sequence of the virus, CDC created and distributed a viable 14 Once an EUA is obtained for a test, CDC typically uses PHLs to verify test kits and provide diagnostic capabilities for the initial phase of a large public health emergency. By necessity, commercial diagnostic manufacturers provide the large-scale diagnostic testing for clinical purposes and to meet the needs of the entire health care system. 15 For test kits to be verified by PHLs, they must meet the expected performance results (positive, negative, or extraction) described in the EUA. 16 For example, quality control testing of different test kit components could be occurring at the same time. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 6 COVID-19 test kit in 2 months, as shown in Figure 2 on the previous page. 17 HOW WE CONDUCTED THIS AUDIT Our audit period covered January through March 2020. To conduct this audit, we obtained an understanding of CDC's process for developing the test kits and created a flow chart that documents this process. 18 (See Appendix B.) We focused our audit efforts on reviewing the internal controls involved in this process. To conduct a review of CDC's internal controls, we performed several steps. First, we interviewed CDC and HHS personnel who participated in the development of the COVID-19 test kits. We focused mainly on the CDC laboratories that developed the test kits. Second, we requested and reviewed white papers, final and draft policies and procedures, strategies, and departmental emails and memos that each laboratory supplied. Third, we reviewed the results of CDC's root cause analysis, which sought to determine the cause of CDC's failure to detect problems with the initial test kit before distribution, in addition to HHS's Office of the General Counsel's report of its investigation into the initial test kit failure. 19 The Standards for Internal Control in the Federal Government (Green Book), published by the Government Accountability Office, sets the internal control standards for Federal entities. 20 The Green Book defines internal control as a process used by management to help an entity achieve its objectives. Specifically, it states that "[i]nternal control comprises the plans, methods, policies, and procedures used to fulfill the mission, strategic plan, goals, and objectives of the entity" (Green Book Overview 1.03). Although there are different ways to present internal control, the Green Book approaches internal control through a hierarchical structure of 5 components and 17 principles, as shown in Appendix A, Figure 3. The Green Book states that a deficiency in the design of internal control exists when: (1) a control is missing or (2) an existing control is not designed properly. A deficiency in the implementation 17 In past public health emergencies caused by infectious disease (e.g., Ebola and Zika), CDC possessed decades of experience refining and developing disease-specific tests. According to CDC, even though it did not have the same level of historical research on coronaviruses, because of advancements in molecular technology, it was able to produce the COVID-19 test kits faster than it had produced test kits in the past. 18 This included reviewing CDC's decision to develop its own COVID-19 test kit instead of using WHO's COVID-19 test kit. Specifically, CDC's test kit and the test kit distributed by the WHO were created simultaneously, and this timing, along with U.S. regulations for test kit approval, factored into CDC's decision. We reviewed the process CDC used to develop a test kit and did not evaluate the scientific decisions CDC made during the COVID-19 pandemic. 19 CDC's root cause analysis titled Unanticipated Failure of the 'CDC 2019-Novel Coronavirus (2019-nCoV) Real-Time RT-PCR Diagnostic Panel' was issued on Mar. 24, 2020, and updated on Oct. 5, 2020. The CDC root cause analysis is an internal, nonpublic report. HHS's Office of the General Counsel's report summarizing the findings of its investigation, issued on June 19, 2020, is also a nonpublic report. 20 Standards for Internal Control in the Federal Government, GAO-14-704G, published September 2014. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 7 of internal control exists when a properly designed control is not correctly implemented (Green Book Overview 3.05). The Green Book provides that, in evaluating operating effectiveness, management determines whether controls were applied at relevant times during the period under evaluation, the consistency with which they were applied, and by whom or by what means they were applied. Specifically, a deficiency in operations exists when a control does not operate as designed or when the person performing the control does not possess the necessary authority or competence to perform the control (Green Book Overview 3.06). Although we evaluated all 5 components and 17 principles of internal control identified in the Green Book for this audit, we only cite specific Green Book internal control principles used to develop our findings in this report. 21 We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix A contains the details of our scope and methodology. FINDINGS Ultimately, CDC developed a viable COVID-19 test kit within 2 months of China publishing the genome sequence of the novel virus that caused the COVID-19 outbreak. However, some of the initial COVID-19 test kits that CDC developed and distributed to PHLs could not be verified by the PHLs, and CDC initially identified multiple potential causes of this failure. We identified weaknesses in CDC's COVID-19 test kit development processes and the agencywide laboratory quality processes that may have contributed to the failure of the initial COVID-19 test kits. Specifically, we identified the following weaknesses in CDC's COVID-19 test kit development processes: • CDC had inadequate policies and procedures for developing a COVID-19 test kit. • A lack of established processes to prioritize the need for personnel and laboratory space led to the allocation of minimal resources to the lead laboratory. • Insufficient oversight of the laboratory-based response existed when CDC's response efforts escalated from a Center-Led Response to an Agency-Wide Response. 21 The five internal control components are the control environment, risk assessment, control activities, information and communication, and monitoring. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 8 In addition, the following weaknesses in CDC's agencywide laboratory quality processes may have contributed to the failure of the initial COVID-19 test kits: • the lack of a laboratory document control system and • the absence of a laboratory quality management system (QMS). Without effective internal controls, CDC may: (1) experience delays in the development of test kits when responding to future public health emergencies; (2) not identify problems in a timely manner when developing test kits; and (3) risk damaging public trust, which could undermine its ability to accomplish its mission. CDC ULTIMATELY DEVELOPED A VIABLE COVID-19 TEST KIT Ultimately, CDC developed a viable COVID-19 test kit and delivered it to PHLs to test patient samples for COVID-19 within 2 months of the World Health Organization's (WHO's) identification of an unknown virus outbreak. Despite facing multiple challenges during the process of developing the COVID-19 test kit, such as a lack of resources, pressures from within CDC to create the test kit quickly, and a lack of readily available information regarding this coronavirus, RVD Lab's lead scientist and scientific team developed and distributed COVID-19 test kits to 110 PHLs. In addition, CDC published relevant information about the test kit creation so that others could also develop a test kit. The COVID-19 test kits produced and distributed by RVD Lab were used in the United States for more than a year but have since evolved. CDC'S TEST KIT DEVELOPMENT PROCESS HAD CONTROL WEAKNESSES We identified weaknesses in CDC's COVID-19 test kit development process that may have contributed to the failure of the initial COVID-19 test kits. CDC Lacked Guidance and RVD Lab Lacked Policies and Procedures for Developing a Test Kit The Green Book states that management should design control activities to achieve objectives and respond to risks (Principle 10). Specifically, the Green Book states that management should design control activities in response to the entity's objectives and related risks to achieve an effective internal control system. Control activities are the policies, procedures, techniques, and mechanisms that enforce management's directives to achieve the entity's objectives and address related risks (Attribute 10.02). Although CDC had agency guidance documents related to its response structure during a public health emergency (e.g., the draft GRF and the AHP), these documents do not address the development of a test kit. Similarly, RVD Lab did not have policies and procedures in place for developing a test kit. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 9 CDC Guidance The draft GRF provided guidance for CDC when CDC escalated its response posture. Specifically, it provided detailed guidance for a Center-Led Response. The draft GRF in place during our audit period, which also contained the Operations Handbook, outlined the essential elements, protocols, tools, and services available to a Center but did not address developing test kits. The AHP guides CDC during an Agency-Wide Response. It outlines levels of authority, organizational relationships, and resources available to support CDC's mission. It also identifies roles within the IMS and identifies CDC responders' responsibilities for carrying out actions during a public health emergency. In addition, the AHP includes a checklist of the IMS task functions and responsibilities by division within CDC (e.g., internal and external communications, laboratory operations, financial and administrative support, personnel support, logistics support, strategic direction, risk management, and vaccine support). However, the AHP, including the checklist, does not address developing test kits. RVD Laboratory Policies Generally, CDC's process for developing a test kit is predicated on whether it has an existing test kit for or knowledge of a prior virus. The SARS-CoV-2 virus, which causes COVID-19, was previously unknown to CDC, and therefore CDC could not rely on any familiarity with the virus. Although there are standard steps scientists use for developing and manufacturing a test kit for any virus, the nature of the virus dictates many of the steps scientists take. RVD Lab's lead scientist started with the manufacturing procedures from the laboratory within the Influenza Division (referred to in this report as "the influenza laboratory") as a guide. However, RVD Lab could not use these procedures entirely to develop a coronavirus test kit because they were specific to an influenza virus and a different laboratory. Before the COVID-19 pandemic, RVD Lab had a small staff of scientists who researched respiratory viral diseases other than influenza. Unlike the influenza laboratory, which produces an annual flu test, RVD Lab was a research-focused laboratory that was not set up to develop and manufacture test kits and therefore had no policies and procedures for developing and manufacturing test kits. When the COVID-19 pandemic began, RVD Lab also could not use CDC agency documents such as the GRF and AHP for guidance in developing the test kits because neither contained information related to test kit development. Without established policies and procedures at RVD Lab, the lead scientist had to create the processes and procedures for developing the COVID-19 test kit while the test kit was being developed. The lack of policies and procedures created a high-risk environment that allowed for incomplete processes to occur. For example, the test kits were distributed to PHLs before completing the quality control process to ensure all test kits were viable. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 10 CDC Lacked an Established Process, Which Led to Minimal Resources Allocated to the Lead Laboratory The Green Book states that management should design control activities to achieve objectives and respond to risks (Principle 10). Control activities are the policies, procedures, techniques, and mechanisms (e.g., segregation of duties) used by an entity to achieve its objectives and address related risks (Attributes 10.02 and 10.03). Controls can be either preventive or detective. Preventive controls exist to prevent the entity from failing to achieve objectives or address risks, whereas detective controls discover when risks are not being addressed and correct the actions to enable the entity to achieve its objectives or address its risks (Attribute 10.04). In designing the control activities, management should consider segregation of duties, and if lacking sufficient personnel or other resources, the control activities should be designed in such a way as to minimize errors in the operational process (Attribute 10.14). CDC, however, did not have an established process in place to prioritize the need for personnel and laboratory space when RVD Lab, a smaller laboratory not traditionally involved in the development of a test kit, needed assistance and additional laboratory space to respond to the increased responsibility associated with a public health emergency. Lack of Personnel RVD Lab's disease portfolio consisted mainly of Middle East respiratory syndrome (MERS) and severe acute respiratory syndrome (SARS). MERS occurs mainly in the Middle East, and until 2020, there had been no known cases of SARS since 2004. 22 Before the COVID-19 pandemic, RVD Lab had a small staff of two to three scientists and an annual budget of approximately $200,000 to $300,000 per year. Because RVD Lab was a research-focused laboratory, not a production laboratory, it was able to maintain operations with a small staff and budget. When RVD Lab was tasked with developing a COVID-19 test kit in early January 2020, RVD Lab's lead scientist developed a strategy to accomplish this task and presented this strategy to NCIRD management on January 15, 2020. In that presentation, RVD Lab's lead scientist said that to develop a test kit, the laboratory would need approximately 20 additional personnel with experience in project management, regulatory affairs, quality assurance, quality control, manufacturing, laboratory testing, and information management. The lead scientist requested assistance mainly from staff in the Influenza Division who were familiar with the manufacturing and quality control processes of developing a test kit and had knowledge of and experience with verifying test specifications and applying for FDA approval. Ultimately, RVD Lab received assistance from approximately 13 full- and part-time staff members from various areas within CDC, and not the 20 dedicated staff members requested. Because RVD Lab's lead scientist did not receive the staff identified as necessary to develop the test kit, the lead scientist not only managed but also participated in all test kit development 22 MERS is an illness caused by a coronavirus called MERS-CoV. SARS is a viral respiratory illness caused by a coronavirus called SARS-associated coronavirus (SARS-CoV). CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 11 processes. In addition, when the initial test kit failed at some PHLs, the lead scientist was also responsible for troubleshooting and correcting the problem. Troubleshooting efforts consisted of identifying whether contamination of the test kit existed, recreating test kit components, and working with FDA to obtain an updated EUA for the corrected test kit. Some of the CDC personnel we interviewed stated that RVD Lab did not appear to have enough support from across CDC, and others recalled RVD Lab's lead scientist calling and pleading with individuals in other CIOs to help the test kit development team. CDC personnel stated that they believed that some CIOs did not assist RVD Lab because they were also trying to "keep their own teams together" during the public health emergency. Inadequate Laboratory Workspace When RVD Lab began developing the COVID-19 test kit using the genome sequence posted by China in early January 2020, CDC had no samples of the coronavirus to use to verify the accuracy of the test kit it was creating. To obtain the viral material needed for use in verifying the test kit, RVD Lab had to rely on either CORE Lab or outside manufacturers to create the viral material. RVD Lab initially requested the viral material from CORE Lab. 23 Because laboratory space was limited, and CORE Lab was already in the process of producing the reagents for the COVID-19 test kit, RVD Lab made inquiries to outside manufacturers to create the viral material needed to verify the test kit's accuracy. These manufacturers responded to CDC and indicated that they were unable to assist CDC for at least 2 weeks because they could not stop production on their then-current obligations. Because of the urgent need for a test kit, RVD Lab returned to CORE Lab for the viral material. RVD Lab, which was under pressure to quickly create a test kit for the emerging health threat, insisted that CORE Lab deviate from its usual practices of segregating these two activities and fulfill orders for both reagents and viral material. At the risk of possible contamination, CORE Lab would not usually produce both the viral material and test kit reagents simultaneously in the same laboratory. However, CORE Lab ultimately agreed to make the viral material that RVD Lab needed to verify the accuracy of the test kits. Recognizing that conducting these concurrent processes allowed for the possibility of contamination, CORE Lab implemented strict workflow protocols to mitigate this possibility. 24 CORE Lab then created both the viral material and the reagents for RVD Lab to use for the development of the test kit. Once these initial test 23 CDC's CORE Lab can provide various specialized expert services to researchers in the CDC scientific community. Specifically, CORE Lab fulfills orders from members within the CDC scientific community when they need something for a project. CORE Lab orders are completed based on the specialized instrumentation, platforms, and expertise needed in various areas of biotechnology. 24 CORE Lab has a standard operating procedure that describes safe practices for controlling the potential for biological contaminants in laboratories. This procedure covers surface decontamination and disinfection practices, separated workflow design, and reusable laboratory coat laundering procedures. In addition, CORE Lab instituted strict workflow protocols such as using separate machines in separate rooms, not conducting quality control on the viral strands (to ensure that the viral strands could not mix with reagents), and always working forward through a laboratory process, not backward or concurrent. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 12 kits were shipped to PHLs, some PHLs indicated they could not verify the test kits. CDC scientists then conducted troubleshooting measures to determine the cause of these issues and could not rule out whether contamination at CORE Lab was responsible. CDC conducted an official internal root cause analysis to determine the cause of its failure to detect the initial test kit verification problem before distribution. Its analysis did not determine whether a process error or contamination was at fault for the test kit failure; however, based on our interviews with CDC personnel, contamination could not be ruled out.25 Because CDC's documents to guide response activities did not contain a process to allocate resources, such as personnel and lab space, to an area of need during a public health crisis, both RVD Lab and CORE Lab lacked the resources needed to develop the test kit. Specifically, segregation of duties was lacking by both RVD Lab's lead scientist and other scientists involved in the development of the test kit, as well as CORE Lab personnel due to lack of laboratory space. This caused CORE Lab personnel to conduct processes that deviated from their usual practices. 26 The lack of laboratory space also placed CORE Lab scientists in a position in which they had to produce both the viral material and test kit reagents simultaneously and in the same lab space, putting the test kit development at greater risk. Various CDC personnel stated that they experienced daily pressure to develop a test kit despite being understaffed and lacking available space. They stated there was a daily emphasis on the "National emergency" from all levels of CDC leadership to create the test kit. Some stated that constant requests from NCIRD, CDC, and others for information and updates created a great deal of pressure. CDC personnel stated that there was an understood "make it happen" attitude to create the test because of the urgent need of the public health emergency. They emphasized that those involved in the test kit development worked "around the clock" with "no downtime" while others within CDC did not appear to work with any "urgency." For example, they stated that some people were out of the office when certain tasks needed to be accomplished during the test kit development process. CDC personnel acknowledged that there are many reasons for the pressure they felt, such as the expectation that the agency was "leading the charge" in developing the test kit and in managing the crisis. In addition, personnel said they felt pressure trying to live up to the internal culture that staff are dedicated to helping the public and that everyone was looking to CDC to resolve the problem. 25 Since our audit fieldwork concluded, CDC published a report on Dec. 15, 2021, of its findings from an internal investigation of the root cause of the verification failure the PHLs were reporting. CDC scientists concluded that contamination during the manufacturing of the test kits caused a false positive test result. CDC further concluded that a possible design flaw also caused a false positive test result. This report is available online at https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0260487#sec007. Accessed on Dec. 6, 2022. 26 Segregation of duties includes separating the responsibilities for authorizing transactions, processing and recording them, reviewing the transactions, and handling any related assets so that no one individual controls all key aspects of an event. In the context of the COVID-19 test kit, RVD Lab's lead scientist was responsible for the test kit process, from creation to distribution. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 13 Escalation of COVID-19 Response Efforts Exposed a Gap in Oversight of Test Kit Development The Green Book states that the oversight body should oversee the entity's internal control system (Principle 2). Specifically, the entity or applicable body also considers the expertise needed by members to oversee, question, and evaluate management (Attribute 2.06). The Green Book states that management should establish an organizational structure, assign responsibility, and delegate authority to achieve the entity's objectives (Principle 3). The organizational structure enables the entity to plan, execute, control, and assess whether organizational objectives are being met (Attribute 3.02). The Green Book also states that management should design control activities to achieve objectives and respond to risks (Principle 10). Management may design a variety of transaction control activities for operational processes, which may include verifications, reconciliations, authorizations and approvals, physical control activities, and supervisory control activities (Attribute 10.10). CDC's escalation of the COVID-19 response efforts exposed a gap in oversight of RVD Lab and its development of the COVID-19 test kit while under both a Center-Led Response and an Agency- Wide Response. CDC's Center-Led Response On December 31, 2019, CDC received notification of an outbreak that was categorized as "an epidemic of pneumonia of unknown etiology centralizing . . . in Wuhan, China." Because CDC did not know the specific nature of the outbreak, both the Influenza Division and the Division of Viral Diseases within NCIRD were considered likely to have a role in responding to the emerging public health crisis. In line with the draft GRF, NCIRD officially activated its IMS response on January 7, 2020. At NCIRD's initial IMS meeting on January 7, 2020, the Respiratory Viruses Branch was tasked with taking the lead because, at that time, the virus was categorized as a respiratory virus. On the same day, NCIRD's director assigned RVD Lab's lead scientist the responsibility of creating a test kit development strategy. The lead scientist presented this development strategy at the NCIRD IMS daily briefing on January 15, 2020. While leading the COVID-19 response, NCIRD management provided minimal oversight of the test development process. Although RVD Lab's lead scientist's test kit development strategy included a plan for developing a COVID-19 test kit, it did not contain milestones or performance measures that RVD Lab and NCIRD management could use to track progress and identify any possible setbacks. Also, NCIRD had no documentation to confirm that the test kit development strategy presented by the RVD lead scientist at the IMS daily briefing was approved by NCIRD management. Under routine, non-emergency NCIRD operations, RVD Lab's lead scientist reported to the laboratory supervisor. However, within the IMS structure, the lead scientist's supervisor at NCIRD acted only in an administrative capacity. In addition, CDC provided no documentation to illustrate oversight existed. For example, communication among NCIRD, IMS leadership, and CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 14 RVD Lab's scientific team related only to timelines and the need to focus on creating the test quickly. Also, there was no documentation that RVD Lab scientists charged with developing the test kit discussed the scientific strategy being tested with NCIRD management or that management was involved in approving any such strategy. However, CDC personnel informed us that RVD Lab scientists provided updates to NCIRD's IMS team on the status of the test kit and that NCIRD leadership used this information to inform CDC management about the status of the test kit. No one from NCIRD leadership, however, provided oversight of the actual development of the test kit. NCIRD leadership stated that they trusted RVD Lab scientists creating the COVID-19 test kit because most of the NCIRD leadership were not scientists. CDC's Agency-Wide Response On January 20, 2020, after leading CDC's response efforts for 2 weeks and because the outbreak was rapidly evolving, NCIRD requested that CDC COVID-19 response efforts be elevated to the agency level to increase operational support and better centralize CDC response coordination. CDC activated its Agency-Wide Response on January 21, 2020, and began following the AHP. The agencywide IMS structure transfers all response coordination activities from the center level (NCIRD in this instance) to CDC's EOC. The AHP requires the CDC Director to select an Incident Manager who is responsible for leading CDC's Agency-Wide Response efforts. Unlike the center-led IMS response structure that loosely follows the center's organizational command structure, the EOC IMS structure uses all CDC CIOs that provide emergency management specialists, subject matter experts, and other staff to serve in IMS functional and operational roles. The EOC IMS structure includes CDC personnel from across the entire organization. Various task forces (e.g., vaccine, immunization, laboratory, or infection control) can be established within the IMS framework, but not all task forces are activated for every response. In response to COVID-19, the laboratory task force was activated but functioned only as an administrative body and was not responsible for overseeing the test kit development process to ensure that quality standards were implemented. According to the leader of the laboratory task force when the EOC was initially activated, the laboratory task force was not involved in the scientific process or scientific decision making related to the development of the test kit. It served only as the administrative intermediary between CDC leadership and RVD Lab's scientists to provide updates on the progress and expected completion of the test kit. CDC informed us that it was not accustomed to responding to public health emergencies that were driven by the need to develop a test kit.27 CDC described the COVID-19 response as a laboratory-based or laboratory-focused response effort; however, no policies and procedures existed that provided instructions for CDC to follow in the case of a laboratory-based public health response. Furthermore, neither the draft GRF (including the Operations Handbook) nor the AHP provided guidance. According to CDC personnel we interviewed, laboratories require a specific organizational structure with assigned roles and responsibilities that does not transition 27 For example, the AHP includes key activities for responding to a public health emergency such as strategic decision making, logistics, collaboration, and coordinating requests. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 15 easily into a streamlined IMS structure. These factors compounded and highlighted the lack of oversight of the development of the test kits during CDC's response efforts, which may have contributed to RVD Lab shipping test kits that failed at some PHLs. AGENCYWIDE CONTROL WEAKNESSES IMPACTED CDC'S TEST KIT DEVELOPMENT PROCESS We identified weaknesses in CDC's agencywide laboratory quality processes that may have contributed to the failure of the initial COVID-19 test kits. CDC Did Not Have an Agencywide Laboratory Document Control System The Green Book states that management should design the entity's information system and related control activities to achieve objectives and respond to risks (Principle 11). It defines an information system as the people, processes, data, and technology that management organizes to obtain, communicate, or dispose of information. It states that management should design the information system to obtain and process information to meet each operational process's information requirements (Attribute 11.03). Furthermore, management should evaluate its information processes to ensure information is complete, accurate, and valid (Attribute 11.05). During our audit period, CDC did not have an agencywide system in place to control changes made to documents during the test kit development process. Rather, the CDC laboratories involved in developing the test kit used different document control systems that were not compatible with each other. Instead of having a single, updated document that could be used for various tasks, there were multiple versions of documents containing test kit procedures and results and no way to verify which version was correct. This situation resulted in the scientists using inconsistent quality control procedures throughout the test kit process. A document control system tracks and manages documents to ensure that only one version of a document (e.g., standard procedures or quality control verification procedures) exists and is the only version in use. The Division of Viral Diseases requires that its laboratories use a specific document control system, the Enterprise Laboratory Information Management System. However, RVD Lab's lead scientist requested access to another type of document control system, SmartSolve (which was used by the Influenza Division), to manage the documentation associated with the manufacturing processes involved in the development of the test kit.28 To obtain a copy of SmartSolve, RVD Lab's lead scientist submitted a request to the proper CDC official, who approved and installed the software. Both SmartSolve and the Enterprise Laboratory Information Management System were used in the development of the test kit with CDC's approval and with the understanding that the separate data sets were to be merged once the test kit was completed. CDC's internal root cause analysis identified that these two document control systems were not compatible and created a situation in which staff were using 28 Most CDC laboratories are research laboratories and have processes and document control requirements that differ from those of manufacturing laboratories. For example, research laboratories do not need to maintain manufacturing procedures and instructions or test kit labeling information. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 16 conflicting versions of documents and procedures at the same time, resulting in staff being unable to distinguish between draft, obsolete, and current versions of laboratory procedures and forms. Because no agencywide laboratory document control system existed, the use of multiple laboratory document control systems during the development of the test kit led to different versions of quality control procedures being created and used. Specifically, CDC created and used approximately four different versions of quality control procedures for the test kits. One set of quality control procedures was used to verify the small lot of test kits CDC created before obtaining EUA approval. A different set of quality control procedures was used for the FDA EUA application, and a third set of different quality control procedures was used for requesting approval from the CDC CLIA Compliance Office to initiate testing. All three of these versions of the quality control procedures differed from the fourth set of quality control procedures that CDC provided in the instructions it sent to PHLs for verifying the test kits. As a result of these different document control systems containing different versions of documents, laboratory staff were unable to distinguish between draft, obsolete, and current versions of procedures. Specifically, RVD Lab: (1) used inconsistent information when requesting approval from the CDC CLIA Compliance Office to initiate testing and applying for the FDA EUA, (2) employed a set of quality control standards to test the first batch of test kits that differed from the quality control standards included in the EUA application, and (3) provided inaccurate instructions to the PHLs for verifying the test kits. According to CDC, using the wrong set of quality control procedures contributed to the failure to identify the issues with the initial test kits. CDC acknowledged that it needed to implement a standardized document control system for use by all laboratories and stated that it plans to do so using some of the COVID-19 supplemental funding it received. CDC Lacked an Agencywide Laboratory Quality Management System Internal control is a dynamic process adapted to meet the changing environment of an entity. The Green Book states that management should establish and operate activities to monitor the internal control system and evaluate the results (Principle 16). Internal control monitoring assesses the quality of performance over time and promptly resolves the findings of audits and other reviews. Management should use quality information to achieve the entity's objectives (Principle 13). Specifically, management uses quality information to make informed decisions and evaluate the entity's performance in achieving key objectives and addressing risks (Attribute 13.05). Without a framework to guide laboratory quality processes, RVD Lab did not have an adequate system in place to control all the process changes during the development of the test kit, which then allowed for incorrectly performed quality testing procedures to occur. Absent an agencywide QMS, CDC created an environment in which management lacked quality information it needed to make informed decisions and evaluate performance against intended CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 17 outcomes. According to some CDC personnel, the focus on laboratory quality had been piecemeal and there were many different quality systems at CDC, as well as a lack of oversight of quality processes at CDC. CDC personnel further stated that CDC has followed more of a "ground up" approach rather than a "top-down" approach in ensuring lab quality. Although CDC created an agency policy to be used as a guideline for its CIOs in developing their laboratory QMS for infectious disease laboratories, CDC did not require its CIOs to implement this agency policy, which would have, in turn, required each CIO to develop and implement its own QMS. We were told that some laboratories had instituted their own quality standards when CDC management did not require its CIOs to implement CDC's agency policy regarding quality standards for all laboratories. A QMS is a system of coordinated activities to direct and control an organization's laboratory quality. A QMS provides laboratory management with a road map to validate the consistent quality of the tests performed, the products created, the data generated, and the results reported. Ultimately, the QMS serves as a key component of CDC's internal control system to establish and monitor laboratory quality. By implementing a QMS and ensuring that all processes related to the 12 Quality System Essentials perform as designed, laboratory quality can be assured. 29 CDC's Quality Manual for Laboratories To promote laboratory quality management within its operations, CDC developed the CDC Infectious Diseases Laboratories Quality Manual (CDC Quality Manual, effective October 2013). The CDC Quality Manual defines the policies and quality standards of CDC's QMS for its infectious disease laboratories. Implementation of the CDC Quality Manual requires all CDC infectious disease laboratories to establish written quality goals and objectives that are consistent with the CDC Quality Manual. Although the CDC Quality Manual was effective October 2013, it provides no specific dates for Centers to implement their own policies or for labs to have QMS policies implemented. NCIRD's Quality Manual for Laboratories To align with CDC's Quality Manual, NCIRD created its own quality manual for all laboratories within its Center.30 The NCIRD Quality Manual provides a framework for organizing, creating, and maintaining the necessary processes and procedures to implement the 12 Quality System 29 A QMS consists of a set of building blocks, known as the 12 Quality System Essentials, needed to control, assure, and manage the quality of a laboratory's processes. The 12 Quality System Essentials for laboratory environments are available online at https://www.cdc.gov/labquality/qms-tools-and-resources.html (accessed on May 2, 2022). The 12 Quality System Essentials are: (1) assessments, (2) continual improvement, (3) customer focus, (4) documents and records, (5) equipment, (6) facilities and safety, (7) information management, (8) nonconforming event management, (9) organization, (10) personnel, (11) process management, and (12) purchasing and inventory. 30 NCIRD Quality Manual (effective September 2019). CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 18 Essentials. The purpose of the NCIRD Quality Manual is to define and describe the scope of the QMS within NCIRD, the responsibilities of personnel affected by the QMS, and the activities laboratories will take to address the 12 Quality System Essentials. The NCIRD Quality Manual requires each NCIRD laboratory to implement these quality standards as part of its QMS by June 1, 2023. At the time of our audit fieldwork (May 2020–October 2021), some of NCIRD's laboratories had implemented a QMS and some had not. Specifically, RVD Lab had not implemented a QMS. In addition, for those laboratories that had implemented a QMS, NCIRD had begun to review their compliance with the NCIRD Quality Manual. Although the CDC Quality Manual and the NCIRD Quality Manual exist to support the development of an agencywide QMS within CDC's laboratories, CDC stated that developing and implementing an agencywide QMS has been evolving for the past 5 to 6 years, and there is no timeframe or deadline for when this system will be fully developed and implemented. Furthermore, CDC has not fully evaluated any QMS that may exist 31 because it has no monitoring process to review laboratories for compliance with the CDC Quality Manual. CDC's Quality System Failures Absent the implementation of an agencywide QMS to guide the development of the COVID-19 test kits, test kits were released for distribution to PHLs before quality control and quality assurance testing was completed. CDC's internal root cause analysis also identified quality control failures as one of many failed processes and stated that a discrepancy in the quality testing was noted only after the test kits had been shipped to the PHLs, not during the test kit development process. Historically, CDC focused on research processes rather than on manufacturing processes and, therefore, having established standardized laboratory quality measures in place was not a priority. CDC stated that cost was the primary obstacle to implementing an agencywide QMS, in addition to a secondary obstacle of having the needed information technology resources to support and sustain the variations in quality systems from one laboratory to another. CONCLUSION Within 2 months of China publishing the genome sequence of the then-unknown virus that caused the COVID-19 outbreak, CDC created a viable COVID-19 test kit during a global crisis. Understaffed and with limited resources, RVD Lab created the initial test kit and distributed it to 110 PHLs within 4 weeks of China publishing the genome sequence of the virus on January 10, 2020. Although some of the test kits could not be verified by all PHLs, the PHLs that were able to verify the test kit began using it to test U.S. citizens. In response to the failures of the initial test kits, CDC took corrective action and produced a viable test kit within another 3 weeks and distributed this improved test kit to its network of PHLs by February 28, 2020. 31 As previously discussed, some NCIRD laboratories may have a QMS. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 19 CDC's initial internal root cause analysis identified multiple causes for why CDC failed to detect that some of the initial test kits failed. Specifically, CDC determined that the cause of its failure to detect problems with some of the test kits was because of inadequate document control and incorrect quality control procedures. CDC's initial review did not assess the cause of the initial test kit failure. On December 15, 2021, after the completion of our audit fieldwork, CDC published its findings of an internal investigation on why some PHLs were unable to verify the COVID-19 test kits, in which CDC scientists concluded that contamination during the manufacturing of the test kits caused the false positive test result. CDC further concluded that a possible design flaw also caused a false positive test result. 32 Consistent with CDC's findings, our audit found multiple factors that may have contributed to the failure of the initial test kits. RVD Lab was confronted with creating a viable test kit under mounting pressure, facing multiple challenges with limited resources, and without key internal controls that would have helped the lead scientist, RVD Lab, and CDC. Those challenges included the following: • No formal procedures existed within RVD Lab for developing a test kit, and neither the AHP nor the draft GRF addressed this topic. • Because of RVD Lab's personnel and resource limitations, the lead scientist performed multiple tasks that should have been segregated, and CORE Lab conducted concurrent processes that it would not normally have done. • The test kit process did not have sufficient oversight during the laboratory-based response when it was managed at both the center and agency levels. • The lack of a standardized document control system led to the use of multiple quality control procedures that were not consistent with one another. • RVD Lab had no standardized QMS. Without effective internal controls, CDC may: • experience delays in the development of test kits for public health emergencies, • fail to identify issues with test kits in a timely manner and take corrective action, and • risk damaging public trust in its ability to accomplish its mission and meet public expectations. 32 This report, Analysis of the initial lot of the CDC 2019-Novel Coronavirus (2019-nCoV) real-time RT-PCR diagnostic panel, is available online at https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0260487. Accessed on Dec. 6, 2022. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 20 RECOMMENDATIONS We recommend that the Centers for Disease Control and Prevention: • create policies and procedures for developing test kits that include roles, responsibilities, and oversight; • ensure that the recently finalized GRF addresses the findings we identified in this report; • develop and implement documented processes to ensure that adequate staffing and laboratory space can be obtained for future responses and provide for separation of duties and supervisor controls; • re-evaluate the IMS structure at all levels of CDC's response framework and integrate positions or roles and responsibilities that provide effective oversight of a response effort, including a laboratory-based response effort; • implement a CDC-wide laboratory document control system; and • ensure that all infectious disease laboratories implement and periodically evaluate a QMS. CENTERS FOR DISEASE CONTROL AND PREVENTION COMMENTS AND OIG RESPONSE In response to our draft report, CDC neither concurred nor nonconcurred with our recommendations. Instead, CDC discussed actions it has taken or plans to take to implement our recommendations. Regarding our first recommendation, CDC stated that it has developed a Laboratory Quality Plan (LQP) to ensure CDC laboratories produce high-quality test methods and test results in both routine and emergency operations. The LQP was introduced to the CDC laboratory community in December 2021 and addresses the shortcomings that occurred during CDC's development of its initial COVID-19 diagnostic test. CDC stated that the LQP sets a framework that encourages continuous quality improvement while providing quality assurance checks that ensure high-quality test results. CDC stated it had already begun to implement the LQP to ensure that it delivers on its promise to the American people to "base all public health decisions on the highest-quality scientific data that is derived openly and objectively." For our second recommendation, CDC stated that it published an annex to the GRF Concept of Operations, titled "Guidelines for Managing CDC Laboratory Functions During an Emergency Response," which describes the internal roles and responsibilities among laboratory leadership, Center and Institute organizations, and incident management when CDC laboratories are part of an emergency response. CDC stated that this annex addresses the possible transition steps of laboratory functions among Program-Led, Center-Led, and Agency-Wide Responses, CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 21 including when CDC centralizes coordination of response activities in its EOC. It also addresses laboratory functions when an IMS structure is implemented. In response to our third recommendation, CDC stated that it evaluated its operating effectiveness and revised internal controls, plans, and procedures to improve the separation of duties and laboratory capacity. Specifically, CDC plans to create a new Center for Laboratory Systems and Response (CLSR). CDC stated that CLSR will provide cross-cutting laboratory operation and systems support for CDC's infectious disease laboratories. In addition, CLSR will collaborate across CDC and with clinical and public health laboratory systems, as well as Federal partners to ensure timely and efficient laboratory response and testing. Regarding our fourth recommendation, CDC stated that it elevated oversight of emergency response efforts by establishing the Office of Readiness and Response, which is accountable for CDC's preparedness, readiness, and response strategies and activities, and for strengthening the readiness and response structure across CDC and the broader public health system. With respect to our fifth recommendation, CDC stated that it is continuing the implementation of an electronic QMS (eQMS), which facilitates laboratory quality activities such as documenting and managing corrective and preventive actions, training, equipment maintenance, and standard operating procedures. CDC estimates that the eQMS could be fully implemented within 18 months after awarding the contract. For our final recommendation, CDC stated that once the aforementioned LQP is fully implemented, it will include a variety of evaluation activities to address the OIG's recommendation to implement and evaluate a QMS. We commend CDC on the actions it has taken or is taking to address our recommendations. CDC provided technical comments, which we addressed, as appropriate, in the report. CDC's written comments, excluding the technical comments, are included as Appendix C. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 22 APPENDIX A: SCOPE AND METHODOLOGY SCOPE Our audit period covered CDC's test kit development efforts from January through March 2020. To conduct this audit, we obtained an understanding of CDC's process for developing the test kits and created a flow chart that documents this process. We focused our audit efforts on reviewing the internal controls involved in this process. To conduct a review of CDC's internal controls, we performed several steps. First, we interviewed CDC and HHS personnel who participated in the development of the COVID-19 test kits. We focused mainly on the CDC laboratories that developed the test kits. Second, we requested and reviewed white papers, final and draft policies and procedures, strategies, and departmental emails and memos that each laboratory supplied. Third, we reviewed the results of CDC's root cause analysis that sought to determine the cause of the initial test kit failure, in addition to HHS's Office of the General Counsel's report of its investigation into the initial test kit failure. When documentation was unavailable, we relied on testimonial evidence that we gathered during the audit. The Green Book sets the internal control standards for Federal entities. It defines internal control as a process used by management to help an entity achieve its objectives. Internal control comprises the plans, methods, policies, and procedures used to fulfill the mission, strategic plan, goals, and objectives of the entity (Green Book Overview 1.03). 33 Although there are different ways to present internal control, the Green Book approaches internal control through a hierarchical structure of 5 components and 17 principles, as shown in Figure 3 on the following page. The Green Book states that a deficiency in the design of internal control exists when: (1) a control is missing or (2) an existing control is not designed properly. A deficiency in the implementation of internal control exists when a properly designed control is not correctly implemented (Green Book Overview 3.05). In evaluating operating effectiveness, management determines whether controls were applied at relevant times during the period under evaluation, the consistency with which they were applied, and by whom or by what means they were applied. Specifically, a deficiency in operations exists when a control does not operate as designed or when the person performing the control does not possess the necessary authority or competence to perform the control (Green Book Overview 3.06). Although we evaluated all 5 components and 17 principles of internal control identified in the Green Book, our findings in this report relate only to those Green Book internal control principles specifically identified in our findings. 33 Standards for Internal Control in the Federal Government (GAO-14-704G), section OV2.09, Figure 3, page 9. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 23 Figure 3: The 5 Components and 17 Principles of Internal Control Control Environment Control Activities 1. The oversight body and management should 10. Management should design control activities to achieve demonstrate a commitment to integrity and ethical objectives and respond to risks. values. 11. Management should design the entity's information 2. The oversight body should oversee the entity's internal system and related control activities to achieve control system. objectives and respond to risks. 3. Management should establish an organizational 12. Management should implement control activities structure, assign responsibility, and delegate authority through policies. to achieve the entity's objective. Information and Communication 4. Management should demonstrate a commitment to 13. Management should use quality information to achieve recruit, develop, and retain competent individuals. the entity's objectives. 5. Management should evaluate performance and hold 14. Management should internally communicate the individuals accountable for their internal control necessary quality information to achieve the entity's responsibilities. objectives. Risk Assessment 15. Management should externally communicate the 6. Management should define objectives clearly to enable necessary quality information to achieve the entity's the identification of risks and define risk tolerances. objectives. 7. Management should identify, analyze, and respond to Monitoring risks related to achieving the defined objectives. 16. Management should establish and operate monitoring activities to monitor the internal control system and 8. Management should consider the potential for fraud evaluate the results. when identifying, analyzing, and responding to risks. 17. Management should remediate identified internal 9. Management should identify, analyze, and respond to control deficiencies on a timely basis. significant changes that could impact the internal control system. METHODOLOGY To accomplish our objective, we: • reviewed Federal laws, policies, and guidance applicable to CDC's authority to respond to emerging health threats and public health emergencies, as well as CDC's response processes and test kit development; • sent questionnaires to CDC officials about CDC's processes and procedures related to the development of the COVID-19 test kit and reviewed the responses; • conducted interviews with personnel at HHS agencies involved in the COVID-19 test kit development; and • discussed the results of our audit with CDC officials. We conducted our fieldwork from May 2020 through October 2021. We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 24 sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 25 APPENDIX B: FLOW CHART OF CDC'S PROCESS FOR DEVELOPING AN EMERGENCY TEST KIT Start 1 HHS delegated CDC authority 4 to prevent spread of communicable diseases under Pathogen dictates the PHS Act, which includes lab assignment. developing and producing CDC or contractors diagnostic test kits. manufacture a large lot of Lab develops test kit test kits using 2 manufacturing and quality components. control. CDC, as part of a worldwide network, identifies public health emergencies. Select components with best results for final test kit. Completed test kits are sent to IRR. GRF used to determine breadth, level of effort, and CLIA – Package resources needed for a health Creation and emergency response. Level Approval of response categorized into IRR sends test kits to PHLs. three levels. A small lot of test kits is manufactured for test validation using Response is handled within manufacturing and quality control Y Program- Y the program. Generally, Verification procedures. PHLs begin Led these are expected or 1 successful? using test kit. Response* limited public events. Y Small lot Seek FDA N CDC and/or N successful? authorization 3 contractors Response is handled within (and EUA). PHLs notify CDC and monitor the center. The incident CDC notifies FDA. performance and Center- Y has potential to create address any Led significant impact. 1 performance Response* N Increased response Y CDC troubleshoots. issues. FDA activities are implemented. 4 authorizes? 2 N Identify type of issue. End N CDC CDC activates Emergency troubleshoots. Agency- Y Operations Center to Wide provide a unified Agency- 1 2 3 4 Response* Wide response. 3 CDC conducts many of these steps concurrently. *A CDC response may bypass either of the first two levels and elevate during a response. CDC's Production and Distribution of the COVID-19 Test Kit (A-04-20-02027) 26 APPENDIX C: CENTERS FOR DISEASE CONTROL AND PREVENTION COMMENTS U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Public Health Service Centers for Disease Control and Prevention (CDC) Atlanta GA 30329-4027 TO: Juliet T. Hodgkins Principal Deputy Inspector General FROM: Centers for Disease Control and Prevention (CDC) DATE: August 11, 2023 SUBJECT: Office of the Inspector General (OIG) Draft Report Attached is the CDC response to the OIG draft report, "CDC's Internal Control Weaknesses Led to Its Initial COVID-19 Test Kit Failure, but CDC Ultimately Created a Working Test Kit (A-04- 20-02027)" Sincerely, Mandy K. Cohen, MD, MPH Director, CDC and Administrator, Agency for Toxic Substances and Disease Registry Attachment: Response to the report CDC's Production and Distribution of the COVID-19 Test Kit 27 ( '~ U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Public Health Service ~~~,- Centers for Disease Control and Prevention (CDC) Atlanta GA 30329-4027 The Centers for Disease Control and Prevention (CDC) offer the following corrective actions in response to the Office of Inspector General's (OIG) recommendations. Recommendation 1: OIG recommends CDC create policies and procedures for developing test kits that include roles, responsibilities, and oversight. CDC Response: CDC considers this recommendation closed. CDC developed a Laboratory Quality Plan (LQP) to ensure CDC laboratories produce high-quality test methods and test results in both routine and emergency operations. Introduced to the CDC laboratory community in December 2021, the LQP effectively addresses shortcomings that occurred during CDC's development of its initial COVID-19 diagnostic test. The LQP contains six major components:  Three quality management systems: one for infectious disease laboratories, one for non- infectious disease laboratories, and one for National Institute for Occupational Safety and Health laboratories. Separate quality systems permit more detailed quality specifications tailor-made for different types of laboratories.  Development of the Quality Manual for Microbiological Laboratories (QMML) that describes quality standards for CDC's clinical, surveillance, and research infectious disease laboratories.  An Infectious Disease Test Review Board (IDTRB) that reviews laboratory tests developed within CDC before they are shared outside of CDC to be sure the tests meet high-quality standards and are suitable for their intended purpose. The review process- that can be expedited in emergency responses-involves objective input from at least three CDC laboratory scientists who were not involved in the test development, ensuring test performance characteristics and test documentation meet appropriate quality standards and can be efficiently transferred to another laboratory. This independent review process is designed to ensure tests meet quality criteria appropriate for the intended use of the test before sharing the test with laboratories external to CDC.  Method expert groups that develop method validation standards and method documentation requirements for each type of method (e.g., RT-PCR, ELISA) used in infectious disease laboratories.  The electronic Quality Management System (eQMS), which was created to be flexible and easy to use and facilitates laboratory quality activities such as documenting and managing non-conforming events, corrective and preventive actions, training, equipment maintenance, standard operating procedures, and more.  Biennial external review of all laboratories (clinical, surveillance, and research); clinical laboratories audited to meet Clinical Laboratory Improvement Amendments requirements by Centers for Medicare & Medicaid Services-approved auditors. The LQP sets forth a framework that encourages continuous quality improvement, while providing the quality assurance checks that ensure high-quality test results. While the plan has been developed, this multi-step process requires time to implement. CDC is dedicated to these improvements being fully realized across all laboratories and the workforce to reinforce the premier role of CDC laboratories in public health and public health emergency responses. CDC's Production and Distribution of the COVID-19 Test Kit 28 ( '~ U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Public Health Service ~~~,- Centers for Disease Control and Prevention (CDC) Atlanta GA 30329-4027 CDC has already begun to implement the LQP, including the formation of the IDTRB in March 2022, to ensure CDC delivers on its promise to the American people. CDC has pledged to base all public health decisions on the highest-quality scientific data that is derived openly and objectively. Public health decisions rely on data from CDC laboratories; therefore, laboratories must operate at a gold-standard of quality using advanced laboratory science to ensure high- quality test methods and test results are produced in both routine and emergency operations. Recommendation 2: OIG recommends CDC ensure that the recently finalized Graduated Response Framework (GRF) addresses the findings we identified in this report. CDC Response: CDC considers this recommendation closed. In July 2022, CDC published an annex to the GRF Concept of Operations, "Guidelines for Managing CDC Laboratory Functions During an Emergency Response." The document describes the internal roles and responsibilities among laboratory leadership, center and institute organizations, and incident management when CDC laboratories are part of an emergency response. It addresses lines of authority, decisions, accountability, and other details to ensure laboratory science, quality, and safety are effectively coordinated. The document outlines the possible transition steps of laboratory functions between program-led, center-led, and agency-wide responses, including when CDC centralizes the coordination of response activities in its Emergency Operation Center. The document also addresses the incorporation of laboratory functions when an Incident Management System (IMS) structure is implemented-most commonly at the agency-wide or center-led response level- and provides guidance for implementing strategic, pragmatic, and coordinated laboratory response efforts. The guidance is underpinned by two principles: first, leadership at all levels is essential for the development and rapid scaling of laboratory functions in an emergency response; and second, success or missteps are a matter of organizational responsibility and accountability and not due to any specific person(s) or technologies. Guidance on different aspects of test kit development, as referenced in the OIG report, is not intended to be a part of the annex. Test kit development is being incorporated within the LQP. Under the LQP, the QMML will incorporate guidance and requirements for test validation. As well, the IDTRB has been established and provides a peer review process to ensure test performance characteristics and test documentation meet appropriate quality standards and can be efficiently transferred to another laboratory. Recommendation 3: OIG recommends CDC develop and implement documented processes to ensure that adequate staffing and laboratory space can be obtained for future responses and provide for separation of duties and supervisor controls. CDC Response: CDC considers this recommendation closed. In alignment with the standards in GAO-14-704G (The Green Book), CDC evaluated its operating effectiveness and revised internal controls, plans, and procedures to improve the separation of duties and laboratory capacity. CDC will create a new Center for Laboratory Systems and Response that will report to the Office of Laboratory Science and Safety. CLSR will provide cross-cutting laboratory operation and systems support for CDC's infectious disease laboratories. CLSR will work across CDC's Production and Distribution of the COVID-19 Test Kit 29 U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Public Health Service Centers for Disease Control and Prevention (CDC) Atlanta GA 30329-4027 CDC and collaborate with clinical and public health laboratory systems, as well as federal partners to ensure scientifically advanced, timely, and efficient laboratory response and diagnostic testing for infectious disease outbreaks, epidemics, and pandemics. Specific changes include:  In FY2023, The Division of Laboratory Systems will move from the National Center for Emerging Zoonotic and Infectious Diseases (NCEZID) to CLSR and incorporate the Laboratory Response Network and the Electronic Laboratory Information Management System  NCEZID's Division of Scientific Resources will be renamed to Division of Core Laboratory Services and Response (DCLSR) and incorporate the Biological Rapid Response & Advanced Technology Laboratory and the CDC Biorepository. DCLSR will remain within NCEZID for the remainder of Fiscal Year (FY) 2023 and move to Office of Laboratory Science and Safety/CLSR in FY2024 after certain administrative milestones are met.  Led by the Office of Readiness and Response (ORR), the CDCReady Responder program is identifying and preparing staff for roles before responses occur. The program pre- enrolls individuals into cadres built around the disciplines needed for response positions, including those that will lead and support laboratory functions within an emergency response. Enrolled cadre members will have met certain qualification criteria and be pre- identified to take on emergency response assignments when needed. The laboratory cadre will begin enrollment by the end of FY2023 and into FY2024. Recommendation 4: OIG recommends CDC re-evaluate the Incident Management System structure at all levels of CDC's response framework and integrate positions or roles and responsibilities that provide effective oversight of a response effort, including a laboratory-based response effort. CDC Response: CDC considers this recommendation closed. As part of CDC's Moving Forward reorganization, CDC elevated oversight of emergency response efforts by establishing the ORR, which reports directly to the CDC Immediate Office of the Director. ORR is accountable for CDC's preparedness, readiness, response strategies and activities, and for strengthening the readiness and response structure across CDC and the broader public health system. During all levels of CDC's response framework, ORR's role and responsibility is to provide oversight and assistance in ensuring that each IMS adheres to incident management principles and follows response plans, guidance, and procedures that ensure a consistent approach for all response activities, including laboratory-based efforts. Recommendation 5: OIG recommends CDC implement a CDC-wide laboratory document control system. CDC Response: CDC considers this recommendation in progress. CDC continues implementation of the eQMS, which facilitates laboratory quality activities such as documenting and managing non-conforming events, corrective and preventive actions, training, equipment CDC's Production and Distribution of the COVID-19 Test Kit 30 U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Public Health Service Centers for Disease Control and Prevention (CDC) Atlanta GA 30329-4027 maintenance, standard operating procedures, and more. CDC is working to accelerate configuration and implementation throughout CDC infectious disease laboratories. Dependent upon completion of contract modification, the eQMS could be fully implemented 18 months after award. Recommendation 6: OIG recommends CDC ensure that all infectious disease laboratories implement and periodically evaluate a Quality Management System. CDC Response: CDC considers this recommendation in progress. Once CDC's LQP is fully implemented, evaluation activities will include:  Adherence to quality standards and quality management practices in infectious disease laboratories using an eQMS that provides real-time, retrievable information on important quality indicators. These indicators include performance in proficiency testing programs, trends in the number of nonconforming events, thoroughness of root cause analysis, and corrective and preventive actions.  Biennial external review of laboratory compliance with standards detailed in QMML. The QMML is undergoing review in line with a final version by the end of Calendar Year 2023 and implementation will then ensue.  Increases in the percentage of new CDC diagnostic tests passing IDTRB review. CDC's Production and Distribution of the COVID-19 Test Kit 31