January 2004/Issue 9 Translating Research to Policy Do National Surveys Overestimate the Number of Uninsured? Findings from the Medicaid Undercount Experiment in Minnesota OVERVIEW to cause upward bias in survey estimates of the number of uninsured. When survey General population surveys of health estimates of Medicaid enrollment do not insurance coverage provide timely estimates match administrative data counts, the of uninsurance. These estimates inform discrepancy raises concerns about other resource allocation and policy decisions estimates produced by the survey. made by Federal and state lawmakers trying to make health insurance more accessible General population surveys are the only and affordable. Policymakers and analysts source of estimates on the number of people use survey estimates of coverage and lack of covered by private insurance, those who are coverage to monitor the dynamics of health uninsured, and those who are uninsured but insurance markets, evaluate the success of eligible for public programs. This SHADAC current programs in reaching target issue brief summarizes our recent study of populations, and assess the costs and the Medicaid undercount in Minnesota. benefits of program changes, outreach activities, and other coverage initiatives. EXPLAINING THE MEDICAID These survey estimates are also used in federal formulas that allocate billions of UNDERCOUNT dollars annually to states for the State Comparisons of survey estimates of Children’s Health Insurance Program Medicaid participation to Medicaid (SCHIP). With these factors in mind, the administrative data indicate that anywhere importance of obtaining accurate estimates from 15 to 50 percent of Medicaid cases are of the number of people lacking insurance missed by national population surveys such becomes clear. as the Current Population Survey (CPS), the Survey of Income and Program One area of ongoing concern to researchers Participation, and the Community Tracking is that general population surveys like the Study.1 One might infer from these results Current Population Survey (CPS) that some portion of Medicaid recipients systematically underestimate the number of do not report their Medicaid coverage in individuals known through administrative surveys asking about health insurance records to be enrolled in Medicaid coverage. programs. This discrepancy between survey and administrative counts of Medicaid Medicaid enrollees might provide inaccurate enrollment—or the “Medicaid responses to survey questions addressing undercount”—is problematic, not in and insurance coverage for a number of reasons. of itself, but to the extent that it is thought Some Medicaid recipients may be confused 1 See Lewis, K., M. Ellwood, and J.L. Czajka. 1998. Counting the Uninsured: A Review of the Literature. Washington, D.C.: The Urban Institute. about what program they are in, either because they MINNESOTA’S MEDICAID UNDERCOUNT haven’t accessed health care services in some time, or because their enrollment status changes frequently. EXPERIMENT Others may provide misleading information because To examine the accuracy of Medicaid enrollees’ they are embarrassed to be associated with a welfare- responses to health insurance surveys, SHADAC like public program. Still others may report a source of researchers conducted the Medicaid Undercount coverage other than Medicaid if they: are confused by Experiment (MUE). By asking a random sample of the similarity of the program names (e.g., Medicare known Minnesota Health Care Program enrollees (i.e., and Medicaid); have multiple sources of coverage (e.g., Medicaid, MinnesotaCare and General Assistance Medicare, private third-party coverage); associate Medical Care) about their health insurance coverage in Medicaid coverage with a commercial product because conjunction with a statewide general population survey, they are enrolled in a Medicaid managed care plan; or researchers were able to determine: (1) the frequency think they are covered by a different state-subsidized with which Medicaid recipients accurately reported health care program altogether. their public coverage, and (2) the impact of inaccurate reports on survey estimates of coverage derived from While the research community has established that the the statewide survey. number of people reporting Medicaid coverage is consistently lower than the number enrolled in the As shown in Figure 1, only 37% of known Medicaid program according to administrative records, the enrollees responded accurately to survey questions question remains whether some portion of Medicaid about their health insurance. The remaining 63% were recipients report having no insurance or some other labeled “missed Medicaid cases” due to the following: source of insurance in surveys asking about health communication barriers or refusals (13.8%), lack of insurance coverage. telephone (18.0%), or inaccurate responses to questions about insurance coverage (31.4%). The inaccurate programs, the populations they serve, and in the health responses consisted of 2.8% reporting no coverage care delivery systems that serve them may affect the at all, 7.8% reporting private coverage, and 20.8% outcome of this research. Future work by SHADAC reporting the wrong type of public coverage (e.g., researchers will therefore repeat the MUE in additional Medicare, MinnesotaCare, or General Assistance states to determine the magnitude of the Medicaid Medical Care). About half of the later group (10.2%) undercount and examine sources of the undercount. were eligible for both Medicare and Medicaid, This will allow us to assess the extent to which the and reported having Medicare, but not Medicaid, results can be generalized to other states, and the coverage. feasibility of developing a method for adjusting survey estimates to account for the Medicaid undercount. The policy implications of these findings are important: the Medicaid undercount—at least as measured by the MUE in Minnesota—introduced only a negligible upward bias to estimates of the uninsured produced by the state survey. Specifically, we calculated the bias introduced by inaccurate survey responses among all public program enrollees in the MUE which reduced Minnesota’s uninsurance estimate by only 0.26 percentage points, from 5.29 to 5.03 percent.2 This difference is not significant; therefore inaccurate reports of coverage among Medicaid recipients were found not to bias the estimate of uninsurance. IMPLICATIONS FOR POLICY AND FURTHER RESEARCH SHADAC’s findings imply that while general population surveys like the CPS systematically underestimate participation in the Medicaid program, the effect on estimates of uninsurance may be extremely modest. This seemingly technical result has real policy implications at state and national levels, and is good news for analysts concerned about the validity of survey estimates of those lacking health insurance coverage. Our research suggests that, at least with respect to the survey implemented in the state of Minnesota, health policy and resource allocation decisions have not been misinformed. We recognize the importance of replicating our results in other states, as these findings have implications beyond Minnesota’s borders. We also acknowledge that differences in survey instruments, public 2 State estimate of uninsurance reported in 1999 Minnesota Health Access Survey (MNHA). The State Health Access Data Assistance Center at the University of Minnesota promotes the effective use of available data to inform the debate on health coverage and access. For a complete account of this study, please see: Call, Kathleen Thiede, Gestur Davidson, Anna Stauber Sommers, Roger Feldman, Paul Farseth, and Todd Rockwood, 2002. “Uncovering the Missing Medicaid Cases and Assessing their Bias for Estimates of the Uninsured.” Inquiry. 38(4): 396-408. State Health Access Data Assistance Center (SHADAC) | University of Minnesota School of Public Health 612-624-4802 | fax: 612-624-1493 | www.shadac.org IB-09-104