bRIEF • JUNE 2014 Issues with the Survey-based Measure for Patient Centered Medical Homes for Children Authors Summary Victoria Lynch Several provisions of the Affordable Care Act (ACA) are directed at the establishment and Urban Institute, Washington, DC promotion of the Patient Centered Medical Home (PCMH), a model for evaluating health Lisa Clemans-Cope care quality that originated in the field of pediatrics. With this increasing emphasis on the Urban Institute, Washington, DC PCMH concept, it is important to ensure policymakers have a valid measure for evaluating it, particularly at the state level, which is where national policy goals are primarily evaluated. This brief considers the current standard measure used to study the patient centered medical home (PCMH) for children in the National Survey of Children’s Health (NSCH) and the National Survey of Children with Special Health Care Needs (NS-CSHCN). Using this com- posite measure as it is currently computed, children with special health care needs (CSHCN) are less likely to be identified as having a medical home compared to children without such needs. However, the treatment of missing information in the surveys calls this finding into question. This is because non-response due to inapplicability for any of the PCMH composite domains is treated as through the criterion for that domain is satisfied, and well-children have substantially more of this non-response. In addition, this treatment means that states with a SHARE is a Robert Wood Johnson higher proportion of children needing the range of services covered by the PCMH domains Foundation (RWJF) grant program that funds rigorous research on health will appear to have lower proportions of children with a medical home compared to states with reform at the state level, including state a lower proportion of these needy children, everything else being equal. implementation of national reform. The authors call for a revision to the current standard PCMH measure and suggest that, as SHARE synthesizes the results of this research in order to establish an computed, it should be interpreted as a measure of whether the medical care that children evidence base for state health reform received conformed to the PCMH model as far as it could be measured given the rage of care the and informs policy by making research children received. and analysis accessible to government officials through strategic translation While the authors acknowledge that there are no simple solutions to this measurement issue, and dissemination. their findings should be taken into consideration when designing and interpreting survey SHARE operates out of the State measures of PCMH, whether for children or adults, particularly as the PCMH gains broader Health Access Data Assistance Center traction under health reform. (SHADAC), an RWJF-funded state health policy research and technical assistance center in the Division of Health Policy and Management, School of Public Health, University of Minnesota. | State Health Access Reform Evaluation 1 Introduction (2009), but there is general consensus that the current standard is the National Quality Forum (NQF)-en- For the past several decades it has been a national goal dorsed measure in the National Survey of Children’s to ensure that children receive patient centered care Health (NSCH) and the National Survey of Children within a medical home (PCMH), and there has been with Special Health Care Needs (NS-CSHCN). The widespread effort to develop consensus on how to de- NSCH and the NS-CSHCN measures are nearly fine and measure the PCMH concept (Sia, Tonniges, identical in design and cover the following five do- Osterus, & Taba, 2004; US Department of Health mains (CAHMI, 2009): and Human Services, 2010). The ideal PCMH is said to provide care that is “accessible, family-centered, • Child has a personal doctor or nurse continuous, comprehensive, coordinated, compas- • Child has a usual source of care sionate, and culturally effective” (Health Resources • Child had no problem getting needed referrals and Services Administration, n.d.). The PCMH when needed was first proposed by the American Association of • Child’s parent/guardian gets help with care Pediatrics as a model for an organization of pediat- coordination when needed ric care that delivers the core functions of primary • Child has family-centered, culturally effective care health care (American Academy of Pediatrics, Medical Based on the parent’s/guardian’s responses to one or Home Initiative s for Children with Special Needs more survey questions, each sample child receives Project Advisory Committee, 2004). It was originally a value of “yes,” ”no,” or “legitimate skip because developed for children with special health care needs care not needed”—referred to in this brief as “no (SHCN) and later extended to all children. response”—for each of the five domains. Several questions used to derive medical home status may not Measuring the Survey-based be applicable to children in good health because they PCMH ask about services that the children did not need. For the surveys’ standard measure of PCMH, cases that The Child and Adolescent Health Measurement have no response on a medical home domain because Initiative (CAHMI) notes that future work is needed care was not needed are treated as though they meet to refine the measurement of the medical home the criteria for that domain. Exhibit 1. Total sample children in the 2007 National Survey of Children’s Health (NSCH) and the proportion with data collected on all five domains of the patient centered medical home 90000 80000 70000 60000 50000 40000 83,800 (100%)( 30000 20000 10000 0 11,300 (13.5%)( Total Children with data collected on all five domains Note: Domains include: 1) Personal doctor or nurse 2) Has a usual source of care 3) No problem getting needed referrals 4) Providers help coordinate care when needed 5) Family centered care. 2 | State Health Access Reform Evaluation It is known that small variations in the ways that sur- was determined to not need the service asked about vey items are used to construct measures of PCMH in a domain. The domains that are missing for a large can affect estimates of the prevalence of medical share of the sample are 1) whether they have a prob- homes (Bethel, Read, & Brockwood, 2004). Howev- lem getting referrals to the specialist that their parent/ er, the extent to which researchers take this possibility guardian thinks they need and 2) whether the parent/ into account when making inferences is unclear. In guardian gets extra help coordinating care among this brief we describe how the standard measure of providers and school/daycare when needed. Referrals Exhibit 2. Hypothetical Values for the Five Domains of the PCMH, Two Example Children Child 2 Child 1 Personal doctor or nurse Yes Yes Has usual source of care Yes Yes No problem getting needed referrals No response Yes Providers help coordinate care when needed No response No Family centered care Yes Yes Has a medical home? Yes No PCMH in the NSCH and the NS-CSHCN is sensi- to specialists were only measured for children whose tive to the treatment of missing information on the parent/guardian reported that they needed a referral PCMH domains and how this raises questions about in the last year. Care coordination was only observed what the PCMH estimates represent. We largely re- for children whose parents reported that they went to port on findings from the 2007 NSCH, and we focus a specialist3 and had at least two services. on the sensitivity of the PCMH measure to missing High levels of missingness in surveys are not always information. an indicator of a problem, but there is reason to believe that it is problematic for the PCMH measure. Challenges in Measuring The high levels of domain missingness may bias esti- Children who are by mates of PCMH because the measure is computed by Patient Centered Medical Home definition less likely to treating cases for which there is no response as though Domains they meet the criteria for that domain. What this need a referral or care We found high levels of missingness on the domains 1 treatment means is that children who are by defini- coordination are more of the PCMH measure in the NSCH. Exhibit 1 tion less likely to need a referral or care coordination likely to be classified as shows that only 13.5 percent of NSCH sample are more likely to be classified as having a PCMH.  having a PCMH.  children2 in 2007 had responses on all five domains. The mechanics of this potential problem are demon- Less than 4 percent of our NSCH analytical sam- strated in Exhibit 2. Example Child 1 did not need ple children were characterized as missing because the full range of health services measured in the the respondent did not know or refused to answer PCMH model, so two of the five domains had no questions on at least one domain, while 82.8 percent response and are treated as if the child received care were characterized as having at least one “legitimate that met the PCMH criterion in those domains. In skip” (i.e., legitimate non-response) because the child contrast, Child 2 needed the range of health services measured in the survey-based PCMH model, so 1 “Missingness” refers to the existence of missing data. 2 Our analytical sample included only those who were 3 Mental health service provider or other specialist includ- insured or uninsured the full year (n=83,754). ing therapists and home health care providers. JUNE 2014 | www.shadac.org/share 3 Exhibit 3. Estimated Prevalence of Children with a Medical Home among States Before and After Adjusting for Relatively High1/Low Rates2 of Response on the Referral and Coordination Domains of the Medical Home States with Relatively High Rates of Response Original Adjusted Virginia 68.8 75.4 Maine 67.8 73.6 Massachusetts 66.2 72.1 Delaware 50.2 56.3 New Jersey 69.7 76.6 States with Relatively Low Rates of Response Original Adjusted Utah 65.3 62.7 Montana 64.1 60.4 Nevada 65.2 63 Illinois 57.2 53.2 Wyoming 61.1 58.5 1 Adjusted by assuming the same rate of response as in the state with the lowest rate of response. 2 Adjusted by assuming the same rate of response as in the state with the highest rate of response. information could be ascertained about the health of medical home care would fall from 59.4 percent to care available to the child in all five domains. In 48.8 percent. 4 effect, Child 1 had three opportunities to be identi- We can infer from the logic demonstrated in Ex- fied as not having care that met the PCMH model, hibit 2 that children with special health care needs while Child 2 had five opportunities and thus a (SHCN) will be less likely to be identified as having a higher probability of being identified as not having medical home because they are more likely than other a PCMH. The exhibit shows that Child 1 was identi- children to need services on all five domains and fied as having a medical home even though the child thus less likely to be treated as a “no response” on a received less care in the PCHM model compared domain. In fact, 28.7 percent of the sample children to Child 2. This is different from determining that with SHCN had responses on all the domains, com- Child 1 had access to care that reflects the PCMH pared to 9.6 percent of the children with no SHCN model (i.e., that the child would get PCMH care if in 2007. That more children without SHCN are he or she needed it). If the sample children with no assumed to satisfy the criteria of the PCMH domains response on the referral and coordination domains raises the question of whether it is true that children had needed those types of services, we do not know without SHCN are more likely to have a medical what proportion of the children would have received home than children with SHCN, as previous research the services. However, we can be fairly certain that has asserted (Zickafoose and Davis, 2013). not all the children would have received the services, which is what is assumed by treating “no response” on the domain as satisfying the domain criteria and what is, therefore, reflected in the estimates. If we assumed, for example, that 80 percent of the children with no information on the referral and coordination domains would have gotten care that satisfied the 4 This estimate is computed for children who were unin- PCMH criteria, then the estimate for the prevalence sured or insured for the full year. 4 | State Health Access Reform Evaluation Implications for State Rankings Conclusion from the Patient Centered Medi- In this study we demonstrated that when survey data cal Home Care Methodology on some of the domains of the medical home are missing, there is not enough information available We can infer from the logic demonstrated in Exhibit to determine whether a sample child has a medical 2 that states with relatively less missingness on the home where he or she can get PCMH care should domains—i.e., states with children who have relative- they need it. Given the high levels of missingness on ly more health care needs—will appear to have lower two domains of the medical home measure in the proportions of children with medical homes com- NSCH, we conclude that the standard PCMH esti- pared to other states. We found empirical evidence mates for children should be interpreted as a measure that the degree of missingness on the domains affects of whether the medical care that children received state estimates of the prevalence of children with a conformed to the PCMH model as far as it could be PCMH. measured given the range of care the children received. The estimated prevalence of children with a medical We also demonstrated that the treatment of cases home among states changed when we assumed higher with missing information on the PCMH domains af- or lower rates of response on the referral and coordi- fects the estimates of the prevalence of children with a nation domains (Exhibit 3). For the five states with medical home and that this treatment logically means the highest rates of response, we assumed that their that CSHCN are less likely to be determined to have rate of response was the same as the state with the a medical home. Finally, we found that assuming lowest level. We then moved the corresponding pro- higher or lower rates of missingness on the domains portion of children in their samples to “no response”, affects conclusions regarding the relative prevalence of which are counted as satisfying the domain criteria. children with a PCMH across states. For the five states with the lowest rates of response, The extensive collaboration of experts involved in we assumed that their rate of response was the same developing the survey-based medical home measure as the state with the highest level. We also assumed for children implies that there are no simple solutions that the random sample of children we moved from to measuring the complex concepts defining the “no response” had the same rate of satisfying the PCMH model. However, these findings suggest that criteria of the domains as the children who originally the measure should be evolved on the treatment of had responded.5 missing data on the PCHM domains. For example, Virginia has relatively high rates of response on the two key domains. When we assumed Virginia’s rates were low, we found that the esti- mate for the prevalence of children with a medical home increased from 68.8 percent to 75.4 percent. Conversely, Utah had relatively low rates of response. When we assumed that it had high rates, we found that the estimated prevalence of children with a med- ical home fell from 65.3 percent to 62.7 percent. On average, the estimated prevalence of having a medical home increased by more than six percentage points among the states with relatively high rates of response and decreased by three points among states with rela- tively low rates of response when their response rates were adjusted in the opposite direction. JUNE 2014 | www.shadac.org/share 5 References American Academy of Pediatrics, Medical Home Initiative s for Children with Special Needs Project Advisory Committee. 2004. “Policy Statement: Organizational Principles to Guide and Define the Child Health Care System and/or Improve the Health of All Children.” Pediatrics 113(suppl 4): 1545-1547. Bethell C., Read D., Brockwood, K. 2004. “Using Existing Population-based Data Sets to Measure the American Academy of Pediatrics Definition of Medical Home for All Children and Children with Special Health Care Needs.” Pediatrics 113 (suppl 5): 1529-1537. Child and Adolescent Health Measurement Initiative. 2009. Measuring Medical Home for Children and Youth: Methods and Findings from the National Survey of Children with Special Health Care Needs and the National Survey of Children’s Health. Retrieved from http://www.childhealthdata.org/docs/medical-home/mhmanual-_body_ sept2009-cb-edit-1-pdf.pdf Health Resources and Services Administration. (n.d.) What is a medical home? Why is it important? Retrieved from http://www.hrsa.gov/healthit/toolbox/Childrenstoolbox/BuildingMedicalHome/whyimportant.html Sia C., Tonniges T. F., Osterhus E., Taba S. 2004. “History of the medical home concept.” Pediatrics 113(suppl 5): 1473-1478. US Department of Health and Human Services. 2010. Healthy People 2020 Maternal, Infant, and Child Health Objectives. Retrieved from http://www.healthypeople.gov Zickafoose, J.S., Davis, M. M. 2013, “Medical home disparities are not created equal: differences in the medical home for children from different vulnerable groups.” J Health Care Poor Underserved 24(3): 1331-1343. 6 | State Health Access Reform Evaluation Report Authors Victoria Lynch, MS Victoria Lynch is a research associate in the Health Policy Center at the Urban Institute in Washington, D.C. Her current analytical focus is simulating eligibility for Medicaid/CHIP using state-based eligibility rules, state enrollment data, and survey data from the American Community Survey (ACS). Ms. Lynch’s current survey methodological focus is on the validity of health coverage related data from the ACS and developing methods and recommendations for maximizing the ACS’s utility for studying the impacts of health reform at the state and local level. Lisa Clemans-Cope, PhD Clemans-Cope is a senior research associate and health economist at the Urban Institute. Dr. Clemans-Cope’s areas of expertise include health insurance; health spending; Medicaid and CHIP programs; Medicaid/Medicare “dual” eligibles; access to health care; health insurance reform initiatives and legislation; health-related survey data; and Medicaid claims data. Her recent work includes both quantitative and qualitative analyses of federal regulation and state implementation of the Affordable Care Act. Other Contributors: Andrea Stronghart and Carrie Au-Yeung provided assistance with document layout and graphics. JUNE 2014 | www.shadac.org 7 State Health Access Data Assistance Center (SHADAC) SHADAC is a health policy research center within the University of Minnesota School of Public Health whose faculty and staff are recognized as national experts on the collection and use of health policy data. SHADAC health economists and policy analysts cover the full range of technical, research and policy expertise involved in using federal and state data to inform health policy, while leveraging hands-on experience working in state government. SHADAC specializes in issues related to health insurance access, use, cost and quality with a particular focus on state implementation of health reform. Work includes providing technical assistance to many agencies and individuals across the country, at both the federal and state government levels. In addition, SHADAC contributes to general health policy literature and debate by conducting timely health policy research, which is translated into issue briefs, reports and peer-reviewed journal articles. For more information, visit www.shadac.org. SHADAC is funded by the Robert Wood Johnson Foundation. For more information, please contact us at shadac@umn.edu, or call 612-624-4802. Our Website shadac.org provides valuable research and resources on health insurance coverage in states, data collection methods, and state health policy. Here you will find: SHARE Resources Data Center Visit the SHARE web page (www. shadac.org provides many resources for A web-based interactive tool shadac.org/share) to learn about analysts to understand the technical and allowing users to customize awarded SHARE grants and policy-relevant issues associated with tables and graphs of health grantee research activities. You measuring health insurance coverage insurance coverage estimates can also access SHARE briefs, and access to care. from the Current Population reports, peer-reviewed publica- Survey (CPS) and the American tions, and podcasts of grantee Community Survey (ACS). presentations. Bridging the gap between research and policy @ www.shadac.org