HEALTH POLICY CENTER RE S E AR CH RE P O R T Building the Evidence Base for Advancing Vaccine Equity Findings from the P4VE Promising Practices Project Ian Hill Zara Porter Juliana Mayer Alaisha Verdeflor with Kristen Brown and DaQuan Lawrence September 2023 AB O U T T HE U R BA N I NS T I T U TE The Urban Institute is a nonprofit research organization that provides data and evidence to help advance upward mobility and equity. We are a trusted source for changemakers who seek to strengthen decisionmaking, create inclusive economic growth, and improve the well-being of families and communities. For more than 50 years, Urban has delivered facts that inspire solutions-and this remains our charge today. Copyright © September 2023. Urban Institute. Permission is granted for reproduction of this file, with attribution to the Urban Institute. Cover image by Tim Meko. Contents Acknowledgments iv Building the Evidence Base for Advancing Vaccine Equity 1 Introduction 1 Approach and Methods 2 Findings 5 Media-Based Outreach Cohort 5 Community-Based Outreach Cohort 14 Vaccine Events and Partnerships Cohort 20 Conclusion and Crosscutting Lessons 25 About the Authors 28 Statement of Independence 31 Acknowledgments This report was supported by Cooperative Agreement CDC-RFA-IP21-2108 from the Centers for Disease Control and Prevention of the U.S. Department of Health and Human Services (HHS) as part of a financial assistance award totaling $18,162,815 with 100 percent funded by CDC/HHS. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by CDC/HHS, or the U.S. Government. We are grateful to the CDC and to all our funders, who make it possible for Urban to advance its mission. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. Funders do not determine research findings or the insights and recommendations of Urban experts. Further information on the Urban Institute's funding principles is available at urban.org/fundingprinciples. We thank our selected grantees for their hard work and commitment to promoting vaccine equity. They generously shared their time, experiences, and findings with us and made this report possible. We also extend our thanks to Brigette Courtot, director of the Partnering for Vaccine Equity Learning Community, who provided wise guidance and input throughout the project and on the content of this report. iv ACKNOWLEDGEMENTS Building the Evidence Base for Advancing Vaccine Equity Introduction The Partnering for Vaccine Equity (P4VE) program, funded by the Centers for Disease Control and Prevention, is addressing racial and ethnic disparities in adult COVID-19 and influenza vaccination. This investment in vaccine equity presents an opportunity to build knowledge and evidence on effectively increasing vaccine coverage for both the current moment and future vaccination efforts. The P4VE program launched in 2021 and will run for up to five years. With support from the Centers for Disease Control and Prevention, the Urban Institute (Urban) participates in the P4VE program through two roles. First, as the group learning manager of the P4VE learning community, Urban cultivates a collaborative environment for hundreds of national, tribal, and local organizations across the country working to build vaccine equity. Second, as a provider of subgrants and technical assistance to community-based organizations (CBOs), Urban supports CBOs' efforts to implement vaccine equity projects and build their broader, long-term organizational capacity. Within P4VE, the Building the Evidence Base for Promising Practices (Promising Practices) project supported CBOs in implementing tailored outreach strategies to advance vaccine equity and uptake in their communities. The project provided grants of up to $25,000 to support 18 P4VE-funded organizations to develop, implement, and evaluate the effectiveness of a promising practice in one of three areas: 1. media-based outreach 2. community-based outreach 3. vaccine events and partnerships Since 2021, P4VE member organizations have been involved in intense learning, capacity building, outreach, and service delivery, working to help communities be safer and healthier in the face of the ongoing COVID-19 pandemic. Equipped with Promising Practices grants, 18 of these organizations have built their capacity to measure the effects of their program activities and, in turn, learn more about the relative effectiveness of various outreach and partnership practices. Overall, the project sought to contribute to the fight against COVID-19 by building a solid foundation of evidence-based practices that promote vaccine equity and reduce disparities among underserved communities. BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 1 Approach and Methods Urban invited proposals from existing P4VE grantees to design and implement promising practices to advance vaccine equity in one of three areas: media-based outreach, community-based outreach, and vaccine events and partnerships. We received 70 applications.1 An evaluation committee composed of eight researchers from Urban's Health Policy Center reviewed and scored each complete proposal based on the following criteria: • distinctiveness of the proposed promising practice from the organization's existing and ongoing P4VE work • likelihood that the proposed project could be implemented within the time frame of the period of performance • quality of the data collection plan and its ability to identify tangible, measurable outcomes • reasonableness of the project's proposed budget and justification Eighteen organizations representing 13 states and Washington, DC, were selected for funding. Many were based in prominent cities like New York, Chicago, and Los Angeles, and others served smaller communities in the Midwest, Northeast, South, and Southwest. Target populations served by the selected organizations included Black, Hispanic/Latinx, East and South Asian, Middle Eastern and Arab, and American Indian communities. Within those communities, some organizations focused their projects on immigrants and refugees, elderly adults, low-income individuals, and individuals with limited English proficiency. The Promising Practices grantees began their six-month period of performance in August 2022. During this time, in addition to all their project implementation work, grantees engaged in the following activities: • Individual kickoff meetings with Urban to welcome the organizations to the project, discuss their applications, and review their expected activities under the grants • Action plan development in collaboration with Urban to outline their project goals, potential challenges and mitigation strategies, and data collection approaches 1 Seven applications were excluded from review and did not advance in the process because they were either incomplete or the organizations submitting them were not in the P4VE Learning Community and thus not eligible for the funding. 2 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY • Three collaboration meetings with three to four other grantees in their cohorts to facilitate peer-to-peer information sharing and implementation support Throughout the project's performance period, grantees were required to submit interim deliverables to demonstrate progress. In some cases, grantees requested one-on-one support for certain tasks, such as reviewing surveys and providing feedback on draft outreach materials. Grantees analyzed their process and outcomes data and developed data submissions and final narrative reports on findings and lessons learned related to their promising practices. As a final wrap-up activity, we held a one-on-one debrief call with each organization to discuss its work plan, findings, and how its project might influence its future work. Table 1 lists the 18 Promising Practices organizations and their projects. BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 3 TABLE 1 Promising Practices Organizations and Projects Organizations Project: Project: Target Population Project: Target Strategy Type Geography Alivio Medical Center Vaccine events Uninsured, working poor families and Chicago, IL; those with health care access barriers MIDWEST Arab American Family Community- Arab, Middle Eastern, North African, Brooklyn, NY; Support Center based Muslim, and South Asian EAST immigrant/refugee communities Beacon Charitable Pharmacy Vaccine events Black and Hispanic nonelderly adults Stark County, OH; MIDWEST Black Equality Coalition Media-based Underserved Black community Pittsburgh area PA; EAST Cancer Justice Network Community- Faith congregations, including ministers Cincinnati, OH; based MIDWEST Edu-Futuro Media-based Hispanic/Latino individuals and families Washington, DC who are non-English speaking or (DMV); EAST w/limited English proficiency El Buen Samaritano Community- Latino residents and communities with Austin, TX; based lower vaccination rates. SOUTH Equal Hope Media-based African American and Latino people in Chicago, IL; cancer treatment and other patient MIDWEST populations Latin American Legal Vaccine Events Latino, low-income, and immigrant Mercer Defense and Education Fund community members Country, NJ; EAST Make the Road Nevada Media-based Latinx individuals, including Las Vegas, NV; undocumented or partially documented WEST National Day Laborer Media-based Latin individuals and families with low- Los Angeles, Organizing Network wage workers and Spanish speakers CA; WEST Northern Indiana Hispanic Vaccine Events Hispanic/Latinx community Elkhart, IN; Health Coalition MIDWEST Philippine Nurses Community- Asian Americans Southeast MI; Association of Michigan based MIDWEST Refugee and Immigrant Media-based Ethnic communities from Burma, BIPOC Des Moines, IA; Vaccine Alliance that are refugees and immigrants MIDWEST Southern California Area Media-based Black individuals aged 65 years or older Los Angeles, National Council of Negro CA; WEST Women The Center for Black Health Media-based Black/African American adults Durham, NC; and Equity SOUTH The Public Good Projects Community- Hispanic/Latinx NATIONAL based Wabanaki Public Health and Media-based Indigenous populations living in or visiting Bangor, ME; Wellness urban areas of Maine EAST Source: Authors' data. Notes: DMV = District, Maryland, and Virginia; BIPOC = Black, Indigenous, and People of Color. All Promising Practices grantees incorporated activities to assess the effect of their project, including qualitative and quantitative data collection (e.g., social media analytic measures, surveys, focus groups, and interviews) and analysis. 4 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY Findings Below are summaries of the P4VE Promising Practices grantees' project activities and findings grouped by grantee cohort (media-based outreach, community-based outreach, and vaccine events and partnerships). Media-Based Outreach Cohort In the media-based outreach cohort, nine organizations aimed to advance vaccine equity by developing media campaigns that combatted misinformation, increased access to community-specific resources, and shared educational information about vaccinations. Although grantees spanned the country, many targeted populations, like immigrant, refugee, Latinx, and Black communities. Grantees employed various strategies for their campaigns, including social media strategies like flyers and videos posted on Facebook, Instagram, and Twitter (now known as "X") and traditional media approaches like newspaper ads, radio ads, and bus wraps. To measure the success of their campaigns, grantees collected data from focus groups, interviews, and surveys, as well as engagement metrics2 for both social media and traditional media. FINDINGS BY GRANTEE Black Equity Coalition The Black Equity Coalition (BEC) sought to increase its engagement with the Black community in the Pittsburgh, Pennsylvania, metropolitan area on issues surrounding vaccine access and equity through a comprehensive media campaign. To achieve this goal, BEC used grant funds to hire a social media specialist/influencer with a public health background who was also part of the local Black community. As a member of the local Black community and having the public health and social media expertise, hiring the media specialist was an effective use of funds. This specialist developed a range of content across social media and traditional forms of media like radio and print. BEC designed its materials to be informative (e.g., either dispelling vaccine myths or encouraging readers to get vaccinated at local events), timely (referencing the winter holidays and Black history month), and accessible (targeting older populations through traditional media and providing younger people easy access to other 2Engagement metrics is an umbrella term for tracking how actively involved audiences are with media content. Engagement metrics can include likes, comments, retweets, and reposts. Impressions are how many times a post appears in someone's feed or timeline. Reach is the number of potential unique viewers a piece of media could have. BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 5 resources through QR codes). BEC put out content through Facebook, Instagram, and Pittsburgh's Black-owned radio station (WAMO) and newspaper (the New Pittsburgh Courier) and distributed print materials at schools and senior centers (figure 1). BEC's total engagement across its social media platforms increased by 1,600 percent, with Instagram up by 1,900 impressions, Facebook up by 2,500 impressions, and the paid reach up by 3,500 impressions. BEC increased the geographic scope of its engagement beyond Pittsburgh to include Philadelphia, Georgia, North Carolina, and Washington, DC. BEC's radio ads totaled 91,000 impressions, the highest number of its campaign compared with its social media posts. Although BEC did not collect metrics for its newspaper ads, the newspaper itself has a readership of over 5,000. After implementing its media campaign, BEC concluded that communication from a trusted source, in this case the social media specialist, was a vital component of the campaign's success. BEC also emphasized the importance of accessible messaging, like targeting older audiences through traditional media rather than social media; targeting younger audiences with QR codes that easily linked to other resources; and taking advantage of media platforms that already had high traffic from its target population, like the local radio station. BEC plans to continue working with its social media specialist to create tailored messaging for its community. 6 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY FIGURE 1 Black Equity Coalition Flyers Source: Reused with permission from the Black Equity Coalition (BEC). Notes: BEC created these flyers (among others) as part of its media campaign to reach the Black community in Pittsburgh. Each flyer included a QR code that linked to BEC's website. BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 7 Center for Black Health and Equity The Center for Black Health and Equity (CBHE) extended its national social media campaign to promote its Truth Check initiative. The Truth Check campaign aims to combat COVID-19 vaccine misinformation spread on social media and increase vaccine confidence by empowering users with the skills to discern factual from misleading information. The campaign included a post-assessment that evaluated users' media literacy skills after completing the Truth Check training. To promote this training, CBHE initiated a paid social media ad campaign to drive traffic to its site and increase participation in the post- assessment. The results from CBHE's assessment were used to determine users' retention of information and media/digital literacy. Of the 38 respondents, 63 percent remembered taking the Truth Check training, with over 50 percent mentioning the training helped them develop practices to more effectively assess social media for misinformation. CBHE learned that it needs to recruit a diverse pool of survey respondents and tailor its messaging based on region, as some respondents were split across the South and Atlantic Northeast. The grantee also reported that community members responded positively to gift card incentives. CBHE noted that much of its project's campaign work involved raising public awareness of COVID-19 and available vaccines. CBHE specifically considers its work as "pre-bunking," or preemptively debunking, disinformation around COVID-19. CBHE elevated the importance of repetition and consistent messaging in its pre-bunking work, declaring, "If we expect people to continue to utilize Truth Check, we must continue to remind them that it exists." Edu-Futuro Edu-Futuro developed six short Familia y Vida videos providing culturally relevant vaccine information in two formats and conducted focus groups to determine which video format was preferred by Hispanic/Latinx parents in Northern Virginia. Three videos followed a host/Zoom format with the host speaking directly to an expert, and three videos used a host/reporter format in which a freelance journalist shared a news story with "on-location reporting." Edu-Futuro distributed the videos via its social media channels (Facebook, WhatsApp, and YouTube) and conducted two focus groups with 59 parents to obtain consumers' opinions. From the focus groups, Edu-Futuro learned that parents' opinions were almost evenly split, with 51 percent preferring the host/Zoom interview format and 49 percent preferring the host/reporter format. However, when Edu-Futuro looked at the preferences by age group, it learned those respondents 40 to 50 years old had more distinct preferences, with 62 percent preferring the host/reporter format and 38 percent preferring the host/Zoom format. Based on challenges finding Spanish-speaking freelance reporters and higher expenses associated with creating the host/reporter 8 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY videos, along with these findings, Edu-Futuro decided to focus on creating host/Zoom videos in the future, but it may create host/reporter videos when it specifically wants to target potential viewers in the 40- to 50-year-old age range. Two particularly surprising findings from the focus groups were the identification of social media as the principal source of news and information for the majority of Hispanic/Latinx parents and, importantly, that parents were not able to distinguish between credible news outlets, like Univision or Telemundo, and their Facebook feed. A large proportion of Hispanic/Latinx participants in the focus groups considered information obtained from social media as credible news, even though it was most often sent to them by family and friends and, in reality, was full of COVID-19 disinformation. To combat this disinformation, Edu-Futuro will focus on creating culturally appropriate videos featuring experts with tips to spot Spanish-language disinformation on social media. In addition, Edu-Futuro plans to conduct focus groups in its P4VE work going forward to continue engaging with its community in the most effective manner. Equal Hope Equal Hope implemented a media-based outreach campaign in Chicago targeting Blacks and Latinos with comorbidities, including cancer. Specifically, this grantee conducted a literature review to identify vaccine-related topics as the basis for developing effective, targeted messages for providers to use in conversations with cancer patients. Several topics emerged from the analyses, including concerns about the speed of vaccine development, lack of trust in government, the importance of trusted messengers, increasing communities' knowledge, and structural racism. Based on the literature review and inspiration from other reputable sources, Equal Hope developed a COVID-19 resource page on its website and culturally appropriate handouts. Equal Hope used email listservs and social media platforms to advertise that providers could download these resources from its website. Although there was some success-4,776 emails were sent, and there were 482 impressions on Instagram, 114 on Twitter, and 28 on Facebook-these contacts resulted in only nine downloads during the tracking period. Equal Hope learned that accessibility is critical in distributing health materials. Initially, downloading the handout required logging onto Equal Hope's website. This extra step was likely a deterrent for some providers. Second, in consulting with media experts, Equal Hope learned the importance of condensing technical and scientific information into a few sentences written at a grade school reading level and using icons and photos to illustrate calls to action to create accessible materials. Lastly, budgeting for social media advertising and "boosting" likely would have increased BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 9 impressions and engagement with the posts. Equal Hope prioritized this strategy in its future work rather than relying on organic views. Make the Road Nevada Make the Road Nevada (MRNV) sought to understand and improve the effectiveness of online vaccine outreach to its Latinx, migrant, and Spanish-speaking target communities in Las Vegas by increasing the resources behind its paid digital media campaign. To do this, MRNV designed social media advertisements for various platforms, tailoring messages throughout its campaign in response to community feedback. MRNV created 20 Facebook posts, 7 Instagram posts, and 13 Twitter posts and sent three digital newsletters to 4,361 recipients. Each month, these social media accounts reached approximately 2,500 people, with about 1,000 people engaging with the posts and 18 new people following. MRNV found that younger audiences using Twitter and Instagram responded best to English posts, while older audiences using Facebook responded best to Spanish posts. MRNV's data showed that Thursdays had the highest engagement, suggesting that is the best day to post. MRNV reported positive feedback from the community for family-friendly events and adjusted its events and messaging accordingly. For example, MRNV hosted "Trunk or Treat," a socially distanced Halloween event where children and families could safely trick-or- treat out of each other's car trunks in a public parking lot rather than traversing their neighborhoods. MRNV's project showed the importance of differentiating language and content based on each social media platform for maximum engagement and reach. Additionally, MRNV emphasized centering its messaging around values important to its community, namely family. MRNV also provided practical insights, like how to vary formats across Twitter, Instagram, and Facebook. National Day Laborer Organizing Network The National Day Laborer Organizing Network (NDLON) sought to use its radio show, Radio Jornalera, to develop culturally competent messaging to combat COVID-19 misinformation and increase vaccination rates within its Los Angeles Latinx community. NDLON employed several strategies, first gathering baseline perceptions about vaccines and existing myths through focus groups and then attacking those myths through testimonial videos from trusted community members. This work uncovered that many sources of misinformation originated from faith-based institutions and online conspiracy theories, both exacerbated by NDLON's immigrant population's hesitancy to rely on governmental sources given reported governmental and societal mistreatment of immigrant populations. Using this information, NDLON produced and launched a series of video stories featuring community members directly impacted by COVID-19 and hosted a town hall with a Spanish-speaking Latina doctor to discuss and dispel myths. NDLON produced three audiovisual products and two radio 10 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY shows and aired these products over six radio stations in the US (in California, Massachusetts, Minnesota, New Jersey, Texas, and Washington State) and three radio stations in Latin America (in Costa Rica, El Salvador, and Guatemala). Through these activities, Radio Jornalera reached 143,500 people during the grant period. Beyond these media efforts, NDLON hosted two vaccine clinics, with an average of 65 individuals participating in each. A post-campaign survey affirmed the success of NDLON's work, suggesting its campaign had a strong reach and positive effect. Specifically, of the 24 day laborers surveyed in a random sample, all but one had seen COVID-19 materials from the campaign, and all but one was vaccinated with at least one dose. Although this is a small sample size, NDLON's project demonstrated how trusted institutions could effectively harness their preexisting networks to combat misinformation, especially when messaging comes from peers like other day laborers or someone who is a trusted member of the target community, like the Spanish-speaking Latina doctor at the town hall. This project also demonstrated the efficacy of accessible messaging and catered messaging. For example, NDLON recognized many day laborers are most likely to listen to and trust a radio show already incorporated into their routine and more likely to trust messaging from Spanish speakers they can easily understand. NDLON plans to continue to center the voices of community members in future campaigns. Refugee and Immigrant Vaccine Alliance The Refugee and Immigrant Vaccine Alliance (RIVA; previously called the Ethnic Minorities of Burma Advocacy and Resource Center) based in Des Moines, Iowa, used grant funds to design, develop, and disseminate 36 videos in 12 languages (Bosnian, Burmese, English, Falam Chin, Hakha Chin, Karen, Karenni, Korean, Mizo Chin, Spanish, Tigrinya, and Zomi Chin). The videos were organized into six series related to COVID-19 and vaccines. The video topics were selected carefully based on conversations with community leaders, partner organizations, and community members in Iowa to create adaptable, accessible, and targeted COVID-19 messaging. RIVA created QR codes on flyers for partners to share at events to link community members to the informational videos. RIVA also created a 15-page communications toolkit that includes guides to plain language, accessible communication, and social media; suggested strategies for communicating with local media; and customizable templates in three languages that partner organizations can adapt for work with their target communities. RIVA disseminated the toolkit to 12 partner organizations. RIVA collected quantitative and qualitative data on video engagement and feedback on the toolkit. The 36 videos had 980 views and 33 likes on YouTube, whereas the videos posted on Facebook had a reach of 13,270 and 1,290 likes and reactions. Partner organizations expressed their gratitude, especially for the plain-language guide, which they described as the most useful resource in the toolkit BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 11 for explaining how to communicate complicated topics clearly, especially for languages with a non-Latin script, such as Burmese, Karen, and Karenni. Partners also shared that the templates and sample messages on respiratory syncytial virus (RSV) in the toolkit displayed clear and easy-to-understand information for the Latinx community. Since distributing the toolkit to its partners, RIVA has received increased requests for technical support. RIVA hopes to build on this momentum and continue providing technical assistance and social media support to CBOs. In addition, RIVA plans to continue engaging with community members and CBOs to incorporate their feedback into products. Southern California Area National Council of Negro Women The Southern California Area National Council of Negro Women (SCA-NCNW) organized and engaged Black organizations and allies to increase the number of COVID-19 vaccinations and boosters among the Black community in Los Angeles County, specifically targeting seniors ages 65 and older. This organization planned and conducted a range of activities, including developing print materials, conducting a media interview with campaign partners, creating a video story on vaccinated Black seniors, hosting a press conference with elected officials, creating a toolkit of culturally relevant outreach materials to distribute throughout Los Angeles County, promoting and hosting sponsored community-based vaccination clinics with partner organizations, and distributing surveys at the events to gauge vaccination status. SCA-NCNW worked with 13 partners, including churches, health care providers, and the Los Angeles County Department of Public Health, and distributed over 1,000 surveys at three events. Over 200,000 Black community members attended these events, where partners provided 147 vaccinations. The project's video story of Black seniors discussing their experiences accessing COVID-19 vaccines and care had over 2,000 views on Facebook. From these activities, the organization plans to continue engaging in collaborative conversations with community members to provide a forum for sharing their experiences. Wabanaki Public Health & Wellness Wabanaki Public Health and Wellness (WPHW) set out to better reach Native Americans living in urban areas of Maine (Portland and Bangor). WPHW serves four federally recognized tribes located in five communities: the Houlton Band of Maliseet Indians, the Aroostook Band of Micmacs, the Passamaquoddy Tribe at Indian Township, the Passamaquoddy Tribe at Pleasant Point, and the Penobscot Nation. To implement its strategy, WPHW created three bus wraps and 26 interior signs that ran for two months in Portland and Bangor, each featuring COVID-19 messages created by its staff and Indigenous artwork created by its community (figure 2). The bus wraps and signs included a QR code that linked to WPHW's COVID-19 web pages and a survey asking questions about COVID-19 12 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY experiences. WPHW also designed corresponding print materials, Facebook and Instagram posts, and a survey to gauge community response to the bus advertising strategy. FIGURE 2 Wabanaki Public Health and Wellness Bus Wraps and Interior Signs Source: Reused with permission from Wabanaki Public Health and Wellness (WPHW). Notes: The image on the left displays artwork created by Wabanaki community member Reanna Sockabasin. The artwork was featured on bus wraps, interior bus signs, and printed materials, as well as posted on Facebook and Instagram. The image on the right shows a bus wrap featuring the artwork in Bangor. WPHW worked with ATA Outdoor to run bus wraps for two months in Bangor and Portland. The estimated number of impressions on the bus wraps was over 300,000 in each location, while the number of impressions on WPHW's Facebook and Instagram posts ranged from 297 to 783. WPHW received an overwhelming amount of positive feedback from the community about the bus ads, especially the Indigenous art. The impressions data and feedback on the bus wraps demonstrated how traditional media campaigns can successfully reach urban Native American populations who use buses to commute to and from work and school. WPHW's use of traditional art created by an Indigenous community member in its campaign created a connection point for those who identified as part of the Wabanaki community and served as a tool to educate and inform the community about the services the grantee offers. WPHW plans to continue building out its website and applying for projects that allow for multiple touchpoints throughout its organization. In addition, WPHW received funding from another source to extend its bus wrap campaign for two more months. LESSONS LEARNED The media-based outreach grantees learned several valuable lessons from their projects. BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 13 The majority of organizations created culturally competent, accessible messaging to engage with their communities, whether it was through social media (Facebook, Instagram, YouTube, WhatsApp) or traditional media (bus wraps, radio ads, toolkits). They found that response and engagement levels with different media sources and strategies varied by age group. For some grantees, creating accessible messaging meant incorporating easy access to additional information. Others facilitated their messaging by taking advantage of already existing, accessible networks. Lastly, many organizations ensured language accessibility for populations that did not speak English as a first language by translating their materials into as many as 12 languages. Although social media are certainly effective, as BEC demonstrated by extending its network beyond the Pittsburgh area to other Pennsylvania cities and other states, many organizations found great success with traditional forms of media, with total impressions often higher in traditional media, such as with WPHW's bus wraps. Many organizations also learned the importance of involving the community in creating the messaging and centering their messaging around the community's needs to increase engagement. The significance of community involvement was demonstrated by feedback shared in focus groups, surveys, interviews, and partner meetings. Many organizations plan to continue centering community voices in future work. Two primary strategies emerged for involving communities and organizations in message development. Some organizations included community members' voices in their work from the start, while others incorporated feedback from the community in their campaigns. Finally, a handful of organizations learned more about the pervasiveness of misinformation throughout social media, especially in Spanish, and are planning activities to combat misinformation in their future work. Community-Based Outreach Cohort With organizations based in the Midwest, Northeast Atlantic, and Southern regions of the US, the five grantees in the community-based outreach cohort implemented and evaluated promising practices designed to directly engage with priority populations. This cohort employed a range of activities, including hosting informational sessions and workshops, conducting surveys, and collaborating with partner organizations to build a health equity–focused coalition. FINDINGS BY GRANTEE Arab American Family Support Center The Arab American Family Support Center (AAFSC) implemented its community-based outreach plan targeting Arab, Middle Eastern, North African, Muslim, and South Asian immigrant and refugee communities in Brooklyn, New York. AAFSC conducted six "The Science behind the COVID-19 Vaccine" 14 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY workshops reaching 114 participants. It developed a culturally appropriate survey tool to assess workshop participants' knowledge of vaccine-related issues, such as the function of COVID-19 vaccines and boosters, sources of reliable information, and knowledge of potential side effects. The surveys were translated into Arabic, Bangla, and Spanish by AAFSC team members with extensive experience with and knowledge of local New York City immigrant communities and the language and phrasing that enhance accessibility and mitigate stigmatization and misunderstanding. AAFSC administered the surveys before and after each workshop to measure the workshop's impact on vaccine literacy and provided $10 gift card incentives for participation. AAFSC found that existing knowledge surrounding COVID-19 vaccinations was higher before the workshops than earlier in the public health emergency (figure 3). The most significant knowledge gained after the workshop concerned vaccine boosters and the degree to which additional doses were effective, safe, and necessary. AAFSC also learned that the greatest amount of fear and mistrust centered around child vaccines, which suggests that family-centered vaccine messaging would be a useful strategy for community engagement. BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 15 FIGURE 3 Arab American Family Support Center Workshop Evaluation Results, Pre/Post Evaluation Outcome Results–Self-Perceived Knowledge/ Attitudes Pre/Post Evaluation Results 100% +12% +12% 90% +12% +2% +21% 80% +13% 70% 60% 50% 40% 30% 20% 10% 0% I understand I understand I have an idea of I have an idea of I am familiar I know where to that the how the what side what side with how go to get factual COVID-19 pediatric effects to expect effects to expect different types information vaccine protects COVID-19 when I get a when my of COVID-19 about the me against vaccine protects COVID-19 children get a vaccines and COVID-19 COVID-19 my children vaccine. COVID-19 boosters can be vaccine. infection. against COVID- vaccine. mixed and 19 infection. matched. % Agree in Pre Test % Strongly Agree in Pre Test % Agree in Post Test % Strongly Agree in Post Test Source: Reused with permission from the Arab American Family Support Center. Notes: Six of the eight survey questions assessed self-perceived knowledge and attitudes related to COVID-19 vaccination and asked respondents to describe their level of agreement or disagreement with a series of statements. This graph depicts the increase in the percentage of respondents who agreed or strongly agreed with each statement, with an average increase of 12 percent. Outcomes demonstrating the most significant increase were knowledge of COVID-19 boosters (+21 percent) and knowledge of COVID-19 side effects for children (+13 percent). Cancer Justice Network Operating in Cincinnati, Ohio, the Cancer Justice Network (CJN) trained six neighborhood health navigators to serve as influential messengers embedded in various religious institutions (e.g., churches, synagogues, mosques) in high-risk communities in Cincinnati. CJN's health navigators engaged with 16 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY congregations and their leadership to learn the needs and priorities of the community and share COVID-19–related information. CJN partnered with other CBOs to host three events focused on broad-based health topics, such as COVID-19 vaccine information, diabetes, cancer, and heart disease. CJN focused on reaching the unvaccinated population and reported data from vaccination and vaccination education events. In all, 1,440 individuals attended CJN's 16 education and 20 vaccination events, and 253 were vaccinated at the events. CJN also created a coalition for health justice that convened leaders of religious institutions, housing projects, and community agencies in areas that lacked health care. The coalition was a new venture, as no existing Cincinnati-based interfaith organizations had committed to changing the health care landscape for underserved populations in their organizations and communities. CJN learned that having community partners and individuals who are trusted community members is important for engaging members of at-risk and marginalized populations. CJN endured challenges regarding misinformation about COVID-19, vaccinations, side effects, and long COVID and fears stemming from religious beliefs, but the organization learned that its trusted messengers (i.e., health navigators) were effective in dispelling community members' fears about vaccination. El Buen Samaritano El Buen Samaritano (El Buen), based in Austin, Texas, expanded its efforts to understand and respond to vaccination barriers that Latino residents in the Austin area experience. El Buen focused on understanding the concerns of unvaccinated community members and the obstacles they encountered. The organization tested new outreach strategies via social media and text messaging campaigns. El Buen also developed and fielded a 17-item survey to 615 Austin area residents to ascertain how community members chose El Buen's vaccine clinics over other vaccination opportunities. Respondents selected El Buen primarily because of existing deep community trust and incentive payments, and most learned about the vaccination opportunity via El Buen's outreach efforts (e.g., text messages, internal referrals from other El Buen programs, social media campaigns, and radio ads). Almost all (94 percent) vaccine recipients were unaware of El Buen's vaccination education efforts before outreach. El Buen found that individuals who wanted more information about vaccines were not necessarily receptive to direct outreach attempts, as very few of these individuals responded to follow-up offers to discuss concerns or learn more about vaccine safety. El Buen expressed the importance of community trust and long-standing partnerships, which were more influential than newer partnerships. El Buen discovered that information sharing within families and incentive payments were important factors in vaccination efforts. After early discussions and testing with partners and community members, El Buen found that focusing on more shorter questions BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 17 (true/false or multiple-choice questions with five or fewer options) rather than fewer open-ended questions generated more complete surveys. El Buen's findings indicate additional research on vaccine- hesitant people is needed. Philippine Nurses Association of Michigan Based in Southfield, Michigan, the Philippine Nurses Association of Michigan (PNAM) sought to build the evidence base on vaccine literacy and acceptance of COVID-19 and flu vaccinations among Asian, Arab, and Middle Eastern communities. PNAM recruited and trained 21 community ambassadors from various Asian and Middle Eastern racial/ethnic backgrounds and adapted an existing vaccine literacy survey to a variety of languages. PNAM's 46-item survey gathered extensive data on vaccine literacy, attitudes and perceptions, and beliefs and behaviors. The survey investigated two kinds of vaccine literacy. Functional literacy refers to questions mainly about language, and interactive-critical literacy refers to questions focused more on problem-solving and decisionmaking. PNAM administered 527 surveys (with a 99 percent return rate). PNAM's survey showed Asian and Arab American respondents had limited functional literacy (mean score 2.36 out of 4) but comparatively higher interactive-critical literacy (mean score 3.07 out of 4). PNAM also found a statistically significant positive correlation between vaccine literacy and educational level and a slightly lower vaccine literacy rate for women surveyed. Other findings indicated that participants had a largely positive attitude about vaccines and considered them safe and effective. The results showed that more than 80 percent of Asian and Arab Americans were up to date on their COVID-19 and flu vaccinations, with only a small percentage of survey takers sharing they wouldn't consider getting the COVID-19 (5 percent) or flu (19 percent) vaccine. Survey respondents were diverse in age, gender, and notably, country of origin, with 21 countries represented. The Public Goods Projects A public health communications nonprofit, the Public Good Projects (PGP), sought to conduct a cross- sectional survey of Hispanic adults across the US to assess whether there was heterogeneity in the use of relevant Spanish words for English terms (e.g., the flu, common cold, COVID-19, masks, and booster shots) and to monitor and analyze trends in COVID-19 and COVID-19 vaccination misinformation. PGP found there was homogeneity in preferred Spanish word choice. The most commonly used terms for flu across all demographics were gripe or flu. For the common cold, for most demographic groups, the most popular words were resfriado or catarro. For vaccines, respondents overwhelmingly used the term vacuna. For mask, the most commonly used words were mascara or mascarilla. The most commonly used words for booster shots or doses were refuerzos, vacuna, and booster. For the survey's misinformation- monitoring component, PGP captured 9.9 million English- and 157,000 Spanish-language posts that 18 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY explicitly expressed misinformation, negative attitudes, or opposition to COVID-19 and/or COVID-19 vaccinations. The English messages had 38.6 billion potential impressions, and the top four negative hashtags were #vaccinedeaths, #vaccineinjuries, #stoptheshots, and #pfizer. The Spanish-language posts had 405,800,000 potential impressions, and the top five negative hashtags were #repentinitis, #yonomevacuno, #covid19, #plandemia, and #vaccinedeaths. Whereas many studies examine Hispanic communities as monolithic, PGP's project was important because it sought to understand diversity (e.g., age and generation, geographic region, and language preference) within the Hispanic/Latinx community (figure 4). PGP learned that it is important to monitor low-level, persistent misinformation. The organization recommends using Hispanic either alone or in conjunction with gender identity–inclusive terms such as Latine or Latinx when referring to the population, and it also suggests intentionally recruiting influencers and/or digital volunteers from rural communities, which had pervasive misinformation compared with urban or suburban communities. FIGURE 4 The Public Goods Project Survey Findings on Preference for Community Terminology by Generation Source: Reused with permission from the Public Goods Projects. Notes: This graph shows respondents' preferences for the terms Hispanic, Latino, Latina, Latine, and Latinx. Among all age groups and generations, the most popular term was Hispanic, followed by Latino/a. LESSONS LEARNED Grantees in the community-based outreach cohort reported several key and consistent themes that will be useful for their organizations' work in the future. Being trusted by the community emerged as a key requirement to accomplish vaccine equity work, and activities designed to build trust (e.g., sharing a meal) are a critical component of relationship building. Culturally sensitive communication is also important to effective vaccine equity work. Creating materials such as surveys and social media campaigns in the language understood best by the targeted population facilitated engagement with vaccine education. Community members should also be involved in the planning stages of material BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 19 development to best capture cultural factors (e.g., history, humor, word choice) that resonate with the targeted audience. Grantees also mentioned the importance of collaboration with partners within and across sectors, assessing how collaboration among groups can preemptively address population health care needs, and integrating lessons learned into concepts and their evaluation processes. Further, grantees in this cohort consistently reported that racial/ethnic groups are not monolithic, and additional work that addresses heterogeneity within groups is needed. Finally, there should be continuous monitoring of low-level, persistent misinformation to support vaccination messaging efforts. Vaccine Events and Partnerships Cohort Within the vaccine events and partnerships cohort, the four Promising Practices grantees aimed to advance vaccine equity by executing and evaluating community-based vaccination events. Although grantees were in different areas of the country, each shared a similar goal to increase vaccinations within their communities, particularly Black, immigrant, and Latino communities. Grantees employed various methods to drive vaccinations, including hosting events in community locations, incorporating cultural elements, and assessing the current practices of their vaccination clinics. To evaluate their success and further understand their communities' needs, grantees administered satisfaction and demographic surveys that collected information on participant experience, health behaviors, barriers to care, and reasons for attending the vaccine event. FINDINGS BY GRANTEE Alivio Medical Center Alivio Medical Center (Alivio) in Chicago sought to coordinate large community events that incorporated cultural elements specific to its target Latino population, such as Mexican Mother's Day and Day of the Dead, while using a strong in-person outreach promotional plan. Alivio coordinated three events throughout its project: Caminando Juntos (Walking Together), a turkey giveaway, and a community and winter resource fair. Twenty-six vaccines were administered at these three events, a lower than hoped-for result. Although Alivio was able to conduct three large events, it could not incorporate the different cultural elements as part of those events. Project staff cited various challenges, such as staffing changes during the project period, which affected their ability to execute the intended plan. Although Alivio could not complete the project's original goal, it showed strengths in community outreach, partnership building with other CBOs, and providing the information needed to make vaccine decisionmaking in a 20 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY culturally competent way. Throughout the project period, Alivio partnered with 35 organizations and participated in 85 community events, where it administered 306 COVID-19 vaccinations, provided COVID-19 rapid test kits, and referred 169 people to resources such as Women, Infants, and Children and the Supplemental Nutrition Assistance Program. During this period, Alivio also had a strong social media campaign that included frequent informational posts about COVID-19 or another health-related topic, Facebook livestreams, and a YouTube rapid test tutorial video. Beacon Charitable Pharmacy Beacon Charitable Pharmacy (Beacon) brought clinical pharmacy services into community settings to increase access to vaccine education and vaccinations among the Black and Hispanic populations in Stark County, Ohio. Beacon first identified geographic areas with high rates of low- to moderate- income residents and low rates of COVID-19 vaccination. Beacon partnered with local organizations within those communities to implement 56 vaccine education events and administer 76 surveys between September 1, 2022, and February 28, 2023. Beacon also launched a Facebook social media campaign to assess engagement levels of whole-health messaging compared with vaccine-only messaging. Beacon posted 137 times between October 1, 2022, and February 27, 2023, on topics such as COVID-19, breast cancer awareness, antibiotic awareness, and other health topics. Overall, Beacon's efforts for community-based approaches produced actionable findings. Attendance was higher and the target population was reached more effectively when events were located within community settings compared with Beacon's location. Despite Black and Hispanic residents comprising only 10 percent of the Stark County population, between 50 and 75 percent of Beacon vaccination recipients identified as Black or Hispanic. In total, 127 attendees received their COVID-19 vaccines and 71 received their flu vaccines, with 55.9 percent of those receiving the COVID- 19 vaccine and 73.2 percent of those receiving the flu vaccine identifying as Black or Hispanic, respectively. Through its surveys, Beacon learned that support for community-based approaches was high: 79.2 percent of respondents indicated that they found it "very important" to have health events in community settings. Although there was less support for COVID-19 vaccines and education in community settings, 62.5 percent still found it "very important," with 25 percent finding it "somewhat important." Beacon will continue to provide vaccine education and outreach in community locations as a result of these findings. The results of Beacon's Facebook campaign varied. Between October 1 and December 31, 2022, posts that included vaccine messages as part of broader health messages had the most impressions, engagement, shares, likes, and clicks compared with vaccine-only or other topics. Yet, between January and February 2023, posts that addressed only general health without mention of COVID-19 had more BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 21 engagement, including comments, likes, and clicks, while vaccine-only messages had the most impressions. However, it was difficult for Beacon to precisely classify its posts as vaccine-only or broad- health messaging as there was significant overlap in how vaccine messages were communicated. Thus, Beacon could not accurately assess the Facebook campaign's original goal. Beacon learned that the key to increased Facebook engagement was for Stark County Health Department to share the posts. As a result of this project, Beacon now works with the Stark County Health Department communications team to share key Facebook posts. Latin American Legal Defense and Education Fund The Latin American Legal Defense and Education Fund (LALDEF) aimed to expand its vaccination efforts outside its headquarters in Trenton, New Jersey, to other community locations to compare attendance and outcomes. LALDEF hosted four events at local churches and other CBOs in three of Trenton's four wards. It worked with 15 schools, businesses, and churches for event promotion; coordinated with other partner organizations to distribute flyers; provided toiletries as incentives to more than 100 attendees; and facilitated the delivery of 37 COVID-19 and influenza vaccines. LALDEF collected 19 verbally administered surveys at these community events compared with 27 surveys collected at its headquarters prior to the Promising Practices project period. LALDEF found that by winter 2023, events at other CBOs had higher attendance than events at its organization headquarters, where it had already held numerous vaccination events. In winter 2023, vaccine events at other CBOs had a slightly higher proportion of Hispanic attendees-LALDEF 's target population-than events at LALDEF headquarters (95 and 89 percent, respectively). Events at other CBOs help to address a barrier in access for clients who may have lived far from LALDEF's headquarters; based on its survey data, 59 percent of survey respondents cited convenience as the reason they received a vaccination (figure 5). Thus, hosting community events and "meeting people where they are" are important to successful vaccination events. However, LALDEF noted that comparing this particular result to results from other headquarters events may be difficult as the surveys were slightly different. LALDEF also noted that convenience was likely a factor attracting attendees to events at its headquarters events, as well. 22 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY FIGURE 5 Latin American Legal Defense and Education Fund Survey Analysis Source: Reused with permission from the Latin American Legal Defense and Education Fund. Notes: Fifty-nine percent of survey respondents cited convenience as the reason they chose to get vaccinated. This percentage is substantially higher than the next most common reasons respondents said they chose to get vaccinated: 18 percent said they chose to get vaccinated because it was free, and another 18 percent cited their health or mentioned that they "needed" the vaccine. Northern Indiana Hispanic Health Coalition The Northern Indiana Hispanic Health Coalition (NIHHC) aimed to evaluate its practices by comparing the wait times at drive-through and in-person vaccination clinics and the wait times of clients with and without appointments. NIHHC held two outdoor drive-through vaccination clinics and three indoor vaccination clinics. During the five clinics, NIHHC administered 265 satisfaction surveys to learn about how participants heard about the event, patient experience, reactions to and preferred outreach methods, and barriers to care. NIHHC found that both types of vaccination clinics were effective. Drive-through clinics were quicker than in-person clinics (18.4 and 20.5 minutes, respectively, from entry to exit). Having an appointment for the outdoor clinics was approximately 2 minutes more efficient than without an appointment (17.8 and 19.5 minutes, respectively), but appointments for indoor clinics were only slightly more efficient (by less than a minute). However, although drive-through clinics were more prompt, indoor clinics provided a different atmosphere and allowed for more engagement. Participants at indoor events were able to learn about other health screenings and resources and could connect with other participants, community members, and organizations, thus creating a stronger sense of community. NIHHC found that although event efficiency was important, how the work is done matters perhaps even more: engaging with and building trust in the community lead to successful vaccination efforts. Based on NIHHC's survey data, 26 percent of survey respondents heard about the event via the BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 23 organization's partnership with a popular Spanish radio channel; 24 percent heard of the event through NIHHC's phone-a-thon, during which community health workers contacted people via phone and answered any questions or concerns that they may have had concerning the vaccine; and 18 percent of NIHHC survey respondents indicated that they heard of the event through their friends and family. Additionally, understanding community needs and providing culturally competent care for the community are crucial. Based on NIHHC survey data, 31 percent of respondents indicated that language was a significant barrier to care; half of all respondents indicated they came to the event because the staff spoke Spanish; and 44 percent indicated they came to the event because of respectful treatment (figure 6). FIGURE 6 Comparison of Participants' Reported Barriers to Care and Reasons for Attending NIHHC Events Source: Reused with permission from the Northern Indiana Hispanic Health Coalition (NIHHC). Notes: Spanish-speaking staff was the top reason why attendees came to NIHHC's COVID-19 clinics (53 percent; n = 141), which correlates to language barrier, which was the third-most frequently reported barrier to health care (31 percent; n = 82). Approximately 35 percent (n = 93) of attendees reported lack of health insurance as a barrier to health care. The same number of attendees (35 percent; n = 93) attended the event because health insurance was not required. LESSONS LEARNED The vaccine event cohort grantees shared common lessons and themes that emerged from their work. Meeting community members "where they are" was a significant lesson for successful events. The grantees found attendance was higher when events were held in community settings than their organization headquarters, especially among their target populations. Additionally, partnering with 24 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY trusted community organizations and using trusted messengers in promoting events were critical. Among all grantees, having established relationships with other CBOs, churches, media channels, and local health departments enhanced their ability to reach the community by assisting with flyer distribution, providing a venue to host events, or boosting their reach to larger audiences on social media or radio. Grantees expect to continue offering events in the community, making a more concerted effort to partner with other CBOs, and providing culturally and linguistically competent care. Conclusion and Crosscutting Lessons The Partnering for Vaccine Equity's Promising Practices project accomplished what it set out to accomplish: it contributed to the evidence base of what works in promoting and achieving more equitable vaccine access. Many of the lessons learned were not new; in fact, most were ones that P4VE- funded organizations have been learning over their years of work with the Centers for Disease Control and Prevention. But this project permitted participating grantees to focus a small but critical amount of resources on testing and evaluating their outreach and access strategies and, ultimately, to document and reinforce the evidence base of what works. In so doing, Promising Practices also built the capacity of these grantees to design and conduct evaluations and collect and analyze data, whether quantitative or qualitative. As described above, grantees strengthened their appreciation and understanding of the importance of the following: • holding events within community settings that are familiar to community members • developing trusted relationships within the community and working closely in and with the communities they wish to reach and serve • tailoring and targeting culturally and linguistically appropriate messages to reach distinct populations and groups • using diverse media and strategies to access different audiences and maximize the overall reach of messaging • promoting COVID-19 vaccine information and education within a broader public health or holistic health context • countering the constant, rampant misinformation in social and print media that is shared among friends and family about COVID-19 and how to prevent and protect oneself from it BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 25 Interestingly, these lessons were not uniquely reported by any single cohort. Rather, they were shared and consistent across most grantees, whether they engaged in media- or community-based outreach or in implementing vaccine events and partnerships: • The importance of engaging and building trust within the community was witnessed through the vaccine events held by LALDEF in the local boroughs of Trenton, which were better attended and more successful than previous events held in its headquarters in the city. BEC succeeded in reaching Black communities in Pittsburgh by placing ads and information with a Black-owned radio station and newspaper. PNAM's community-based efforts succeeded when local Asian and Arab American individuals were hired to administer vaccine literacy surveys to their respective communities in the suburbs of Detroit. • Successful efforts to develop and refine more culturally competent messaging were visible in both media-based outreach efforts (such as RIVA's production of brief testimonial videos in 12 languages to reach its diverse target populations) and community-based outreach efforts (such as the workshops and surveys developed in multiple languages by AAFSC for Arab, Middle Eastern, North African, Muslim, and South Asian immigrant and refugee communities in Brooklyn). Members of the vaccine events and partnerships cohort noted similar successes. NIHHC's drive-through and in-person vaccination clinics were highly attended because, according to surveyed attendees, Hispanic families knew they could obtain care from providers who spoke their language and because the clinics would provide a convenient opportunity to engage with other community members. Perhaps most important, the Promising Practices project will have a legacy that lasts far longer than the grant's brief period of performance. Grantees in all cohorts shared that the skills they developed and lessons they learned through the project will significantly influence their work moving forward. Specifically, they reported that Promising Practices allowed them to build staff capacity and hone data collection and analysis skills. Furthermore, it aligned with and extended their existing P4VE work by permitting experimentation with new strategies and learning about the relative effectiveness of alternative strategies. Many grantees acknowledged the value of investing in paid social media outreach and planned to continue those investments so they could continue to obtain valuable data on the impacts of their messages. Similarly, because the strategy worked so well with their Promising Practices project, some grantees said they would devote resources to hiring a full-time community- based influencer to lead their traditional and social media outreach. The successful deployment of population-specific toolkits for partner organizations convinced some grantees to continue to develop and refine those toolkits. And, equipped with a better appreciation of the value of more formal 26 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY evaluation, grantees consistently reported that they would continue to intentionally incorporate data collection and analysis into future vaccine outreach and access efforts to learn how to maximize those efforts. In conclusion, the Building the Evidence Base for Advancing Vaccine Equity project supported the efforts of 18 community organizations to invest in and test new strategies to improve vaccine equity, collect and analyze data on the effects of those strategies, and build their capacity to work more effectively in their communities. The lessons they learned through Promising Practices can help other P4VE-funded organizations in their work and guide the Centers for Disease Control and Prevention in making similar future investments in community capacity building. BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 27 About the Authors Ian Hill is a senior fellow in the Health Policy Center at the Urban Institute and chair of the Urban Institute's Institutional Review Board. He possesses nearly 40 years of experience directing evaluation projects on health programs for low-income mothers, children, and families. Hill is a nationally recognized qualitative researcher who conducts case studies of health program implementation and focus groups, and other community-engaged methods with health care consumers and providers. He is the coprincipal investigator of the Maternal Opioid Misuse demonstration, evaluating integrated care models for pregnant women with opioid use disorder. He co-led a qualitative investigation of how the COVID-19 pandemic affected maternal health equity, including an analysis of how telehealth improved access to care for some mothers while erecting barriers for others. Hill directed the Strong Start for Mothers and Newborns evaluation, which found that midwifery-led birth centers produced dramatically better outcomes at lower costs than typical Medicaid prenatal care. From that work, he was invited to serve on the Institute for Medicaid Innovation's Midwifery and Medicaid Learning Collaborative National Advisory Council. For 10 years, Hill led evaluations of the Los Angeles Healthy Kids Program and Best Start LA, sponsored by First 5 LA, in his hometown of Los Angeles. He has led various qualitative assessments of Affordable Care Act implementation and the qualitative components of two congressionally mandated evaluations of the Children's Health Insurance Program. Prior to joining Urban, Hill was Associate Director of Health Systems Research, Inc., a senior policy analyst with the National Governors Association, and a Presidential Management Fellow with the US Department of Health and Human Services. His undergraduate studies were at the University of California, Santa Barbara, and he received master's degrees in both Public Administration and Social Work from Syracuse University. Zara Porter is a research analyst in the Health Policy Center, where she conducts qualitative research and analysis focused on maternal and child health, reproductive health, and equitable access to health care. She has been involved in qualitative data collection and analysis for several projects, including Building and Supporting a Black Midwifery Workforce in Oklahoma and the Maternal Opioid Misuse model. She is also a core member of the Partnering for Vaccine Equity Learning Community group learning manager team. Her qualitative data collection experience includes conducting literature reviews, tailoring interview protocols, and participating in key informant interviews and focus groups with a range of participants. Porter holds a BA in public policy with a minor in global health and a Human 28 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY Rights Certificate from Duke University. She previously interned for the Children's Defense Fund and Population Connection. Juliana Mayer is a research assistant in the Health Policy Center, where she conducts qualitative research and analyses on a variety of topics relating to health equity, maternal and child health, telehealth, and health care payment reform. She has supported qualitative data collection through literature reviews, focus groups, and key informant interviews for several projects, including the Maternal Opioid Misuse model, Unequal Treatment at 20, and the Partnering for Vaccine Equity program. Mayer holds a bachelor's degree in public policy with a minor in psychology from Duke University. Previously, she interned for the Death Penalty Information Center and conducted research with the Duke-Margolis Center for Health Policy. Alaisha Verdeflor is a research analyst in the Health Policy Center. Since joining Urban, Verdeflor has used qualitative and community-engaged methods in projects focused on expanding vaccine access among adults, managed care implementation in Medicaid, and health equity. Before joining Urban, they worked as a community health worker and as a policy and community engagement assistant as part of the Drexel University Urban Health Collaborative. Verdeflor holds a bachelor's degree in public health from Rutgers University and a master's degree in public health from Drexel University. Kristen Brown is a senior research associate in the Health Policy Center. Her research takes a multilevel, transdisciplinary approach to understanding how structural and social determinants contribute to racial health inequities. A trained social epidemiologist, her most recent research has focused on assessing the impact of the pandemic, including examining the mechanisms through which racial and economic segregation shaped vulnerability to COVID-19, insurance coverage stability in children, and strategies for mitigating long-term effects in communities of color. She is the deputy director of group learning for the Partnering for Vaccine Equity program, a Centers for Disease Control and Prevention–funded initiative purposed to improve racial and ethnic equity through partnerships that drive community action. In this role, Brown develops public health initiatives that center the voices of those most severely affected and supports CBOs in disseminating accurate, vetted information. Before joining Urban, Brown conducted research at the National Institutes of Health and Emory University. She holds a Bachelor of Arts degree from Rutgers University in biological sciences and psychology, a Master of Science degree from the University of Michigan in human genetics, and a PhD from the University of Michigan in epidemiologic science. BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY 29 DaQuan Lawrence is a former research analyst in the Health Policy Center, where he contributed to policy papers and projects focused on vaccine access in low-income communities, health equity in Medicaid, and state human services structures. Before coming to Urban, Lawrence was a graduate student researcher focused on human rights and international public policy. He has worked in the Dominican Republic, Haiti, Indonesia, Liberia, and South Africa. Outside Urban Institute, Lawrence is cofounder and chairman of Strong Men Overcoming Obstacles Through Hard-work (SMOOTH) Inc., a nonprofit that aims to educate, develop, empower, and organize young Black men to improve their graduation and retention rates during their public school or college matriculation. Lawrence holds a bachelor's degree in sociology with a minor in philosophy and criminal justice from Morgan State University and received his master's degree in international public policy from the Johns Hopkins University School of Advanced International Studies. He is currently pursuing a doctoral degree in African studies with a focus on public policy at Howard University. 30 BUILDING THE EVIDENCE BASE FOR ADVANCING VACCINE EQUITY STATEMENT OF INDEPENDENCE The Urban Institute strives to meet the highest standards of integrity and quality in its research and analyses and in the evidence-based policy recommendations offered by its researchers and experts. We believe that operating consistent with the values of independence, rigor, and transparency is essential to maintaining those standards. As an organization, the Urban Institute does not take positions on issues, but it does empower and support its experts in sharing their own evidence-based views and policy recommendations that have been shaped by scholarship. Funders do not determine our research findings or the insights and recommendations of our experts. Urban scholars and experts are expected to be objective and follow the evidence wherever it may lead. 500 L'Enfant Plaza SW Washington, DC 20024 www.urban.org