Findings from an Innovative Teen Pregnancy Prevention Program Evaluation of BUtiful: An Internet pregnancy prevention intervention for older teenage women in New Orleans, Louisiana Final Impact Report for Tulane University School of Public Health and Tropical Medicine August 31, 2015 – Updated November 11, 2015 Prepared by Patricia Kissinger, Ph.D. Norine Schmidt, M.P.H. Jakevia Green, M.P.H. Recommended citation: Kissinger P, Schmidt N, Green J, Latimer J, Madkour A, Clum G, Johnson C (2015). Evaluation of BUtiful: An Internet pregnancy prevention for older teenage girls in New Orleans, Louisiana. Acknowledgements: We wish to thank our partners at Delgado Community College, Southern University of New Orleans, Dillard University, Xavier University, The Tulane Drop In Clinic, and numerous community venues throughout the city of New Orleans. We would like to acknowledge the BUtiful research team: Carolyn Johnson, Ph.D., Aubrey Madkour, Ph.D., Gretchen Clum, Ph.D., Jennifer Latimer, M.P.H., Taylor Johnson, M.P.H., Steffani Bangel, B.A., Brittany McBride, M.P.H., and all of the student research assistants who were invaluable in conducting this research. We would also like to acknowledge our consultants, Gina Wingood, Ph.D. and Ralph DiClemente, Ph.D., along with their staff for their instrumental support. Dr. Kissinger, who is co-PI, was the lead evaluator on this project. There is a potential conflict of interest, because Dr. Kissinger also had oversight of all study operations and activities. However, we feel that this potential conflict did not arise since the intervention was done online, outcome measures were done via audio computer-assisted self-interview, and the analysis plan was acknowledged by OAH and not altered. This publication was prepared under Grant Number TP2AH000013 from the Office of Adolescent Health, U.S. Department of Health & Human Services (HHS). The views expressed in this report are those of the authors and do not necessarily represent the policies of HHS or the Office of Adolescent Health. EVALUATION OF ESIHLE: AN INTERNET PREGANCY PREVENTION INTERVENTION FOR OLDER TEENAGE WOMEN IN NEW ORLEANS, LOUISIANA: FINDINGS FROM AN INNOVATIVE TEEN PREGNANCY PREVENTION PROGRAM I. Introduction A. Introduction and study overview While unintended teen pregnancy (UTP) rates have been declining in the U.S., they remain high for minority and economically disadvantaged women. The rate of UTP is 5 times higher for women living in poverty than women in the highest socio-economic level 1 and youth in New Orleans are 1.8 times more likely to live in poverty than the rest of the U.S. 2 In 2008, the rate of UTP was highest among 18-19 year-olds (2.5 times higher than teens ages 15-17). 3 At the start of our study, Louisiana had seen a smaller decline in teen births than the U.S. as a whole (4% vs. 6% from 2011-2012)3 and the rate of teen births in 2012 was 1.5 times higher in Louisiana than the rest of the country (43.1 per 1,000 vs. 29.4 per 1,000). 4 Similarly, human immunodeficiency virus (HIV) and sexually transmitted infections (STI) disproportionately affect minority youth under 25. By the age of 25, 50% of all youth will have acquired an STI. 5 In 2011, 53% of all Chlamydia trachomatis (Ct), 43% of all Neisseria gonorrhoeae and 22% of new HIV infections occurred in the 18-24 age group. Compared to Whites, African-Americans are more likely to have Chlamydia, gonorrhea, HIV and UTP, constituting a notable health disparity.5,6,7 UTPs and STIs have individual and societal consequences. Young women who have UTP are more likely to live in poverty, drop out of school, have substance use disorders, and are less likely to engage in prenatal care and breastfeeding. 8 Women with Chlamydia or gonorrhea infections are more likely to have pelvic inflammatory disease, ectopic pregnancy, become infertile and acquire HIV. 9 The financial burden of these health conditions is enormous. 1 Taxpayers spend approximately $12 billion annually on publicly financed medical care for women who experience an UTP 10 and another $11.0-$20.6 billion on STIs including HIV. Many older adolescents have not received essential knowledge and skills to protect themselves against UTP and STIs. Half of UTPs occur among women who use contraception inconsistently or incorrectly,10 suggesting an ongoing need for teens to obtain factual education regarding sexual intercourse and contraception. Interventions to prevent UTP, HIV, and STIs are greatly needed. Prevention programs that young women may have had access to are mostly school-based and provided at younger ages. A prevention program that uses digital media could provide a model for UTP programs that can be conveniently disseminated to older teens in various settings and would assure standardization of the content, provide flexibility of delivery and be appealing to youthful audiences. In addition, evidence suggests that Internet interventions are more effective than face-to-face interventions. 11 Be yoU, Talented, Informed, Fearless, Uncompromised, and Loved (BUtiful) is an internet- delivered pregnancy prevention intervention that was developed with funding from a Tier 2 Office of Adolescent Health award to implement and rigorously evaluate new and innovative programs to prevent teen pregnancy, especially within high-risk, vulnerable, and culturally under-represented youth populations. Tulane University grantees developed the BUtiful website in consultation with the creators of Sisters, Informing, Healing, Living, and Empowering (SiHLE), a group-session intervention targeting African-American teen females. The evaluation of BUtiful was performed using a randomized control design whereby women were randomly assigned to either BUtiful or to the attention control website, Diversity, Individuality, Vitality, Activity and Strong (DIVAS), which is a general health and wellness website also developed by Tulane staff. “Attention control” means that participants randomized to the DIVAS group 2 received similar access to their website and interaction with the staff; however, the content of DIVAS did not overlap the content of BUtiful. BUtiful has not been previously evaluated. This report describes the implementation and impact of BUtiful through six months post-intervention completion. Data collection is ongoing for outcome measures at twelve months post- intervention. Impact analyses including this longer follow-up period will be reported in the future. B. Primary research question(s) The primary research question was: What was the impact of the BUtiful intervention, relative to the DIVAS attention control, on the consistent use of reliable contraceptive methods among 18- and 19-year-old African-American women 6 months after intervention completion? C. Secondary research questions The secondary research questions were: 1) What was the impact of the BUtiful intervention, relative to the DIVAS attention control, on the rate of pregnancy among African-American 18- and 19-year-old women 6 months after intervention completion? 2) What was the impact of the BUtiful intervention, relative to the DIVAS attention control, on the rate of Chlamydia or gonorrhea among African-American 18- and 19-year-old women 6 months after intervention completion? II. Program and comparison programming A. Description of program as intended BUtiful is an Internet-delivered intervention. The website (http://www.butifulonline.com) was designed to be accessed via any computer with an Internet connection. The website has 8 interactive sequential sessions that are indicated by tabs at the top of the webpage. The 8 sessions are: 1) BUtiful Beginnings – introduction to website and characters, role models, women in music and media; 2) Be yoU! – understanding values and setting goals; 3) BUtiful Body – 3 female reproductive anatomy, menstrual cycle; 4) BUtiful Choices – hormonal, non-hormonal, barrier and natural birth control options and availability; 5) BUtiful and Informed – STI/HIV information and prevention; 6) BUtifully Communicate – types of communication and negotiation strategies; 7) BUtiful Relationships – relationship choices, abuse and substance use; and 8) We are BUtiful –wrap-up. Completion of a session unlocks the next session (users cannot skip ahead), and all completed sessions remain available during the activation period. Each session is approximately 30 minutes long, depending on the participant’s level of interaction. For example, the interface in the sessions on contraception and STIs are pictorial grids. Hovering over an image with a mouse pointer brings up the name and a small description of the method or STI. Clicking on the cell opens a page that gives more detailed information about that method (e.g., dosage, effectiveness, common side effects, etc.) or STI. Participants receive a $5 Walmart gift card for each session completed. Information is presented via video, text, and interactive activities; BUtiful has 48 activities that are found throughout the eight sessions. Completion of the activities within a session are optional and will not affect the ability to move forward in the session. On certain pages, questions to the participants, called “BUtiful Shares” prompt them to interact with the subject matter (e.g., “What birth control methods work best for you?”). Responses are viewable to all participants. Five African-American female characters are present throughout BUtiful. Four of these young women portray women from the target population who have diverse looks and varied backstories; all are working through their particular issues related to contraception, relationships, etc. The fifth character is a slightly older African-American woman who serves as a moderator; she presents medically-accurate health information as well as anecdotes of her history of dealing with relationships, pregnancy and STI infection. 4 At enrollment, staff demonstrates the website and asks the participant to create her individual username and password. Activation on the BUtiful website starts on the day of enrollment and ends 4 weeks later. Once activated, the participant chooses where and when she logs on to BUtiful throughout the 4-week activation period. At enrollment, women are not given any hard-copy handouts or information sheets to retain session information. After deactivation, session material is reinforced by quarterly newsletters delivered via mail or email from which participants may opt out. To reduce contamination, this website requires credentials, and participants are asked not to share their log on information. Participants are provided study contact information to use if they need assistance with any questions about the site or to manage technical difficulties (i.e., log on difficulties, content disruptions). Staff also maintained contact with the participants through text messages and phone calls to positively reinforce progress on the site, remind participants of the 4-week activation timeline, and to update contact information as necessary. B. Description of counterfactual condition The control website (http://www.divasonlinesite.com) is designed as an attention control to BUtiful and has a general health and nutrition curriculum created by the study team. DIVAS and BUtiful follow the same procedures. Like BUtiful, DIVAS is designed to be accessed via computer with an Internet connection. DIVAS also has 8 sequential interactive sessions: 1) What is Healthy? – introduction to website and characters, looking at health and healthfulness; 2) Better Choices – making healthy food choices; 3) Fast Food – healthy snacks, drinks, calories; 4) Sugar, Fat and Salt – BMI, hypertension, diabetes; 5) Get a move on – exercise; 6) Live Well – stress management; 7) Beauty – sleep, foods and beauty; and 8) Living healthy – wrap-up. 5 DIVAS is presented via video, text, interactive activities, and questions and answers (Q&A); it has 40 activities that are found throughout the eight sessions. Completion of the activities within a session are optional and will not affect the ability to move forward in the session. On certain pages, questions to the participants, called “DIVAS Dish” prompt them to interact with the subject matter (e.g., “What are your barriers to exercising?”). Responses are viewable to all participants. Five African-American female characters are present throughout with four young women portraying women from the target population who have diverse looks and varied backstories; all working through their particular issues related to general health, nutrition and exercise. The fifth character is a slightly older African-American woman who serves as a moderator and whose character was a nutritionist that presents relatable anecdotes about her and her family’s health. The DIVAS website operatess identically to the BUtiful website. The 4-week activation period starts on the day of enrollment, and sessions are self-paced and accessible from any location where a computer with Internet access is available. Session length is approximately 30 minutes; completion of a session unlocks the next session (users can not skip ahead), and all completed sessions remain available during the activation period. Participants receive a $5 Walmart gift card for each session completed. Hard-copy handouts or information sheets for information retention are not provided to participants, however session material is reinforced in quarterly newsletters from which participants are able to opt out. Program staff were available to assist with any questions about the site or to manage technical difficulties. Staff also maintained contact with the participants through text messages and phone calls to positively reinforce progress on the site, remind participants of the 4-week timeline, and to update contact information as was necessary. 6 III. Study design A. Sample recruitment Women were recruited from September 2012 through September 2014 as a single cohort. Potential participants were recruited through partner sites which were chosen to allow participation by African-American women from a range of socio-economic backgrounds and experiences (a community college, three historically black universities, and an adolescent drop-in clinic). At these sites, study staff was allowed to circulate on campus/onsite and directly recruit potentially eligible participants using street intercept recruitment techniques. Several student organizations and on-campus health centers invited staff to participate in health fairs, welcome fests, and other student service programming. Three of the campus sites also welcomed staff to present the program opportunity in freshman-focused seminars and courses, and those students not directly eligible were offered passive recruitment materials to pass along to potentially interested friends or family. Participants were also recruited through community events such as health fairs and health walks, or passive recruitment materials such as flyers posted at places where young African- American women were likely to congregate, for example: bus stops, hair supply stores, shopping centers with retail and restaurant locations primarily staffed by or targeting youth, swimming pools, and skating rinks. A pseudonym, “You Geaux Girl!” (YGG!), was created for the eSiHLEstudy in order to generate a more friendly and age appropriate “face” of the study; all promotional recruitment materials bore the YGG! branding. The study was presented as a health education and empowerment program for 18- and 19-year-old African-American women. Women expressed their interest directly to study staff during recruitment events or by phone or email. Potential participants who expressed interest were contacted approximately 2-3 days later by study staff to review eligibility criteria and discuss the activities and commitment of the program. 7 This recruitment model was chosen for several reasons: 1) Improved retention: By having a delay of at least one day between contact and enrollment, participants are demonstrating an investment in the program, 2) Staff/Participant logistics: Enrollment appointments took about an hour and needed to be scheduled to allow sufficient time, and 3) Privacy: Detailed eligibility criteria prior to enrollment may not have been asked at initial contact if there were privacy concerns and eligibility questions were not appropriate in such a setting. If a young woman was eligible and agreed to enroll, staff arranged an enrollment visit with the potential participant at her convenience at the partner site from which she was recruited or the study office. The visit served as an opportunity to fully describe study procedures, answer any questions, confirm eligibility, and obtain a signed written informed consent form. Eligibility criteria were: African-American women, 18 or 19 years of age, and living in Orleans Parish or Jefferson Parish (counties), Louisiana. Women were ineligible if they were currently pregnant or intending to become pregnant in the next year or identified as intending to have sex with women exclusively. If a young woman was ineligible because she was less than 18 years old or reported that she was pregnant at the time of recruitment, she had the option to contact the staff once she became eligible. Women who provided informed consent at the enrollment visit but were then deemed ineligble after consenting were provided with a small incentive ($10) to compensate them for their time. During the informed consent process, women were told that YGG! is a study evaluating two online programs that look the same but contain different content and that there was a 50/50 chance of being randomized to one site or the other. The names of the two websites were disclosed at that time. Staff obtained consent from 656 women (see Table III.A.1). 8 After informed consent was obtained, the enrollment survey was administered and biological specimens to test for pregnancy, Chlamydia, and gonorrhea were collected. Pregnancy results were available within 24 hours of the enrollment visit. If it was determined that a participant was pregnant, she was de-enrolled from the study but retained her enrollment compensation. As compensation for the enrollment visit, women received a $50 gift card and a promotional bag with items relating to the specific arm of the study to which they were randomized. For example, women on both arms received a journal, lunchbag and headphones, but women in BUtiful received a condom case (related to the condom-use content) and women in DIVAS received a tape measure (related to the waist-to-hip ratio lesson). All BUtiful promotional items were branded with the BUtiful logo, and likewise, all DIVAS promotional items were branded with the DIVAS logo. Table III.A.1 Outcome of eSiHLE Recruitment Efforts from Sept. 2012 through Sept. 2014 Recruitment result Number of unique individuals Total number women reached out to 1,562 Unresponsive to contact for eligibility screening 315 Declined participation prior to being screened for eligibility 176 Recuited but never contacted for enrollment because of time contraints 97 Successfully contacted and screened for eligibility 974 Did not pass screening criteria 184 Eligible for random assignment 790 Refused to enroll 58 Set enrollment appointment but did not attend 76 Eligible, Enrolled, Consented, and Randomized 656* *Does not reflect analytic sample. See Appendix B, Table B.1, for analytic sample size. 9 B. Study design eSiHLE is a randomized controlled trial comparing the outcomes of women assigned to the intervention website, BUtiful, with those assigned to an attention control website, DIVAS. Randomization occured at the participant level using blocked randomization with randomly selected block (subgroup) sizes. Blocked randomization is a method used to reduce selection bias and potential confounding between the arms and to produce a better measure of treatment effects. Randomization allocation was determined using Statistical Analysis System (SAS) software. Five separate randomization schemes were created to accommodate each of the 5 recruitment sites along with a scheme for community recruitment. There was a 50% probability of assignment to the treatment group. Pre-prepared, sealed envelopes with study arm assignments were sequentially numbered (one series for each recruitment site). Randomization disclosure to the participant was done by program staff during the enrollment visit. A single randomization envelope was assigned to a single study number in sequential order. Neither staff nor participant knew which arm was being assigned until the randomization envelope was opened to reveal the assignment. Staff then demonstrated the assigned website and assisted the participant with log on procedures. An implementation evaluation of the study was also performed. Adherence, content and context were measured using data collected during each individual participant’s activation period by Google Analytics. C. Data collection Data collection occurred at enrollment, during the website activation period, at approximately 6 months post-intervention completion (with a range of 5-9 months), and at approximately 12 months post-intervention completion (with a rangeof 11-15 months). (See Table III.C.1.) 10 Table III.C.1. Timing of data collection efforts used in the impact and implementation analyses of eSiHLE Data collection effort Timing Screening and enrolllment data collection September 2012 – September 2014 Website activation period September 2012 – September 2014 Impact data collection at 6-month follow-up February 2013 – June 2015 Impact data collection at 12-month follow-up August 2013 – December 2015 1. Impact evaluation Sources of data for the impact evaluation are listed in Table III.C.2. Data sources included: 1) screening survey at enrollment; 2) impact surveys atenrollment, and 6 and 12 months after intervention completion 3) urine pregnancy tests at enrollment, 6 and 12 months; 4) urine Nucleic Acid Amplification Test (NAAT) Chlamydia and gonorrhea test at enrollment, 6 and 12 months. Regardless of study arm, all data was collected in the same manner. Table III.C.2: Sources of data for the impact evaluation and timeline for collection . Enrollment visit 6-month follow up 12-month follow up Survey data . . . Screening survey X . . Impact survey X X X Biological data X X X Urine pregnancy test X X X Urine NAAT test for Chlamydia and gonorrhea X X X The survey data and urine samples were collected during the 6- and 12-month follow-ups for which participants received compensation in the form of $75 Walmart gift cards. Enrollment surveys and impact surveys at 6 and 12 months post intervention. Enrollment, 6- month, and 12-month surveys were typically conducted in the field, in person. At enrollment, the screening survey was administered first, then the impact survey. Both were conducted after the participant signed her informed consent form. Women who were not available for the duration of 11 their follow-up window (e.g., students at another location during summer recess) were given the option to complete a distance follow-up at another location. All surveys were taken on a computer. Participants were instructed to ask staff any questions at the time of the survey. Participants who completed the survey at another location had contact information for staff. Biological data. At enrollment, 6- and 12-month follow-ups, participants were asked to provide a first void urine sample to be screened for pregnancy, Chlamydia and gonorrhea. Results for the pregnancy screening were available within 24 hours of the testing. Results for Chlamydia and gonorrhea screenings were available within one week of shipment. An attempt was made to discose all positive results in person as soon as possible after staff received the results and was convenient for the participant. Whenever possible, results were disclosed by the same field team member who conducted the enrollment/follow up appointment and collected the biological specimen to maximize participant comfort. All positive results disclosures included connection to medical care, confirmation of social support, and risk reduction and partner disclosure counseling. Results were data entered in a password protected database on a password protected and encrypted computer. No names or other personal identifiers were used in the results database. Positive results were reported to the Louisiana Office of Public Health by the testing laboratory. Participants who completed their follow-up appointments by distance were given two options for the biological screening; either request a specimen collection kit from research staff (specimen collection supplies and protocols that detail collecting and shipping specimens back to research staff), or, participants could receive testing at a clinic within their vicinity. Staff requested that the participant disclose the results from her clinic visit and provide documentation; documentation was indicated in the database. 12 2. Implementation evaluation Data collection for the implementation evaluation was identical for the intervention and counterfactual conditions. Fidelity data were not collected since BUtiful and DIVAS were delivered electronically, content was consistent and standardized for each arm, and no substantial and/or unplanned adaptation occurred. Data collected regarding sessions received and content delivered were recorded on Google Analytics in real time and used by study staff for implementation monitoring and evaluation. Log-on/log-out times, session completion/duration, and activity completion/duration were recorded for each participant. Additionally, responses to prompts (i.e., Shares and Dish) were monitored daily. Additional questions regarding participants’ participation in other Teen Pregnancy Prevention programs were added to the impact survey. Please see Appendix A for detailed information on data collected. D. Outcomes for impact analyses The outcome measure for the primary research question is the consistent use of reliable contraception (i.e., condom, birth control pill, birth control shot, ring, patch, implant or IUD) during vaginal sex in the past 3 months as reported in the 6-month follow-up survey by arm. For women who reported condom use only, the measure took into account the consistency with which condoms were used by each partner for all vaginal sex acts (see Table III.D.1). For women who reported birth control pill or hormonal shot, condoms needed to be worn consistently for the first month. Sponge, cervical cap and diaphragm were not considered reliable forms of birth control. Participants who reported not engaging in sex were retained in the denominator. If these woman reported using a reliable contraceptive method during that time, she was considered a reliable contraceptive users using the same adherence algorithm as for women who did have sex. Participants who responded that they used a reliable form on birth control but did not have usage information were classified as missing and removed from the analysis. 13 Table III.D.1. Behavioral outcomes used for primary impact analyses research questions Timing of measure Outcome name Description of outcome relative to program Consistent use of Dichotomous (yes/no) measure of participant self-report of using condoms, birth 6-months post- reliable control pills, shot, vaginal ring, patch, implant, or intra-uterine device as a method of intervention contraception contraception. All participants regardless of whether they engaged in vaginal sex method were included in the analyses. Use assessed by survey question: • “Are you currently using any of the following types of birth control?” [Participants must indicate: Condoms, Birth Control Pills, The shot, The ring, The patch, IUD, Implant, Sponge, Diaphragm, Cervical Cap, and/or Other.] Consistency of use was calculated by method based on responses to the following survey questions: Condoms [Only] For each male sexual partner reported from the past 3 months, the following questions were asked: • In the past 3 months, how many times have you had vaginal sex with [partner]? • In the past 3 months, how many times did you use a condom when having vaginal sex with [parter]? • In the past 3 months have you EVER had vaginal sex without you or [partner] using a condom? • Did you use a condom the last time you had vaginal sex with [partner]? To be deemed a consistent user, the number of times used a condom must match frequency of vaginal sex acts, and subsequent questions must indicate that a condom was used for every vaginal sex act for all male partners reported in the past 3 months. Birth Control Pills • During the past three months, did you ever miss 2 or more pills in a row during the time when you were supposed to be taking your pills? • Did you have vaginal sex on the days that you missed 2 or more pills in a row in the past 3 months? • [If yes] Did you use another method of birth control at that time? Birth Control Shot • During the past three months, were you ever late in getting your birth control shot? • Did you have vaginal sex on the days that you were late getting your birth control shot? • [If yes] Did you use another method of birth control at that time? The Ring • During the past three months, were you ever late in replacing your ring? • Did you have vaginal sex on the days when you were replacing your vaginal ring? • [If yes] Did you use another method of birth control at that time? • In the past 3 months has your ring ever fallen out? 14 Timing of measure Outcome name Description of outcome relative to program Consistent use of • Did you replace your vaginal ring immediately? 6-months post- reliable intervention contraception • Did you have vaginal sex on the days before you replaced your vaginal (continued) method ring? (continued) • [If yes] Did you use another method of birth control at that time? Birth Control Patch • During the past three months, were you ever late in replacing your patch? • Did you have vaginal sex on the days that you were replacing your patch? • [If yes] Did you use another method of birth control at that time? Birth Control Implant • During the past three months, were you ever late in replacing your implant? • Did you have vaginal sex on the days that you were replacing your implant? • [If yes] Did you use another method of birth control at that time? IUD IUD users were deemed consistent users. The variable is constructed as a dummy variable where respondents who consistently used a reliable form of contraception are coded as 1 and all others are coded as 0. Missing data are coded as missing and not included in the analyses The outcome measures for the secondary research questions are: 1) pregnancy test positive screening or self report of pregnancy at the 6-month follow up by arm, and 2) infection with chlamydia or gonorrhea at follow-up or self-report of an interim infection with either of those two organisms. (see Table III.D.2). Participants who had missing data for either of the secondary outcomes were not included in those respective analyses. Participants who denied sexual activity were retained in the denominator. 15 Table III.D.2. Behavioral outcomes used for secondary impact analyses research questions Timing of measure Outcome name Description of outcome relative to program Pregnancy Dichotomous (yes/no) measure of pregnancy at time of follow-up, or self-report of 6-months post- pregnancy between enrollment and follow up visit on the survey. Measure is intervention constructed from: • A positive result from the staff-administered pregnancy urine screen. • Survey question: “Have you been pregnant since we saw you last?” The variable is constructed as a dummy variable where respondents who test positive for pregnancy or self-report pregnancy in their survey are coded as 1 and those who tested negative for pregnancy as 0. Those with missing data were not included in the analyses. Chlamydia Binary measure of STI infection at time of follow-up or self-report of infection 6-months post- infection or between enrollment and follow up on the survey. Measure is constructed from: intervention Gonorrhea infection • Positive result from laboratory testing of participant urine specimen. • Survey question: “Since we saw you last, have you been told by a doctor, nurse, or health professional that you had Chlamydia or gonorrhea?” Subjects were counted as “yes” for the Chlamydia or gonorrhea outcome according to what was reported. Variable is constructed as follows: positive testing or interim self-report of Chlamydia or positive testing or interim self-report of gonorrhea are coded as 1, while negative results are coded as 0. Missing data are coded as missing and not included in the analyses. 16 E. Study sample A total of 656 women were enrolled and randomized for eSiHLE from September 2012 through September 2014. Eight women were administratively de-enrolled: 6 women were determined ineligible post-consent (3 were pregnant at enrollment screening, 1 was over 19 years old, 2 women had sex exclusively with women), and 2 women did not have proper Health Insurance Portability and Accountability Act documentation. An additional 23 women were excluded from the analyses conducted for this report because at the time of this report, the Tulane Institutional Review Board was reviewing the recruitment process for these subjects. The analytic sample for this report varied depending on the outcome (i.e. reliable contraception N=517, Chlamydia or gonorrhea N=505, and pregnancy N=508). Please see Appendix B, Table B.1, for information about the flow of the sample from the start of the study to the follow-up survey. All analyses were run on subjects who had data for the the outcome being considered. For example, for the primary outcome of interest, all subjects were included in the analysis except those who did not provide information on contraceptive use. For pregnancy (a secondary outcome) the analyses were run for those who had complete data for pregnancy. And for Chlamydia or gonorrhea infections (a secondary analysis), analyses were run for those who had complete STI information. Sample sizes for the three outcomes of interest were: 517 women for the outcome of consistent use of reliable contraception, 505 women for the outcome of Chlamydia or gonorrhea infection and 508 women for the outcome of pregnancy. F. Baseline equivalence All women on both arms were African American and aged 18-19. Baseline equivalence was assessed on age, age of sexual debut, number of male sex partners in past 3 months, pregnancy history, STI history, mother’s age at the birth of her first child, and education status of mother 17 and/or father. Two-tailed t-tests, chi-squared tests, and Fisher’s exact tests (where applicable) were used to determine any group differences at enrollment. All women included in each analytic sample have been included in the analysis of baseline equivalence for that analytic sample. Baseline equivalency tables demonstrated that all variables were evenly distributed between arms for the samples used for the primary outcome of interest and the two secondary outcomes (see Tables III.F.1a.-1c). In all three datasets, the samples had similar rates of selected variables at baseline by arm (i.e., ever had vaginal sex, mean age at first vaginal sex, had more than one sex partner in the last three monhts, had some college education, were ever pregnant, were ever diagnosed with an STI, her mother was < 20 years of age at the birth of her first child, mother education and father’s education). 18 Table III.F.1a. Summary statistics of key baseline measures for youth completing 6-month data collection with no missing data for the primary outcome of interest: consistent, reliable contraceptive use (n=517) Intervention Intervention Comparison versus versus Intervention mean mean or % comparison comparison or % (standard (standard mean p-value of Baseline measure deviation) deviation) difference difference Mean age (s.d.) 18.93 (0.54) 18.85 (0.55) 0.08 0.101 Ever had vaginal sex 74.2 76.2 -2.00 0.595 Age of sexual debut in years (s.d.) 16.09 (1.57) 15.96 (1.62) 0.13 0.419 More than 1 male sex partner in last 3 11.9 10.6 0.630 months 1.30 Some college education 93.7 91.7 2.00 0.395 Ever pregnant 7.1 11.3 -4.20 0.102 Ever diagnosed with STI 15.4 18.4 -3.00 0.337 Mother was < 20 years of age at birth 46.4 45.5 0.824 of first child 0.90 Mother’s education > high school 60.2 65.0 -4.80 0.234 Father’s Education > high school 43.6 48.8 -5.20 0.253 Use of reliable contraceptive method 53.8 59.9 -6.10 0.168 Chlamydia or gonorrhea 6.1 8.5 -2.40 .308 Sample size 252 265 . . Note: Analytic sample size reflects those with non-missing values for the primary outcome measure, reliable contraceptive use. Note: 31 randomized subjects were administratively removed from this analysis post randomization They were removed because they were found to be ineligible post-randomization or because of regulatory issues (e.g. indirectly enrolled from a non-approved site). Note: s.d. means standard deviation †Fisher’s exact used when cell sizes were too low for chi-square to be assessed. 19 Table III.F.1b. Summary statistics of key baseline measures for youth completing 6-month data collection with no missing data for the secondary outcome of interest: pregnancy (n=508) Intervention Intervention Comparison versus versus Intervention mean mean or % comparison comparison or % (standard (standard mean p-value of Baseline measure deviation) deviation) difference difference Mean age (s.d.) 18.93 (0.53) 18.85 (0.55) 0.08 0.149 Ever had vaginal sex 74.0 76.0 -2.00 0.608 Age of sexual debut (s.d.) 16.07 (1.59) 15.95 (1.63) 0.120 0.500 More than 1 male sex partner in last 3 12.2 10.3 0.500 months 1.90 Some college 93.9 92.0 1.90 0.400 Ever pregnant 7.7 11.5 -3.80 0.155 Ever diagnosed with STI 15.8 18.6 -2.80 0.414 Mother was < 20 years of age at birth 45.9 45.6 0.933 of first child 0.30 Mother’s education > high school 60.1 65.0 -4.90 0.255 Father’s education > high school 43.0 49.0 -6.00 0.221 Use of reliable contraceptive method 53.9 60.2 -6.30 0.157 Chlamydia or gonorrhea 10.2 10.7 -0.50 0.847 Sample size 246 262 . . Note: Analytic sample size reflects those with non-missing values for the secondary outcome measure, pregnancy. Note: 31 randomized subjects were administratively removed from this analysis post randomization. They were removed because they were found to be ineligible post-randomization or because of regulatory issues (e.g. indirectly enrolled from a non-approved site). Note: s.d. means standard deviation †Fisher’s exact used when cell sizes were too low for chi-square to be assessed. 20 Table III.F.1c. Summary statistics of key baseline measures for youth completing 6-month data collection with no missing data for secondary outcome of interest: Chlamydia or gonorrhea (n=505) Intervention Intervention Intervention Comparison versus versus mean or % mean or % comparison comparison p- (standard (standard mean value of Baseline measure deviation) deviation) difference difference mean age (s.d.) 18.93 (0.54) 18.85 (0.55) 0.08 0.141 Ever had vaginal sex 74.3 75.8 -1.50 0.700 Age of sexual debut (s.d.) 16.07 (1.59) 15.95 (1.63) 0.12 0.480 More than 1 male sex partner in 12.2 10.4 0.509 last 3 months 1.80 Some college education 93.9 92.3 1.60 0.488 Ever pregnant 7.8 11.2 -3.40 0.193 Ever diagnosed with STI 15.9 18.8 -2.90 0.403 Mother was < 20 years of age at 46.1 45.6 0.899 birth of first child 0.50 Mother’s education > high school 59.9 65.1 -5.20 0.230 Father’s education > high school 42.8 48.6 -5.80 0.205 Use of reliable contraceptive 53.8 60.3 0.141 method -6.50 Chlamydia or gonorrhea 10.2 10.8 -0.60 0.836 Sample size 245 260 . . Note: Analytic sample size reflects those with non-missing values for the secondary outcome measure, Chlamydia or gonorrhea infection. Note: 31 randomized subjects were administratively removed from this analysis post randomization. They were removed because they were found to be ineligible post-randomization or because of regulatory issues (e.g. indirectly enrolled from a non-approved site). Note: s.d. means standard deviation †Fisher’s exact used when cell sizes were too low for chi-square to be assessed. 21 G. Methods 1. Impact evaluation Since there were no differences on selected characteristics for those who had all data compared to those who had any missing data, no multivariable analyses were conducted and unadjusted rates and means were reported. Impacts were assessed at 6 months post intervention and calculated as the difference between the unadjusted outcomes for the intervention and control groups. Statistical significance of the differences was determined based on p-values. 2. Implementation evaluation The implementation evaluation was focused on the number of sessions and activities completed by participants.. User data included the number of sessions started , completed, activities completed and time on the website. Frequencies, means, medians and proportions were generated. See Appendix C for a detailed description of methods used in the implementation evaluation. IV. Study findings A. Implementation study findings Adherence Eight sessions were available on both the BUtiful and DIVAS websites. Participants of BUtiful completed a mean of 5.1 sessions; 58.5% completed 6 or more sessions, and 55.7% completed all eight sessions. DIVAS participants completed a mean number of 5.4 sessions with 63.6% of women completing 6 or more sessions and 62.0% completing all 8 sessions. Table IV.A.1 shows the percentage of participants who completed each number of sessions by arm. 22 Table IV.A.1: Percentage of participants completing each number of sessions, by arm Number of sessions Intervention – BUtiful (n=307) Control – DIVAS (n=318) completed Percentage of participants Percentage of participants 0 23.8 22.7 1 5.7 4.1 2 5.2 3.1 3 1.3 1.3 4 3.6 3.1 5 2.0 1.3 6 1.6 1.6 7 1.0 1.0 8 55.7 62.0 The percentage of BUtiful participants who engaged in at least one of the 48 activities on the website was 57.6%; 61.5% of DIVAS participants engaged in at least one activity on that website. The mean number of activities completed by participants in BUtiful and DIVAS was 26.7 and 24.1, respectively. Context A small percentage of participants reported at enrollment that they had participated in other teen pregnancy prevention programs in the past (6.9% of BUtiful and 7.3% of DIVAS participants). There were no adaptations to the BUtiful website once implementation began. There were also no major issues with delivery of the website that would have affected participant use of the site. 23 B. Impact study findings Participants in the intervention arm were similar to those in the control arm for use of reliable contraceptives at their 6-month follow-up (62.2% vs. 63.2%, p-value=0.81). There were also no significant differences in pregnancy rates (3.3% vs. 1.9%, p-value=0.34). The STI rate among the intervention groups was somewhat lower than the control group (5.3% vs. 9.2%, p- value=0.09), but the difference was not statistically significant. Table IV.B.1. Post-intervention estimated effects using data from 6-month survey to address the primary research question Intervention compared to Intervention Comparison comparison Mean difference (p-value of Outcome measure % % difference) Consistent use of reliable contraception 62.2 63.2 -1.00 (0.81) Sample Size 251 266 . Source: 6-month follow-up data. Data collected 5-9 months after completion of intervention/control. Table IV.B.2. Post-intervention estimated effects using data from 6-month follow up to address the secondary research question of incident pregnancy Intervention compared with Intervention Comparison comparison Mean difference (p-value of Outcome measure % % difference) Pregnancy 3.3 1.9 1.40 (0.34) Sample Size 246 262 . Source: 6-month follow-up data. Data collected 5-9 months after completion of intervention/control. 24 Table IV.B.2. Post-intervention estimated effects using data from 6-month follow up to address the secondary research question of incident Chlamydia or gonorrhea Intervention compared with Intervention Comparison comparison Mean difference (p-value of Outcome measure % % difference) Chlamydia or gonorrhea 5.3 9.2 -3.90 (0.09) Sample Size 245 260 . Source: 6-month follow-up data. Data collected 5-9 months after completion of intervention/control. V. Conclusion No significant differences were found between intervention and control groups for the primary or secondary outcomes at 6 months (short term) post intervention. This is a different finding than that of the RCT evaluating the original SiHLE, which found a somewhat higher consistent condom use among those in the intervention group versus those in the control group (this difference was not statistically significant) and a lower rate of self-reported pregnancy for those in the intervention group compared to those in the control group at six months (this difference was statistically significant). 12 It also contrasts with the findings in a meta-analysis that found internet interventions to be more effective than face-to-face interventions. 13 One explanation for the lack of significant differences for the main and secondary outcomes could be dose, since 44.3% of the sample in the intervention arm did not complete the intervention. Another possible reason is that the effectiveness of SiHLE was a result of personal contact with the interventionists and internet delivery is not a good modality for this intervention. It is possible that a hybrid of personal contact and internet intervention would be better. The SiHLE study included younger adolescents (i.e., 14-18). It is possible that the intervention is not as relevant for older teens, particularly those in college or that our adaptations for older teens were not relevant. 25 Analysis of the long term impact and of the qualitative data as an “as treated” analysis may elucidate the reason for the negative findings. VI. References 1 Kost K. Unintended Pregnancy Rates at the State Level: Estimates for 2002, 2004, 2006 and 2008. 2013. 2 Mack V. New Orleans Kids, Working Parents, and poverty http://www.datacenterresearch.org/reports_analysis/new-orleans-kids-working-parents-and- poverty 3 Martin, JA. Osterman MJK, Curtin SC, Mathews TJ. Births: Final Data for 2012. In: Statistics. DoV, ed. Vol 62. Atlanta, GA, 2013. 4 Ventura SJ, Hamilton BE, Matthews TJ. National and state patterns of teen births in the United States, 1940-2013. National vital statistics reports : from the Centers for Disease Control and Prevention, National Center for Health Statistics, National Vital Statistics System. Aug 2014;63(4):1-34. 5 Centers for Disease Control and Prevention NCfHA, Viral Hepatitis, STD, and TB Prevention, Division of STD Prevention. 2012 Sexually Transmitted Diseases Surveillance. 2012. 6 Centers for Disease Control and Prevention NCfHA, Viral Hepatitis, STD, and TB Prevention, Division of STD Prevention. Diagnoses of HIV Infection in the United States and Dependent Areas, 2011. 2012. 7 Hamilton BE, Mathews, M. S., & Ventura, S. J. Declines in state teen birth rates by race and Hispanic origin. 2013; http://www.cdc.gov/nchs/data/databriefs/db123.pdf. Accessed July 15, 2014. 8 Finer LB, Zolna MR. Shifts in intended and unintended pregnancies in the United States, 2001-2008. American journal of public health. Feb 2014;104 Suppl 1:S43-48. 9 Gottlieb SL, Xu F, Brunham RC. Screening and treating Chlamydia trachomatis genital infection to prevent pelvic inflammatory disease: interpretation of findings from randomized controlled trials. Sex Transm Dis. Feb 2013;40(2):97-102. 10 Owusu-Edusei K, Jr., Chesson HW, Gift TL, et al. The estimated direct medical cost of selected sexually transmitted infections in the United States, 2008. Sex Transm Dis. Mar 2013;40(3):197-201. 26 11 Mosher WD, Martinez GM, Chandra A, Abma JC, Willson SJ. Use of contraception and use of family planning services in the United States: 1982-2002. Advance Data. 2004(350):1-36. 12 Bailey JV, Murray E, Rait G, et al. Computer-based interventions for sexual health promotion: systematic review and meta-analyses. International journal of STD & AIDS. Jun 2012;23(6):408-413. 13 DiClemente RJ1, Wingood GM, Harrington KF, Lang DL, Davies SL, Hook EW 3rd, Oh MK, Crosby RA, Hertzberg VS, Gordon AB, Hardin JW, Parker S, Robillard. Efficacy of an HIV prevention intervention for African American adolescent girls: a randomized controlled trial. JAMA. 2004 Jul 14;292(2):171-9. 27 Appendix A: Implementation evaluation data collection Table A.1. Data used to address implementation research questions Types of data used to assess Frequency/sampling of data Party responsible for data whether the element of the collection collection intervention was implemented as Implementation element intended Adherence: How many sessions were Data measures include: number of User data is captured via Google Google Analytics measures individual offered? sessions accessed Analytics as users interact with the and aggregate user data from the website (i.e., activity is recorded in website. (The BUtiful website contains 8 real time). sequential sessions to be completed Program staff exports and by participants within a 4-week Staff monitors session progress daily, synthesized these data for the activation period. Average session and deactivates participants at the Evaluation staff to analyze. length is approximately 30 minutes.) end of their 4-week activation period. User data is downloaded to generate user reports which are compiled into a single database. Adherence: What and how much was Data measures include: individual User data is captured via Google Google Analytics measure individual received? log-in/out dates and times, session Analytics as users interact with the and aggregate user data from the and activity start and stop times, website (i.e., activity is recorded in website. number of activities and length of real time). time spent to complete activities and Program staff exports and sessions. synthesizes these data to analyze. Adherence: What content was Data measures include: number of User data is captured via Google Google Analytics measure individual delivered to youth? sessions completed, number of Analytics as users interact with the and aggregate user data from the activities completed, and number of website (i.e., activity is recorded in website. Share opportunities completed. real time). Program staff exports and synthesizes these data to analyze. Adherence: Who delivered material to Staff clanendars and monitoring Staff visits with participants for Implementation staff, Program youth? reports. enrollment, follow-up, and test results Coordinator, Program Manager, Co- disclosures are recorded as they Investigators, and Co-Principal occur.. Investigators. Program Coordinator, Program Manager, and Investigators review staff training, protocols, and field experiences in meetings (monthly). 28 Types of data used to assess Frequency/sampling of data Party responsible for data whether the element of the collection collection intervention was implemented as Implementation element intended Quality: Quality of youth engagement Program staff monitors session 100% of user data is reviewed and Program staff, Evaluation staff with program progress. User data captured by recorded as available. Google Analytics is downloaded from the website to generate user reports Data are synthesized and analyzed that are then synthesized for analysis. approx. every 6 months. Data measures include: log-in/out dates and times, session and activity start and stop times, and length of time spent to complete activities and sessions. Counterfactual: Experiences of Data measures include: number of User data is captured via Google Google Analytics measure individual comparison condition sessions accessed. Analytics as users interact with the and aggregate user data for both the website (i.e., activity is recorded in intervention and control websites. real time). Program staff exports and Program staff monitors session synthesizes these data for the progress daily, and deactivates Evaluation staff to analyze. participants having reached the end of their on-site allotment. As participants complete the website or reach the end of the 4-week allotment, user data is downloaded to generate user reports that are then compiled into a single database. Context: Other TPP programming Surveys administered at enrollment This information is gathered from Program Staff, Evaluation Staff available or offered to study and 6- and 12-months post- program participants on an individual participants (both intervention and intervention assess whether basis at 3 points in time (enrollment comparison) participants have exposure to other and two follow-up visits). TPP programming within the community. 29 Appendix B: Study sample Table B.1. Youth sample sizes by intervention status – individual-level assignment designs Total Intervention Comparison Total sample Intervention Comparison response response response Number of youth Time Period size sample size sample size rate rate rate Randomized Baseline 656 328 328 . . . Administratively de-enrolled Baseline -31 -21 -10 . . . Retained in study Baseline 625 307 318 . . . Contributed the main outcome (consistent reliable 6-months post contraception use) baseline 517 251 266 78.6 76.5 81.1 Contributing to STI testing 6-months post and/or survey data baseline 505 245 260 76.7 74.7 79.3 Contributuing to pregnancy 6-months post testing and/or survey data baseline 508 246 262 77.2 75.0 79.9 30 Appendix C: Implementation evaluation methods Table C.1. Methods used to address implementation research questions Implementation element Methods used to address each implementation element Adherence: How many sessions The total number of sessions delivered is the number sessions completed by participants enrolled. were offered? Adherence: How much was Average number of sessions attended is calculated as the average of the number of sessions that each participant received? completed. Percentage of subjects completing > 75% sessions completed is calculated as number of people who completed 6 or more sessions divided by all enrollments. (gross measure) Percentage of activities completed is calculated by the number of activities completed divided by the total number of activities offered. (fine measure) Counterfactual: Experiences of Implementation research questions for the counterfactual condition are analyzed in the same manner as the intervention. counterfactual condition Context: Other TPP programming Responses to a question on impact survey regarding participation in other TPP program is reported. available or offered to study participants (both intervention and counterfactual) Context: External events affecting Number of time technical issues closed down the websites for a period of time (generally no longer than a couple of hours) implementation causing non-access to sites is reported. Context: Substantial unplanned None adaptation(s) TPP = Teen Pregnancy Prevention 31