Acquiring Opinions of a High School Dropout Prevention Program: Conducting Focus Groups With Adolescents

Project Overview and Context

In late 2014, I was a brand new doctoral student in a PhD program focusing on educational policy and evaluation at a large state university in the United States. My background was in higher education, both professionally and academically; however, I wanted to make the transition from higher education to focus on and research secondary education. One of the first projects I worked on as a new doctoral student was an analysis of a dropout prevention program (DPP) at a low-income urban high school in Arizona. This study was what I considered my first real foray into academic research, and my advisor asked me to lead the project, which included facilitating several focus groups with high school students, as she thought it would be a good entry into academic research for me. Although I had some research experience stemming from my undergraduate days in psychology and my masters degree in higher education, I was still incredibly new to research in educational settings—especially secondary education settings. The thought of talking to high school students—and, more importantly, getting them to talk to me—was quite intimidating.

In the United States, students failing to earn a high school diploma or subsequent General Education Development (GED) certificate is a pressing issue. Compared to students who graduate high school, students who do not finish high school are more likely to be unemployed or underemployed (U.S. Bureau of Labor Statistics, 2015b; Vedder, Denhart, & Robe, 2013), work jobs that pay lower wages (U.S. Bureau of Labor Statistics, 2015a), live in poverty (DeNavas-Walt & Proctor, 2015), have health problems (Cutler & Lleras-Muney, 2010), and/or be incarcerated (Lochner, 2011; Lochner & Moretti, 2004).

The DPP at the focus of this study was created to support students at risk for not graduating from high school due to low academic achievement and/or high absence rates in persisting to earn their diploma. Many students at this high school were from low-income and/or minority backgrounds, and both low-income and minority students are more likely to drop out of and never graduate from high school compared to their more affluent or non-minority peers (Stark, Noel, & McFarland, 2015). The DPP started in 2012 and was in its third year of operation during the 2014-2015 school year. What made this DPP unique was that it was a combination of efforts among teachers at the high school, student-teachers from the nearby state university (where I was enrolled in my doctoral program), and staff from the local National Basketball Association (NBA) team, the Phoenix Suns. The DPP goals were supported through academic tutoring and mentoring by the university student-teachers, with academic and emotional support from the high school teachers and financial and logistical support from the Phoenix Suns. The Suns also provided incentives to active DPP participants, such as team apparel and game tickets. Since this program was costly to run (funded by the Suns at approximately US$250,000 per year), the Suns had already analyzed the efficacy of the program prior to its third year in operation and found that, overall, it was meeting its goals as students in DPP were getting better grades, doing better on the state’s standardized tests, and graduating from high school at an increased rate (Dialynas, 2015).

However, although DPP had been found to be effective within the high school in terms of reducing dropout rates and increasing graduation rates, the Suns realized that they had no idea how the program participants actually felt about the program. Up until fall 2014, the high school student participants, the student-teacher participants, and the supporting high school teachers had not once been asked for their opinions of the program. The Suns wanted to ensure the program was not just effective but was also meeting the perceived needs of the students, student-teachers, and high school teachers involved. Thus, in conjunction with administrators from the high school, administrators from the Phoenix Suns Charities, the arm of the Suns supporting DPP, reached out to faculty, including my advisor, at the local state university to see whether somebody at the university would be willing to evaluate the program participants’ feelings and opinions about DPP.

Research Design and Practicalities

One of the first things my advisor and I needed to determine was the best method to get participants’ feedback about DPP. We quickly determined we would create an online survey for the high school teachers and student-teachers, as we wanted to ask them similar questions to compare their responses. Given both groups’ busy schedules, we also wanted to allow them to give their opinions whenever it was most convenient for them, so an online survey made sense. We also thought about surveying the students who participated in DPP, but we realized that the data gleaned from an online survey would not be as rich as we would like. Given the main purpose of DPP was to support and help students, we knew we needed to talk with them face to face to get the best data. I had some previous experience in survey design and survey item analysis, but I had no experience with focus groups. As such, this case study focuses solely on my experiences with the focus group aspect of this research study.

Before my advisor and I could conduct the focus groups, we had several issues to navigate. Some of our major questions included the following:

• How many students will (or should) participate in the focus groups, overall and per group?
• When and where should the focus groups be held?
• Should any high school administrators be present during the focus groups?
• What questions should we ask the students?

My advisor and I found out from the high school administrators that over 1,200 students participated in DPP yearly, much to our surprise, as we had incorrectly assumed the number was much smaller. Typically, focus groups are best conducted with a small number of participants per session to allow for all participants to be involved in the conversation, so we knew it was not feasible to talk to all or even the majority of students participating in DPP. We also realized that in order to try to have a representative sample of all DPP participants, we would need to talk to different types of participants. Confirming our assumption with the high school administrators that not all students participated in the various aspects of the DPP at the same frequency, we determined that it would be beneficial to talk to students who participated very frequently, occasionally, and very rarely in DPP, as we assumed different levels of participation could affect participants’ views and opinions of the program. Additionally, we knew that the high school administration kept a log of the number of hours students participated in DPP (i.e., tutoring and mentoring sessions), so we had a tangible data point to use to pull our sample from. With the high school’s Vice Principal’s assistance, we randomly identified 36 students from the 1,200 total participants as potential focus group participants: 12 who were very active in DPP, 12 who were moderately active, and 12 who were very rarely active.

Since we were not offering these students any incentives for participating in our focus groups, we knew we had to make the groups as inviting and accessible as possible. Therefore, we decided to hold the focus groups in an unused conference room within the high school during one of the students’ free periods during the school day. We also decided that, given the nature of the questions we planned to ask, it did not matter whether any high school administrators wanted to be or were present during the focus groups. In some cases, the presence of extraneous people, especially people who have a stake in the study’s outcome, can alter interview or focus group conversations, thus potentially biasing the data. This can especially be the case if the topic at hand is sensitive in nature. In our case, no high school administrators felt the need to be present, so the discussion between me and my advisor was moot.

The last big decision we needed to make before conducting the focus groups was to determine what questions we wanted to cover during the sessions. We knew we wanted to conduct the focus groups in a semi-structured manner, meaning we wanted the conversation to flow naturally and allow for follow-up or clarifying questions if needed (Fylan, 2005). We wanted the focus group discussions to feel relaxed and casual, so participants felt comfortable and open in voicing their opinions, as potentially richer data can be generated during this kind of discussion than in one where participants are uncomfortable and/or questions are rigidly asked with no space for real dialogue.

When my advisor and I were initially brought into the project, the Phoenix Suns and the high school administrators had shared with us some of the feedback they wanted to get from the students. Several areas of interest included the students’ perceptions of the student-teachers and what they really liked about the program, as well as how they felt about a specific program component related to standardized testing preparation. My advisor and I quickly drafted three questions based on that information, but then were left to determine what other information would be helpful for the Suns and the high school administrators to learn. My advisor had been gracious enough to allow me to participate in all meetings she had with the Phoenix Suns and high school administrators to discuss the logistics and plan for this research study, so I was privy to a lot of the rationale and specific details surrounding this study, which ultimately was incredibly helpful in forming additional focus group questions. I put myself in the shoes of both the high school administrators and the Phoenix Suns and asked myself, “What do I want to know about the program from these students?” As cliché as it might sound, asking myself that question from the high school and Suns administrators’ perspective was incredibly helpful, and I immediately began to draft questions. Most of the questions were general in nature and included things like “What is your favorite part of the program? Least favorite?” and “What are some things the program doesn’t include that you wish it did?” Given the focus groups were all about student perceptions, I also thought it would be beneficial to ask students about program aspects that we already knew were successful, such as improving grades. For example, one question was, “Do you think the program helped you get better grades? Why?” Although the prior analyses (Dialynas, 2015) indicated that DPP did indeed increase participants’ grades, we wanted to determine whether the students believed an increase in grades (if applicable) was related to DPP, and if so, why they thought that was the case.

Method in Action

Once the potential students for the focus groups were identified, my advisor and I had created our list of questions, and we had secured times, dates, and locations to hold the focus groups, we were ready to go. We had scheduled six focus groups over a 3-day period and anticipated anywhere from a few students to close to all 36 participating in the sessions. Each of the 3 days had two groups scheduled, and my advisor planned to facilitate one, while I planned to facilitate the other.

Prior to each focus group actually beginning, however, we needed to get informed consent from each participant. Typically, for research participants under the age of 18, a parent or legal guardian must provide consent as well. In our case, since our research posed such minimal risks, the high school was willing to act in loco parentis for students who were under the age of 18—meaning the high school would act in place of the parent in terms of giving consent. While this was approved by our university institutional review board (IRB), the high school administrators also sent a note home with all 36 students to inform them and their parents/guardians of this study and the possible participation of their child. The note also indicated how parents could contact the school if they felt their child should not participate.

On each day of the focus group, we did not know which students would be participating until they actually entered the focus group rooms. At the onset of each focus group, students entered the room at or near the schedule time it was slated to begin. However, since we did not have a set number of confirmed participants, I was always unsure whether each student who entered my room would be the last one to enter or whether there would be more. This led to an issue one time, where I had already closed the door to the room and started the session, and an additional student proceeded to join the group. Luckily, I was still going over the informed consent and general focus group instructions, so it was not problematic.

At the beginning of the first focus group I facilitated, I made two mistakes. This was the very first time I was facilitating any sort of focus group or group discussion in any context, and although I had the book knowledge of how to conduct one, my lack of experience showed in these two instances. First, I assumed that all students knew they would be participating in a focus group. After introducing myself, I began to go over the informed consent process, which included each participant providing written consent to participate. I saw some blank or confused looks, but assumed it was due to the students just being typical teenagers. I then jumped immediately into the first question, which was about participants’ general thoughts about DPP. Yet, I quickly realized that some of my participants had come to the room because their “teachers told them to” and were mostly unaware of what they would be doing. Second, and related, was that I assumed all students knew about the DPP of which they were a part. After explaining to the students why they were in the room with me (i.e., to participate in a focus groups), a few of the participants also asked what DPP was. My internal reaction was one of shock and disbelief; how could these students not be aware (a) that they would be participating in a focus group and (b) they had been participating in DPP all along? I came to realize (through additional conversation with the students and my own reflection) that many of the participants had never heard of a focus group before, so they did not make the connection that they would be participating in a group discussion about DPP. I also realized that the students knew they had been participating in additional tutoring and mentoring with student-teachers from the university, but they were not aware that those activities were a part of a specific program known by a specific name.

Once I put my initial blunders past me, I was able to try to focus on the matter at hand. During each session, which I had initially assumed would last anywhere from 30 to 60 min, I found myself having trouble building rapport with the students. I had come into the situations automatically assuming the students would want to talk to me, as I had been a willing (and even not so willing) participant in several focus groups, and always actively participated, regardless. Additionally, I also assumed I would easily be able to find commonalities that would allow me to build rapport. However, I could not have been more incorrect. Where I thought I was close in age (as I was in my late 20s), they saw an adult who was 10 years older than them. Where I thought I was close to them academically, as I had been in high school only 10 years ago, they likely saw a person with a college degree and now in a graduate program. Where I naively thought I understood their academic experiences as I had done reading and research about educational inequities for low-income and minority schools and students, they possibly saw a White male who had gone to school in different circumstances.

In addition to struggling to generate rapport with the students, I also had a hard time allowing the conversation to ebb and flow. I had my list of questions that I wanted to cover, but I found myself asking the questions in the order I had them listed and having a hard time asking participants to elaborate on certain answers they gave. I was nervous about biasing the conversation, especially since I knew the effect of DPP on the students’ academics (i.e., it was beneficial). I was also thrown off by some of the participants’ reactions at the beginning of the session, about being unaware of what they were doing in the room or not being specifically aware that they were participating in DPP. Looking back, I think I was also nervous about wanting to fit in with the participants or have them see me in a positive light and want to talk with me. I think I was quite cognizant of the fact that I was not similar to the participants, in terms of age, race, or education level. Although I was able to solicit responses to all of the questions that I intended to cover and although those responses did provide some rich data (which I confirmed during the data analysis), I definitely had the feeling that I was in dire need of some practice to work on and hone my focus group facilitation skills, regardless of who the participants might be.

Practical Lessons Learned

• Do not assume potential participants will want to participate. One of my biggest assumptions was that I thought all or nearly all 36 students would be willing to participate in the focus groups. I had assumed that they would be happy to share their opinions about a program that benefitted them, especially if what they shared could affect change for the better. After reflection and some discussion with the high school administrators, I later realized that the students had no real incentive to participate. My advisor and I did not offer any incentive to participate, as is the case with many research studies, such as food or other incentives (e.g., gift cards). Since some students were not even aware that their participation in tutoring and mentoring sessions with the university student-teachers was a part of an official program, it was also quite possible that the students did not realize DPP was benefitting them, which presumably also did not encourage them to participate. If I had to redo the study, I would likely go about the logistics of confirming participation much differently. Although nobody can ever confirm participation in any research study until the participant walks into the room (and even then, any participant can leave at any time), I believe my advisor and I could have worked with the high school administrators in a more purposeful manner to confirm how many students would be participating, they all knew what they would be participating in, and they would show up on time and ready to actively participate.
• Do not assume participants know what you will be discussing. As mentioned, several students in the focus groups did not seem to be knowledgeable of DPP or their involvement with it, which I had assumed they would. One thing to keep in mind is that it is possible the language used for any program or study topic differs between participants and researchers. In my case, I knew about and referred to DPP by its official name. In my talks with the high school administration and the Phoenix Suns, they did the same—so I incorrectly had assumed that the involved students knew about DPP by name. I quickly found out during the focus groups that the students did indeed know about the program and their involvement in it, but they referred to it (during the focus groups with me and among themselves) in different terms. After the first focus group, I learned that a useful strategy at the onset of each focus group was to explain the program by name and then ask whether the students (a) knew about the program, (b) thought they participated in it, and (c) called it by any other name or referenced it differently. I found this especially helpful in the subsequent discussions as I could immediately get a sense of how much participants knew of the program and their involvement in it, as well as tailor my questions and my references to the program as needed (which had the additional benefit of improving my rapport with the students).
• Do not assume participants, especially adolescents or younger children, will willingly talk to you. Another one of my biggest assumptions was that I thought students would willingly and actively participate in the discussions. I assumed they would share their opinions about DPP and answer my questions in detail, elaborating to describe their thoughts and experiences. Reflecting on the experience, I was able to identify several reasons why my participants might have been hesitant to do so. One, and possibly the most obvious, is that they might have been nervous to talk to a complete stranger. Additionally, some of these students were not even aware of what they were in the room with me for! In addition to me being a virtual stranger, there were seemingly few similarities between myself and the students, as we appeared to be of different ages, ethnicities, and education levels. I now realize that, regardless of real or perceived differences, it would have been a great idea for me to try to build more rapport with the students prior to starting my “official” questions about DPP. This would have allowed me to loosen up and get rid of some nerves, as well as more importantly allow the students to gain trust in me and relax, and see that the discussion would be casual and possibly enjoyable, rather than be filled with anxiety or fear.
• Be mindful of how you might lead the participants to certain answers. This applies both to the initial question formation and during the focus group discussions. Since my advisor and I knew the program was effective, in terms of improving students’ grades, test scores, and graduation rates, we did not want to phrase any questions in such a way that would have led students confirm the program was helpful for them. We purposefully phrased our questions in a neutral manner to try to avoid this (e.g., asking “Has the program affected your grades?” rather than “How has the program helped with your grades?”). However, trying to ensure a lack of bias during the discussion was another challenge. In any focus group or interview, the facilitator or interviewer must be aware of the potential bias he or she can bring to a discussion, and this was something I was especially cognizant of, possibly to a fault. I was so conscious of my word choice that I feel like it hindered the quality of the conversation and my ability to build rapport with the participants. This leads me to my next point.
• Practice your focus group questions, asking for clarification, and asking follow-up questions. I cannot stress the importance of this enough. I was, and still am, used to doing minimal practice for talks or presentation. I would practice until I felt comfortable with the information without needing to refer to note cards or look at a slide on the presentation screen. Similarly, once I could recite the 10 focus group questions without needing to look at them, I (incorrectly) assumed I would be fine during the focus group. While I did remember the 10 questions during the focus groups, what was quite disconcerting was my lack of ability to effectively ask follow-up questions, probe for detail, and ask for clarification. Part of this might have been due to my high sensitivity of not wanting to bias participants’ responses (see above), but this was largely due to lack of practice. In addition to practicing questions—at the very least, read them out loud to make sure they sound how you want them to—it is incredibly beneficial to practice, with friends or colleagues, facilitating dialogue. This can even be with one other person, and even with somebody who has no idea about the topic at hand. Had I practiced my questions with somebody and had they made up answers to my questions, at the very least, I would have gotten more comfortable with facilitating dialogue and conversation.

Conclusion

In this research study, my advisor and I wanted to get feedback from participants of a high school DPP. Specifically, we wanted to talk to all parties involved—the students who were benefitting from the program, university student-teachers who provided tutoring and mentoring to the high school students, and the high school students’ classroom teachers. This case study centered on our attempts, and specifically my experience as a novice researcher, at eliciting feedback from the students involved in the program via focus groups.

Utilizing focus groups can be an effective research method to get rich data from multiple participants at one time. Additionally, focus groups allow participants to discuss ideas with and challenge each other, thus eliciting possibly richer data than an individual interview alone. While focus groups might seem easy to conduct, effective facilitation can be difficult, something I realized during this research study.

Ultimately, for any research project using any method, the researcher should be as prepared as possible. In addition to reading about a particular method, such as focus groups in my case, or talking to colleagues about best tips and tricks, it is incredibly beneficial to practice your method. Additionally, it is just as beneficial to assume as little as possible—something I learned the hard way. The experience I gained from leading this project, and especially from facilitating the focus groups, was invaluable. Although I do not plan on often utilizing focus groups with my future research, I now have the first-hand knowledge of what it takes to successfully facilitate discussions with diverse participants.