The aim of this case study is to highlight the importance of piloting or pre-testing the research design (in this case, semi-structured interviews and narratives). Most textbooks offer minimal guidance about pre-testing the research design, and published reports rarely report whether the research was piloted and, if so, what were the results. Therefore, this case study argues that researchers need to report on their pilot study so that others embarking on the research can learn from it. In this case study, I explore the concept of piloting and pre-testing the research, explain the process of collecting the qualitative data using semi-structured interviews, and demonstrate the complexity of analyzing the data when using a narrative approach, especially when the data are unexpected. In addition, I discuss the pilot study process and the lessons I learned.
By the end of this case students should be able to
- Have a better understanding of the importance of piloting their research;
- Have an insight of how to conduct qualitative interviews;
- Have a better understanding about the challenges researchers may face when the outcome from the collected data is not the “expected” one;
- Recognize the useful role that theoretical models may play in shaping the research.
As part of my professional doctorate program, I was asked to carry out a pilot study to “try out” the ideas and theoretical framework I proposed to apply in my main research. The term “pilot study” refers to a small version of the main research, whereas pre-testing involves a particular method or methods applied in the research (Baker, 1994). Pre-testing can include observations, questionnaires, interview questions, or surveys. Many researchers have suggested that it is often sensible to pilot the theoretical framework and to pre-test interview questions before the start of the main research because potential practical issues may be identified. Despite the generally accepted importance of the pilot study, it has received little methodological attention. For this reason, Edwin R. van Teijlingen and Vanora Hundley (2001) argue that researchers have an ethical obligation to report the results of the pilot study so that others embarking on the research can learn from it.
Subsequently, in this case study, I reflect on the pilot study I conducted prior to embarking on my main research regarding university experience and its impact on the professional work of early years’ practitioners (i.e., child care or nursery workers). The objectives of the pilot study were informed by my experience teaching and assessing students in the early years–related degree programs, mainly female mature students who lacked confidence in their academic ability but excelled at work. In this case study, I reflect on the way I collected the qualitative data, and then I demonstrate the complexity I faced in analyzing these qualitative data. Finally, I discuss the impact of the pilot study on my research and the lessons learned in the process.
Recent evidence shows that participation in Employer Based Higher Education (EBHE) early years–related programs in the United Kingdom and in Europe (Devins, 2013) is diverse in terms of age, race, ethnicity, entry qualification, and professional roles (National Union of Students [NUS], 2012). The main characteristic of this program is that students embarking on EBHE need to be employed (paid or unpaid) in an early years field in order to learn practical and professional skills relational to the relevant theory and policies. Early years policy and its articulation in the field of higher education, however, have been in flux since 2006, during which time I have been involved in teaching “working students.” Not only was my teaching approach shaped, but I have also developed a deep responsiveness toward these learners. Therefore, the rationale to focus on students studying on the EBHE program as opposed to other programs is entirely connected with my experience of teaching these students, who are mainly female due to the occupation in which they work. The main research focus, therefore, was to investigate a diverse group of university students’ emotional experiences and to explore their battle to “fit in” to the academic environment while studying on the EBHE program.
Since the end of the 1980s, there has been a rapid expansion of nurseries, or child care facilities (DfES, 2005) due to women gaining paid employment globally. Alongside this expansion has come an awareness of the need for the employment of a larger but skilled early years workforce, as both the quality and quantity of child care services depend in large measure on the workforce. The New Labour government in the United Kingdom under the leadership of Tony Blair between 2006 and 2010 invested heavily into the early years field; the government introduced financial help on several fronts (e.g., the Graduate Leader Fund) to help the process of upskilling the workforce. However, in recent years, the whole early years sector has undergone major “refashioning.” For example, more emphasis has been placed on professionalizing the field, with the introduction of the Early Years Professional Status (EYPS) in 2006, which in 2013 was modified to Early Years Teacher Status (EYTS). The introduction of the new qualification came with the withdrawal of bursaries and funds, leaving the workforce to rely on the financial support offered through government-monitored loans.
One of the manifestations of this refashioning is the mode of delivery of vocational degree courses. Part-time courses have been transformed into full-time courses but with the same expectation from higher education to meet “quality.” In other words, students on the course before the educational policy and funding changes enjoyed full bursaries for their higher education and employers’ support (paid mentor support) to attend lessons; they now face higher tuition fees and a shorter timescale (by 1 year) to fulfill the same university workload. This came with the rise of emotionally related issues within the university environment, not only because of the academic requirements that need to be met but also because the value and the purpose of the EBHE course in early years have received differing levels of engagement and understanding from employers (Mukadam & Mikuska, 2016). Therefore, to understand the ways we, as teachers, can and need to support these students, I decided to investigate students’ emotional experience on the EBHE course. I inevitably adopted a qualitative approach to explore the participants’ voices. I decided to use semi-structured interviews to gather the opinions of early years practitioners; this gave me an opportunity to engage with them and to prompt an answer when needed.
Piloting the main study is strongly recommended by Edwin R. van Teijlingen and Vanora Hundley (2001), as this “might give advance warning about where the main research could fail, where research protocols may not be followed” (p. 1). Therefore, I hoped that by pre-testing my research instrument (semi-structured interview), I would be able to assess whether my proposal was workable. Because I am a lecturer on the EBHE course and a new researcher, it seemed reasonable for me to select participants I already know.
After my pilot study was ethically approved by the University Ethical Committee, I approached three students I had previously taught. They all agreed to take part in my research after I contacted them by e-mail. I also attached an information letter about the pilot study to my e-mail (Appendix 1). Joseph Maxwell (2005) refers to this type of sampling as purposeful sampling, whereby the setting and persons are deliberately chosen to provide the information required.
It is important, however, to address that interviewing participants you already know may affect the data. For example, Nonie Harris (2002), in her work, reflected about her experience interviewing friends and stated that using a pre-prepared research introduction appeared contrived and unnatural. Furthermore, Harris reported on the power dynamics between the researchers and researched. As the researcher, she was concerned about loyalty to her friends and maintaining her friendship; she felt that her power was increased by the prior knowledge she had of her respondents and that this could lead to the interview data being misinterpreted. Consequently, the experience of interviewing people I already knew added another dimension to my pilot study, as I felt that participants were occasionally very conscious about how and to whom their story was told and what was told, paying additional attention to ensure that they answered my questions thoroughly, even seeking confirmation after each answer by asking me “Was it enough information? Do you need more to tell?”
Pat Drake (2010) also suggests that the identity of the researcher is likely to influence the research. Certainly, I have an identity with regard to my professional role at the university that partly explains my interest in studying students’ experience, but I also had a personal identity as the respondents’ former teacher.
With this in mind, I conducted three semi-structured interviews which were held over 2 days in July 2014. The participants were recently graduated early years practitioners who started their careers as parent helpers (someone who volunteers or works at the day care setting her children attend, or attended); all were females, aged between 35 and 43 years, who described themselves as White English. They were all employed in the early years sector at the time the interview took place.
The interviews focused on (a) the reason why the respondent joined the course, (b) the last 2 years of the respondent’s experience as a student and practitioner, (c) the way parent helpers are positioned and positioning themselves within their work and the university environment, and (d) the role of the respondent in the setting. The interviews were 35 min in length, on average, took place at the university, and were recorded using an audio digital recorder. I transcribed all the obtained data using the thematic narrative approach for data analysis (Riessman, 2000).
Jennifer Mason (2002) and David Silverman (2013) point out that good qualitative interviewing requires careful planning; therefore, for my pilot study, I used an interview schedule which proved to be very effective. The interview schedule, on which I listed the topics to be covered and my potential questions, assisted me during the interview. Prior to the interview, I sent the three participants the interview questions and an informed consent form through e-mail. At the beginning of each interview, I discussed the informed consent form with the participant, and once the participant granted consent, I asked her to sign the form.
Some of the prepared questions were as follows: “Would you tell me why you decided to join the university? or What prompted you to study?” and “Would you tell me what your experience was at the university and work place, focusing on the last two years of your study/job?” (Appendix 2).
The following example is an extract from the first interview I conducted:
Q: So, where do you work?
A: I am based at Nursery. In XXXX yeah.
Q: Is it a full day care setting?
A: Yes. Full day care. Yeah.
Q: Day care setting, full day. So that’s …
A: Past seven to half six, five days a week.
Q: Is it privately owned?
A: Yeah, privately owned.
Q: Day care setting. How big is this setting?
A: We have 50 children.
Q: Fifty children a day?
Q: So, age?
A: Between six months and five years.
Q: Okay. Six months, five years to school age.
Q: And how many staff members you have?
A: Full time workers, about 16.
Q: Sixteen. And your job role is?
A: I’m a preschool room leader.
Q: OK, so you are a preschool room leader. Can you tell me a bit about your and your staff background? Are they white British?
A: I’ve got one lady who’s Polish, one lady’s Italian. The others … I’ve got one lady who’s Pakistani. And all the other ladies are all White British.
Despite having an interview schedule, I was not following my predesigned plan. Perhaps due to nervousness and the thought that I needed to look confident in what I was doing, I deviated from my plan. I only realized after listening to the interview and after reading the first transcript that I was actually asking closed questions, and that I did not allow this participant to start her own narrative. It was both a disappointment and a surprise for me as I thought I knew how to conduct an interview. Learning from my own mistake, I better prepared myself for the second interview, but still found that I had plenty to learn. For example, my questions were answered, but I failed to prompt the respondent to shed light on the issues or facts raised. For example,
Q: So, would you say that the children in the nursery are fairly diverse?
A: Yes, yeah. I’ve got a range of children. Some are sort of East Europe, like Romanian and Polish. Some are from places like Pakistan, India, and those kinds of countries. And others just sort of White British. I’ve got a range of different cultures and different children really coming to us. Some are from what you could define as deprived backgrounds. They are friendly and they need that support when I was working full time. Sort of the doctors and those kinds of professions. So, we’ve got a diverse range of care. Yeah.
The answer clearly shows that the children attending the nursery are diverse. Not only diverse, but some of the children come from deprived backgrounds. Reflecting on this question, I missed the opportunity to prompt this topic, such as, “Can you describe what do you mean by ‘deprived’?” or “ Can you give an example of which kind of support they may need?”
Overall, reflecting on the three interviews I conducted to pre-test my research, I can certainly say that they helped me to develop my interviewing skills. They also taught me to follow the interview schedule more rigorously and that it is acceptable to read my own notes. Therefore, conducting a pilot study was vital for me not only to evaluate the feasibility and appropriateness of my interview questions prior to the performance of a full-scale research project but also to learn how to listen to the respondents. The next part of this case study describes the process I used for data analysis.
Once the three interviews were conducted and the data transcribed, I looked forward to bridging the gap in the knowledge, by asking myself “what to do now?” I read the data several times. Then, I re-visited the suggested literature on narrative analysis (Riessman, 2000; Earthy & Cronin, cited in Gilbert, 2008), in which it was suggested that a narrative approach consists of not only the story-telling component but also the social interactions between the interviewer and interviewee. Therefore, the first step in my data analysis process was to decide what aspect of the narrative to explore and how the data would help me do this. I decided to use a thematic approach as it involves comparing the accounts from a sample of interviewees with similar experiences. The next step was coding the data, that is, building a set of themes by looking for patterns within the data, which was followed by labeling and grouping them (Riessman, 2008; Weber, 1990, cited in Gee, 2014).
On reading the transcripts, I found many comments relating to motherhood (maternal discourse). I was surprised and realized that I had ignored this recurrent theme when I designed the research. For example, when I asked the question “Would you tell me what your experience was at the university and work place, focusing on the last two years of your study/job?” all the answers started with “When I gave birth to my child eight years ago …” or “After giving birth I suddenly felt that I need to resign, and seven years later …” Not only was the theme unexpected, but so too were the similarities between the respondents. They all started their early years professionals career as volunteers in the child care setting their children attended.
Therefore, I re-visited the literature, which helped me understand that unexpected findings are part of the learning process of analyzing and contextualizing the data. Since the aim was to listen to each respondent’s “voice” and because the motherhood theme was evidently important to the participants, I decided to explore the recurrent theme of motherhood more closely. For example, further comments relational to motherhood were made:
I was pregnant so went to look around day nurseries …
I turned into a mother role, mothering others …
… because I am a Mother, I am a better practitioner.
Thus, within this theme of motherhood, the text was further coded to determine similarities and differences in the various narratives.
After coding the theme of motherhood (maternal discourse), I then divided my findings into two subgroups: First, how discourses of motherhood are mobilized to construct the professional self; and second, the intersections of gender, age, class, and motherhood in constructions and performances of the professional self. This discussion is based on my understanding of the maternal discourse in the early years profession within the UK context (Ailwood, 2008; Butler, 1999; Osgood, 2012) and takes into account the perspectives of the early years practitioners who started their careers not only as volunteers but also as mothers.
Discourses identified in the narratives I collected were related to the construction of the professional self and the way in which early years practitioners who started their careers as volunteers positioned themselves within the work environment. Sophie stated,
I didn’t want anyone else to think “oh because she has got a son there,” “he is sitting on her lap again,” or this sort of things … and because other people had children in the setting and they would say “oh look, she is with her daughter all day and she is following her around.” So, I was quite conscious to keeping that professional and personal space more separate you know.
For Sophie, it was very important to be seen as a professional. The notion of professionalism is considered “modern” within the early years context, since it became a hot topic with the introduction of the EYPS in 2006 and EYTS in 2013. Furthermore, this quote from Sophie also suggests the intersection between the private and public split which is an ongoing feminist debate between the dualism of rational/emotional, mind/body, public/private (Hochschild, 1983). Within Sophie’s narrative, this split is clear, as she is explaining the way of separating mothering from the professional self. This split is also present in the following extract from Gill, in which she explains that achieving the balance between the professional and personal self was an emotional practice:
I think you know … I work with my emotions. What I do, my work with children and so many difficult children yeah … with everything … wear my heart on my sleeve … but it is different when I work (than being a mum). At work, the children aren’t yours, and you can do so much with them, but I do feel that you are forming such a close relationship with them, especially with some of those children (in need) …
This extract indicates the role of emotions in professional work where the maternal discourse was mobilized to compare her feelings for her own children with those for the children she cares for. Arlie Russell Hochschild (1983) coined the term “emotional labour,” claiming that a feeling or emotional response is self-induced and provides the basis of “acting” or management of emotions. She also argues that emotional cues may be among the most important in human interaction as in the previous extract where Gill describes the emotional involvement as “I wear my heart on my sleeve,” indicating that the emotional investment in her job is a big part of her life. Furthermore, Hochschild argues that women are more often employed commercially for emotion-based work (such as nurses); however, it can be argued that nursery workers, as in the case of the participants in my pilot study, entered child care careers by default, needing child care for their own children (Osgood, 2012).
An interesting theme in maternal discourse is highlighted in the following extract, in which mothering or a mothering attitude seems to be seen as a resource to gain power and deal with uncertainty:
Being a student, I found it difficult because I’m always the person in charge, and you go into a classroom, and suddenly, I am not in charge and you’re very much at the bottom of the pile, and I think I was older as well in group where there were lots of younger practitioners I felt a bit uncomfortable to start with … I think I probably had a bit of lack of tolerance for some of them, and I wanted to look after them, I wanted them to do well when they necessarily didn’t me want to do well and I could still see that I was trying to encourage them and help them develop and you know people would sort of e-mailing me saying “Can I send you my work?” so I could read it and I found myself turning into the mother role in the group as well, which is interesting because I didn’t think that would happen. Yeah … it is actually happened. (Angie)
Judith Butler’s (1999) view on the maternal discourse is that it is also performative, whereby individuals’ subjectivity changes over time and through the ways individuals are positioned and position themselves, offering a reason for Angie’s performance. For Angie, the maternal experience (she has six children, with the oldest being 12 years old) has helped her overcome the issue of “not being in control.” By displaying or performing a mothering attitude toward younger classmates, Angie positioned herself in such a way that she gained confidence in her class and was viewed as “knowledgeable” by others seeking advice from her. However, Ruth Wodak and Michael Meyer (2009) state that “knowledge is conditional, i.e. its validity depends on people’s location in history, geography, class relation and so on” (p. 34). As knowledge and identity are strongly connected, the knowledge of “mothering” is (re)constructed in the university experience. Furthermore, Angie clearly specifies how she feels about being a mother and practitioner:
Because I am a mother, I am a better practitioner … my emotions are totally different to someone who hasn’t got children but who works as a practitioner with lots of children because I see it as the other’s (mother) perspective and I think that I am more emotional probably because I am a parent, more so … I feel guilt that I kind of left him (child in the nursery), as I haven’t been there, I haven’t read or focused on him; I probably could have helped him, and perhaps, other practitioners haven’t so much.
Here, the discourse of motherhood is used to construct the image of an ideal early years professional; in other words, being a mother is the same as being a good early years practitioner. The intersection between mother and good practitioner is clear in the everyday life of Angie. Stephanie Lawler (2008) clarifies this point by stating that multiple identities coexist and that social class identities are marked out by “wrong” or “right.” Here, the “mother” is seen as doing right things, and this becomes the hegemonic norm. Therefore, the understanding of relevance to social class within the UK context, as attitudes toward others in this case, is based on motherhood. Furthermore, this quote is in line with the assumption that being a mother makes one a better child care worker and that mothering can be (has been) used as a form of power.
The following extract highlights the issues involved in intersecting strands of difference (class, gender, age) in constructions of motherhood. Although I had not asked any questions about ethnicity and class, the notion of the middle class being privileged in professional discourses and practices was evident:
They knew I haven’t worked in the early years before, but I thought I can show my actions and by modelling that I know what I am talking about you know … I am a middle class older women, older mother, and they were very young teenagers. They always come to me to talk to the parents, so I almost felt I was mothering a little bit you know … which was OK to a certain point. (Sophie)
Not only does the reference to class stand out in this quote, but also the way Sophie positioned herself with regards to being an older mother; so the intersections of age, class, and motherhood in constructions and performances of the professional self are significant factors.
My pilot study clearly highlighted that by using a narrative approach to explore students’ experience and by listening to their voice, I have failed to acknowledge the maternal discourse in the early years’ context. As a result, I need to revisit the literature review to address the debates around maternity and early years professionals and I need to reshape the interview schedule. On reflecting on the narrative approach used in my study, I recognized that the thematic narrative approach can generate new ways of understanding maternal discourse, evidenced by the finding that motherhood can be used as a form of power within the work and classroom environment. Therefore, the assumption of “because I am a Mother, I am a better practitioner” can be problematized.
In addition, I also need to revisit the demographics of the participants in my main study, as I believe that it would be necessary to include a sample group of non-mothers in the main study in order to compare and contrast the similarities and differences between their narratives.
Overall, this pilot study has highlighted the gaps in the knowledge in the literature as well as the need for me to adjust my research design prior to conducting the main research. So, when something goes “wrong” or not as you planned, you will discover that it is part of the learning process and it is more common than has been reported from researchers.
It is also important that you record your interview and transcribe it as soon as you can while it is fresh in your memory. By doing this, you will learn the data that may highlight new topics you had not considered; therefore, you have an option to revisit the interview schedule and to adjust the interview questions and your topics. Please remember, there is no such thing as “wrong interview” as every interview gives you new knowledge and new skill.
- This case has revealed the importance of carrying out a pilot study. I highlighted two reasons: you can practice your interviewing skill, and you may identify gaps in the literature.
- Discuss in small groups other possible benefits of conducting a pilot study.
- Discuss in small groups possible disadvantages of conducting a pilot study.
- In this case study, I knew all of my participants. Discuss in small groups whether knowing the participants is actually an advantage or not.
- I found that the data gathered through the semi-structured interviews offered rich data. What other data analysis method could you suggest? Why?
- In small groups, design a new set of interview questions addressing the maternal discourse. Once you created a minimum of three questions, try them out with your colleagues or fellow students.
I would like to invite you to participate in my doctorate pilot study. This research is looking to investigate how the undergraduate degree may have changed you as a professional. I am particularly interested in discovering what your emotional experience was throughout the last 2 years as a student and as an early years professional. Also, how these emotions changed/shaped you as a person.
If you choose to take part in this study, then please reply to this e-mail. You will be invited to complete an informed consent form in order to take part. The study has only one part which is to participate in a face-to-face interview conducted by me. The interview will take place at a location of your choice and at a time convenient to you. The interview will last between 30 and 45 min, and the data may be used for publication purposes.
There are no known risks to taking part in the research. Any information that you give during the study will be kept confidential and anonymous in accordance with data protection legislation. You have the right to withdraw your participations at any time before July 31, 2014.
The study is approved by my supervisor in accordance with the University’s Ethical Policy Framework. If you have any questions about this study, then do let me know by e-mailing me.
- Would you tell me why you decided to join the university/what prompted you to study?
- What prompted you to study to obtain the degree? (financial, self-esteem)
- Would you tell me what you do in your job? What is your job role?
- How long have you been working in this field?
- What was your job role before?
- Would you tell me what your experience was at the university focusing on the last 2 years of your study?
- Would you explain how it has changed you as a person?
- In what way it changed your professional practice?
- Would you tell me whether your job has changed since you obtained a degree? If your answer is yes, can you describe in what way?
- Would you tell me how do you feel now when you completed your study?
- Can you tell me a bit more about that?
- What do you mean by “emotion”?
- What do you mean by …
- Background (job role, power relationship between manager/practitioner, social class, gender, age)
- Emotionality (combining work with study and other duties—care for children, parents)
- Identity (conscious or tacit) as a student and as a professional