This case study describes the methodology used to analyze qualitative data accumulated from 50 professionals representing six different health and human service professions: social work, public health, nursing, community psychology, law, and medicine. They were brought together for a day of simulated exercises and reflections to collaborate on a community health issue. This case study describes the fundamentals of qualitative coding and analysis, including the integration of computer-aided ATLAS.ti qualitative analysis software. The program was used to code and analyze 21 transcribed group deliberations. This case study is a practical application for Master’s level students, doctoral candidates, academics, and practitioners on the use of qualitative coding software and its relevance to social work and public health scholarship, exploring the intersections of inter-disciplinary collaboration in community health settings.
By the end of this case, students should be able to
- Understand the development of a research project on inter-disciplinary collaboration through simulated focus group exercise
- Understand and apply use of grounded theory for qualitative analysis
- Explain and make recommendations for creating qualitative coding systems
- Learn descriptions of a qualitative coding system related to inter-disciplinary collaboration
- Apply technical advancements in qualitative analysis for current inter-disciplinary research and exploration for the health-related professions
This case study explores the use of the ATLAS.ti qualitative analysis software program to analyze research based on an exercise that addressed the social conditions of a simulated neighborhood with numerous community health and social problems. The goal of the study was, first, to analyze how professional background, experiences, and values affect the process, strategies, and outcomes of collaboration around specific community health problems, and, second, to examine the impact of professional identity on inter-disciplinary community collaboration (ICC), defined as projects that are designed to improve the social and health conditions of communities and populations at risk.
This case study will describe the use of ATLAS.ti, qualitative methodology, and present select findings related to community collaboration to illustrate the value and challenges related to qualitative research in general, and the use of ATLAS.ti in particular.
The research team selected and invited a cohort of 60 professionals (10 from each of six health related disciplines (medicine, law, psychology, public health, nursing, and social work) to a day of structured simulation of community health problem-solving. They served as informants on the subject because of their expertise and insight into inter-disciplinary dimensions of collaboration. The researchers and others knew them as being knowledgeable and experienced in ICC. Ultimately, 51 professionals participated: nine lawyers, 10 physicians, seven nurses, seven psychologists, nine public health professionals, and nine social workers. The cohort included 11 men and 40 women (22% and 78%, respectively). Their ethnic breakdown, based on the authors’ knowledge of the participants, was 33 Caucasian (66%) and 18 people of color (34%), (2 Asians, 9 Black/African Americans, and 7 Latino/as). The participants had an average of 15 years of ICC experience (with a range from 10 to over 40 years).
Researchers designed a hypothetical case study, a well-established qualitative research method, of a composite urban neighborhood problem and asked study participants to deliberate on the same case study, both in groups with members of their own profession, and then in inter-disciplinary groups (Quinn Patton, 2002; Wayne, 2013; Yin, 2009). The single case study included information about a fictional community with a number of health and environmentally related problems including community demographics (see Appendix 1).
The descriptive aspect of this research as well as the small sample size and direct measurements of sample characteristics do not allow us to make direct inferences to a larger population (Kachigan, 1982). Nevertheless, a qualitative study with 40 informants is considered to be a relatively large study (Hudelson, 1994). Thus, the sample size in this study exceeds those recommendations.
A total of 21 group dialogues were held on two different days. There were 12 mono-disciplinary groups (two groups from each of the six professions), who deliberated first on the case study, and then broke into 9 multi-disciplinary groups with a mix of these professions, who deliberated for a second time using the same case study. We use the term multi-disciplinary here to represent the short-term collaboration that took place in this study’s focus groups. These 21 group discussions were transcribed from audio recordings into Microsoft Word documents and afterwards imported into ATLAS.ti software, to be discussed below. In addition to the transcripts of 12 mono-disciplinary group and 9 multi-disciplinary group simulations, the methodology also included the collection of additional datasets, including separate memos written by the coders noting the process and outcomes of each group based on the multiple readings of the transcripts.
In addition, at the end of the day, they completed a self-administered questionnaire from which we derived both qualitative and quantitative findings. Participants also engaged in a closing debrief with all the groups, facilitated by researchers.
ATLAS.ti, a qualitative software program, was chosen because it allows for electronic project management, multiple possibilities of query and data analysis, easy creation of codes, renaming codes, viewing co-occurrences, that is, how often two codes occur within the text of themes within the data, and identifying trends in relatively easy-to-use software (Friese, 2002). In practical terms, the software was chosen because the primary documents were lengthy, multi-layered in content, generating much data to be analyzed. This resulted in diverse interest areas from the multiple researchers involved, and who wanted to create scholarly work related to inter-disciplinary collaboration among health professionals, community building, and the teaching of collaboration to other professionals. For example, Dr. Martha Garcia was able to use the same transcripts uploaded into ATLAS.ti to explore communication methods and strategies related to management of conflict among the inter-disciplinary collaboration in community problem-solving (Garcia, 2013). ATLAS.ti allowed researchers to focus on particular units of analysis allowing for cross-referencing of codes and categories to be explained further.
The codebook was created using the grounded theory analysis method (Corbin & Strauss, 2008). With the goal of understanding the process of collaboration among and between different professions, grounded theory was selected as it provides a methodology that assists in theory development through the process of analysis.
Through the use of constant comparative methods, similarities and differences within the text were identified that could help answer questions related to inter-disciplinary collaboration without relying solely on prior research or hypotheses. Grounded theory method allows researchers with previous knowledge of subject matter to test theories, hypotheses, and previous research (Gilgun, 2013).
New Codes were created as themes began to emerge after multiple readings of the text. This method includes: (1) reading and re-reading the textual database; (2) identifying and labeling of variables into specific categories, properties, and concepts; and (3) examining these categories and concepts to determine their inter-relationships. In the beginning analysis stages, content analysis was carried out, while analyzing transcripts on paper. This allowed for the emergence of manageable classifications and coding schemes (Quinn Patton, 2002), and to the creation of a preliminary codebook. These documents were uploaded into ATLAS.ti providing greater ease for analyzing codes qualitatively and quantitatively, and allowing for sharing documents and analyses more easily through saving and sharing files. Inter-rater reliability was integral from the beginning of the process. Each transcript was assigned a primary coder, and a secondary coder who reviewed the initial coding, comments, and questions, adding their own insight where necessary, or weighing in on questions the primary coder had about coding. When the secondary coder wasn’t sure about a question or when there was a difference in interpretation, a third coder was brought in to help settle the question at hand. Any differences were entered into “Comments” section of ATLAS.ti that a third coder reviewed and reconciled when necessary.
Research assistants transcribed the 21 taped simulations into Word documents. These Word documents were uploaded in ATLAS.ti software. Each of the 51 participants received a unique Identity Code, for example, M065FH-profession (M = Medicine), number from 1 to 51 of participants (06), followed by which of the 2 days they attended (5 = May), followed by Gender (F = female), and Race/Ethnicity (H = Hispanic). These codes allowed researchers to seek out any themes that emerged by different demographic and professional classifications.
Researchers developed a codebook that is a list of terms based on key themes, trends, and categories within the data, and based initially on the following five questions the participants used in the problem-solving simulation (see Appendix 1). The codes used are in brackets:
- Identify a community health problem that should get highest priority. [Code: Problem] [Code: Priority]
- Identify the group’s goals and objectives to address the problem. [Code: Goals]
- Develop a strategy and/or a program to solve this problem. [Code: Strategy] [Code: Solution]
- Define and measure your success. [Code: Evaluation]
- Identify other professions and/or organizations, you think need to be involved in order to achieve the goals. [Code: Organizations/Professions]
The four primary researchers read through the transcripts several times and identified various themes using Grounded Theory (Corbin & Strauss, 2008). Grounded theory allows for the categorization of data through a systematic process of creating codes only as themes emerge in multiple pieces of text within the dataset, thus grounding codes and interpretations to text. Grounded theory allows for discovery of new insights directly linked to data. In accordance with grounded theory, method codes were assigned to a theme, which were directly linked to selected text to classify that text and provide meaning to data. For example, the theme of involving individuals who reside in the community in the decision-making process emerged from the data first identified by one of the researchers. After agreement by additional researchers, we developed a code assigned as “Community Collaboration.” These researchers came together to reconcile differences in coding and interpretation of data as they read through additional transcripts related to this new theme. They then identified new codes to be added to the initial five in the codebook: “problems,” “priorities,” “strategies,” “solutions,” and “goals” bracketed above. Researchers checked the validity of one another’s coding of the data by confirming conformity to the definitions of codes. The research team would reconcile interpretations of text related to the codebook. Based on what we observed in the transcriptions, we created categories for similar codes such as codes that are related to community involvement in the decision-making processes and solutions. These codes in quotations were identified references to the call for “Community Needs Assessment,” “Community Participation,” and “Community.” Since the context generally called for the involvement of the community as part of the strategy to resolving the problems, we created a family of community codes.
The authors incorporated several a priori categories into their coding based on theories related to the level of problem identified to address the community health problem(s) such as micro, mezzo, or macro (described below). We also utilized “in vivo” coding, a method that allows new codes to emerge from the data as it is read and re-read. Both a priori ideas and in vivo codes are aspects of the grounded theory approach (Ryan & Bernard, 2003).
The a priori coded categories were based on the questions included in the case study listed above, many of which came from the literature or from the researchers’ experiences: (1) Community Health Problem(s) Identification, (2) Priority Identification among the problems, (3) Strategies to Resolve Community Health Problem(s), (4) Solutions to the Community Health Problems, and (5) Goals. The following definitions were used to guide the coders:
- PROBLEM: Based on the case study, the participants identified problems they saw, such as housing, social determinants of poverty, environment, lack of education, and so on.
- PRIORITIES: These were the specific problems that participants identified as the most important; for example, some participants stated that housing was a priority because they defined it as a community health problem.
- STRATEGIES/SOLUTIONS: These were the main tasks, activities, and interventions participants said were needed to resolve a community health problem; for example, one strategy suggested was to conduct a more comprehensive community needs assessment with additional stakeholders. Other strategies included increasing health access, more community involvement, and seeking additional funding.
- GOALS: These were identified as the desired end result after implementation of their suggested strategies: for example, building a community health center, or reducing infant mortality.
- PROCESS: The actions used to enhance the ability of groups from diverse backgrounds to come together to find creative solutions to the problems posed by the exercise. It also refers to the process of forming as a group and managing differences (Garcia, 2013)
An example of in vivo coding emerging from their responses to Question 5 was to add a code for the involvement of “Other Professions/Organizations.” The participants identified other members of the community they would invite beyond the categories of organizations and professions that were posed; namely, they consistently added involving “the community” generically or specific groups in the community such as residents or youth.
The codebook was continuously revised and transcripts were re-reviewed when new codes were identified. Codes were classified and grouped into families of codes. The team continued to refine and re-code the transcripts, reconciling questions and identifying additional new themes, or further refining themes, when necessary. The main file in ATLAS.ti consisted of the raw dataset and all subsequent coding.
The term “family” refers to groupings of individually unique codes that can be combined but still maintain their distinct assignation. Multiple codes can be combined into families. Documents can be combined into families and families of codes can be combined into larger families. This allows for rich analysis since multiple “problems” identified by participants each received their own individual code, such as “Infant Mortality-Problem” and “Access to Health Care-Problem.” The ability to create families of codes allowed researchers to identify trends with the data without having to re-code individual data since codes linked to text remained. Researchers grouped together various codes into families of codes. These families of professions were created to group together each of the six different professions while still allowing for the retention of each individual participant. For example, there were nine individual social workers who each had their own identifying code. A family of “social workers” was created allowing for data analysis related to social work identity.
A family of codes was created to denote the theme of “collaboration” that emerged as a strategy, and as a broader response to Question 5 posed. As noted earlier, many of the participants mentioned that “the community” should be used to improve health conditions and/or that the community residents should lead the assessment and intervention process to solve the community health problem. These responses went beyond the originally posed question asked: to name additional organizations and professions. Hence based on in vivo coding, the researchers then created a number of codes around the theme of collaboration as a strategy. These included “Collaboration-Community,” “Collaboration-Professional,” and “Collaboration-Organization/Sector.” All of these codes were then classified under the code family: “Collaboration.” Professional collaboration included working with other professions such as doctors, lawyers, social workers, and so on. Organization/sector collaboration included agencies and institutions such as businesses, elected officials, and religious institutions. Community collaboration was added later to include the involvement of “the community” broadly defined, and/or reference to the inclusion of residents or members of the community as a whole, or a subpopulation such as youth or immigrants. The top left hand corner of Example #2 illustrates a sample of groups of codes into families in ATLAS.ti.
As the coding progressed, the coders discovered that the participants sorted out the “Priority,” “Problem,” “Strategy,” “Goals,” and “Solutions” on a variety of levels. It became clear that a broader classification of codes was needed to indicate the level at which a strategy should occur. These were classified into micro, mezzo, and macro interventions:
- Micro interventions indicated specific health- or medical-related services or issues, such as diabetes, or a narrow, more limited approach that focused on the individual, family, or small group clinical level.
- Mezzo interventions indicated mid-level management of broader issues, such as a health-related or population-based problem. Mezzo interventions include preventative and educational strategies and also the development of a health clinic, or improve access to health services for the residents without insurance.
- Macro interventions indicated systemic issues focusing on the community as a whole or on the underlying causation of the specific problems. It is the broadest level of intervention and includes approaches related to the “social determinants” of health that included housing, employment, racism, and poverty.
Using the ATLAS.ti query tool enables a researcher to analyze data in segments. Researchers read every quotation that appeared in that query for greater understanding of content within the larger context of the transcript. This review of text allowed for greater understanding of each participant’s approach or values in answering the hypothetical question. For example, researchers ran a query for each profession together with the “community collaboration family” in both the multi-disciplinary and mono-disciplinary groups. Then they analyzed how each professional cohort and each individual within that profession identified the concept of collaboration in general. This allowed the research assistants to see that in one transcript, participants used a range of related terms for the same concept, such as community collaboration, community empowerment, and community needs assessments—in general expressing a strong belief that the community must identify the problems for themselves in order to get the community to collaborate with professionals for change. Therefore, researchers were able to identify a major theme, that of community collaboration, in the findings.
The computer screenshot image of the ATLAS.ti query tool (see Figure 1) captures how frequently the family of physicians’ codes co-occurred with the individual code for Community_Colloboration_Strategy (upper right box) providing direct links to coded text (bottom right box). This illustrates how often physicians were coded as discussing collaboration and the involvement of the community. The upper left box illustrates families of groups of codes, while the lower box illustrates some individual codes.
A co-occurrence is when two or more codes exist within the same coded text. Using the ATLAS.ti “query tool,” we were able to see how many times each of the six profession codes co-occurred with coded text (i.e., Problem, Priority, Strategy, Solution, Goal). We analyzed the co-occurrence of code families within families such as Micro, Mezzo, Macro, and Community Collaboration. The accuracy of quantitative and numerical queries was cross-compared by running frequency counts for codes, leading us directly to the transcripts to read for the content. See Table 1 below for the co-occurrence of how often macro, mezzo, and micro codes were coded for each professional cohort.
As part of the on-going coding process, a separate document “Memos” was created. As is suggested in grounded theory analysis (Corbett & Strauss, 2008), the primary coder of the transcript wrote up summary memos that reflected their overall impressions of each transcript, with a focus on how the codes reflected the course of each groups’ deliberations. ATLAS.ti has a built-in memo function allowing for memos to be tracked in ATLAS.ti by date and author. It also focused on the process each group used in reaching conclusions, the level and intensity of study participants’ involvement, commentary on the various professions’ participation, and whether they completed the assignment (i.e., specifically addressing the five posed questions). In addition, one member of the research team also read the 21 transcripts and then provided her reaction to the coders’ memos—agreeing and/or providing a different interpretation of processes and outcomes.
In examining the frequency that each code family appeared in the group simulations, the researchers found that the participants discussed solutions far less than the other four main categories (goals, problems, priorities, strategies). They also found that strategies were by far the most frequently coded. After coding for solutions and strategies separately, the research team eventually collapsed those two codes together because study participants seemed to be using strategies and solutions as relatively interchangeable terms; both were used to indicate a means to an end. Additionally, some codes could exist within multiple levels (micro, mezzo, and macro) depending on the context in which it was being discussed. For example, the creation of a health clinic could be considered both a micro and a macro solution since it would lead to greater health access for the community. In order to mitigate this, the research team attempted to locate each code in one of these three levels, depending on the content of the quote in which it was included.
We would like to share our insights of the process through our experience and our recommendations:
- Researchers implemented both a priori and in vivo coding for data analysis and chose ATLAS.ti for qualitative data coding and analysis to allow for shared files among researchers and mutual creation of a codebook by a research team by using one file.
- ATLAS.ti allowed for development of a codebook throughout the coding process by adding new codes, renaming codes without having to re-code text since ATLAS.ti embeds a code in selected text.
- We recommend tracking the length of a codebook and reclassifying codes as coding progresses and develops by combining similar codes and creating families of codes. This helps limit long and unwieldy code lists.
- Researchers used memos recorded in ATLAS.ti as they coded and analyzed data, recording their impressions in real time that served as a communication tool between researchers allowing additional opportunities to agree and/or provide different interpretation of processes and outcomes.
- Utilize the memo function of ATLAS.ti to record notes on the creation, edits, and deletions of codes identifying themes grounded in the text.
- Researchers did not all have the same level of experience using ATLAS.ti and it was important to communicate, clarify, and explain ATLAS.ti terminology.
- Establish shared use of terms related to qualitative research and ATLAS.ti terminology to understand the relationship of ATLAS.ti terminology and tools such as the concept of classifications of codes within families, co-occurrences of codes, and frequency counts.
- Researchers on occasions were not able to open ATLAS.ti files if they attempted to work with a file that was updated by another researcher using a more recent version of the software.
- Update ATLAS.ti as soon as the new version is released.
- Save back-up files including versions of the codebook for reference.
- ATLAS.ti was very helpful in allowing multiple researchers to code and analyze a dataset to answer their research questions about inter-disciplinary collaboration from related but independent viewpoints. For example, Dr. Garcia (2013) analyzed the dataset related to communication techniques among professionals. Dr. Mizrahi is in the process of analyzing the identification of strategies and solutions among the various professionals related to community collaboration. Researchers and research assistants were able to access the documents from multiple locations, which was a major benefit allowing for flexible and independent work hours.
- Researchers recognize the value of insight gained from research assistants. It was they who noticed contradictory interpretations, as well as new emerging themes. The collaborative process between those engaged in research design and those who input data into ATLAS.ti is essential. The ability to learn from research assistants is a significant lesson for principal investigators.
- What are the benefits and challenges of having multiple coders on a qualitative analysis project? We have included articles in our references below that address the usefulness of memos, supervision related to coding and reflexivity (Engward & Davis, 2015), and usefulness of qualitative data analysis software (Drisko, 2013).
- What are the benefits and challenges for research teams when using particular software for analysis?
- Given the multiple levels of analyses ATLAS.ti is capable of doing, at what point do researchers decide to stop coding and re-categorizing codes and decide one overall approach? In other words, should they decide on the unit of analysis first?
In dear memory of our partner Marcia Bayne Smith, Associate Professor, Queens College.
SOCIAL CONDITIONS IN OUR NEIGHBORHOOD AFFECTING COMMUNITY HEALTH STATUS
Here is a description of the health and social conditions in an imaginary community. Please use this example in order to answer the following questions:
- Please identify a community health problem that should get highest priority.
- What are your group’s goals and objectives to address the problem?
- Develop a strategy and/or a program to solve this problem.
- How will you define and measure your success?
- Which other professions and/or organizations do you think need to be involved in order to achieve your goals.
Your group will be asked to present your plan to a council of funders afterwards.
The City Health Planning Department has just released new data about our community. The profile of the neighborhood includes the following:
- Our population is growing from its present 100,000; over 40% of the population is under age 21. About 15% of the population is aging, and many are over 80 years old.
- Many people in our neighborhood are undocumented immigrants. Many of whom do not speak English well, and some do not speak English at all. The major languages of the new immigrants are Spanish, Chinese, and Creole.
- We have one of the largest numbers of single-parent families in the City, an increasing incidence of teen pregnancy, low birth weight babies and a higher rate than the city wide average of infant mortality. Some of these families are reliant upon public assistance.
- There are many large apartment complexes in our area. Most are privately-owned and in disrepair. An increasing number of old buildings are being abandoned by their owners. While some tenant and block associations exist, there is limited participation.
- There are many voluntary groups and social service agencies.
- Local elected officials are trying to attract City and State funds for housing rehabilitation.
- There is a new community policing patrol program operating in our precinct. The day laborers have experienced harassment from the police.
- There is high unemployment, but many families receive income from small businesses they operate out of their homes.
- Our health care facilities are limited and have had to cut services in their specialty clinics and primary care unit.
- As a result visits to local hospital emergency room have increased dramatically, specifically amongst immigrants who are unable to afford primary care.
- We have a large park, although due to budget cuts, it is not well maintained, and is becoming a site of more and more criminal activity and stray dogs.
- There is a visible presence of drug dealers and sex workers in the park and other parts of the community.
- There is a community college nearby which specializes in entry level technical human and health service programs.