Qualitative Analysis to Examine Primary Care and Public Health Integration Success Determinants in the United States

Learning Outcomes

By the end of this case, students should be able to

• Discuss the impact of data saturation on sample size in qualitative analyses
• Define content analysis and describe research limitations associated with this method
• Justify the importance of exclusivity and exhaustivity in codebook development
• Differentiate between primary and secondary data
• Justify the importance of operationalizing in the research writing process

Project Overview and Context

The Association of State and Territorial Health Officials (2014) collaborated with more than 60 organizations and 200 individual partners to develop an Integration Forum aimed at promoting and supporting primary care and public health integration in the United States. Inspired by the array of collaborative efforts displayed in the repository and the unique ability of the information contained within to strengthen the body of knowledge of when, where, and under what contextual circumstances collaboration and integration have proven or been perceived effective in the United States, I decided to analyze the written secondary data in a qualitative dissertation. A major purpose of my work was to identify determinants of successful collaboration between primary care and public health entities in the United States. Few formal analyses of the efficacy of such efforts exist. Empirical evidence suggests integration efforts can strengthen healthcare by improving health-related outcomes, access to care, chronic disease management, and communicable disease control (Martin-Misener et al., 2012).

Research Design

As a relatively inexperienced researcher, the first deliberate decision I made was to focus on secondary data. I made this decision because existing literature suggests primary data collection often prolongs the research process (Jacobsen, 2017). Secondary data are distinguished from primary data and primary sources that include original works or raw data without interpretation. The research design for this study was a sampling and systematic retrospective review of published case reports of collaboration between primary care and public health entities in the United States. Jacobsen (2017) posits that systematic reviews use a predetermined searching and screening process to identify relevant articles. The study employed content analysis to identify and summarize commonly self-reported determinants of success. Cooper and Schindler (2014) define content analysis as a systematic process for coding and drawing inferences from units of data that guards against selective perception of content. Therefore, it was selected for its documented breadth, flexibility, and rigor as well as the overall richness characteristic of qualitative analyses.

As stated previously, due to the number and diverse nature of cases in the repository, ASTHO Integration Forum reports were used as a primary source in data collection. The first case reported to the Integration Forum in each state, beginning on the west coast and moving eastward, was used for the study, provided content was available that specifically addressed the research question. Since most cases were reported on in 2013, I decided to consult other sources for updated cases. Four databases (PubMed, MEDLINE, CINAHL, and ABI/INFORM) were searched from 2014 to 2017 employing a general search strategy and key words. Additional peer-reviewed literature was gained using the related articles function. The Practical Playbook (Michener, Koo, Castrucci, & Sprague, 2016; Practical Playbook, 2017) was also consulted for updated cases. Databases, key words, and general search strategy were developed and modified with the assistance of a health professions librarian.

Statements identified as integral to the primary research question were extracted from cases and designated units of meaning, a term derived from previous empirical studies, such as that conducted by Graneheim and Lundman (2004). A codebook was developed based on previous literature. Individual units of meaning were coded in three distinct phases beginning with the systemic, organizational, or interactional designation. The second phase of coding included categorization of units of meaning into subgroups that comprise the first phase systemic, organizational, and interactional categories. The third and final phase of coding grouped the units of meaning into the various categories that exemplify codes of the second phase. If units of meaning were not detailed enough to be described via three phases of coding, coding concluded at the appropriate level. Units of meaning that did not fit explicit codebook definitions were noted, reported, and used to inform study findings and discussion.

To increase the reliability and validity of the study, I recruited a second coder to assist with the coding process; she was selected for her Masters-level education in a non-healthcare-related field of study. After I trained the second coder on codebook development and function, she and I independently reviewed and coded a small number of extracted statements and collaborated to establish consensus and coder reliability. Inter-rater coder reliability was recorded and reported. Initial reliability between two independent coders for all coding levels was 84%. After reconciling coding discrepancies through discussion, the two coders reached complete consensus. I then proceeded to code the remainder of the 57 cases.

Once the initial 57 cases were coded, I coded an additional 20 case study reports to make sure no new information would be added to my research. In qualitative analyses, it is critical that sampling resume if the breadth and depth of knowledge of the topic continues to expand. Data saturation is said to occur when such expansion ceases. Once I was confident data saturation was achieved, code frequencies were tabulated and reported for included cases. Below is a short excerpt of the codebook for visual reference. As an example, every time “regular meetings” was extracted from a case, it was assigned the code 3.3.4 and the final frequency (12 cases) was reported.

• 3.
  Interactional Determinants

  • 3.1.
    Shared purpose, philosophy, and beliefs: defining terms:

    • 3.1.1.
      Early successes
    • 3.1.2.
      Enthusiasm
    • 3.1.3.
      Similar philosophies of care among partners
    • 3.1.4.
      Belief in the value of collaboration as related to community health
    • 3.1.5.
      Shared mission, vision, goals
  • 3.2.
    Clear role and positive relationships: defining terms:

    • 3.2.1.
      Quality professional relationships
    • 3.2.2.
      Clear roles and responsibilities
    • 3.2.3.
      Clear knowledge of one another’s roles
    • 3.2.4.
      Understanding of and capacity for interdisciplinary teamwork
    • 3.2.5.
      Previous positive relationships
    • 3.2.6.
      Developing new linkages
    • 3.2.7.
      Partnerships
    • 3.2.8.
      Collaboration
  • 3.3
    Effective communication and decision-making strategies: defining terms:

    • 3.3.1.
      Direct and open communications
    • 3.3.2.
      Direct decision making
    • 3.3.3.
      Brief unscheduled visits to overcome barriers of time and scheduling
    • 3.3.4.
      Regular meetings

Two additional research terms were considered in this study to ensure codebook reliability and validity: exclusivity and exhaustivity. Exclusivity described observations, sampling elements, or units of meaning that could be classified into only one classification category (Shi, 2008). Exhaustivity was defined as the state where observations, sampling units, or units of meaning could be classified via the designated categories (Shi, 2008). In other words, I had to ensure that the codebook was inclusive enough to accommodate each unit of meaning while simultaneously eliminating duplication or overlapping codebook categories.

Definitions are integral in research as they facilitate transparency and potential for research duplication. Operational definitions are definitions developed to provide understanding of abstract constructs while simultaneously providing a concept of measurement (Cooper & Schindler, 2014). To examine the primary research question in this study, I first had to construct, at minimum, operational definitions of primary care, public health, collaboration, integration, units of meaning, and systemic, organizational, and interactional success determinants. These were mostly derived from previous literature and, in terms of success determinants, what secondary data was revealed in case studies. Intercoder reliability was also operationalized in this study. Intercoder reliability “requires that two or more equally qualified coders operating in isolation from each other select the same code for the same unit of text” (Campbell, Quincy, Osserman, & Pedersen, 2013, p. 297).

Research Practicalities

Most of the case reports utilized in this study were publicly available and easy to access via the Internet. Planning involved allotting time to consult a research librarian and developing a general research strategy to identify relevant research articles. Despite readily accessible data, this project presented several obstacles that were ultimately deemed study limitations. Research limitations are inherent and should always be conceptualized as they affect the generalizability of the work.

Many of the ASTHO cases were authored in 2013. When I began this project, my intent was to contact documented case authors for any information that was missing or unclear as related to my research questions. Perhaps due to the time lapse, I found much of the contact information to be outdated. Virtually no case authors responded to my inquiries. Therefore, due to time constraints, I decided to focus exclusively on the secondary data that were readily available. Elements that were missing from a case study subsequently became limiting factors. For example, not all cases included in this study detailed type or degree of integration. Therefore, I recognized limitations inherent in the work such as the inability to distinguish large from small integrations or urban from rural integrations. Other study limitations included recognition that extracted units of meaning were derived from the author’s interpretation of case reports and were, therefore, subject to misinterpretation. In addition, self-reported cases are characterized by inherent confounds. Although self-reports can provide essential research information, they have also been criticized as a weak evaluative strategy (Hewson, Copeland, & Fishleder, 2001) as they can reflect bias of professionals’ perceptions (Willumsen, Ahgren, & Odegard, 2012). Finally, while data categorization increases analytic efficiency, it also sacrifices a certain level of data detail.

Method in Action

Overall, I think the qualitative research process progressed quite smoothly and proved very effective. Results are displayed in Table 1. At the systemic level, results revealed adequate funding as an essential component of collaboration initiatives. Organizationally, multiple stakeholder engagement, leadership, data and information sharing, pooling resources, community engagement, and steering committees emerged as common themes. At the interactional level, effective communication; regular meetings; shared mission, vision, and goals; previous positive relationships; and collaborations and partnerships were typified (McVicar, 2018). These results largely reinforced previous conceptualizations that were introduced in the literature review and discussed in conclusions. Whenever possible, I tried to provide detail to the frequencies reported. For example, I thought readers would find it useful to know in how many cases federal, state, and miscellaneous funding were coded concurrently.

In addition to reinforcing concepts introduced in the literature review, this study also revealed areas for future research and suggestions for researchers attempting to conduct similar research. For example, sample case studies utilized multiple reporting formats. While this lack of standardization complicated the data extraction process, I was able to recommend optimal reporting format in the discussion and conclusion sections of the case.

Practical Lessons Learned

I learned many important lessons from this research project, some basic and some method-specific. Ultimately, the value of planning was reinforced. Program faculty encouraged me to select a topic early and continue to develop it. I evaluated most major research assignments in my doctoral program for future dissertation potential.

Furthermore, it is important to accept that each step in the research process is replete with inherent delays. For example, when other contributors (co-authors, editors, etc.) are reviewing your work, it is important to remain productive. The primary researcher has many logistical responsibilities such as paperwork, communications, and so on that can be accomplished while other contributors are reviewing content. If handled correctly, I believe these gaps can be truly synergistic as the overall project can be advanced in multiple ways simultaneously.

As with any research project, it is very important to begin with a thorough review of existing literature. This applies to method and content. In terms of qualitative analysis specifically, I think my biggest takeaways included the importance of operational definitions of key terms and recognition of study limitations. These parameters create transparency and help readers understand what the project is and is not. For example, I found the semantic barriers related to previous research on this topic difficult to wade through. Previous authors used varying combinations of the terms integration, collaboration, partnership, as well as determinants, factors, elements, and so on. Not to recognize such inconsistencies in my work would have been a mistake. I found it useful to compose the limitations section of my paper throughout the process because potential limitations are identified at various stages. I suspect I would have forgotten something important if I had not kept a running list.

When composing discussion and conclusion sections, I found it helpful to continually reflect on who I thought would most benefit from reading my research. I felt that doing so enhanced my discussion and conclusion sections. For example, since my work revealed important determinants other than funding, I was able to suggest that funders of future collaborative efforts demand emphasis on those elements. My initial conceptualizations were aimed solely at primary care and public health entities aspiring to form successful collaborations. Similarly, if prospective authors wish to publish their manuscripts in an academic journal, I suggest having the journal and its requirements in mind as early as possible. This information can be helpful when working with an editor and it can dictate number of acceptable tables, figures, and so on.

This entire research process spanned approximately 1 year. At the beginning, I worked exclusively on weekends and this progressed to working on specific details daily for the final few months. I spent around 3 months preparing the literature review and introduction, finding articles, and constructing the codebook. Approximately 3 months were allocated to reading cases and extracting content. Coding took another 1–2 months and the rest of the time was spent writing, revising, and responding to dissertation committee members and an editor. I found it very easy to lose momentum if I disengaged for any length of time. Maintaining momentum is essential. I believe total completion time would have been extended had I not consulted a copy editor to make sure everything adhered to both American Psychological Association (APA) and university guidelines. I hired a faculty-recommended editor early in the process to review my completed prospectus (first three chapters of the dissertation) and again at the conclusion of the process to revise the completed manuscript. There are different types of editors; therefore, I recommend establishing exactly what the expectations are from a prospective editor up front. I was lucky in this area as the person I selected was very explicit about what her services would and would not include in our initial communication.

The last practical lesson I reflect upon is the involvement of key players on the dissertation committee. I was lucky enough to secure an expert in qualitative analysis from Central Michigan University, a public health expert from the Centers of Disease Control and Prevention, and a primary care expert from Duke University. Keeping four (including myself) researchers informed and cohesive requires diligence and adequate two-way communication. Each step of the communication process presents inherent delays that must be planned for in advance; otherwise, the timeline will suffer an overwhelming level of uncertainty. To illustrate: in one of our first telephone meetings, my committee decided a 2-week response time for all correspondences was appropriate enough to accommodate our busy schedules while also keeping the project moving at an acceptable pace. Two weeks worked well for us. Accordingly, I recommend insisting that committee members alert you right away if they anticipate deadline delays. This way, you can begin to formulate alternate plans in advance. I also recommend the primary researcher (student) assume the lead on all communications, meeting times, formats, and so on. I made every effort to make the process as easy as possible for committee members. I believe they appreciated this. If a new researcher is unsure where to begin to find suitable committee members, I suggest contacting authors of previous literature on the same topic.

Conclusion

In terms of study design, this research project demonstrated use of the methodological and contextual concepts of qualitative design, retrospective review, content analysis, data extraction, codebook development, coding extracted units of meaning, and operationalization. As with all research, it is important to construct operational definitions of key terms and recognize study limitations. Doing so creates transparency and helps readers understand the explicit project parameters. Qualitative analyses are emphasized for their richness and these concepts were employed in this research to inform the systemic, organizational, and interactional success determinants reported by primary care and public health collaborations in the United States. I’m pleased with the results and generalizability of this research project and I think the applications related to method, research practicalities, and practical lessons learned could prove useful to future researchers.

Exercises and Discussion Questions

• 1.
  How does a qualitative researcher know when data saturation has occurred?
• 2.
  Discuss limitations inherent among the content analysis research technique. Research a content analysis software package and suggest how an automated process may have impacted this work.
• 3.
  What is the importance of operationalizing in the research writing process? How might this study have been impacted had key terms not been operationalized?