Skip to main content
SAGE
Search form
Auto-Scroll PDF
  • 00:05

    [Increasing Validity in Qualitative Research]

  • 00:11

    DENISE POPE: Hi.I'm Dr. Denise Pope, and I'm a senior lecturer at the GraduateSchool of Education at Stanford University. [Dr. Denise ClarkPope, PhD.Senior Lecturer, Graduate School of Education, StanfordUniversity] And today we're goingto talk about a tutorial to increase validityin qualitative research.Let's get started.So, lots of strategies to increase validity.In this tutorial, students will understand the controversy

  • 00:34

    DENISE POPE [continued]: in the qualitative research field concerning validity.They're going to learn specific methods to increase validityin qualitative research, including triangulation,negative case sampling, grounded inferences,do a subjectivity audit, and how to recognize patterns and doa fair representation of the data that you're finding.

  • 00:55

    DENISE POPE [continued]: [Validity in Qualitative Research]

  • 00:60

    DENISE POPE: Validity in qualitative researchhas been a hotly debated topic.We can't necessarily employ the same exact techniquesthat are used by quantitative researchers.So this tutorial is going to explain a little bitabout the tensions that exist in the field.It's going to offer several methodsqualitative researchers can use to increase validityin their work.

  • 01:20

    DENISE POPE [continued]: So, what are the issues?What's going on in terms of validityin qualitative research?So, traditional criteria of validity and reliabilityfrom quantitative research can be problematic.And you're going to see in a second here why that is.Validity traditionally means that youcan show that you're measuring what you intended to measure,

  • 01:42

    DENISE POPE [continued]: and you've obtained accurate data.And reliability means the extent that the studycan be replicated.When you're doing qualitative research,you can't really replicate exactly whathappened in that classroom when you were theredoing your observation or interviewing people.It's very different from giving a survey.It's very different from quantitative methods

  • 02:05

    DENISE POPE [continued]: of counting.We are looking at qualities.And that can't necessarily be replicated.It also causes some problems whenyou look at this traditional notion of validity-- that,can we really show we're measuring what we intendedto measure-- sometimes we find completely different thingsfrom what we were intending.So you can see why the traditional criteriaof validity and reliability from quantitative research

  • 02:25

    DENISE POPE [continued]: can be problematic.And because of this, there are some qualitative researcherswho just say, you know, these are not relevant,and we don't have to prove it.But that still leaves the problemof how do we know that what we're doingis valid and rigorous.So there's lots of other criteriathat qualitative researchers use.They are trying to get rigorous data, rigorous methods

  • 02:46

    DENISE POPE [continued]: to get very good data, and very strong analysis.They're after, really, how well can I help someoneunderstand the phenomenon?How trustworthy do I come across as a researcher?How trustworthy is the data?How plausible or believable is it?And how credible is it?Of course, the problem with this is sometimes,

  • 03:07

    DENISE POPE [continued]: things can be very credible.And as Wolcott says, there's nothing ascredible as a swindler's tale.So you have to be careful that you're notfalling into that trap.But those are some of the other criteria usedby qualitative researchers.[Triangulation]

  • 03:23

    DENISE POPE: So one of the main things that we look atis triangulation.And triangulation is really how to cross verify.How you see that you've collected the data this wayfrom this source, and this way from this source,and if the two match that's a good signthat it's probably legitimate data.So one form of triangulation is multiple data sources.If you heard from three different people

  • 03:45

    DENISE POPE [continued]: a similar story, or a similar kind of phenomenon, that'sa good kind of note to yourself, gosh, that sounds about right.I think I've got it right.You can also use multiple data methods.So you might do interviews.You might check that what you've heard in the interviewsyou are actually seeing in real life, in your observations.And you might check what you're seeing in your observations is

  • 04:07

    DENISE POPE [continued]: actually coming through on the documentsthat you're looking at.Someone's websites, or the emails that they're sending.That's called multiple data method triangulation.And then the other thing, and thisis kind of rare in the field, but it'sstarting to get more and more common,is to have multiple investigators.And I do this with my own studentsin my qualitative research class.We sent two or three people out to the site.

  • 04:28

    DENISE POPE [continued]: And you're all taking field notes.And then you're looking to see, where did these fieldnotes triangulate.That might be a better way to figure outwhat really happened.[Negative Case Sampling]

  • 04:41

    DENISE POPE: Another method is negative case sampling.And negative case sampling is you're basicallyasking yourself throughout qualitative research, whatis the story this data is telling me?Not what is the story I want to tell.Right, that's not fair.That's not getting at trustworthinessand credibility.But what is the story that the data is telling me?What's the story here?

  • 05:02

    DENISE POPE [continued]: And then you have to ask yourself,OK, let's take a step back.Are there are other stories that can be told from this data?Someone who is trying to prove a completely different point,maybe an opposite point, would theybe able to look at some of the evidenceand tell a different story?Make sure that you're looking at the negative cases.Right, not just the stuff that ishelping to confirm what you want to say.

  • 05:24

    DENISE POPE [continued]: And in the field we basically say,make sure you aren't drawing the bullseye around the target.That's cheating.That's not fair.Basically going and saying, oh, I found what I was looking for.That's not being open.That's not really showing that you'rea trustworthy and credible researcher.[Grounded Inferences]

  • 05:44

    DENISE POPE: Grounded inferences is somethingthat you'll hear a lot in the field of qualitative research.What we're basically saying is, insteadof starting with a hypothesis and looking for evidenceto prove or disprove that hypothesis,in qualitative research people whoare doing grounded methodology are looking from the bottom up.Inductive reasoning.What do I see here?

  • 06:05

    DENISE POPE [continued]: What are some of the themes?What are some of the ways I could code that data,and then I form a proposition.So we talk about low level descriptions.You're going to be using verbatim quotesfrom your interviews to help prove your points.You're going to be using actual notes from your observation.That's your evidence.That's your data.That's what you're going to be codingand using to form your propositions.

  • 06:27

    DENISE POPE [continued]: And that's going to give you the most valid or credibleconclusions.[Subjectivity Audit]

  • 06:36

    DENISE POPE: We also ask that you do a subjectivity audit.And let me explain what subjectivity is.Subjectivity, also known as reflexivity,is really the biases and personal lensesthat you bring as you're looking at an observation,as you're interviewing.You can't help this, you're human.Right, so there's going to be things thatare going to get you excited.There's going to be things that are

  • 06:56

    DENISE POPE [continued]: going to get you very upset.Someone is going to say something that you completelydisagree with in an interview.And instead of saying, I completely disagree with you,you're going to listen and then you'regoing to calmly be taking notes, and tryto steer the conversation to learn more about that person'spoint of view.So as a way of making sure that the data is credible,that the data is valid, and that you're trustworthy

  • 07:19

    DENISE POPE [continued]: as an instrument of qualitative research,you're going to be constantly auditingand monitoring your biases to keep them in check.Maybe you are reading a story about a kid in a certain way,because something about that kid is driving you crazy,or reminding you of your own teenager at home.Right, so the idea here is to be-- not as

  • 07:42

    DENISE POPE [continued]: neutral as possible, you're not a robot--but you're really focusing on those hot and cold spotsto keep your biases in check as much as possible.To let the data really speak for itself,to look for the actual story that the data is telling.Not the story that you want to tell.[Pattern Recognition and Fair Representation]

  • 08:03

    DENISE POPE: The same is true for pattern recognitionand fair representation.When you're coming up with a claim in qualitative research,you're basically saying, I saw a pattern of thisso I feel comfortable making this claim.But what constitutes a pattern how many times youhave to see something to say, thisis happening in this classroom, or thisis happening in the field.

  • 08:23

    DENISE POPE [continued]: It can actually be a one-time occurrence,but it's so salient that you havelots of people talking about it, that you havelots of conversations about it.So maybe it only happened once, but it constitutesas something significant.That's what we say is sort of an N of 1.And N of 1 is maybe it's a case of one school, or one person,doing something that is going to resonate with others.

  • 08:45

    DENISE POPE [continued]: That's OK.But in many times you might see lots of it incidences.And that's where you could call it a pattern.And you feel like you are really strivingfor a fair representation and honest reporting of the data,and the outliers.Just because something is an outlierdoesn't mean you can ignore it.In qualitative research, we usually embrace the outliers

  • 09:05

    DENISE POPE [continued]: and say, wait a minute, that means something else mightbe going on here.Let's figure this out.What's another story to tell?And when you don't know, or you have questions, or lotsof limitations in how you're collecting the data,you want to tell that.You want to tell that outright when you're writing up

  • 09:25

    DENISE POPE [continued]: your piece so that people know.Hey, I might be biased because I'm a white female professor.Or I might be biased because we could onlysee the kids on certain days of the week,and we don't know what was happeningon the other days of the week.Or we have some questions because some of thingsweren't resolved.That's how you're trying to show that you're really

  • 09:46

    DENISE POPE [continued]: doing the best job you can do to be credible and trustworthy.[Conclusion]

  • 09:54

    DENISE POPE: So although qualitative researchers can'treally rely on traditional quantitative methodsto assess validity, we can't assurethat it's going to be that test of reliability,we can't assure that they're goingto find the same exact things if someoneelse goes into that classroom and asks similar questions.We can use all the strategies discussed in this tutorial

  • 10:14

    DENISE POPE [continued]: to help achieve rigor and high quality research.The ultimate is you want people to believe from the datathat your story is credible.That it's trustworthy, that it's oneway of looking at something that you hopewill resonate with others.Lots of people writing on this.Lots of debates in terms of what'sthe right way to examine validity

  • 10:36

    DENISE POPE [continued]: in qualitative research.You can look at the Johnson article, whichis in the Education volume 118.Harry Wolcott in Eisner and Peshkin'sbook Qualitative Inquiry in Education talksabout a lot of ways he tries to buildin strategies for validity.And then, of course, Buddy Peshkin,The Color of Strangers, the Color of Friends,writes all about subjectivity and really monitoring

  • 10:57

    DENISE POPE [continued]: that subjectivity to make sure what you're saying is fairand that you're keeping your own biases in check.Good luck as you strive for validityin qualitative research.Make sure to do what's right and true for you.And do the best job you can to have a rigorous pieceof qualitative research.

Video Info

Publisher: SAGE Publications Ltd

Publication Year: 2017

Video Type:Tutorial

Methods: Validity, Triangulation, Subjectivity, Negative cases

Keywords: honesty; logic; practices, strategies, and tools; repetition; Self-awareness; trust ... Show More

Segment Info

Segment Num.: 1

Persons Discussed:

Events Discussed:

Keywords:

Abstract

Professor Denise Clark Pope discusses research validity and best practices for data collection. She also outlines approaches for avoiding research bias.

Video Info

Publication Info

Publisher:
SAGE Publications Ltd
Publication Year:
2017
Product:
SAGE Research Methods Video
Publication Place:
London, United Kingdom
SAGE Original Production Type:
SAGE Tutorials
ISBN:
9781473991828
DOI
http://dx.doi.org/10.4135/9781473991828
Copyright Statement:
(c) SAGE Publications Ltd., 2017

People

Academic:
Denise Clark Pope

Segment Info

Title:

Segment Num: 1

Keywords:

Segment Start Time:

Segment End Time:

People

Things Discussed

Organizations Discussed:

Events Discussed:

Places Discussed:

Persons Discussed:

Methods Map

Validity

The extent to which something that has been studied can be said to have been accurately depicted.
Validity
Increasing Validity in Qualitative Research

Professor Denise Clark Pope discusses research validity and best practices for data collection. She also outlines approaches for avoiding research bias.