Skip to main content
Search form
  • 00:11

    JOHN HITCHCOCK: Hi.I'm John Hitchcock, and I'm the directorfor the Center for Evaluation and Education Policy.And I'm also an associate professorof instructional systems technology, both of whichare housed in the School of Educationat Indiana University.And I'm here to give you a very brief overview of mixed methodsdesigns.

  • 00:31

    JOHN HITCHCOCK [continued]: So a couple of language caveats at the outset-- sometimesI refer to mixed methods research.Some of my colleagues in the fieldsalso like the phrase "mixed research."And I use both.And another big caveat is that mixed methods research can getinto some pretty complex stuff.And I'm only here to give you some of the most introductory

  • 00:52

    JOHN HITCHCOCK [continued]: of ideas, and I hope that this will inspireyou to learn a little bit more about this way of engagingin research.Big picture-- I'll cover the following points.I'll get into discussing the so-called paradigm wars.Then I'll get into talking through letting your researchpurpose and questions drive your design choice then

  • 01:14

    JOHN HITCHCOCK [continued]: how mixed methods can often be usedto conceptualize and address more complex questionsthan you could typically pursue if you used only onetype of research approach, such as a qualitative approachonly or a quantitative approach only.And then finally I'm going to introduce two very basic designideas that you might think about when you're planning

  • 01:36

    JOHN HITCHCOCK [continued]: your mixed methods research.And at the outset I also like to say why this is important.In essence mixed methods can createa kind of synergy between your qualitative inquiryand your quantitative inquiry suchthat you can have the strengths of one approach offset

  • 01:57

    JOHN HITCHCOCK [continued]: the weaknesses of the other and vice versa.Using qualitative work allows you to engagein a lot of exploratory work.Quantitative design allows for a lot of statistical analyses.And you can often triangulate the kinds of findingsthat you have across your qualitativeand quantitative inquiry.

  • 02:18

    JOHN HITCHCOCK [continued]: By triangulation you might be looking for whether or notyou're getting the same story from the different approaches,and a lot of times you find that you're not.And that can yield some interesting new insightsthat you may not have been able to achieve if you used just oneapproach on its own.To give you an example of what I mean,think about standardized testing.In the United States, you will engage in standardized testing

  • 02:41

    JOHN HITCHCOCK [continued]: all the time.In grades K through 12 standardized testingis used to help determine whether or notyou can attend college and which ones.Standardized testing is used in many other professions,and it's very much a part of our culture.But think globally.There are countries that are out there right now that are juststarting to engage in standardized testingin its various forms.

  • 03:02

    JOHN HITCHCOCK [continued]: And imagine a group taking the timeto determine what the citizens of that countrythink about adopting a standardized testing approach.Well, one of the things I would probablydo in a situation like that would be to develop a survey.And I might give that survey to a groupand ask them what they think about the testing procedures.

  • 03:26

    JOHN HITCHCOCK [continued]: What do they like about it?What worries them?And now consider for a moment that even if I'm a good surveywriter, it may not occur to me to askall the pertinent questions that are at hand, particularlybecause I'm working in a different culture.I may spend a lot of time focusingon things like the benefits of the testing, howdifferent subgroups are performing on the test.

  • 03:49

    JOHN HITCHCOCK [continued]: But maybe I missed something really important.Perhaps standard test security practicesprevent people from getting a sense of trustin the testing process.And I failed to think through that.And that creates a kind of coverage problem with surveyitems that I would like to be able to avoid, particularlybecause I'm working in a new environment.

  • 04:10

    JOHN HITCHCOCK [continued]: And so from a mixed methods point of view,one of the things that you could dois start out by engaging in a lot of exploratory qualitativework.You can do interviews with stakeholdersand ask them lots of exploratory questions.Try to identify and uncover the different thingsthat you should be asking about later on in a survey.You can engage in observations of the testing approaches.

  • 04:32

    JOHN HITCHCOCK [continued]: You can look at documents such as newspaper accountsof the testing approach that's being done,maybe test materials that developers put together.And engaging in all of this form of qualitative inquirycan often yield an awful lot of insightsinto what kinds of items you wantto write about in your survey and how they should be written.

  • 04:52

    JOHN HITCHCOCK [continued]: Of course, one of the drawbacks to qualitative inquiryis it's really hard to generalizethe findings from even a lot of interviewsto a wide group of people.There's a much more efficient design typically for that,and that would be the survey approach.And so think about combining your qualitative work--your interviews and your focus groups

  • 05:13

    JOHN HITCHCOCK [continued]: and your observations and document analyses--and combining that with your standard survey work.And it puts you in a position whereyou can, in essence, use both.Indeed why not use the best of both worlds?In fact once you get to the survey stage,you could start to engage in things like random samplingtechniques and whatnot, so that you can have some confidence

  • 05:34

    JOHN HITCHCOCK [continued]: that whatever you observe amongst the participants whorespond to your survey is likely to generalize backto some larger population.Meanwhile because of your prior qualitative work,you can have some confidence that you're actuallyasking the right questions and in the right way.

  • 05:55

    JOHN HITCHCOCK [continued]: So with that background, there are some proponentsagainst mixed methods.I think that a lot of this is borne outof the so-called paradigm wars.And the paradigm wars in essence are a kindof philosophical argument.It deals with how we know what we now.And at the root of it, it's basically the idea

  • 06:15

    JOHN HITCHCOCK [continued]: that one class of research methods or methodologies--such as qualitative work-- is somehowor another superior to another class,such as quantitative work.And there's even a thesis that a clear philosophicalunderstanding of how one can view the world, in essence,makes these approaches incompatible.If you engage in qualitative inquiry,

  • 06:37

    JOHN HITCHCOCK [continued]: you can't use quantitative work or vice versa.And clearly as a mixed methods researcher,I don't agree with that.And part of the reason why is I liketo think of the hammer analogy.And what I mean by that is if you were talkingwith a carpenter, they'll see any one of their toolsexactly as that.And I guess the basics there is that you

  • 06:58

    JOHN HITCHCOCK [continued]: would use the hammer for its intended purpose.You wouldn't try to cut wood with it.You wouldn't try to screw a nail into a board with a hammer.You would use a hammer when you had to hammera nail into something.I tend to think of research methods as different toolsthat you want to use them for their intended purpose.So if you want to explore a topic with a few people,and you really want to get their insights

  • 07:18

    JOHN HITCHCOCK [continued]: into something and in an open-ended manner,that's great.But then use an interview.If you think you need to determine whether or nota new reading approach is better than some standardizedinstructional package, then maybe usea randomized controlled trial.If you think it's important to learn

  • 07:38

    JOHN HITCHCOCK [continued]: how a culture thinks and behaves in given ways,then use ethnography.From a mixed methods perspective,my point is that use the tools that are available.And then try to stitch together the various forms of evidenceso that you can generate a big picture of what's happening.Final point about paradigm wars.

  • 07:59

    JOHN HITCHCOCK [continued]: You see that in the literature.I think the word wars is overwrought.Some of my colleagues have written about this.In fact one in particular has actually been in a war zone.And I think after that experience,he's always going to use the word paradigm arguments.We're not in a war.Instead, they're different ideas that are out there,and people who buy into the paradigm argument

  • 08:20

    JOHN HITCHCOCK [continued]: have legitimate points.But overall from a mixed methods perspective,I encourage anybody who's watching this videoto be open to the different kinds of methodologies thatare at their disposal so they can answer the questionsthat they want to pursue.Another criticism against mixed methodsis the notion that there's not enough ofa philosophical backing to it.And so in the qualitative world, you

  • 08:41

    JOHN HITCHCOCK [continued]: have constructivism, for example,as a kind of a philosophical orientation.I'm not going to go into that because of time.There's something called postpositivismthat aligns nicely with a lot of quantitative research.But in the mixed methods world, we have pragmatism.And we also have something called dialectical pluralism,which as the name implies is meant

  • 09:02

    JOHN HITCHCOCK [continued]: to promote this idea of a constructive dialoguebetween different ways of looking at evidenceand different ways of looking at the worldand seeing if you can promote unityand seeing if these different viewpoints can helpunderstanding a phenomena better thanif you just embrace one way of looking at something.

  • 09:25

    JOHN HITCHCOCK [continued]: A key issue when considering mixed methodsis to always follow this idea of letting your purposedrive your design choice.And in fact the alignment between your research questionsand the purpose of your questionsand the overall design decisions that you makeis a real marker of good research quality.And as an example-- I'm a professor.

  • 09:47

    JOHN HITCHCOCK [continued]: And a lot of times I'll have studentswho approach me and say somethinglike, I want to use qualitative research methodsfor my dissertation.Or I want to do mixed methods research.Or I want to do a survey.And I could tell you that always makes me a tad bit nervous,and the reason why is because I wantto know your question first.Let me know your question, then we'll talk through the design.Otherwise you're sort of putting the cart before the horse,

  • 10:10

    JOHN HITCHCOCK [continued]: if you will.With that background, trouble canoccur when there's a disconnect between what you want to knowand what your design choice is.And I can also tell you that you want to watch outfor question drift.And what I mean by that is your initial questions that youwanted to pursue end up morphing into something else by the timeyour study is over.

  • 10:30

    JOHN HITCHCOCK [continued]: And that can often lead to problems.Always keep your question in mind,particularly when you're engaging in mixed methodsresearch.That's something that scholars in this field pay attention to.Because when you're using multiple methods,you're looking at things a lot of different ways.That can become more of a danger.

  • 10:52

    JOHN HITCHCOCK [continued]: So another point to raise about mixed methodsis that using these kinds of approachescan help you conceptualize more complex questionsand answer them.And a point that I want to raise first in this section isdo not confuse complexity and rigor.Those are two different things.

  • 11:13

    JOHN HITCHCOCK [continued]: In fact I've had the pleasure and the privilegeof being a co-principal investigatorof the few randomized controlled trials.And I like to think that they've metsome high standards for research,and they were big studies.But at their heart they asked really simple questions.Does using one particular reading technique yieldbetter outcomes than another?

  • 11:35

    JOHN HITCHCOCK [continued]: And a lot went into answering that question.The question itself was pretty simple and straightforward.With that in mind, though, if youstart to get into mixed methods thatcan allow for conceptualizing and answeringmore complicated questions.For example, we might get into the issueof can we develop a new intervention or a new treatment

  • 11:55

    JOHN HITCHCOCK [continued]: that you work in a culture that we haven't worked in before?If you get into the issue of how can Icreate a new treatment package?And how can I assess it in a brand new arena?A lot of times you're going to need ethnographic techniquesor, if you will, qualitative techniquesto help guide your thinking.And of course, if you're engagingin a randomized control trial, that

  • 12:16

    JOHN HITCHCOCK [continued]: tends to be aligned with more experimental designand a quantitative perspective.You start to get into questions like, how can we best assesswhether or not that new approach or treatment or interventionis working?And a lot of times you find that youhave to create new assessment tools thatcan work not only with people who've been exposed

  • 12:38

    JOHN HITCHCOCK [continued]: to that treatment package but also folks who could havereasonably not have been exposed to itto see if there's a meaningful differencethat that treatment has made.Again, in order to identify assessments-- or maybe evencreate assessments-- a lot of timesyou have to rely on qualitative workso that you can understand the context in which you'reoperating before you can get into the quantitative designelements.

  • 12:60

    JOHN HITCHCOCK [continued]: Indeed, you could even get into exploratory workso you can try to understand all the different stakeholders whoare involved in a particular intervention.My point to that is that when youstart to think about work from both a qualitative lensand also a quantitative lens, youcan start to identify a lot of complexities thatare hard to even identify if you just

  • 13:21

    JOHN HITCHCOCK [continued]: stick with one particular approach to doing research.But with that, remember your first principle.Just because you could mix methodsdoesn't mean that you should.Again, remember what is your question?And if you have a question that warrantsthe use of mixed methods, then go ahead and pursue them.

  • 13:42

    JOHN HITCHCOCK [continued]: I want to leave you with a couple of basic designconsiderations if you're going to pursue mixed methods work.This is just to give you a feel for how these things canstart to operate.So consider balance.Is your work more qualitative in orientation,or do you think it's more quantitative in orientation?Sometimes you can have an equal balance between the two.Another issue that you might consider

  • 14:02

    JOHN HITCHCOCK [continued]: is sequencing, for example.Many times studies call for doing qualitative inquiry firstso that you can explore a phenomenaand get familiar with the contextbefore you follow up with quantitative work.Other kinds of studies can be done where quantitative work isdone initially, and then you mightuse some qualitative techniques to better understand

  • 14:22

    JOHN HITCHCOCK [continued]: what happened.Even you can engage in iterative techniques.One of my colleagues, Dr. Shannon David,developed a measure of trust at one pointwhere as it turned out, that was a pretty complex measure shehad to create.And what she ended up doing was starting her workby doing some qualitative interviewing.Then she used that to develop a series of items

  • 14:44

    JOHN HITCHCOCK [continued]: to create an instrument.Then she used a number of psychometric techniquesto ascertain whether or not the measure was assessingwhat she thought it was.It turned out that wasn't the case.So she started to do some more qualitative workto figure out why.She got a new sample to do more quantitative workand bounced back and forth between these,so that over time she developed what I think

  • 15:05

    JOHN HITCHCOCK [continued]: is a really solid instrument.The point there is that when it comes to mixed methods,sequencing is a complex issue.And you want to be able to try to think about thatto the degree that you can in advance so that you can pursuethe answers to your questions.So in conclusion, I hope you learned a little bit

  • 15:26

    JOHN HITCHCOCK [continued]: about the paradigm wars and learned a little bit about someof the philosophical orientation to mixed methods research.I hope you will walk away with this sensethat your purpose really should drive your design choice.And I also hope you have an initial sense of howmixed methods can help you conceptualize

  • 15:46

    JOHN HITCHCOCK [continued]: some pretty complex questions and be able to pursue them.Again, I can't cover all you needto know on mixed methods in the space of a few minutes.So I hope that you'll be able to read about these and move on.And in the end, I hope you're inspiredto learn about this great set of design choices.

Video Info

Publisher: SAGE Publications Ltd

Publication Year: 2017

Video Type:Tutorial

Methods: Mixed methods, Paradigms

Keywords: practices, strategies, and tools

Segment Info

Segment Num.: 1

Persons Discussed:

Events Discussed:



Professor John Hitchcock discusses mixed methods and the best situations for this approach. Mixed methods combine qualitative and quantitative methods to use the strengths of both. Hitchcock discusses the paradigm wars, how to know when to use mixed methods, and research design questions.

Looks like you do not have access to this content.

An Introduction to Mixed Methods Research

Professor John Hitchcock discusses mixed methods and the best situations for this approach. Mixed methods combine qualitative and quantitative methods to use the strengths of both. Hitchcock discusses the paradigm wars, how to know when to use mixed methods, and research design questions.

Copy and paste the following HTML into your website