Skip to main content
Search form
  • 00:10

    ELIZABETH MANCUSO: My name is Liz Krumrei Mancusoand I'm associate professor of psychologyat Pepperdine University.I'm a clinical psychologist and Iteach in the areas of psychotherapy,advanced research methodology, and psychology and religion.The purpose of today's tutorial isto focus on how to get useful information outof survey research.We'll be looking specifically at research

  • 00:31

    ELIZABETH MANCUSO [continued]: that makes use of self-report measures, in which participantsrespond to questions either verballyor in writing on a survey from the researcher.In order to accomplish our goals,we're going to focus on four particular topics.First we're going to examine what the usefulness isof survey research.Second, we'll consider what types of information

  • 00:52

    ELIZABETH MANCUSO [continued]: can be gathered through surveys.Third, we'll examine how to design a survey.And fourth, we'll consider what types of individualsshould complete a survey.[Survey Research]Let's start by focusing on the purpose of survey research.When researchers are describing a particular research

  • 01:12

    ELIZABETH MANCUSO [continued]: methodology, they tend to make a distinctionbetween experimental research and non-experimental research.Experimental research is designedto assess cause and effect relationshipsby looking at whether changing one variable resultsin changing another variable.For example, seeing whether administering a drugwill result in symptom relief.

  • 01:34

    ELIZABETH MANCUSO [continued]: Non-experimental survey on the other hand,does not look at cause and effect relationshipsbetween variables, and survey research, togetherwith many other forms of research,is an example of a non-experimental formof research.It may seem odd to start by defining survey researchby what it is not, but this is an important pointbecause if survey research is to be effective,

  • 01:54

    ELIZABETH MANCUSO [continued]: it needs to be employed for the purpose for which itwas designed.So if not to look at cause and effect relationships,then what is the purpose of survey research?Well there are two common forms of researchthat tend to employ surveys, one is descriptive research,and one is research that examines non-causal linksbetween variables.Descriptive research, as it sounds,

  • 02:16

    ELIZABETH MANCUSO [continued]: is a type of research that describes phenomena or people.For example, we may be interested in knowingwhat percentage of college studentshave experienced sexual assaults,or we may want to know what proportion of the US populationspeaks more than one language fluently.Alternatively, a form of research that uses surveysis research that examines non causal relationships

  • 02:37

    ELIZABETH MANCUSO [continued]: between variables.This can be for example changing our question of what proportionof college students have experienced sexual assaultsto asking is there a difference in the proportion of studentswho experience sexual assaults if they liveon campus versus off campus.In this case, we're looking at whether thereis a link between nature of housing

  • 02:58

    ELIZABETH MANCUSO [continued]: and the experience of sexual assaults,but we're not examining whether one of these variablescauses the other.Or we may be interested in askingis the extent to which a person is religious relatedto how likely a person is to help another individual.And if we want to throw in a third variable,we might consider does this also relateto how much time the person has or whether the person is

  • 03:19

    ELIZABETH MANCUSO [continued]: in a hurry.Again, examining links between these variableswithout looking at whether one causes the other.[Information Gathering]Next let's look at what types of informationcan be gained through survey research.If you are asking people to complete self-report measures,it's important that first of all individuals

  • 03:40

    ELIZABETH MANCUSO [continued]: are aware of the type of information that you're asking.For example, I know that my favorite color is blue.But if you were to ask me on a surveyto explain why my favorite color is blue,I might not really be able to put that into words.I could explain how the color makes me feel,or the fact that I like it, but whyblue instead of another color?This is something that I wouldn't reallybe able to explain, and so not a topic

  • 04:02

    ELIZABETH MANCUSO [continued]: you would want to assess with survey research.Similarly, we know from previous researchthat individuals are not very good at predictingtheir behavior.For example, when you ask individualshow likely it is that they will conform to a group,most individuals will say that they will notconform to a group.However, we know from a number of experiments or studies

  • 04:23

    ELIZABETH MANCUSO [continued]: that a larger percentage of individualsactually do conform to a group when they are in a group thatis behaving in a uniform way.So for example, the ash experimentsasked participants to match up the length of two lines thatcould be done correctly 100% of the time by participants.But when the research participantswere put in a group of Confederates who

  • 04:43

    ELIZABETH MANCUSO [continued]: were planted there by the researcherand told to give incorrect answers,a substantial proportion of the research participantsactually started giving incorrect answers in orderto conform to the group answers.So survey research may ask individualsto predict how likely it is that they are to conform to a group,but if individuals aren't very good at predictingfuture behavior, this is the type of topic

  • 05:05

    ELIZABETH MANCUSO [continued]: that you would not want to assess through a survey.In addition, to knowing the type of informationthat you're assessing through a survey,it's important for survey respondentsto be able to remember this information.For example, individuals are usually aware of everythingthat they eat on a given day or in a given week.But if you were to ask them at the end of a dayor the end of a week to report everything that they've eaten,

  • 05:26

    ELIZABETH MANCUSO [continued]: they may not be likely to remember everything.And actually we know from researchthat participants who are overweighttend to under-report what they've eaten.So in situations like this, it maybe more valuable to have individuals create a foodjournal or a log, and do this along the waythroughout the day and week, and then submit thatrather than to respond to a survey at the end of the day

  • 05:47

    ELIZABETH MANCUSO [continued]: or the end of the week to indicatehow much they've eaten.Worse yet would be to ask them how many calories they'veeaten, which relates to the first pointsthat this is the type of informationthat they may not know.Finally, if you have a topic that participants know,and have insight into, and are able to remember,you also need to make sure that it'sa topic that survey respondents are willing to give

  • 06:08

    ELIZABETH MANCUSO [continued]: honest information about.Participants may be hesitant to give informationabout very personal topics, such as sexual behavior or substanceabuse, in particular if the survey is not anonymous.In addition, survey research may bestavoid topics that are highly loadedwith social desirability.For example, if you ask individuals are you

  • 06:29

    ELIZABETH MANCUSO [continued]: against child abuse, there are not many peoplewho would say no to that.So it might not be a useful topicto study through a survey.[Survey Design]Once you have a topic that can be used for a surveyresearch that is the type of information thatis useful to gain through surveys,

  • 06:50

    ELIZABETH MANCUSO [continued]: the next step is to either select a surveyor design a survey.Most forms of research need to make use of surveysthat have been validated.This means that the survey is reliable,that it's internally consistent, and that it'sbeen shown to be appropriate for the target population that'sbeing assessed by the survey.Needless to say the quickest and most convenient

  • 07:11

    ELIZABETH MANCUSO [continued]: option would be for a researcher to selecta survey that's already been designed and validated.However, if you're studying a topic for whichno surveys exist, then it might beuseful to design a new survey.We're not going to go into all the ins and outsof how to validate a survey, but wewill cover some common pieces of adviceabout how to write good survey items.

  • 07:32

    ELIZABETH MANCUSO [continued]: First, you want to make sure that all of your survey itemsare addressing one common topic.In order to make this happen it'simportant to have a strong conceptualization of whatthe purpose is of your research, and thento make sure that every survey item actuallyassesses this topic.Second, you want to make sure that every survey item isaddressing only one question.

  • 07:54

    ELIZABETH MANCUSO [continued]: So once you write your survey items,you can look through them and see if any of themuse the words and, or, or but.When this is the case, they're likely to be assessingmore than one topic and we call this a double barreled item.For example, the survey question might ask individuals to ratetheir agreement or disagreement with the statementthat the government should invest more in art and nature.

  • 08:16

    ELIZABETH MANCUSO [continued]: Well how is a respondent supposedto react if they agree that the government should invest morein nature but not in art?You can see how this might be problematic.Third, you want to make sure that all of your survey itemsare clear to participants and avoid ambiguous terminology.So any type of key terms that can be interpretedin more than one way.

  • 08:37

    ELIZABETH MANCUSO [continued]: In general, it's valuable to keep it short and simple.An example would be a research question asked participantsto indicate whether they've ever had a relationship with someoneof a different ethnicity.This might leave the research participant wonderingwhether this involves only a romantic relationship,or rather the researcher is thinkingof any kind of personal relationship,

  • 08:57

    ELIZABETH MANCUSO [continued]: or what about a business relationship?In this case, the burden should be on the researcherto define clearly what the question is asking for sothat the information gained by the researcherwill be valuable.Fourth, it can be helpful to phrase questionsin both positive and negative ways.For example, you might have a question asking aboutwhether individuals perform regular maintenance

  • 09:19

    ELIZABETH MANCUSO [continued]: on their cars.You could also add a second questionasking whether individuals sometimes don't changethe oil in their car on time.This helps to respond to what we call response sets, or responsetendencies, where individuals might have the tendencyto always agree or always disagreewith survey statements.So with example that I just gave about changing

  • 09:40

    ELIZABETH MANCUSO [continued]: the oil in the car and performing maintenance on time,these items would balance out with one anotherif a participant has a tendency to always say yes or alwaysagree with most survey items.Finally, it's important to avoid making assumptionsabout your research participants.For example, the survey item that I justgave asking participants whether they perform

  • 10:00

    ELIZABETH MANCUSO [continued]: regular maintenance on their carsmight be problematic if you're asking individualsto rate the extent to which they agree with this itemif those individuals don't own a vehicle.You can see how that might be problematic for participants.A second point about sampling is that researchersmust be cautious about non-responsive bias.The response rate is the proportionof individuals who are recruited for a survey who

  • 10:23

    ELIZABETH MANCUSO [continued]: end up completing the survey for the researcher.If you recruit a group of individualsto complete a survey and only 10% of them participates,then this leaves a lot of room for this samplenot to be representative of the population.This can be problematic because therecould be systematic reasons that those 10% of individualsagree to participate and the other individuals did not

  • 10:45

    ELIZABETH MANCUSO [continued]: agree to participate.For example, you might have a sample that'svery altruistic in nature.Or you might have a sample among which participantsfelt very strongly about your topic of interest whichcould bias your results.[Designing a New Survey]The first step in designing a new surveyis deciding what type of response options to use.

  • 11:08

    ELIZABETH MANCUSO [continued]: Survey response options can rangefrom open to closed ended.Open ended items allow participantsto respond to the survey in whatever way theywish using their own words.An advantage of the open ended formatis that it doesn't constrict participants,that participants can decide what type of informationis valuable on a given question, and also

  • 11:29

    ELIZABETH MANCUSO [continued]: that the information gained is much richer and more detailed.In contrast, researchers can make use of closed ended items.And with closed ended items, research respondentsare given a set number of choices from which theycan select an answer.For example, a common closed ended type of format

  • 11:49

    ELIZABETH MANCUSO [continued]: is a Likert scale where individualselect from a group of numbers or statements.Close ended formats can also include visual skills.For example, the visual analog skillwhere participants can select a positionon a line that represents a continuum between twoanchor points.I recently traveled through an airportwhere I had the opportunity to respond to one item surveys

  • 12:12

    ELIZABETH MANCUSO [continued]: in various parts of the airport thatwere closed ended in nature.In different parts of the airport,for example customs and baggage claim,there were screens that asked what my experience wasin those parts of the airport, and Icould select between a smiling face, a neutral face,and an unhappy face.This is another example of a closed ended survey format.Closed ended response formats are valuable because they

  • 12:35

    ELIZABETH MANCUSO [continued]: are easier to convert to numbers and therefore easierto analyze statistically, but a limitationmay be that they make participantsfeel more confined in the type of response they can provide.[Sampling]So the last topic will consider ishow to know who the surveys should be administered to.In order to understand this, it's

  • 12:56

    ELIZABETH MANCUSO [continued]: important to know the distinctionbetween a population and a sample.A population consists of all of the individualsthat a researcher is interested in learning about.So this might be everyone in the US,all the members of a particular organization,everyone who has a particular career and so forth.Some populations are small enough

  • 13:17

    ELIZABETH MANCUSO [continued]: that a researcher can survey the whole group.For example, a professor who administers a courseevaluation to all of the students in the class.On the other hand, some populationsare much too large to survey everyone, and in this casea researcher will select a subset of the populationto survey, and we call this group a sample.The size of the sample is less important than the nature

  • 13:39

    ELIZABETH MANCUSO [continued]: of the sample and knowing whether the sampleis representative of or can speakfor the population as a whole.For example, you might be interested in studyingthe differences between introverts and extroverts.Well if you go and gather surveysfrom 10,000 young adults in bars and clubs,you have a huge sample, but it may notbe representative of all the individuals

  • 13:59

    ELIZABETH MANCUSO [continued]: that you're interested in studying becausein the settings that you've lookedyou may find more extroverts than introvertswho are actually at home reading a book during that time.You're also targeting a very particular agegroup in that example.When it comes to sampling, what mattersis whether the sample is representativeof the population as a whole.We tend to know whether the sample is representative when

  • 14:20

    ELIZABETH MANCUSO [continued]: we use probability samples.A probability sample is a type of samplein which the researcher knows the exact probability or chancethat any given person in the populationends up in the sample.This is in contrast to a convenient sample,where the researcher targets a particular groupwithout knowing what the likelihood isthat any given person in the population

  • 14:41

    ELIZABETH MANCUSO [continued]: ends up in the sample.There are different types of probability samples.Perhaps the strongest is a random sample.In order to gather a random sample,a researcher needs to know every individual in a population,and then ensure that each of the individuals in the populationhas an equal chance to every other individualin the population of being selected into the sample.

  • 15:03

    ELIZABETH MANCUSO [continued]: Random samples can also be stratified,meaning that the proportion of relevant groupsare represented equally within the random sample.So for example, taking a random sample of menand a random sample of women so that eachequals 50% of the total sample.Samples can also be made proportionateso that subgroups within the sample

  • 15:24

    ELIZABETH MANCUSO [continued]: represents particular proportionswithin the total population.For example, making sure that the samplehas an equal representation of racial groupsas the population as a whole.You can imagine that getting a random samplemight be a difficult task because it requiresknowing everyone who exists within the given population,and having contact information for every individual who

  • 15:46

    ELIZABETH MANCUSO [continued]: ends up in the sample.For this reason, random samples are not alwaysavailable to a researcher.An example of a nonrandom form of a probability sampleis known as a cluster sample.This is often used in opinion researchwhen polling companies take a random sample of zip codeswithin a given country, then take a random sample of streets

  • 16:06

    ELIZABETH MANCUSO [continued]: within those particular zip codes,then take a random sample of addresses within those streets,and then ask those individuals to complete a survey.[Review Questions]So here are some of your questions for this tutorial.One, explain the difference between experimental andnon-experimental research.

  • 16:28

    ELIZABETH MANCUSO [continued]: Two, name some examples of types of researchthat may make use of surveys.Three, explain some advantages and disadvantagesof open versus closed ended survey response options.Four, summarize some tips for writing effective survey items.Five, explain the difference between a population

  • 16:49

    ELIZABETH MANCUSO [continued]: and a sample.Six, define what a probability sample is.Seven, explain what a random sample is.Eight, give an example of how you might come upwith a cluster sample.And nine, explain why response bias isa concern in survey research.

  • 17:09

    ELIZABETH MANCUSO [continued]: [Conclusion]As we wrap up, the takeaway messages from this tutorialare that in survey research it's firstimportant to understand that thisis a form of non-experimental research.It's great for describing people or phenomena,or for studying non causal relationshipsbetween variables.

  • 17:30

    ELIZABETH MANCUSO [continued]: In addition, before conducting survey researchit's important to make sure that the topics being studiedare topics that the survey respondents have insight into,are able to remember, and are reallyto report honestly about.If a researcher wants to conduct survey research,options are to select a survey that has previouslybeen developed and validated or to design a new survey.

  • 17:52

    ELIZABETH MANCUSO [continued]: And we covered some common tips for how to designappropriate survey items.Finally, it's important to rememberthat survey research is only valuableif an appropriate sample of population is surveyed.I hope that you found this information useful.

Video Info

Publisher: SAGE Publications Ltd

Publication Year: 2018

Video Type:Tutorial

Methods: Survey research, Quantitative data collection, Sampling

Keywords: group beliefs; practices, strategies, and tools; representation and representative

Segment Info

Segment Num.: 1

Persons Discussed:

Events Discussed:



Elizabeth Krumrei Mancuso discusses using survey methods in research. Designing a survey, information gathering, and proper sampling techniques are explained.

Looks like you do not have access to this content.

Learning to Design a Survey Study

Elizabeth Krumrei Mancuso discusses using survey methods in research. Designing a survey, information gathering, and proper sampling techniques are explained.

Copy and paste the following HTML into your website