Skip to main content
Search form
  • 00:14

    IPEK BILGEN: My name is Ipek Bilgen and I'm a researchmethodologist at NORC at the University of Chicago.[Ipek Bilgen, PhD] I received my PhD and Master's degrees fromthe Survey Research and Methodology departmentat the University of Nebraska, Lincoln.And I obtained seen my bachelor's degreein the Statistics department at Hacettepe University Ankara,

  • 00:34

    IPEK BILGEN [continued]: Turkey.[What is meant by the term "interviewer effect"?]

  • 00:40

    IPEK BILGEN: My dissertation researchwas on interviewer experience impact on interviewerand respondent verbal behaviors during data collection,and the combined impact of interviewer experienceon verbal behaviors on item non-response.Specifically, don't know responses.

  • 01:01

    IPEK BILGEN [continued]: My dissertation research was funded by the CannellFund in Survey Methodology.Interviewer effect means that the interviewercan impact data quality in any stage of the data collectionprocess.They have a lot of roles during the data collection,

  • 01:22

    IPEK BILGEN [continued]: from locating households, creatingsampling frames, contacting households and selectingeligible respondents, motivating respondents, explainingthe questionnaire and the survey and the surveyaims to the respondents, responding to any questionsand confusions respondent may have,

  • 01:42

    IPEK BILGEN [continued]: and-- if the interview is computer-assisted--troubleshoot technical problems, as well as enter dataand deliver data to researchers.[What is meant by the term "data quality"?]Data quality and survey error sources goes hand in hand.

  • 02:05

    IPEK BILGEN [continued]: Coverage error occurs when the sample frame does notcover some of the units defined in the target populationor a sampling frame covers more units than what'sdefined in target population.Sampling error does occur due to using a sample rather

  • 02:25

    IPEK BILGEN [continued]: than a population.And then the third error source is non-response,which occurs when not all the sample frame or samplemembers do not respond to the questionnaire or our survey.Measurement error is the survey areathat is introduced during the data administration period,

  • 02:47

    IPEK BILGEN [continued]: either because of the instrument or interviewer or respondent.And the last error in the survey life cycleis the processing error, which happens due to errorduring processing of the data or the delivery of data.Interviewers can have an impact on all of these errors sources,

  • 03:12

    IPEK BILGEN [continued]: other than the sampling error.So they may have not been covering the sample unitwhen they were creating the sample frame.Or they may be impacting respondents motivationto be responding to a survey.Or measurements, definitely, is very much

  • 03:33

    IPEK BILGEN [continued]: associated with interviewer effectswhere interviewer presence or behaviors or characteristicsmay impact data quality.And lastly, during data entry interviewersmay have an impact on data quality.Data quality is highly associated with all these error

  • 03:56

    IPEK BILGEN [continued]: terms.And high quality data means that all the five error termsare minimized, if not eliminated, during the surveys.[What is interviewer experience?]

  • 04:13

    IPEK BILGEN: Three different types of interviewer experienceare defined in the literature.The first one is the general interviewer experience,which is the experience interviewersgain during their lifetime.The second interviewer experienceis defined as experience gained during the interviewer's time

  • 04:33

    IPEK BILGEN [continued]: in a specific survey organization.And lastly, interviewer experience,also known as interviewer exposure,is the within study interviewer experience.Meaning, the interviewer's experiencegained throughout the data collection period.[What is meant by item non-response?]

  • 04:59

    IPEK BILGEN: Item non-response isfailure to obtain responses from one or more itemsin the questionnaire.The failure may cause systematic differences between respondentsthat responded to that specific item versus non-respondentswho did not respond to that specific item.

  • 05:21

    IPEK BILGEN [continued]: These differences may then cause distortion in survey inferencesduring data analysis after data collection.[What is the relationship between interviewer experienceand item non-response?]

  • 05:35

    IPEK BILGEN: Findings in the literature are mixed.While some find positive relationshipbetween interviewer experience and item non-response,others found negative relationships,while some other literature found no relationshipbetween the two concepts.

  • 05:56

    IPEK BILGEN [continued]: The competing theories include, while experienced interviewersdo better-- because practice makesperfect-- on the other hand, inexperienced interviewersstick to the scripts a little betterand follow respondent instructions much betterthan the experienced interviewers.

  • 06:17

    IPEK BILGEN [continued]: [What was your research question?]During my dissertation I examinedtwo different types of interviewer experience--impacts on interviewer verbal behaviorsand respondent verbal behaviors, and the combined impactof interviewer experience and interviewer

  • 06:41

    IPEK BILGEN [continued]: and respondent verbal behaviors on item non-response,specifically don't know responses.The two experiences I have examinedwere general interviewer experience--the experience gained throughout the interviewer's lifetime--and within study interviewer experience, whichwas the experience gained during the data fielding period.

  • 07:05

    IPEK BILGEN [continued]: I looked into interviewer experience impacton item non-response also in different typesof questionnaires.One is the calendar interview, whichis a flexible interviewing technique.And the second interviewer techniqueis the standardized interviewing,where the interviewers do stick to the script closely.

  • 07:30

    IPEK BILGEN [continued]: [What methods did you use?]

  • 07:36

    IPEK BILGEN: This study included two stages.The first stage was the computer assisted telephone interviews.629 respondents were randomly selected among a panel surveyof income dynamics respondents.These respondents are randomly assigned to 26 interviewers,

  • 07:57

    IPEK BILGEN [continued]: and each interviewer was assigned-- randomly--to the interviewing technique-- calendaror standardized interviews.The second stage of the study did include verbal behavioranalysis of these interviews, whichwas recorded and transcribed during the data collection

  • 08:21

    IPEK BILGEN [continued]: period.Our coders coded randomly selected165 calendar interviews and 162 standardized interviews.We have measured the coder reliabilityusing Kappa analysis, and we did double-blind code

  • 08:42

    IPEK BILGEN [continued]: at these studies.After a verbal behavior coding, wedid multi-level data analysis whereI looked into two different interviewer experienceimpacts on each verbal behaviors, that were combinedinto groups of three.Which was first, interviewer and respondent communication

  • 09:06

    IPEK BILGEN [continued]: and rapport.The second was interviewer deviationfrom the script and the questionnaire.And the third verbal behavior groupingincluded interviewer and respondent retrieval behaviors.After the verbal behavior data collectionperiod, we did analyze these verbal behaviors

  • 09:29

    IPEK BILGEN [continued]: and looked into which verbal behaviors go together.And at the end, we came up with three different verbal behaviorgroupings.The first one was interviewer deviationfrom the questionnaire, which included digressions,interviewer's positive or negative feedback-- which

  • 09:52

    IPEK BILGEN [continued]: may or may not impact respondent behaviors.The second verbal behavior includedinterviewer and respondent inter-communication dynamics,which included laughter, neutral feedback of interviewers,empathy, joking, and so on and so forth.

  • 10:12

    IPEK BILGEN [continued]: And the last verbal behavior groupingincluded interviewer and respondent retrieval behaviors,which was pulling from the literature of memory.For instance, an example of the behavioris parallel when the respondent is talkingabout their education history, theyrefer to their marriage or their job history.

  • 10:37

    IPEK BILGEN [continued]: [What were the findings of your research and why does itmatter?]

  • 10:42

    IPEK BILGEN: The interviewer and respondent verbal behaviorassociation with item non-response matteredwhen the behaviors were used in the interviews,whether or not they were used by experienced or inexperiencedinterviewers, or which interviewing techniquethey were being used in.

  • 11:03

    IPEK BILGEN [continued]: And the takeaway message is, this definitelydoes explain the mixed results in the literature.So it doesn't necessarily matter if your interviewer isexperienced or not experienced, but ratherwhich types of behaviors they have been conductingthroughout the data collection.Additionally, in my research, I found that item non-response

  • 11:27

    IPEK BILGEN [continued]: and interviewer experience relationshipdiffered between calendar and standardized interviews.The interviewer experience impact on item non-responsedid diminish during calendar interviews, whichwas because inexperienced interviewers had more room

  • 11:51

    IPEK BILGEN [continued]: to be flexible and to deviate from the scripts.So interestingly this did not reallydecrease the data quality.So what one main finding was, flexible interviewsdid a little better, in terms of diminishinginterviewer experience impact on item non-response.

  • 12:14

    IPEK BILGEN [continued]: And this didn't translate to an increase in item non-response,but rather it did translate to interviewersbeing a little bit more comfortableduring the interview.And respondents, therefore, providing more responsesto the item.[What are the most exciting innovations currently in survey

  • 12:36

    IPEK BILGEN [continued]: research?]

  • 12:40

    IPEK BILGEN: In my opinion, the roleof technology and how this impacts survey researchis starting to be one exciting research area.At NORC, I'm currently involved in the web initiative, wherewe look into how web data collection methods and web

  • 13:00

    IPEK BILGEN [continued]: recruitment methods impact data quality and survey errorsources.Also, one other exciting researchthat we have been conducting under the umbrella of webinitiative, is the use of non-probabilistic techniquesfor collecting data.For instance, the use of social media and search engine

  • 13:24

    IPEK BILGEN [continued]: techniques during data collection.So the increase in technological use among the US populationdoes enable researchers to come up with creative and new waysto use survey methods.Also, tying this to interviewer effects, a new and exciting

  • 13:48

    IPEK BILGEN [continued]: research is the use of avatars during data collection,in lieu of interviewers.And different gamification techniquesare also a new and exciting opportunityfor survey researchers to keep respondentsengaged and motivated during data collection.

  • 14:11

    IPEK BILGEN [continued]: In my opinion, it's very interestingthat survey researchers are now using the use of technologyamong households to their advantage to increaserespondent motivation.And also, to engage respondents, as well aschange the public opinion about survey research

  • 14:33

    IPEK BILGEN [continued]: and how survey researchers conduct it.

Video Info

Publisher: SAGE Publications Ltd

Publication Year: 2017

Video Type:Video Case

Methods: Interviewer effects

Keywords: innovation; motivation; practices, strategies, and tools; verbal behavior; verbal communication

Segment Info

Segment Num.: 1

Persons Discussed:

Events Discussed:



Dr. Ipek Bilgen discusses her research using verbal behavior analysis to examine the impact of interviewer experience. Interviewer experience is how the interviewer's level of experience affects his or her research. Bilgen discusses interviewer experience, her research methods, and her research findings.

Video Info

Publication Info

SAGE Publications Ltd
Publication Year:
SAGE Research Methods Video
Publication Place:
United Kingdom
SAGE Original Production Type:
SAGE Case Studies
Copyright Statement:
(c) SAGE Publications Ltd., 2017


Segment Info


Segment Num: 1


Segment Start Time:

Segment End Time:


Things Discussed

Organizations Discussed:

Events Discussed:

Places Discussed:

Persons Discussed:

Methods Map

Interviewer effects

Influences on the responses given during an interview that can be attributed to the characteristics of the interviewer.
Interviewer effects
Researching the Impact of Interview Experience Using Verbal Behavior Analysis

Dr. Ipek Bilgen discusses her research using verbal behavior analysis to examine the impact of interviewer experience. Interviewer experience is how the interviewer's level of experience affects his or her research. Bilgen discusses interviewer experience, her research methods, and her research findings.

Copy and paste the following HTML into your website