Skip to main content
SAGE
Search form
  • 00:11

    TAREK AL BAGHAL: My name Tarek Al Baghal.I work at the University of Essex,where I work on designing one of the largestsurveys in the United Kingdom.I designed the questionnaire, and I'vetaught the questionnaire design course hereand the University of Nebraska, in the United States.In this tutorial, I'll be talkingabout theoretical and practical toolsfor designing a questionnaire.Specifically, I'll be talking about the response process,the four maxims of conversation in a survey, question wording,the different types of questions, question ordering,visual design, and, in particular,for internet and mobile surveys.

  • 00:46

    TAREK AL BAGHAL [continued]: We ask questions in everyday situations to everyday people.For that reason, we tend to think that writing a surveymay be easy.However, what we have learned is that thereare a lot of things that can affect survey results,including the wording.And it's not as easy as one may think.Therefore, it's important for us to consider these issues as wego forth and design our survey.

  • 01:11

    TAREK AL BAGHAL [continued]: The response process is made up of four steps thatgo through your cognitive psychology,while you're answering a question in a survey.The first step is comprehension, where it's necessary for youto understand what the question is askingand what is being asked of you to report.After you comprehend the question,the next step is retrieval.

  • 01:31

    TAREK AL BAGHAL [continued]: And that is retrieving all of the information thatis in your memory, or in your mind thatis pertinent to the question, or at leastthat you're willing to respond to.The next step is judgment.And this is where you're taking all of the informationthat you've retrieved up to that pointand deciding what is pertinent.What do I want to tell the interviewer or the surveyabout, and what am I going to say?

  • 01:53

    TAREK AL BAGHAL [continued]: The final step in the response processis the formulation of response.This is where you're taking all the information that you'vedeemed pertinent and willing to respond,and formatting it to the response that hasbeen given to you in a survey.For example, you might be asked to answer a questionin whole numeric terms.But you think about it in terms of some vague quantity.

  • 02:14

    TAREK AL BAGHAL [continued]: So you have to translate that vague quantity into 5 or 10,or whatever the question is asking for.When responding, people tend to follow conversational norms.This is because we are used to conversations whereboth parties are following a cooperative principle, wherethey're trying to add to the conversationand make the conversation relevant to both sides.

  • 02:40

    TAREK AL BAGHAL [continued]: This conversation is structured around four maxims.The first is the maxim of manner,where everyone in the conversationhas to comprehend what is being said.The next maxim of the conversationthat every one holds to, is the maxim of relation.And that is where everything is relevant to the conversationbeing held.The third maxim is that of quantity.

  • 03:01

    TAREK AL BAGHAL [continued]: That is, where everything that is being said is of relevanceand is necessary to the conversation, but not more so.You wouldn't say more than is necessary for the conversationto be relevant.And the last maxim is that of quality.Whereas, you don't say anything false,nor something that does not have any factual basis that youknow of.

  • 03:23

    TAREK AL BAGHAL [continued]: When respondents are sitting in a interview situation,whether it be in front of an interviewer,or filling out a survey themselves,they are following these cooperative principlesin these maxims in order to fill out the survey.Therefore, it's a conversation between you and the respondent.When partaking in this conversation,there are four types of questions that are generallyasked in surveys that respondentsare asked to answer to.

  • 03:51

    TAREK AL BAGHAL [continued]: The first type is that of knowledge.Knowledge is just asking people whether they know something,such as, do you know who the prime minister is?The second type of question is that of demographics.These are simple questions about the respondent,such as, what is your age?What is your sex?And what is your education?The third type of question that is normally asked in surveysis that of behavioral, or factual types of questionstowards the respondent.

  • 04:16

    TAREK AL BAGHAL [continued]: These are asking about life events, or other instancesin the respondents life, and haveto do with factual information that they must retrieveduring the response process.This requires the respondent to recalldifferent kinds of information pertinent to the question.So, for example, if you ask to respond to how many times theywash their hands in the past five years,that would be near impossible for anyone to recall.

  • 04:42

    TAREK AL BAGHAL [continued]: However, if you ask them how many timesthey wash their hands in the past day,that may be more possible.This is the kind of information that you should thinkabout when writing questions.How is the memory structured, and how is it most likelythat they'll be able to recall the information that you'reasking for?The last type of question is that of attitudes.Attitudes are very frequently made up on the spot,except for those kinds of attitudesthat are very strongly held.

  • 05:07

    TAREK AL BAGHAL [continued]: These types of questions are affected by wordingin a significant way.Largely, because the way the questionis worded can affect what kind of informationis being retrieved.So you want to be able to word the questions in such a waythat you get the attitude that you're actuallyinterested in measuring.For example, if you ask a question about monarchyin the United Kingdom, that may be totally differentthan if you ask about Queen Elizabeth or Prince Charles.

  • 05:33

    TAREK AL BAGHAL [continued]: It's two different institutions.And one is individual, and the other is institution.You might think that they're the same,but may end up with totally different results.When asking these different types of questions,it's very important to think about the wordingthat you're going to use in the questions that you write.This is not just true for attitude questions,but, also, for any other of the types of questions.

  • 05:54

    TAREK AL BAGHAL [continued]: For example, one of the things that I find really interestingis that if you ask somebody how old they are,they can get that wrong more often than you might imagine.However, if you ask them what year they were born,they rarely get that wrong.So even for a simple question like that, asking for what yearthey're born may be better than how old theyare because they're less likely to get that wrong,and you can do the math.

  • 06:20

    TAREK AL BAGHAL [continued]: One of the big things to avoid, or at least considerwhen you're writing it, is the word, and, for example.So, and, is a conjunction, and that leadsto double barreled questions.So you tend to ask about two things at once,when, really, only you wanted to know about one,or respondents have difficulty because they're onlyanswering about one.

  • 06:40

    TAREK AL BAGHAL [continued]: For example, you might say, how satisfied are youwith work and family?Now, a respondent may be satisfied with their workand not with their family, or vice versa, with both,or with neither.And you only have asked them one questionto talk about two things.And that can be very confusing for the respondentand lead to less quality data for you.

  • 07:01

    TAREK AL BAGHAL [continued]: So be very careful when you see things like the word, and.Other wording can really affect responsesas well, particularly, in attitude types of questions.For example, do you think that the United Statesshould allow speeches against democracy, leadsto very different results than when you ask,do you think the United States should forbid speechesagainst democracy?

  • 07:22

    TAREK AL BAGHAL [continued]: Even though these tend to be just opposites,they lead to very different conclusions.One of the important things is that wordingis something that's under your control as the researcher.Unlike the cognitive processes of the respondents,or the maxims that they may be following in a conversation,this is something that you can manipulate.The wording, however, can affect the response processesand the maxims that they're following.

  • 07:48

    TAREK AL BAGHAL [continued]: So this is something that you should really, reallycontemplate.Not only can it lead to bad data,but it can really improve your dataif you know what you're doing and take into considerationthe respondent when writing the questions.Besides wording of the questions,the order of the questions in the questionnairecan have a large effect on results.

  • 08:10

    TAREK AL BAGHAL [continued]: For example, a questionnaire in Germanythat asked how typical of a drink in Germany is milk,was affected greatly by the preceding question.When the preceding question asked about how many Germansdrank vodka, milk was rated quite highas a typical type of German drink.When the question was preceded by how many Germans drink beer,the typicality of milk decreased significantly.

  • 08:37

    TAREK AL BAGHAL [continued]: This is called the context effect.This occurs when the question that precedes,or the questions that precede a particular question,affects the context, and affects what is recalled in your mindwhen you're answering any given question.Therefore, it is important to thinkabout how previous questions in your questionnairemay influence the one that you'reasking at any given moment.

  • 08:59

    TAREK AL BAGHAL [continued]: They may not, but they may, Such asin the particular example about Germans and beer and vodka.The mode of survey administrationcan also affect the way questions are answered,including the ordering, or including the types of effectsthat we've discussed previously.So there are two main types of ways of administering a survey.

  • 09:21

    TAREK AL BAGHAL [continued]: Either by the interviewer, and thiscan be either the interviewer being at a person's houseor over the telephone, or the respondent fills outthe survey themselves.And this is generally done through a mail questionnaire,or now, more and more frequently, over the internet.So when a interviewer calls on your telephoneor in your house, the inflection of the interviewer's voice,or the speed that they read the question,or the rapport that they develop with the respondent,can all influence responses.

  • 09:48

    TAREK AL BAGHAL [continued]: In a self-administered questionnaire,where the respondent is filling itout themselves, either on a piece of paperor on the internet, the visual design of the questionnairecan have large impacts on the responses.This is particularly important as we move more and moreto self-administered questionnaires,through either the internet or mobile phones.For example, an internet survey wasasking about how healthy people thought they were.

  • 10:12

    TAREK AL BAGHAL [continued]: Either the survey included a picture of an athlete running,or a picture of a person sickly in a hospital bed.People who saw the picture of the healthy athleterated their health as lower than peoplewho saw the picture of the sickly personin the hospital bed.They tended to rate their health higher.Therefore, the graphics that are includedin a internet, or any kind of visual survey, for that matter,can have large impacts on results.

  • 10:40

    TAREK AL BAGHAL [continued]: Generally, because of things like this, we tend to say,avoid graphics in your visual design of internet and mailsurveys.Even when answering the questionnaire on their own,for example, on the internet, respondentsare still having a conversation with you, the researcher,even if you don't realize it.They're trying to understand what you'reasking of them by inferring, through the questionnaire,and through the design of the survey.

  • 11:05

    TAREK AL BAGHAL [continued]: In this first example, an internet questionis asked in two different ways.This is a check all that apply question,meaning that a respondent says, I did all of these things,or none of these things, dependingon their personal situation.The same question was asked a different respondentin a different way.In this first way that the question was asked,the check all that apply was separatedin two different categories.

  • 11:28

    TAREK AL BAGHAL [continued]: Respondents tended to choose at least one of the checkall's from both of the two categories.In this second presentation of the same question,there's only one grouping of the check all that applies.In this grouping, respondents tend to choose lessof the check all's.This tends to be the case because respondentstry to infer what you, the research, are asking for.

  • 11:51

    TAREK AL BAGHAL [continued]: In the first question, respondentsare inferring that the researcherwants to get groupings of different kindsand wants you to select from both.In the second question, respondentsinfer that the researcher's just asking about one groupingand feels free to select just from that one.In this second example, you can seehow visual design can be impacted by learningand cultural differences.

  • 12:17

    TAREK AL BAGHAL [continued]: In this first example, asking for a person's date of birth,you can see that this is how the graphical design would lookfor a United States respondent.That's because the first box is larger than the other two,and indicates that you should write the month in this space.In the second question, you can see that this is generallywhat would be asked for many other countries,particularly in Europe, where the day comes first,followed by the month, and then the year.

  • 12:43

    TAREK AL BAGHAL [continued]: To continue with mode, one of the thingsto think about when designing the visual componentsof your survey is things that make it easier for usability.For example, select fonts that are easy on the eyes,both for the respondent or the interviewer,depending on what types of survey you're using.Also, when designing a survey, tryto put only one question per page.

  • 13:04

    TAREK AL BAGHAL [continued]: This will help both the interviewer or the respondent,depending who is conducting the survey, in flowand answering the questions.Another thing to avoid in internet questionnairesare questions that require scrolling down.Respondents frequently don't scroll, and missresponse options that normally you would want them to answer.These visual design issues are becomingeven more important as mobile surveys become moreand more prevalent.

  • 13:29

    TAREK AL BAGHAL [continued]: As can be seen in these examples,the differences between computers and mobile devicescan be great.This is largely due to differences in screen sizesand differences in technology thatare required to do surveys on the different devices.These differences in screen sizes and softwarecan dramatically affect survey responses.

  • 13:50

    TAREK AL BAGHAL [continued]: Taking these into account, and designingsurveys that adapt to mobile design,is increasingly important as you design questionnaires.This tutorial pointed about that designing a surveyis not as straightforward as one may think.

  • 14:10

    TAREK AL BAGHAL [continued]: This has been a very brief introductionto what goes into questionnaire designand has only touched the surface of thingsthat we know and have learned.It's best that you take the time to think about these things,and look more into best practices and the literaturethat has developed over the yearsin determining how to design your questionnaire.

Video Info

Publisher: SAGE Publications Ltd.

Publication Year: 2017

Video Type:Tutorial

Methods: Questionnaire design, Survey research, Online surveys

Keywords: administration; attitudes and behavior; cell phones; cognitive psychology; communication challenges; comprehension; conversation; internet; judgment and decision making; knowledge; language usage; memory; nonverbal communication; rapport; relevance (education); vocabulary; word order ... Show More

Segment Info

Segment Num.: 1

Persons Discussed:

Events Discussed:

Keywords:

Abstract

Questionnaire writing and design may seem simple, but many factors can affect the quality of response data. Dr. Tarek Al Baghal describes the cognitive processes that respondents undergo while taking a survey. He also stresses the importance of appropriate word choice and order to minimize unintended context effects.

Looks like you do not have access to this content.

An Introduction to Questionnaire Design

Questionnaire writing and design may seem simple, but many factors can affect the quality of response data. Dr. Tarek Al Baghal describes the cognitive processes that respondents undergo while taking a survey. He also stresses the importance of appropriate word choice and order to minimize unintended context effects.