Skip to main content
SAGE
Search form
  • 00:12

    DINA SMELTZ: But we will be releasingthat in Washington DC as usual.And we're going to plan to do it at the Wilson Centerthis year, hopefully in conjunctionwith a media partner.And we will be presenting those resultsbefore the first presidential debate on September, 26th.I think we have at least two weeks.

  • 00:33

    DINA SMELTZ [continued]: I'll let you know for sure.But, yeah.That sounds like perfect timing.So that's great.Thanks.Hi.My name's Dina Smeltz. [Dina Smeltz, Senior Fellow, PublicOpinion & Foreign Policy] And I'mthe senior fellow of public opinionand foreign policy at the ChicagoCouncil on Global Affairs.The Chicago Council on Global Affairs was founded in 1922.And its goal is to educate the public

  • 00:56

    DINA SMELTZ [continued]: and try to engage the public in discourseabout the global issues that matter today.It's a nonpartisan, independent think tank in that regard.And we convene a lot of programs.But the study side of the house, which is where the survey live,is more of a think tank.

  • 01:17

    DINA SMELTZ [continued]: From 1974 to 2004, every four years the Chicago Councilconducted a survey of opinion leaders.That would be business leaders, government leaders, peoplefrom the executive branch, media leaders, journalists,academics, religious leaders, labor union officials.

  • 01:38

    DINA SMELTZ [continued]: So in 2004 was the last time we had that done.It was done by telephone.It was super expensive.We've never been able to do it again until 2014.We were able to revive the leaders survey because wecould do an online platform and do a lot of itin-house in 2014.

  • 01:59

    DINA SMELTZ [continued]: So that was really interesting because wesee so many partisan divides among the general public.But they're even wider among foreign policy elites.And that's really interesting because foreign policyused to be an area where there weren'ta lot of partisan divides.There have been on domestic issue since the '70s.

  • 02:20

    DINA SMELTZ [continued]: But we're only recently seeing them on foreign policy issuesas well.The survey itself is 40-- over 40 years old.It's one of the longest running surveys of American attitudeson foreign policy.So it's been a pleasure to work on such a treasure.

  • 02:42

    DINA SMELTZ [continued]: I worked for 15 years at the State Department's Officeof Opinion Research.And I was lucky to have worked in EasternEurope and the Balkans after the Berlin Wallfell in '91 until around 2002.

  • 03:04

    DINA SMELTZ [continued]: And then I also worked in the Middle East and South Asiaafter we invaded Iraq.And after the September 11 attacks, obviously.And for me, it was really satisfying on several levelsworking both in Kosovo, and Bosnia, and also in Iraq.And really in all the countries thatwere developing survey practices, some of them

  • 03:26

    DINA SMELTZ [continued]: had a long history.But we were able to work with peopleand actually help them develop their own researchorganizations that became viable businesses themselves.So that was a-- gave me a great deal of satisfaction.And then seeing what is really happening on the groundand listening to people's concerns.

  • 03:48

    DINA SMELTZ [continued]: And being able to pass those concerns onto policymakerswas incredibly satisfying.And especially when I was working in Baghdad,we didn't do the interviews ourselves obviously,but we worked with the research organization there.And it was the only way that some of those policymakers

  • 04:08

    DINA SMELTZ [continued]: knew what average people were thinking.Because it was very unsafe for them to leave the green zone.I feel very satisfied being able to bea voice for those people in those situations.And to convey what their most important concerns and needswere.But, yeah.I just wanted to make sure that we were all

  • 04:29

    DINA SMELTZ [continued]: set for our fielding this May, June for our 2016 poll.In our staff meetings, what we usually discussis the status of various projects.Our overall goal is always-- we're always writing proposalsto try to get funding.We're a nonprofit, independent, nonpartisan think tank.

  • 04:50

    DINA SMELTZ [continued]: So we're always looking for funding for our big studies.So the proposal process-- really,what we cover in those meetings is everything from the startand finish, and the different stages of all the tasksthat are involved in fielding a survey.So we'll talk about the proposals.We talk about questionnaire design.

  • 05:11

    DINA SMELTZ [continued]: We discuss various questions.It's something that we think mightbe really important to ask in January,by the time we field in May, might not be that important.So we're constantly monitoring, updating the questionswe're going to field.We discuss what presentations are out thereand who's going to give those presentations.

  • 05:32

    DINA SMELTZ [continued]: How people can help each other prepare those presentations.And then we have a blog that we update several times a month.And so we discuss who's going to write on what topic.And we collaborate very often on some of those articles as well.So that's our next step.I don't think it'll take us too much more work.

  • 05:54

    DINA SMELTZ [continued]: But maybe one of you guys could take the leadon working on that.The essentials of survey design, let'ssay there's five essentials.The first is knowing what your research question is.The second is drawing an accurate and representativesample.The third is designing a survey questionnaire that is balanced.

  • 06:15

    DINA SMELTZ [continued]: And in an order in which the questions don't contaminatethe succeeding questions.Fourth is fielding the survey itself.And fifth, coding the data properly, data processing,and finally analyzing the survey results.If I had to pick one very crucial element-- but there's

  • 06:38

    DINA SMELTZ [continued]: so many.But one very crucial element is the sample itself.And this has been getting a lot of attentionbecause of the elections that are happening right now.And some of the primary polling like in Iowacame out with different results than actuallyhappened on the ground.

  • 06:60

    DINA SMELTZ [continued]: And this is really a challenge for pollsters today.I'm particularly happy that I am notin that kind of political pollingwhere I'm trying to predict who's going to win elections.But because it's very hard now.The response rates are really low, in the single digits.And it's very hard to draw an accurate sample because

  • 07:26

    DINA SMELTZ [continued]: of the amount of people that are on cell phones now.It used to be landline-based sample.And a lot of people don't-- they can screen their calls.First it was answering machines.Now it's cell phones where you can see who's callingand actually just not answer the call.So it's been very difficult to reach people.And then to make sure that your sample is

  • 07:48

    DINA SMELTZ [continued]: comprised of people that represent the total population.So waiting after the sample is collectedhas been very important in trying to predict elections.So I think the sample is really a very important placeto start.It's really-- if you have a bad sample,then everything else in the survey

  • 08:10

    DINA SMELTZ [continued]: is pretty much gone to pot.Then they're running two of the questionsthat we had proposed with some wording changes.So I'll make sure I get that into our new revisedquestionnaire.But we'll still have to cut a lot.So designing a survey and decidingwhich mode of interview is best is really a trade-off

  • 08:33

    DINA SMELTZ [continued]: between cost and coverage.Or the way you can-- the broadest samplingcoverage that you can get.So for us, we switched to online surveysin 2004 mostly because of cost.But what we did was parallel surveys of telephone and onlineat the same time to make sure thereweren't very large differences in the responses.

  • 08:57

    DINA SMELTZ [continued]: And after that experiment, we felt comfortable switchingto online.Another reason online is good for our surveyis because the survey questions are complicated.They're quite long.And it would be very hard to have somebodyon the phone for 20 to 25 minutesanswering surveys about foreign policy.Now, telephone or face to face is sometimes better

  • 09:19

    DINA SMELTZ [continued]: for if you have a more sensitive topicthat you would want an actual personto speak to the other person.Especially in working abroad in certain countries,face to face really is the only wayto do it because there is no online sample that'srepresentative enough to reach everybody.And perhaps not everybody has a telephone.

  • 09:41

    CRAIG: Do we know when our partners wantto field their questionnaires?Because last time I remember the Japaneseand the Koreans fielded them very early, like in March.And then we didn't field ours until the summer.

  • 09:51

    DINA SMELTZ: I didn't think it was a problem last year that itwas a little bit staggered.So I think that would be fine.You know, it's always best to field around the same time.But we'll kind of take what we canget since each partner organization ispaying for their own.The most important thing to considerwhen drafting questions for a given surveyis to keep your research focus in mind.

  • 10:15

    DINA SMELTZ [continued]: You don't want to ask any tangential questions.You want your questionnaire ideallyto be as short as it can to get to reach your objectives.The second-- there's so many things.But what I would point out right nowis that the question wording needs to be balanced.There's a certain skill in designing survey questions.

  • 10:35

    DINA SMELTZ [continued]: For example, if you want to ask about a policy,you don't say, do you favor this policy-- doyou favor this particular policy,and then follow up with yes, no, maybe.You should say, do you favor or oppose this policy?Or does it not make a difference to you?You need to make sure that all of those optionsare included in the survey question.

  • 10:57

    DINA SMELTZ [continued]: And the other really important thingis to make sure that questions are in a languagethat your target responding can understand.You don't want it to include jargon.You want it to be simple enough, in our case,for an average person to be able to comprehend.And that's a lot of the issues that we

  • 11:18

    DINA SMELTZ [continued]: talk about with my staff in designing surveys that are alsogoing to be asked abroad is whether questions in Asiawill translate for the American public and vice versa.So we've been discussing the best wayto design some of those questions as well.I thought it was interesting that younger Australians aren't

  • 11:38

    DINA SMELTZ [continued]: as keen on democracy as you would think.But I don't know if it was so relevant for whatwe're looking at.

  • 11:45

    SPEAKER 1: Yeah.

  • 11:46

    DINA SMELTZ: Probably the most important methodthat we use here at the council is called trend analysis.And that's because since 1974, we've been conducting surveyshere at the council.It gives us a really rich repositoryof data that we can look at it over time because we've askedidentically worded questions.And so that's called trend analysiswhen you look at over time and look for different patterns.

  • 12:08

    DINA SMELTZ [continued]: And that you'll see some beautiful graphs that goup, or down, or stay the same.But that is one method we use.Another is we use statistical packages called SPSS.Some people use R. Some people use Stata.Our default here is SPSS.And we conduct mostly basic data analysis using crosstabs.

  • 12:31

    DINA SMELTZ [continued]: Craig and I have been working on a lot of immigration work.And we've been running crosstabs thatshow Republicans, Democrats, and independents,and how their views differ or arethe same on immigration policy.Immigration and climate change are the two issueswhere-- two foreign policy issueswhere we see the widest differences between Republicans

  • 12:53

    DINA SMELTZ [continued]: and Democrats.So we have been looking at what'smotivating those differences and those trends over time.So that is probably a mainstay of our work here.And then in rare cases, we'll alsorun regressions or maybe even factor analysisto identify key drivers of opinion.

  • 13:13

    CRAIG: Splitting the difference between illegal immigrantsand undocumented immigrants.And how it shows pretty strong effectsfor support for citizenship among Republicans.

  • 13:22

    DINA SMELTZ: Yeah.That's a great idea.It's trying to push forward immigration reformamong conservatives, in particular.So just would be helpful.The results of our surveys are used in many ways.For one thing and importantly, all of our dataare made public to academics, to students.They form the backbone of several important academic

  • 13:43

    DINA SMELTZ [continued]: books, dissertations.So one way is just that we make the raw dataavailable to everybody.The other, we write various reports every yearwhen we finish our survey.It's usually in the summertime.We write all summer.And by September, have a big report that we release in DC,

  • 14:05

    DINA SMELTZ [continued]: and Chicago, and then several other citiesaround the country.And try to make people aware of what other Americans thinkour foreign policy should be, especially in this electionyear.So presentations is another aspect of that.We also write opinion editorials in the Washington Post,

  • 14:25

    DINA SMELTZ [continued]: The Wall Street Journal, Foreign Policy,and some local newspapers.So we try to also influence public discourse notto take a stand, but to be active,and engaged, and interested in foreign policy issues.

Video Info

Publisher: SAGE Publications Ltd

Publication Year: 2017

Video Type:In Practice

Methods: Survey research, Online surveys, Cross-tabulation

Keywords: blogs; cell phones; climate change; Democratic Party; editorials; foreign policy; immigration; Iraq; leadership (political); non profit organizations; opinions; partisanship; policy evaluation; policy making; Republican Party; Serbia; Staff meetings; think tanks ... Show More

Segment Info

Segment Num.: 1

Persons Discussed:

Events Discussed:

Keywords:

Abstract

Dina Smeltz of the Chicago Council on Global Affairs discusses her professional experience in survey research. She highlights best practices for survey design and details the day-to-day activities that go on at the think tank where she works.

Looks like you do not have access to this content.

Political Surveys: The Chicago Council on Global Affairs

Dina Smeltz of the Chicago Council on Global Affairs discusses her professional experience in survey research. She highlights best practices for survey design and details the day-to-day activities that go on at the think tank where she works.