Skip to main content
SAGE
Search form
  • 00:04

    [Introduction to Experimental Design]

  • 00:11

    MAHTASH ESFANDIARI: My name is Mahtash Esfandiari.I'm on the faculty in the Department of Statisticsat UCLA . [Mahtash, Esfandiari, PhD.Professor, Department of Statistics,UCLA] And today I'm going to be talkingabout experimental design.And there are three major designs I want to talk about.[Presentation Topics Observational studies,Real experimental designs, Quasi experimental designs]Observational studies, real experimental designs,and quasi-experimental designs.

  • 00:32

    MAHTASH ESFANDIARI [continued]: [Observational Studies]I'm going to start with observational studies.Observational studies involve [No maniupulation,intervention, or experiment] no manipulation, intervention,or experiment.[No random assignment of participantsto experimental and control group]There is no random assignment of subjectsto experimental and control groups.[Can be descriptive or hypothesis testing]They can be descriptive or they can be hypothesis testing.

  • 00:52

    MAHTASH ESFANDIARI [continued]: [Can be retrospective or prospective]They can be retrospective or they can be prospective.With retrospect studies, that's somethingthat they have already happened in the past.It's like looking at an educational record of a studentor a patient's medical history.Prospective studies, they usuallyinvolve an intervention.They want to answer like, a specific question.

  • 01:13

    MAHTASH ESFANDIARI [continued]: You collect baseline data.You collect data over time.And you collect data at the end of intervention.And potential sources of biases or confoundersare less of a danger than they are in prospective studies.In observational studies, sometimes ethical reasonsdo not allow assignment of participantsto the experimental and control groups.

  • 01:35

    MAHTASH ESFANDIARI [continued]: And since they cannot be randomly assignedto experimental and control groups,therefore drawing causal conclusions will involve bias.Conclusions that you draw from observational studiesare not causal.They simply show some kind of an association or iteration shapebetween the variables involved.

  • 01:57

    MAHTASH ESFANDIARI [continued]: And observational studies sometimesinvolve a control group.But it doesn't mean you can draw causal conclusions from them.For instance, if you're looking at the depression levelof elderly with and without a pet,that's not causal conclusions, because you cannot randomlyassign the elderly to have or not have a pet.And observational studies usually

  • 02:18

    MAHTASH ESFANDIARI [continued]: have multiple confounders.In research, we also refer to them sometimesas nuisance [nuisance or extraneous variables] variablesor extraneous variables.And they usually hinder the outcomeand hinder drawing causal conclusions.I'm going to give you some sample conclusions fromobservational-- from sample questions from observationalstudies. [Sample Questions] For instance,we can say [Is there any relationship between level

  • 02:38

    MAHTASH ESFANDIARI [continued]: of education and political affiliation] is therea relationship between level of education and politicalaffiliation? [Is there any relationship betweenparticipation in support groups and level of depression afterhaving a mastectomy?] Is there a relationship betweenparticipation in support groups and their level of depressionafter having a mastectomy?So therefore, we can come to the conclusion that we cannotconclude that level of education is the cause of whetheryou're going to be voting Democratic or Republican.

  • 02:59

    MAHTASH ESFANDIARI [continued]: And as I mentioned before, ethically speaking,we cannot randomly assign women to either participate or notparticipate in a support group after they go througha mastectomy.Therefore, we cannot conclude that support groups leadto lower depression after mastectomy.All we can say is that they are related to or to each other.I'm going to site a study on gender role and smoking

  • 03:23

    MAHTASH ESFANDIARI [continued]: behavior.The outcome variable-- I'm going to talkabout what's the outcome variableand what's the independent variable.Outcome variables in research sometimesare called dependent variable or they're alsocalled response [response variable] variables.They're synonyms.And the independent variable is sometimes called

  • 03:44

    MAHTASH ESFANDIARI [continued]: independent or explanatory [predictor variable]or predictor variable.In this particular study, smoking vs. not smokingis the outcome.And Type A behavior is the predictor.Now there are some variables, some confounders,that we control.We keep them fixed.And those now then we would call control variables.In this particular study, for example, age,

  • 04:06

    MAHTASH ESFANDIARI [continued]: level of education, and employment status,et cetera was looked upon as controlled variables.Now you wonder what's Type A personality?Type A personality is a person who's intensive.They're striving for achievement.They're competitive.They get easily provoked.They're impatient.They are over committed etc.And Type B personality is like kind of being relaxed,

  • 04:29

    MAHTASH ESFANDIARI [continued]: less stressed, flexible, emotional, expressive, toocasual, procrastinator, etc.And sometimes you have Type A/B personalitywhich is like something in the middle.Now this research showed that Type A behavioris significantly higher in males as compared to females.Therefore, we cannot say that being a male leads to Type A

  • 04:52

    MAHTASH ESFANDIARI [continued]: personality or Type A personality--or being a male is the cause of Type A personality.All we can say is that they are related to each other.And so this study implied that the womenwho have Type A personality are more likely to smoke.So there is a relationship there.

  • 05:14

    MAHTASH ESFANDIARI [continued]: It cannot be concluded that Type A behavior causes smoking amongwomen.All that we can say is that thereis an association between Type A personalityand smoking among women.So conclusions that we can draw from the gender rolestudy on smoking, we can say that womenwith masculine gender role orientation

  • 05:36

    MAHTASH ESFANDIARI [continued]: have a higher odds of being a smoker than not smoker.Now one of the things that is used highlyin research, whether in causal or non-causal studiesare odds ratios.And I'm going to talk about them a little bit.[odds ratios] Odds ratios is a very simple statistical conceptthat is used to clarify things.

  • 05:58

    MAHTASH ESFANDIARI [continued]: I'm going to give you an example of how it can beused in an observational study.[Education & Endorsing Women for Public Office]So, if you're looking at the relationship between the levelof whether people are college educated or non collegeeducation, and whether they endorse women for public officeor not.So we're looking at the relationshipof two categorical variables with two levels.

  • 06:18

    MAHTASH ESFANDIARI [continued]: They're college educated, they're not.They endorse women, they don't.And if you have, for example, of the college educated people,582 endorse, 111 do not, the odds ratio is 524.Non-college educated people, a total of 776 that endorseand 358 that do not.

  • 06:40

    MAHTASH ESFANDIARI [continued]: That ratio is 2.42.And then, the ratio of those two ratiosis called odds ratio, which is 2.41.And then you can conclude and say,the odds of endorsing women for public officeby college educated participants is 2.41 times

  • 06:60

    MAHTASH ESFANDIARI [continued]: higher than the non college educated participants.And this is a technique that is used very widely in medicine.For instance, someone can go to the doctor and they say,if you use such and such medication,the odds of having you drop your blood pressure is three timescompared to if you don't.[Experimental Studies]

  • 07:24

    MAHTASH ESFANDIARI [continued]: Experimental studies generally involve[Involve random assignment and control group]random assignment of subjects to experimental control.And this random assignment is cheap.They are prospective. [Are prospective].They happen in the future. [Involve hypothesis testing]They involve hypothesis testing.And usually they include [Include an interventionor experiment] an intervention, sometimes called an experiment.They are the same thing. [Make causal conclusions possible]

  • 07:45

    MAHTASH ESFANDIARI [continued]: And they make causal conclusions possible.So the major difference is that here, youcan make causal conclusions.In observational you can't.Here, you have random assignment,and there you do not.Now it is very important for us to beable to minimize the role of confoundersor the potential factors that are going

  • 08:05

    MAHTASH ESFANDIARI [continued]: to affect the outcome, so that we can make surethat our conclusions are OK.Now the thing is that random assignment isnecessary, but not sufficient.Because sometimes random assignmentdoes not take care of creating equivalence between the controland the experimental group.Suppose we want to examine the effect of a medication

  • 08:27

    MAHTASH ESFANDIARI [continued]: on lowering cholesterol.A potential confounder that affects cholesterolis the level of weight.Therefore, the question is, how do we take care of that?We can use blocking [blocking] as a means of controllingfor the confounder of weight.What are we going to do? [Schematic Showing Blockingby Weight] We're going to take peopleand divide them on their weight based on being obese,

  • 08:49

    MAHTASH ESFANDIARI [continued]: about average weight, and below average weight.And then what we do once we have those threeblocks of above average, middle, and below average,then we take people from there and we randomlyassign them to the experimental and the control group.And that's how we're going to do it.Let's say we have 120 patients whohave high cholesterol level.Then we're going to divide them into three groups of 40.

  • 09:12

    MAHTASH ESFANDIARI [continued]: And each of those 40 then are randomly assigned.20 go to take their medication and the other 20go to take the placebo.And then in medicine or in agriculture also,there are other means of lowering, you know, bias.And they're called blind and double blind.

  • 09:33

    MAHTASH ESFANDIARI [continued]: [blind experiment] What they mean by blindis that the experimenter-- so, say the doctor-- does notknow who's taking the placebo and who'staking the medication.And they will double blind [double blind experiment]is an example when neither the doctor nor the patientknows who's taking the medication.I can give you another example of double blindfrom agriculture.Let's say that you have 10 plots of land

  • 09:55

    MAHTASH ESFANDIARI [continued]: and you want to plant tomatoes on them.Then what you're going to do, you'regoing to randomly assign them to eitherhave a fertilizer or a placebo.And the person who is actually giving the fertilizerhas no idea which plot of land is getting the placeboand which plot of land is getting the actual fertilizer.And of course, you're going to ascertain

  • 10:17

    MAHTASH ESFANDIARI [continued]: that you're going to-- they-- the plots of land thatget a fertilizer and the placebo areequivalent with suspect to other variablesthat-- other confounders that couldaffect the growth of tomatoes.Like air or light, etc.

  • 10:40

    MAHTASH ESFANDIARI [continued]: Another way to lower bias is [baseline data]to collect baseline data.What do we mean by baseline data?For example, let's say we want to find outthe relationship between hypertensionand we want to find out the effect of a new medicationon lowering blood pressure.But we know hypertension is related to blood pressure,

  • 11:00

    MAHTASH ESFANDIARI [continued]: and therefore that's a confounder.So what we do, we're going to collect baseline dataon hypertension and ascertain that our groups-- the groupsthat are taking the medication and the groups thatare taking the placebo-- are similar at baselinewith respect to hypertension.Another example would be a study that Idid on the effectiveness of simulations

  • 11:24

    MAHTASH ESFANDIARI [continued]: on teaching hypothesis testing.One of the things that I was thinking aboutwas that prior knowledge of statisticalwas a potential confounder.Therefore, what I did, I matched the participantson their baseline knowledge of prior knowledge of statisticsbefore assigning them to the control and experimental group.So that's how you ascertain that the groups are pretty similar.

  • 11:47

    MAHTASH ESFANDIARI [continued]: Another thing is example we call--example we call the example of matching.[matching] What do we mean by matching?For example, let's say that-- again, Igo back to the same problem where I was telling youI want to match them based on their prior knowledgeof statistics.So let's say I have, just to be simple, to make it simple,let's say I have four students only.I give them a test of knowledge of statistics

  • 12:10

    MAHTASH ESFANDIARI [continued]: and I'm going to rank them.I'm going to say, OK, the higher score is 40.The next higher score is 30.And the next is 20, and the next is 10.So 40 and 30 become one pair.And then 20 and 10 become another pair.So I randomly send the 40 and 30,one of them to the experimental, one of them to the control.And I send the 20 and 10, one to the experimental

  • 12:31

    MAHTASH ESFANDIARI [continued]: and one to the control.So that way, I'm going to match thembased on their knowledge of statisticsbefore I assign them to the experimental and control group.Another example of matching is whenyou're looking at a weight loss program.So we want to find out if a couple of weight loss programs

  • 12:52

    MAHTASH ESFANDIARI [continued]: are equally good, two weight loss programs.So prior weight is an important confounder.So what do I need to do?I'll pair them up, based on their weight.And I randomly assign each of them, one to the control,one to the experimental.Then that way, the average weightin the two groups, experimental and control,become almost similar before they actually

  • 13:14

    MAHTASH ESFANDIARI [continued]: start the intervention.So I'm going to talk about what we call a pretest[pretest-posttest design] posttest design.That means, like, let's say I have 40 people.And I'm going to randomly assign them to the experimentaland to the control group.By pretest/posttest, it means I test them beforeand I test them after the intervention is finished.

  • 13:34

    MAHTASH ESFANDIARI [continued]: And that is called a pretest/posttest design.I'm going to talk about a study herethat was done on the effect of St. John's Wort on ADHD.And this study, you should probablybe able to find it in the literaturejust if you google St. John's Wort

  • 13:55

    MAHTASH ESFANDIARI [continued]: and hyperactivity disorder, you should be able to find it.This was a very interesting study in the sensethat they wanted to look at the efficacyof an herbal medica-- on an herbal thing for lowering ADHD.So what they did, they had 57 participants in the 6-

  • 14:16

    MAHTASH ESFANDIARI [continued]: to 17-year-old age range.And they divided them randomly into two groups.One took St. John's Wort, another one took the placebo.And then it was double blind in the sensethat neither the parents of the kids nor the doctorsknew who's taking the medication and who's taking the placebo.

  • 14:40

    MAHTASH ESFANDIARI [continued]: And then what they did, they observed themon a hyperactivity scale from baseline up to eight weekslater which means baseline week one, week two, week three, weekfour, up to week eight.And then, once you look at the article,you're going to see that [Mean ADHD RS IV TotalScores at Each Study Visit] thereis a plot that shows the mean all the way from week onethrough week eight.

  • 15:01

    MAHTASH ESFANDIARI [continued]: And there's a confidence interval,which is an error bar for all the weeks.And the mean of the control group has a hollow dot.And the mean of the experimental group has a black dot.And what happens, those dots are very similar.The error bars are very similar.Which means that St. John's Wort over timehad exactly the same effect as the medication.

  • 15:22

    MAHTASH ESFANDIARI [continued]: And so the conclusion was that St. John's Wort was noteffective in treatment of ADHD.Now this is an example of not just the pre-post,but it's a repeated measure.Because it was repeated so many times.But if you look at the baseline when it was a baseline,and when it was done at week eight,

  • 15:43

    MAHTASH ESFANDIARI [continued]: you can consider the baseline pre, and week eight as post.And when statistical analysis wasdone on the before and after of week eight,nothing significant was found.So the conclusion that we are goingto draw from the St. John's Wort studywas that the drug and the placebo grouphad similar means and confidence intervalsfrom baseline all the way to the end of the study.

  • 16:07

    MAHTASH ESFANDIARI [continued]: Now one of the topics that's veryimportant in experimental design is the conceptof internal validity.[Internal Validity.When an experiment is internally valid,we can be certain that the independent variable (e.g.method of teaching, medication, etc.) caused the outcomeof the study (e.g. learning of math concepts, improving sleep,etc.)] Internal validity means that the result that we draware attributable to the experiment and notthe confounders.You can say the result are because of method of teaching.The results are because of the medication.And not because of the confounders.

  • 16:30

    MAHTASH ESFANDIARI [continued]: We enhance internal validity when our subjects are randomlyassigned to experimental and control.And the other things that we do to enhance internal validitywas stuff we talked about.It was blocking, matching, blinding,using control, and all of that stuff,that help to minimize the error and maximize

  • 16:51

    MAHTASH ESFANDIARI [continued]: the internal validity of your study.Now there is a topic that I'm notgoing to have time to discuss in this lecture.But sometimes the confounders cannot be controlledexperimentally.But we can control them through statistical control.And that's called using covariance-- or covariance

  • 17:12

    MAHTASH ESFANDIARI [continued]: analysis, analysis [analysis of covariance]of covariance.But this would be the topic of another lecture.[External Validity.Relates to generalizing experimental resultsto groups that were not included in the study] External validityrelates to generalizing the experimental resultsto groups that are not included in your study.Factors that hinder external validity[Factors That Hinder External Validity]are lack [Lack of representation from diverse populations]of representation from diverse populations.

  • 17:34

    MAHTASH ESFANDIARI [continued]: For instance, if you just have a white population,you cannot you know, generalize it to African American or Asianor whatever.[Lack of randomness] Lack of randomness creates a problem.[Attrition] Attrition.It means when participants need to drop out,that's a major problem.Sometimes in some of the people studies I have conducted,I start with 50.

  • 17:55

    MAHTASH ESFANDIARI [continued]: I end up with 50.But not all of those 50 match.Sometimes only 20 of them match.So that another issue.[Quasi-Experimental Designs]Now that to have done-- we have dealtwith observational experimental studies,and explained the difference between them,

  • 18:17

    MAHTASH ESFANDIARI [continued]: I'm going to move on to the third class of design whichis called quasi-experimental design.And quasi means as close as it can getto real experimental design.But it's not exactly experimental design.The major difference is that in experimental--in true experimental designs, we randomly

  • 18:40

    MAHTASH ESFANDIARI [continued]: assign the subject to the experimental and the controlgroup.In quasi-experimental designs, [Quasi Experimental Designs,Subjects are not randomly assigned to experimentaland control group] this random assignment cannot happen basedon students.It happens [The treatment is randomlyassigned to the experimental and control group]based on the group.For instance, let's say I have two classes.And I want to use clickers in one of them.So I go toss up a coin.I say, if I get a head, then my 10 o'clock class

  • 19:02

    MAHTASH ESFANDIARI [continued]: is going to be my experimental and my 12 o'clock classis going to be in my control.And with this type of studies, [Needto be cautious with causal conclusions]you need to be a little more cautiouswith causal conclusions.And they are [Mostly used with intact groups suchas classrooms] basically used more in intact groups,means groups that cannot be touched, like classrooms.An example of a quasi-experimental design

  • 19:22

    MAHTASH ESFANDIARI [continued]: from my work.In a study I wanted to examine the effect of law relatededucation program on the attitude of sixth gradestudents toward police and authority.And the study was conducted for one semesterin a school with four [Schematic of pre post quasiexperimental design] sixth grade classes [Four sixth gradeclasses randomly assigned to] with about 40 to 50 studentseach.So what I did, I had to randomly assign [Experimental classes

  • 19:44

    MAHTASH ESFANDIARI [continued]: one and four] the intervention [Control classes two and three]to two out of the four classes.I cannot send students randomly all over the place.Because students belong to an intact group,and they need to stay there.The experimental group studied the regular social studiesbook, plus cases.They read in law related education.While the control group only studied the social studies

  • 20:07

    MAHTASH ESFANDIARI [continued]: book.After the classes were randomly assigned to the experimentaland the control group, the students [Pre, Post, Pre, Post]were pretested first week of class and post tested last weekof class on attitude toward police authority.And so you start with four classes.You send two of them to the experimental.

  • 20:28

    MAHTASH ESFANDIARI [continued]: You send two of them to the control randomly.Then you pre-post the control.You pre-post the experimental.[Conclusion]If I wanted to summarize, I wouldsay that [Summary, Observational studiesinvolve no random assignment to subject or control groupand only allow for relationship conclusions]observational studies involve no random assignment of subjects

  • 20:48

    MAHTASH ESFANDIARI [continued]: to experimental and control groups.Even if they do have a control group, it is not random.And then all you can do is you canmake association or relationship conclusionsbased on those studies.And confounders play a major role there.[Experimental studies are more rigorous,have random assignment, and allow for causal relationshipconclusions] Experimental studies are much more rigorous.

  • 21:10

    MAHTASH ESFANDIARI [continued]: They involve random assignment of subjects to groups.They have a control group.They allow you to draw causal conclusions.And usually you can use additional stepslike blocking and matching, and you know, stuff like that.Collecting baseline data, collecting data over time,

  • 21:31

    MAHTASH ESFANDIARI [continued]: like repeated measures, in order to ascertainthat the results that you get arethe cause of the experiment.And in educational research, or in other type of research whereyou have intact groups that you cannot separate,then you go from experimental design to what we callquasi-experimental design.

  • 21:52

    MAHTASH ESFANDIARI [continued]: [In quasi experimental design, random assignmentis not based on the subject] In quasi-experimental design,the random assignment is not based on the subject.The random assignment is assignment of interventionto the particular group.That is the end of this lecture for today.

Video Info

Publisher: SAGE Publications Ltd

Publication Year: 2017

Video Type:Tutorial

Methods: Experimental design, Quasi-experimental designs, Observational research

Keywords: ADHD; attitude formation; classrooms; gender roles; placebos; police; practices, strategies, and tools; Smoking ... Show More

Segment Info

Segment Num.: 1

Persons Discussed:

Events Discussed:

Keywords:

Abstract

Professor Mahtash Esfandiari identifies and describes three main types of experimental design: observational studies, real experimental designs, and quasi-experimental designs. She provides several examples from actual research projects.

Looks like you do not have access to this content.

Introduction to Experimental Design

Professor Mahtash Esfandiari identifies and describes three main types of experimental design: observational studies, real experimental designs, and quasi-experimental designs. She provides several examples from actual research projects.

Copy and paste the following HTML into your website