Skip to main content
Search form
  • 00:02

    ALISON GIBBS: In this video, we'llintroduce some of the ideas behind using experimentsto collect data.Experiments are the gold standard,allowing us to make causal conclusions.They allow us to consider questions such as,if I observe fewer infections in a group that wasn't vaccinatedversus a group that was, can I conclude

  • 00:23

    ALISON GIBBS [continued]: that it was the vaccine that caused lower infection rate?Or could it be something else?Or say, a new teaching method results in better learningoutcomes, can I definitely attribute the improvementto the new method?We'll start by introducing some of the vocabulary thatis used in experiments.The outcome-- or response variable--

  • 00:45

    ALISON GIBBS [continued]: is the variable that we are interested in, measuredon each of the individuals or entitiesparticipating in the study.It is sometimes called a dependent variablebecause we are typically interested in how it dependson the values of the other variables, whichare the explanatory variables.Explanatory variables-- or predictors or independent

  • 01:07

    ALISON GIBBS [continued]: variables--are variables that we believe affect the response.The key feature that separates an experimentfrom an observational study is that in an experiment,the researcher manipulates the explanatory variablesto see the effect on the response variable.

  • 01:28

    ALISON GIBBS [continued]: An example of an experiment that we've already seenis the PATRICIA study to examine the efficacy of a vaccineat preventing infections with HPV.In the PATRICIA study, we were interested in whether or notthe subject in the study acquired an infection so thatis our response variable.

  • 01:48

    ALISON GIBBS [continued]: Response variables can be categorical or quantitative.In this situation, we have a categorical response variablewith two categories.Yes, the subject acquired an infection in the study periodor no, they did not.In this example, the explanatory variable is also categorical.

  • 02:09

    ALISON GIBBS [continued]: We believe that getting the vaccineaffects the chance of getting an infection.And the explanatory variable is the vaccination statuswith possible values, yes or no.In an experiment, the investigatorobserves how a response variable behaveswhen he or she manipulates one or more explanatory variables.

  • 02:34

    ALISON GIBBS [continued]: Since there are only a finite numberof values of the explanatory variablethat the researcher can possibly study,explanatory variables in experimentsare typically categorical and arecalled factors in this setting.The values of a factor that are studied in an experimentare its levels.In our HPV vaccine example, the factor

  • 02:56

    ALISON GIBBS [continued]: is vaccination status with levels, yes or no.A particular combination of values for the factorsis called a treatment.When there is only one factor-- as in the HPV study--the treatments are the levels of the factors.But it's also possible that an investigatormight want to simultaneously manipulate

  • 03:17

    ALISON GIBBS [continued]: two explanatory variables to see the effect on the response.As an example, imagine a study comparing two drugs--we'll just call them drug A and B--each at two different doses, high and low.Then in this situation, we have two factors,each with two levels, but four different combinations-- drug

  • 03:39

    ALISON GIBBS [continued]: A in high dose, drug A in low dose, drug B in high,and low dose.So in this case, there are four treatments.In experiments, the participating subjects,or individuals, are called experimental units.Sometimes, treatments are given to groups of people or things,like all of the students in a class receiving a new teaching

  • 04:02

    ALISON GIBBS [continued]: method.Or all of the plants in one pot receivinga new type of fertilizer.The experimental unit is then, notthe individual student or plant, but the whole class or pot.Using this vocabulary, the key feature thatseparates an experiment from an observational study again,

  • 04:23

    ALISON GIBBS [continued]: is that in an experiment, treatmentsare imposed on experimental units by the researcherso that the effect of the treatments on the responsecan be seen.Extraneous factors are factors thatare not of interest in the current study,but are thought that they might affect the response.They need to be controlled, to rule out the possibility

  • 04:46

    ALISON GIBBS [continued]: that the extraneous factor is causing any observeddifferences in the response.To control for an extraneous factor,we have a couple of different options.The first option is to hold it constant.Say we think that the HPV vaccine might work differentlyon males and females.

  • 05:06

    ALISON GIBBS [continued]: We can then make the decision to say, study only females.If we think it might work differently in different agegroups, we could choose to study only one particular age group.This limits the generalizability of the study,but it also eliminates the potentialof having the extraneous variable confound the results.

  • 05:30

    ALISON GIBBS [continued]: The second method of controlling for extraneous factorsis to use blocking.A block is a group of experimental unitsthat are similar in the extraneous factor.And all treatments are randomly assigned to experimental unitswithin each block.If we wanted to study the effects of the vaccineand we wanted to include different age groups which

  • 05:52

    ALISON GIBBS [continued]: might have differential response,we can treat age group as a blockand randomly assign treatments within each age group.So in each age group, we have subjectswho both receive the vaccine and do not receive the vaccine.But what about extraneous factorsthat can't be controlled?And even more of a concern, what about the extraneous factors

  • 06:15

    ALISON GIBBS [continued]: that haven't been identified just because wemissed something or perhaps, because there arepotential extraneous factors--maybe a genetic mutation--that hasn't yet been studied so we don't knowthat it's an extraneous factor?The solution is to use randomisation.By randomly assigning individuals

  • 06:35

    ALISON GIBBS [continued]: to treatment groups, we can ensurethat any differences that exist between the groups,in any possible extraneous variables,are just due to chance.And the fact that we expect to seesome differences due to this chance variation,is part of our statistical model.After randomisation, when we average out

  • 06:57

    ALISON GIBBS [continued]: this chance variation, the treatment groupsare essentially the same.Once we have a randomized experimentto compare treatments, we have a studyfor which, if we observe differences among the treatmentgroups, we can conclude that it wasthe different treatments that causethe difference in the response.The idea here is that we've eliminated

  • 07:19

    ALISON GIBBS [continued]: any other differences between the groups.So if the response variable is different,the only explanation is the different treatments,and cause and effect conclusions can then be made.There are three fundamental principlesof experimental design.Control the identified extraneous variables

  • 07:39

    ALISON GIBBS [continued]: by blocking or holding them constant.Randomly assign experimental units to treatment groups.And use replication.We haven't talked about replication yet,and here, I'm not talking about replicating an experimentto check whether a result found in one studyalso holds when you do another study.

  • 08:01

    ALISON GIBBS [continued]: While this is an important part of the scientific process,we're currently only talking about single studies.So what I mean by replication here,is that we need to apply each treatment to morethan one experimental unit.Having replicates allows the researcherto estimate variability in the measurement of the response,

  • 08:21

    ALISON GIBBS [continued]: which we can't do if we only have oneobservation for each treatment.And ensures that the treatment groups are more comparablein values of the extraneous factors,by having the opportunity to havedifferent values of the extraneous factorswithin each treatment group.The word, control, also has another meaning in experiments.

  • 08:43

    ALISON GIBBS [continued]: Experiments often have what we call, a control group.If the goal of an experiment is to showthat a treatment affects the response,you need to have at least one other group for comparison.It's possible that the simple actof studying an experimental unit causes it to change.Studying at least two groups undergoing the same experiment,

  • 09:06

    ALISON GIBBS [continued]: allows us to compare them under the same circumstances.A comparison group in the study could get a different treatmentor no treatment.It is called a control group if it either does notreceive a treatment or receives the current standard treatmentif there is one.The control group is itself, a treatment groupand if there is only one factor, control

  • 09:27

    ALISON GIBBS [continued]: is considered a level of the factor.So the only way to establish a causal relationshipis to carry out a randomized controlled experiment.So why would anyone carry out a study that's not an experiment?The answer is simple, it's just notalways possible for ethical or practical reasons.

  • 09:51

    ALISON GIBBS [continued]: If we want to study the effect of smoking on human mortality,we can't randomly assign some people to smoke a pack a dayand some people not to--that would be unethical.For studies on humans, we need to have sufficient doubtabout the benefit of a treatment,that we do not feel that we are unfairly exposing some

  • 10:11

    ALISON GIBBS [continued]: of the subjects to an inferior treatment in orderto carry out in experiment.If we're interested in how the explanatory variableGDP affects the response of life expectancy for countries,we can't manipulate the GDP of a country.So carrying out an experiment is not practically possible.

  • 10:31

    ALISON GIBBS [continued]: Let's look at one more example of an experiment.This story is slightly adapted from a project presentedby two Montreal students at the 2011 Canada Wide Science Fair.This project was the winner of the Statistical Societyof Canada award for Excellence in the Use of StatisticalMethodology--which they won for their excellent exampleof a well-designed experiment.

  • 10:53

    ALISON GIBBS [continued]: The students who are interested in ischemic preconditioning,which is a technique to create resistance to loss of oxygenthrough loss of blood supply to tissues.Ischemic preconditioning works by applying brief episodesof restricted blood flow in orderto protect against damage from a subsequent longer episode.

  • 11:13

    ALISON GIBBS [continued]: The students were interested in whether a similar techniquecould be used to improve sports endurance.They provided ischemic preconditioningusing a blood pressure cuff.They applied a pressure of 20 poundsto some of their subjects and for comparison,minimal pressure, nominally zero pounds, to other subjects.

  • 11:33

    ALISON GIBBS [continued]: They also compared applying pressurefor 10 minutes versus 20 minutes.So there were a total of four treatments.The four combinations of the two pressuresand the two lengths of time applied.Each treatment was applied to 10 teenage malesso there's our replication.Since there are four treatments, therewere 40 participating teenagers, and these

  • 11:55

    ALISON GIBBS [continued]: are the experimental units.They were chosen to be of similar athletic ability.After application of the treatment,the length of time the teenagers could stay in a wallsquat position was measured.This is the response.Note the use of a control group here.The students needed to know if ischemic preconditioning worked

  • 12:17

    ALISON GIBBS [continued]: so they gave a sham treatment of zero poundsto some of their experimental units,some of whom got zero pounds for 10 minutes and somefor 20 minutes.Moreover, to control for extraneous factors,only a specific group of people were studied.Namely, male teenagers of similar athletic ability,

  • 12:37

    ALISON GIBBS [continued]: chosen because they participate in sports at school.Although this wasn't done, another waythey could have controlled for extraneous factorswas to use blocking.For example, they could have enrolled both athletesand non-athletes in the study and assign the four treatmentsto subjects within each of the athletic groups.

  • 12:58

    ALISON GIBBS [continued]: Another of the principles of experimental designis randomization.And the 40 experimental units were randomlyassigned to which of the four treatments they received.This ischemic preconditioning experimentalso illustrates some other characteristicsof excellent design, which should be usedwhen appropriate and possible.

  • 13:19

    ALISON GIBBS [continued]: The students used blinding.Blinding-- if it can be used in an experiment--reduces the potential for bias since peopledon't know if a treatment is in place or not.Subjects can be blinded meaning theydon't know which treatment they received, the person measuringthe response can also be blinded if he or she does not

  • 13:40

    ALISON GIBBS [continued]: know which treatment was given to which participant.Experiments can be single-blind meaning only one type of blindwas used.Or double-blind, if both types of blinding were used.The ischemic preconditioning experiment was single-blind.Subjects did not know which treatment they received.And they used the placebo.

  • 14:02

    ALISON GIBBS [continued]: People often show change when participating in an experiment,whether or not they receive a treatment.This is known as the placebo effect.For this reason, the control groupis often given a placebo which is somethingthat is identical to the treatment receivedby the treatment groups, except that itcontains no active ingredients.

  • 14:23

    ALISON GIBBS [continued]: So the subjects aren't aware of whether or notthey're receiving a treatment.In our ischemic preconditioning experiment,the placebo was the application of zero pounds pressureto some of the experimental units.Randomisation placed two key rules in data collection,this table summarizes those two rules.

  • 14:43

    ALISON GIBBS [continued]: By using random sampling, we get a representative sample,and ensure that we can generalize our resultsto a larger group or population free of any selection bias.Later on in our inferential procedures,we'll use a statistical model thataccounts for the random variation thatcauses our sample to differ from other samplesthat we could have possibly chosen

  • 15:03

    ALISON GIBBS [continued]: in another random selection.Randomization also plays a key role in experiments.Randomly assigning experimental units to the treatmentsthey receive eliminates the effects of extraneous factors,allowing us to make causal conclusions.While the extraneous factors may varyfrom treatment group to treatment group somewhat,

  • 15:24

    ALISON GIBBS [continued]: randomisation ensures that these differencesare due to chance variation only and notany systematic difference that couldconfound our interpretation of the effect of the treatment.So if we want to make causal conclusions that wecan generalize to an entire population,we need to randomize both in our selectionof our experimental units and in assigning treatments

  • 15:46

    ALISON GIBBS [continued]: to experimental units.

Video Info

Series Name: Understanding Data

Publisher: Alison Gibbs and Jeffrey Rosenthal

Publication Year: 2013

Video Type:Tutorial

Methods: Experimental design

Keywords: covariates; practices, strategies, and tools

Segment Info

Segment Num.: 1

Persons Discussed:

Events Discussed:



Alison Gibbs explains the various types of experimental variables and how they affect research design. Gibbs demonstrates these principles by analyzing a sample experiment.

Looks like you do not have access to this content.

Data Collection - Experiments

Alison Gibbs explains the various types of experimental variables and how they affect research design. Gibbs demonstrates these principles by analyzing a sample experiment.

Copy and paste the following HTML into your website