[Learning to Design an Experimental Study]
THERESA DE LOS SANTOS: Hello, my name is Theresa de Los Santos.[Theresa de los Santos, Assistant Professor]I am an assistant professor of communicationat Pepperdine University in Malibu, California.In my personal research on the effects of emotional framingof news stories, I primarily conductquantitative experiments and content analyzes.In this tutorial, I will cover the purpose
THERESA DE LOS SANTOS [continued]: of experimental research, the key elementsof a true experiment, and the main benefits and limitationsof using this research method.As you're watching this video, keep these key pointsabout experimental research in mind.Key point number one--the central purpose of experimental researchis to test hypotheses of cause and effect.
THERESA DE LOS SANTOS [continued]: Key point number two--three key elements of all true experimental designsare manipulation of the independent variable, control,and random assignment of participants.And key point number three--experimental researchers need to beaware of several internal validity threats.
THERESA DE LOS SANTOS [continued]: Expanding on the first key point,one of the strongest ways for a researcherto be able to make causal claims is to conduct an experiment.Unlike survey researchers, experimentersintroduce variables, called manipulating variables,that were not present to begin with.They then can assess the impact of these variables.
THERESA DE LOS SANTOS [continued]: Survey studies try to identify associations among variables,but experimenters add variables to an existing settingto see what effects might be found.Thus if the guiding thought for survey researchis, let's ask people, the guiding thoughtfor experimental research is, let's do something
THERESA DE LOS SANTOS [continued]: and see what happens.Remembering the three general rulesfor establishing causality can helpto understand the purpose of experimentsand move us to defining the three key elementsof true experimental designs.First, variables must be related.This one is easy.Before we can show that we have a causal relationship,
THERESA DE LOS SANTOS [continued]: we have to show that we have some type of relationship.Second, we must establish time order.This means the independent variable has to come firstor the experimental stimulus must come before the effect.Keep this one in mind.And finally to show we have a causal relationship,
THERESA DE LOS SANTOS [continued]: we have to rule out other explanations or causes.It must be shown that a different variable is notcausing the change in the two variables of interest.This is why elements of control and random assignmentare very important.And finally, to show we have a causal relationship,we have to rule out other explanations or causes.
THERESA DE LOS SANTOS [continued]: It must be shown that a different variable is notcausing the change in the dependent variable of interest.This is why elements of control in random assignmentare very important.This takes us to the second key point of this tutorial.There are three key elements to a true experiment.
THERESA DE LOS SANTOS [continued]: In true experiments, researchers manipulatethe independent variable while attemptingto maintain control over all other factors thatmay affect the results.How exactly is this done?Let's break it down.Manipulation of the independent variablemeans the researcher divides the variableinto different conditions or treatment groups.
THERESA DE LOS SANTOS [continued]: To make this clearer, let's explore an example.Let's say a researcher wanted to testwhether exposure to different frames of a news issuecaused people to spend more time reading about it.The researcher might have three conditions--a fear-frame group, a hope-frame group,and a neutral-frame or control group.
THERESA DE LOS SANTOS [continued]: The independent variable in this study then is frame type.Remember those three frames I just told you, hope, fear,and neutral.And the dependent variable is reading time, probablyin seconds.Participants in each condition would onlybe given their assigned article to read.The researcher would then compare mean scores
THERESA DE LOS SANTOS [continued]: on the dependent variable measure across conditionsto see if differences exist.The control group I just mentionedis not the same as experimental control.Control groups are groups not exposedto any experimental variables and are used as a baseline
THERESA DE LOS SANTOS [continued]: to measure any changes in groups that are exposedto experimental variables.Experimental control, on the other hand,means keeping all independent variable and all other studyconditions the same except for the independent variableunder investigation.This is the second element of all true experimental designs.
THERESA DE LOS SANTOS [continued]: For the news frame study example we started earlier,this would include environmental conditions as well asother factors of the independent variable,including the length of the articles,the facts included in the articles,and how the articles are delivered to participants,maybe whether they're on a computeror whether they're printed in red.
THERESA DE LOS SANTOS [continued]: All of those sorts of other factorsof the independent variable would be held constantor controlled or kept the same.When all factors are controlled and kept constant,you can conclude that any measurable differencesin your results are due to changesin the manipulated variable.This allows science to isolate the effects of treatment.
THERESA DE LOS SANTOS [continued]: When we get rid of nuisance variables,all groups are equal except for the manipulation.At this point, we have two of the three key elementsof a true experiment.The third is random assignment of participants to conditionsof the independent variable.This means participants must have an equal chanceof ending up in each condition.
THERESA DE LOS SANTOS [continued]: The reason why this is important is because it probabilisticallymakes the groups equal before manipulation.It involves using the role of chance to balance groups.Without random assignment, you would notknow if the groups were equivalentbefore a study began.The quest for high internal validity orients researchers
THERESA DE LOS SANTOS [continued]: to design experiments in which treatmentmanipulations can be tightly controlledand extraneous variables or those variablesthat compete with an independent variable that, remember,is manipulated in explaining the outcome of a studyare minimized.This takes us to key point number three.While survey research and sampling
THERESA DE LOS SANTOS [continued]: are most concerned with external validity or generalizability,in terms of validity, the traditional importanceto experimenter's is establishinghigh internal validity.Internal validity is concerned with the rigor, and thusthe degree of control of the study design.
THERESA DE LOS SANTOS [continued]: Specifically, internal validity is evidencethat what a researcher did in a studycaused what they observed to happen.It is evidence that changes in the independent variable or xare responsible for observed variationin the dependent variable or y.Without internal validity, results are uninterpretable.
THERESA DE LOS SANTOS [continued]: When researchers utilize less powerfulpre-experimental designs or don't conduct a true experimentproperly, alternative explanations or threatsto internal validity become possible.Threats to internal validity are mattersthat jeopardize an experiment.
THERESA DE LOS SANTOS [continued]: The first threat to internal validity is history.These are specific events in the environmentexternal to a study that happen and affect its outcome.For example, following the news-framing example,a new development on the topic being studiedcan be widely reported before or while a study isbeing conducted.
THERESA DE LOS SANTOS [continued]: If the event changes variable scores overall or changesthe scores of one group more than another,it threatens the researcher's ability to say x caused y.The second threat to internal validity that you shouldbe aware of is maturation.This thread particularly focuses on study participantswho can change during the course of an experiment.
THERESA DE LOS SANTOS [continued]: During short-term studies, participantsmay go from being in a good mood at the beginning of a studyto a bad mood.They also may become fatigued or tiredor grow suspicious of the study.In more long-term studies lasting days, months,or even years, participants will grow older in their skill setand might change in their attitudes or ways.
THERESA DE LOS SANTOS [continued]: The question is for researchers, how confidentare you that observed changes in the dependent variableare due to the treatment and not changes in your participants?The next threat is testing effects.This threat is an issue in pretest, post-test experimentalresearch design.
THERESA DE LOS SANTOS [continued]: You might have noticed that although they provideuseful and sometimes mandatory information on change,a pretest is not a required element for true experiments.When a pretest is included, the first testmay prime or make participants more aware on the second test.Thus differences in scores between groups
THERESA DE LOS SANTOS [continued]: on the dependent variable may be due to pretest sensitizationrather than the independent variable manipulation.This becomes a more viable threatas the time between a pretest and post-test is shortened.The next threat is instrumentation.This threat arises when actual physical measuring
THERESA DE LOS SANTOS [continued]: instruments-- for example a stopwatch--or observers are used in an experiment.How instruments are calibrated or how observers performmay change over the course of a study and account for results.Next we have mortality or attrition.Especially over the course of long-term studies,
THERESA DE LOS SANTOS [continued]: participants frequently drop out,or as the name of this threat implies, die off.Reasons for participants dropping out of a studycan vary from them moving to becomingtoo busy to participate or growingtired with the study, topics, or tasks.This is particularly a problem if participants don't drop outrandomly and they drop out of one group or condition
THERESA DE LOS SANTOS [continued]: more than another.When dropout rates are higher in one group compared to another,it becomes more difficult to concludethat any observed differences aredue to the treatment and not the dropouts.Next we have statistical regression, alsocalled regression to the mean.This thread deals with participantswho have extreme scores, either high or low,
THERESA DE LOS SANTOS [continued]: on a dependent variable measure.Due to natural performance and also chance,when scores are particularly high or low,there is a tendency for scores to move or regresstowards the mean or middle.So in pretest/post-test designs, extreme scores on a pretest are
THERESA DE LOS SANTOS [continued]: not likely to occur on a second testing.Thus if the sample you are dealing with in your studyis likely to include people with extreme views or scoreson a dependent variable, you needto carefully consider this threat to internal validity.The next threat deals with participants.It's the Hawthorne effect.
THERESA DE LOS SANTOS [continued]: Participants behave differently when theyknow they are being studied.This reactive effect may cause participantsto not behave normally and thus cloud whether a treatmentor participant's modified behaviors or attitudes areresponsible for observed differences.And finally we have experimenter bias.
THERESA DE LOS SANTOS [continued]: Experimenter bias is the last threat to internal validitythat I will discuss in this tutorial.This broad threat includes any typically unconsciousexperiment or behaviors or expectationsthat may lead to changes in the dependent variable that are notdue to treatment.We do not want to be misled by results
THERESA DE LOS SANTOS [continued]: that are due to an experimenter's knowledge,beliefs, or simply enthusiasm.The bottom line is that experimental researchersshould be aware of and consider threats to internal validityas they design their studies and again when theyanalyze their data and draw conclusions.To summarize, this tutorial breaks down
THERESA DE LOS SANTOS [continued]: the purpose and key elements of experimental researchcompared to other research methods.In order to be a true experiment and increase internal validityan experiment involves the manipulationof the causal independent variable while controllingall other factors and random assignment of participantsto independent variable conditions.
THERESA DE LOS SANTOS [continued]: One major limitation of experimental researchis that studies are artificial in two ways.They are typically conducted in artificial laboratory settingsand may create situations that do not represent real life.While this helps to ensure internal validity,it is often at the expense of external validity.
THERESA DE LOS SANTOS [continued]: The principal advantage of experimental researchis that it provides the opportunityto identify cause and effect relationships.Like when selecting all research methods,the method should match your question or predictionand the goal of the research.Hopefully this tutorial will help you figure outif an experiment is the right method for your research needs.
Publisher: SAGE Publications Ltd
Publication Year: 2018
Segment Num.: 1
Theresa de los Santos explains experiential research and how to use experimental research effectively.
Looks like you do not have access to this content.
Theresa de los Santos explains experiential research and how to use experimental research effectively.