- 00:05
[An Introduction to Experimental Designs I]

- 00:11
MAHTASH ESFANDIARI: My name is Mahtash Esfandiari.I'm on the faculty at UCLA Department of Statistics.And I'm also the Director of the Statistical ConsultingCenter in my department. [Dr. Mahtash Esfandiari, Professorof Statistics] In this video, I'mgoing to be talking about the statistical concepts thatare needed in order for you to be able to followdifferent experimental designs including observational,

- 00:33
MAHTASH ESFANDIARI [continued]: experimental, and quasi-experimental.And I'm going to go over case studies.[Basic Statistical Concepts for Experimental Designs]Before I do that, I want to talk about some basic statisticalconcepts that we need to know.The first concept is the concept of population statistics.

- 00:55
MAHTASH ESFANDIARI [continued]: Population includes all of the elements in a data set.For example, we can say all of the people whoare eligible for the next presidential election,all the patients diagnosed with hypertension,all of the seniors in Los Angeles Unified School Districtor any other school district.And then, we're going to talk about the concept of sampling

- 01:16
MAHTASH ESFANDIARI [continued]: statistics.A sample is a fraction of the population.The sample can be random.It can be a sample of convenience like the studentsenrolled in your class.Or the sample can be people who volunteeredto participate in your study.We show-- we designate population characteristics

- 01:39
MAHTASH ESFANDIARI [continued]: with what we call a parameter.And we show the parameters with Greek letters.They're usually unknown.We try to estimate them by using sample values,coming up with confidence intervals, et cetera.The concepts that-- one concept we'd want to talk aboutis descriptive statistics.Descriptive statistics deals with computationof certain statistics like percentages, means, medians,

- 02:05
MAHTASH ESFANDIARI [continued]: standard deviations about the sample.For example, if I have a class of 50 students,I do their average height, weight,the percentage of females-- that is a statistic.Inferential statistics deals with generalizing that datafrom the sample to the population.And I'm going to be talking about some Greek letters thatdisplay parameters in the population.

- 02:27
MAHTASH ESFANDIARI [continued]: Mean we show with mu.Variance we show with sigma squared.Standard deviation we show with sigma.Slope we show with beta.Intercept we show with alpha and so on.Now, when we want to describe the sample,we use different symbols.We use x bar for the mean.We use s squared for the variance.We use s for standard deviation, b for the slope,

- 02:49
MAHTASH ESFANDIARI [continued]: and a for the intercept.I'm going to start with the simplest experimental design.It's called a one-shot case study.And it's not even an-- it's not even consideredan experimental design.We call it a pre-experimental design.What you do, you have a single group.It undergoes a treatment or an intervention.And after the treatment is completed,

- 03:11
MAHTASH ESFANDIARI [continued]: you measure something about them.So basically then, as an example,suppose you want a short course.You have a short course.You want to teach the participantabout the theory and right methods of conflict resolution.And after the course, you want to measure their attitudeabout the right methods of conflict resolution.

- 03:32
MAHTASH ESFANDIARI [continued]: And this is like a one-shot case study.[Null & Alternative Hypothesis]In statistics any time you want to makea decision about anything, we aregoing to be talking-- in inferential statistics--we're going to have an alternative-- we'regoing to have a null hypothesis aand what we call an alternative hypothesis.

- 03:52
MAHTASH ESFANDIARI [continued]: And definitely always what we do,we examine the null and we make a conclusionabout the alternative.And you have to remember that both the nulland the alternative are about parameters.You cannot make a null hypothesis about a samplestatistic because you already know what it is.For instance, suppose we are conducting a study

- 04:14
MAHTASH ESFANDIARI [continued]: where we want to find out whether showing pictures isgoing to help kids with the teaching of new vocabulary.Our null hypothesis can be that-- well, they'regoing to be learning seven new words.So mu is going to be equal to 7.That is our H0.Our alternative hypotheses is that well, wehope they learn more than seven words.

- 04:35
MAHTASH ESFANDIARI [continued]: So we say H of a is mu larger than 7.We call this a one-tailed test because we say it's larger.And so when we say it, and in hypothesis testing,it always helps to say it in symbolsand then say it in words within context.So we're going to say the average number of vocabularythat 4-year-olds learn by showing them

- 04:58
MAHTASH ESFANDIARI [continued]: pictures is 7 out of 10.And the alternative hypothesis isthat the average number of words that 4-year-oldslearn by showing pictures is at least 7 out of 10.And saying things within context is a big deal in statisticsbecause people have to follow what your research is about.So let's say we choose a sample of 40 4-year-olds.

- 05:20
MAHTASH ESFANDIARI [continued]: We teach them 10 new words.And at the end of the intervention,we test them on how many correct words they learned.The first step to testing the null hypothesisis calculation of the sample mean.So what you need to do is you need to add all the 40 scores.And then you need to divide by 40.

- 05:40
MAHTASH ESFANDIARI [continued]: So your x bar is going to be the sum of x from 1 to 40divided by 40.That is your sample mean.The next thing you need to do is to compute the sample standarddeviation, which we said we show with s.In order to do that, you subtract the meanfrom each observation.

- 06:01
MAHTASH ESFANDIARI [continued]: You square it.You add those squares.You divide by n minus 1.And you take the square root.And that is called the standard deviation.So the question-- the thing is, what's the question we'reasking in hypothesis testing?Let's say that the average of the mean of the samplebecomes 8.

- 06:22
MAHTASH ESFANDIARI [continued]: And the value that we assume for the parameter was 7.So the $9 million question is, is the differencebetween the sample mean of 8 and the population mean of 7large enough so that we can call it statistically significant?And so therefore here, there are a number of things

- 06:43
MAHTASH ESFANDIARI [continued]: you need to talk about.And the one that I'm going to concentrate onis type I error and p value.So what's type I error?Type I error is making the error of rejecting the true null.Let's say the method of vocabulary teachingdoesn't work.

- 07:03
MAHTASH ESFANDIARI [continued]: I'm going to say it works.This is a type I error.The method of weight loss doesn't work.I say it works.That's a type I error.Another example of a type I errorwould be like when someone tells you, you pay $100,your SAT score is going to improve by 50 points.The SAT method of teaching does not work.

- 07:27
MAHTASH ESFANDIARI [continued]: And you paid $100.You lost.So that is your type I error.And in hypothesis testing, we call the type I erroror alpha the risk we're willing to taketo reject the true null.And it is referred to as alpha.And you can put it at 5%, 1%, 0.1%.

- 07:49
MAHTASH ESFANDIARI [continued]: And it's under the control of the experimenter.And the magnitude of it depends on what you're doing.If you're shooting a rocket, you want a much, much lower levelof alpha.If you're looking at comparing two teaching methods,you can have a higher alpha.And now, what's the p value?The p value is the risk we actually

- 08:10
MAHTASH ESFANDIARI [continued]: take to reject the true null.So there's the actual risk, and there isthe risk we're willing to take.If the p, which is your actual risk, is less than the alpha,the risk you're willing to take, you reject the null.If your p value is more, then you don't reject the null.Now a lot of times, people ask for guidelines for the p value.

- 08:31
MAHTASH ESFANDIARI [continued]: If your p value is like more than 0.1,you don't have much evidence against the null.If it's between 0.05 and 0.1, yousay I have moderate evidence against them null.If it's between 0.01 and 0.05, you have strong evidence.If it's less than 0.01, then you have very strong evidenceagainst the null.

- 08:52
MAHTASH ESFANDIARI [continued]: And now, I'm going to talk about the formulafor finding out the p value for the one sampletest of the mean, which I called the one-shot case study.That t value is equal to x bar-- the mean in the sample--minus mu-- which is the mean in the population-- dividedby standard error of the mean-- which is the s, which

- 09:12
MAHTASH ESFANDIARI [continued]: is standard deviation of the sample-- dividedby the square root of n.So we're going to calculate the t valuefor that particular example.And the mean in the sample is 8.The standard deviation in the sample is 1.And so when we replace the relevant values, 8 minus 7,1 over the root of 40, we find the t value to be 6.32.

- 09:36
MAHTASH ESFANDIARI [continued]: So how are we going to estimate what does the 6.32 mean?You should know from your previous lectures and knowledgeof statistics that about 95% of the data in the normal modelare within two standard deviations of the mean.So any value pretty much larger than 2

- 09:57
MAHTASH ESFANDIARI [continued]: is going to have a value of less than 2 and 1/2%.So that way, you see that 6.32 is a very large value.And the area above 6.32 is almost going to be zero.So your p value is almost going to be equal to zero.So that means your actual risk of rejecting a true null

- 10:18
MAHTASH ESFANDIARI [continued]: is almost zero.So this is going to be less than the 5% nominal risk whichyou're willing to take to reject the true null.So you're going to do that.And the 6.32, what you do, you actuallystandardize the difference of 8 and 7 into a value of 6.32.

- 10:40
MAHTASH ESFANDIARI [continued]: One of the most important things in statisticsis you can do a lot of these calculations on any softwaretoday, so the most important thing is OK, what does it mean.So we decided we were going to reject the null.So we're going to say, we're going to conclude,that on average the showing of pictures

- 11:01
MAHTASH ESFANDIARI [continued]: help the kids learn at least seven words.And one thing I'm going to introduce you tois some applets that have been developedby two of our colleagues, Rossman and Chance.And you can Google these applets online.

- 11:21
MAHTASH ESFANDIARI [continued]: And they're going to be showing youexactly how to run all this analysis without carrying outany computation.So you Google Rossman and Chance, and then you go.You see all of their applets.And then, you go to the theoretical applet.And then among the theoretical applet,

- 11:43
MAHTASH ESFANDIARI [continued]: you choose-- you click on the one sample test of the mean.When you click the one sample test of the mean,there is a place you enter x bar.There is a place you enter that 8.There is a place you enter the standard deviationof the sample, which is 1.There is another place that you enterthe sample size, which is 40.On the right side, it allows you to enter the mean,

- 12:06
MAHTASH ESFANDIARI [continued]: so you enter 7 for the population mean.For the alternative, you say greater than 7because you assume that the mean is larger.Then it's going to give you a p value and a t value.And they're going to be exactly the same as what we calculated.So basically you can do that on those free appletsand practice.

- 12:26
MAHTASH ESFANDIARI [continued]: [Confidence Interval in Hypothesis Testing]The next thing they give you on that appletis called a confidence interval.And a confidence interval is an intervalwhere we make a range of values associated with a parameter

- 12:47
MAHTASH ESFANDIARI [continued]: because the point estimate is useless.Like the x bar from the sample is 8.That's a point estimate.That's useless.But you can look at the confidence interval,which goes from 7.6 to 8.3.So the conclusion you draw, you sayI am 95% confident that the average number of words

- 13:10
MAHTASH ESFANDIARI [continued]: that 4-year-olds learn by showing them picturesis between 7.6 and 8.3 words.Being a statistical consultant in the world,I can tell you that confidence intervals go a much longer waythan just sample estimates.For instance, if somebody goes to a doctorand they say if you take this particular pill,we are 95% confident that your blood pressure

- 13:32
MAHTASH ESFANDIARI [continued]: is going to drop between so many points to so many points.So they're going to be able to follow thatwithout knowing any statistic.And the p value and the confidence intervaltell you the same story.If you reject the null, then your confidence intervalis not going to include the value under the null.We said mu is 7, and your confidence interval

- 13:54
MAHTASH ESFANDIARI [continued]: goes from 7.6 to 8.3, which means it is not including--sitting to the right of seven.If your method did not work and your confidence intervalwent from 5.6 to 6.3, that meant on averagethe kids learned between 5.6 to 6.3 words, whichis less than 7.

- 14:15
MAHTASH ESFANDIARI [continued]: Calculation of the 95% confidenceinterval-- by the law of the land, any time you wantto calculate a confidence interval,you're going to have the sample statistic in the middle.You're going to go one margin of error to the right.And you're going to go another margin of error to the left.The margin of error is the value of the z or t multiplied

- 14:39
MAHTASH ESFANDIARI [continued]: by s over the root of n.So it's x bar, the mean of the sample, plus or minus 2s over the square root of n.And if we do the calculations, weget exactly what you found in the applet, whichis 7.6 to 8.3.[Pretest-Posttest Design]

- 14:60
MAHTASH ESFANDIARI [continued]: Now, this one-shot case study or one sample testof the mean from experimental design point of viewis a very weak design because there is no baseline data.There's no pretest data.So we have no idea what happened.And there's no randomization.And that's all that happens.

- 15:20
MAHTASH ESFANDIARI [continued]: The next design, which is a little bit better,is called a pretest-posttest design.That means you get a bunch of participants.You test them before.You test them after.So they serve as their own control.And this way, you lower the bias a little bit.I'm going to give you an example.I'm going to give you a fictitious example on algebra,

- 15:41
MAHTASH ESFANDIARI [continued]: let's say.So you have a bunch of students.You measure their knowledge of algebra before.You measure their knowledge of algebra after.And then you subtract the after knowledgefrom the before knowledge or posttest minus pre.And you call that a difference.So although you start with two observations,

- 16:01
MAHTASH ESFANDIARI [continued]: but you create one after subtracting them.So any statistics book that you look at, in a lot of them,they put the pre and post under the one sample test of the meanbecause you end up creating a single observationby subtracting-- by saying the after minus the before.So the null hypothesis is that the new method of teaching

- 16:22
MAHTASH ESFANDIARI [continued]: algebra was not effective or average gain in the knowledgeof algebra was zero.The alternative hypothesis is that the new method of algebrawas effective.And the average gain in the knowledge of algebrawas greater than zero.The null hypothesis here is that the mu of d, dbeing the difference, is zero.And the alternative is that the mu of d is not equal to 0.

- 16:45
MAHTASH ESFANDIARI [continued]: And the formula is pretty much a extension of the one sampletest of the mean.because you say t is equal d bar minus mu of d bar--but you're not going to see mu of d barbecause that's 0, that disappears-- dividedby s of d-- we said s of x, here we say s of d-- dividedby the square root of n.

- 17:06
MAHTASH ESFANDIARI [continued]: Now here, we need to calculate the d bar.So the d bar are all the differences addeddivided by the sample size.Standard deviation is exactly as we did itbefore-- each deviation score minus the average deviationsquared, all squares added, divided by n minus 1.

- 17:27
MAHTASH ESFANDIARI [continued]: You take the square root.So only instead of s of x, now you get s of d.The next thing you're going to calculateis going to be the t value.And your t value is going to be d bar dividedby s of d, divided by the square root of n.Again, you can go to Rossman and Chance.Again, you go under the one sample.

- 17:48
MAHTASH ESFANDIARI [continued]: But this time, instead of putting the mu equal to 7,you're going to put it equal to 0.And you see that you get exactly the same thing.And here, your confidence intervalis going to go from 7.9 to 10.9, which means on average gain,as a result of this new method of algebra,

- 18:08
MAHTASH ESFANDIARI [continued]: the scores increased anywhere between 8 pointsto about 10 points.And again here, the margin of erroris a sample statistic plus minus margin of error.But your sample statistic here is not x bar.Your sample statistic is d bar, which isthe difference of pre and post.So we're going to say that we're 95% confident

- 18:29
MAHTASH ESFANDIARI [continued]: that the average gain in the knowledge of algebrais between 8 and 10.And you have to remember to say the average therebecause this is about average.[Two-Sample Test]We move on to what we call the two-sample test of the mean.This can be either a true experimental design

- 18:51
MAHTASH ESFANDIARI [continued]: or a quasi-experimental design or an observational study.Example from the previous step, wecan go back to observational study.It's the ladies with pet, the ladies without pet,the level of depression.That a observational study with two-sample test.Quasi-experimental design was when we randomly assigned

- 19:12
MAHTASH ESFANDIARI [continued]: the treatment to the classes.And a real one is when you randomlyassign the participants to the experimental and the control.Independent variable, I'm going to give you an example.I wanted to teach some statistical method.I wanted to actually teach experimental design.And I was looking at two teaching methods.

- 19:34
MAHTASH ESFANDIARI [continued]: One is lecture only.The other one was having students listen to the lectureand get together and do some problems.So the dependent variable was the knowledgeof experimental design.So basically, you have a bunch of students.You randomly assign them into control and experimental.And you look at their knowledge of how much they learned

- 19:55
MAHTASH ESFANDIARI [continued]: about experimental design.So I have some numbers, fictitious numbers, here.The mean of the lecture only was 60.The mean of lecture plus problem was 70.The standard deviations in both groups was 4.And the sample size in both groups was 50.So again, the $9 million questionis, is the difference between 70 and 60 large enough

- 20:19
MAHTASH ESFANDIARI [continued]: to call it statistically significant.So here the null hypothesis becomesthe mean in the population of the lectureonly minus the mean of lecture plus problem is equal to 0.The same as saying this mu is equal to the other mu.And the alternative is that the mean of the lectureonly is lower than the mean of the lecture plus problem.

- 20:41
MAHTASH ESFANDIARI [continued]: Or lecture plus problem is larger than lecture only.So if you want to say it in words,we're going to say adding problemsto lecture on experimental designhas no effect on enhancing students' knowledgeof experimental design.And the alternative would be no, that adding problemsto lectures on experimental design

- 21:02
MAHTASH ESFANDIARI [continued]: is going to improve students' knowledgeof experimental design.Then you can again go to applets by Rossman and Chance.But this time, you choose the two-sample mean.And you enter the relevant data.And you're going to get a p value for the two-sample testof the mean.And you see that the p value is less than 0.

- 21:24
MAHTASH ESFANDIARI [continued]: And the mean of the other group is larger.So you're going to say the students whowere taught by lecture method scored significantlylower on the test of experimental designthan those who taught by lecture only.And it's important not to forget the on average thing.Now, you look at the 95% confidence interval

- 21:48
MAHTASH ESFANDIARI [continued]: and you see that the confidence intervalgoes from about negative 11.6 to negative 8.4.That means you are 95% confident that on average the studentswho were in the lecture only groupscored about 11.6 to 8.4 points lower than thosewho did the lecture only.

- 22:09
MAHTASH ESFANDIARI [continued]: And one of the questions that the students always askis, which one should I subtract from the other one?There's no word of God.It doesn't matter.If you had subtracted-- if you had said 70 minus 60,your confidence interval instead of negative 11.6to negative 8.4 would have been plus 8.4 to plus 11.6.And you would have concluded that this group scored

- 22:31
MAHTASH ESFANDIARI [continued]: that much lower-- that much higher than the other group.Now, as far as the formula goes, the key formulais mean of group one minus mean of grouptwo minus the mean of population one minus population two--that's equal to 0, so it disappears--divided over the variance of group one

- 22:51
MAHTASH ESFANDIARI [continued]: over the size of sample one plus the variance of group twoover the size of sample two.And again, the 95% confidence intervalis going to be just like before.The sample statistic, which is now x bar 1 minus xbar 2, plus margin of error this wayminus margin of error the other way.Formula being x bar 1 minus x bar 2

- 23:13
MAHTASH ESFANDIARI [continued]: plus t multiplied by square root of s1squared over n1, s2 squared over n2.Let's say if you're comparing men and women on how much theymake per year and you find a confidence interval to be5K to 10K and on average men make more,

- 23:33
MAHTASH ESFANDIARI [continued]: you say I'm 95% confident that for the same jobmen make between $5,000 to $10,000 more per yearthan women.And you make sure that you put the confidence intervalwithin context with reference to the relevant variables.

- 23:54
MAHTASH ESFANDIARI [continued]: For further reading, please check outIntroductory Statistics- Exploring the World ThroughData by Robert Gould and Colleen Ryan,Statistics by Freedman, Pisani, and Purves,and Introduction to the Practice of Statisticsby David Moore and George McCabe.Thank you very much.

### Video Info

**Publisher:** SAGE Publications Ltd

**Publication Year:** 2017

**Video Type:**Tutorial

**Methods:** Experimental design, Descriptive statistics, Statistical inference

**Keywords:** context communication; mathematical applications; mathematical formulas; pay differentials; risk; teaching methods
...
Show More

### Segment Info

**Segment Num.:** 1

**Persons Discussed:**

**Events Discussed:**

**Keywords:**

## Abstract

Professor Mahtash Esfandiari explains basic statistical concepts like population, sample, and mean. She also discusses how these concepts are used in experimental, quasi-experimental, and observational research.