Skip to main content
Search form
  • 00:00


  • 00:14

    JONATHAN COOK: My name's Professor Jonathan Cook.I'm an Associate Professor at the University of Oxford.I'm based in the center for statistics and medicine.Been working as a statistician for about 20 yearsand focusing on clinical trials, and especially randomizedcontrolled trials and those that we might call,

  • 00:36

    JONATHAN COOK [continued]: late phase studies.I'm going to talk about sample sizecalculations for randomized controlled trials.The particular type of randomized control trialswe think about are what is sometimescalled definitive trials.Those are aiming to come up with an answer, whichyou could implement.There are other types of trials.

  • 00:58

    JONATHAN COOK [continued]: Either early phase, you might say, or pilot, or feasibilitystudies, which aren't really trying necessarilyto give you a definitive answer, but they're justreally preparing for the next study in your evaluationprocess.And they have quite a different approach.

  • 01:20

    JONATHAN COOK [continued]: During my undergraduate, I was fortunate to get placementat the Health Services Research Unitat the University of Aberdeen, which have a long standinginterest in conducting randomized controlled trials,particularly those which are what you might sayas non-pharmacological, so not drug treatments,but those that involve over things

  • 01:41

    JONATHAN COOK [continued]: like surgery or physiotherapy or policy decisions.And I got involved a project there relatedto a surgical trial when I was in my last yearof undergraduate.And through that, I undertook a PhD there as well,which was on a related topic.And really, from then, I've had an interest

  • 02:02

    JONATHAN COOK [continued]: all the way through, in randomized controlled trials,how they're conducted, and more recentlythat's focused quite heavily on sample sizecalculations for randomized control trials,and how you go about doing that, and howwe could improve practice.

  • 02:27

    JONATHAN COOK [continued]: Put very simply, sample size is just the number of peoplethat you need to include in your study,and you need to determine how you're going to do that.Typically, what you have is what it's called,the sample size calculation.There is some sort of numerical calculations in simpler studydesigns, that can be a fairly straightforward formula, that

  • 02:50

    JONATHAN COOK [continued]: tell you the number that you need,so that you can be confident that your study is goingto provide an answer, which is the aim of the projectas a whole.There are different ways to do that.There's certainly more complicated approaches,but most people do what's sometimes called,the Neyman-Pearson approach or conventional

  • 03:10

    JONATHAN COOK [continued]: or to statistical hypothesis testing approach.And practice, in most areas, is overwhelminglystill that conventional approach.And that there's various inputs that you need, but possiblywhat you might say is most importantis what sometimes called the target difference.So that is for the outcome that you've

  • 03:30

    JONATHAN COOK [continued]: chosen to be your main way that you'regoing to assess the study and the comparison between twotreatments, whether is's drug A and drug B, or surgeryof one type, versus maybe rehabilitationof physiotherapy.How big a magnitude of the differencesare you interested in?And how big a study do you need to be confident that you're

  • 03:52

    JONATHAN COOK [continued]: going to pick that up?There are different ways to calculate such a targetdifference, but important point is the sample sizeis often very sensitive to it.So it's very important that you getthat right and the other inputs you might put into your samplesize calculation.

  • 04:20

    JONATHAN COOK [continued]: The Neyman-Pearson approach.The name comes from two statisticians, Neymanand Pearson who had slightly different approachesto deal with your analysis and howyou might choose to design your study,but those have kind of come together over time,and are regularly used.

  • 04:41

    JONATHAN COOK [continued]: In fact, would be what you'd typicallyget taught in a stats one to one or an introductory statisticscourse when you're dressing sample sizefor any sort of clinical study.What it tries to deal with is basically the datathat you want to get.

  • 05:01

    JONATHAN COOK [continued]: Which implies an outcome how you've measured it,and then, the question is if you do that analysis,what is your result?And it's based on trying to balance two risks.There's a risk of not being able to detect a genuine difference.And that's where the target difference comes in,what difference do you really want to detect

  • 05:22

    JONATHAN COOK [continued]: and an outcome if it's blood pressure or a reductionin complications after surgery.So you're trying to make sure youhave a good chance of picking up if it actually exists,but also you're trying to not cometo a false positive result. If there's generallyno difference just because you're doing analysis,

  • 05:44

    JONATHAN COOK [continued]: you run a risk that you might come to a false positive.So you tend to control those two types of errors.They're often called type 1 error, type 2.Type 1 error is controlled by what'scalled the significance level, which you usually set to 0.05,and is usually what they call two sided.

  • 06:04

    JONATHAN COOK [continued]: You're interested in a difference in both directions.You don't mind if it's A greater than B or B greater than A.You are interested in either of those differences.On a practical level that significance level helpsyou then interpret the analysis result, which is usuallyunder most analysis that are done,

  • 06:24

    JONATHAN COOK [continued]: you produce an estimate of what you're interested in.Say I mean, difference in blood pressure.You do your analysis, you could get a confidence interval,but that confidence interval has some degree of uncertaintythat addresses usually a 95% level, whichrelates to a 5% type 1 error rate in your sample size

  • 06:46

    JONATHAN COOK [continued]: calculation.The other error is the possibilitythat you don't detect a genuine difference.That is typically said something like 10% or 20%,which we usually talk about that actuallyas a complement of that.So we talk about statistical power.

  • 07:08

    JONATHAN COOK [continued]: So we like to frame it in the positive direction.How likely are we to take this differenceif it actually exists?And usually one level of 80% or 90%,which implies a risk of this type 1 error of 20% or 10%.

  • 07:31

    JONATHAN COOK [continued]: It's a function of any data that youlook at if you look to see something,it might be there what you're interested inor there might be some interesting finding,but what we're typically dealing with almostalways is measurements are made with error.So we don't have perfect data, and we don't have probably

  • 07:53

    JONATHAN COOK [continued]: a perfect amount of data.We don't have an infinite amount of data,we have a small amount.What that means is the little bit of datathat you have might be slightly misleading.It might seem like drug A is better than drug B,but that might just be a play of chance that you were just--the data that you happened to collectseemed to suggest that A was better than B,

  • 08:15

    JONATHAN COOK [continued]: and actually B is better than A. Is a real truthor that there's no difference at all.It's just a function of doing any analysis,you have this possibility of having a spurious finding,which we sometimes call a false positive in this settingbecause we're thinking about preplanning thatand controlling that in advance.

  • 08:35

    JONATHAN COOK [continued]: And people do tend to control that in most areasoutside of clinical trials, but itmakes more sense than a clinical trialand potentially a randomized control trialbecause is, in essence, an experimentthat you've preplanned, you set up in advance,and you plan your main analysis.In your sample size, should address your main researchquestion or questions, making sure

  • 08:56

    JONATHAN COOK [continued]: that the studies can be big enoughand enable you to be able to explore that with confidence.There's various inputs that are needed for a typical samplesize calculation.

  • 09:18

    JONATHAN COOK [continued]: The trial design, there's different waysyou can do a randomized controlled trialdesign, sort of standard design, which usually implies two arms.So just an A and B not an AB and a C,but just an A and a B treatments.Also you need to be explicit about whatyou're going to analyze.What measurement is it?Is it blood pressure or is actually stroke or something

  • 09:41

    JONATHAN COOK [continued]: else that's going to be your main outcome of interest,which we usually call our primary outcome.Also you need to know what analysis you're planning to door at least the rough type of analysis.Possibly, not the exact form but at least, in essence,do a calculation that matches the type of analysisyou're going to do.So that has implications as well,

  • 10:02

    JONATHAN COOK [continued]: which should be related to the outcomeand how the outcome is measured, and how you're going to,in essence put it into the statistical analysis whenyou analyze.And then, there's also this differencethat I mentioned, the target differenceand related to that issue depend on what outcome

  • 10:23

    JONATHAN COOK [continued]: you have there's often another parameter whichwould be that you need to specify as well to geta simple calculation.So that is often say a standard deviation,which is a measure of variability,which would go with anything like blood pressure.In fact, blood pressure is a good example,and it's something that actually can be measured that precisely.

  • 10:45

    JONATHAN COOK [continued]: So there's always quite a bit of variability.But it's also true for other outcomes thatsay it's event based outcome, so youdid or didn't have a complication,or you did or didn't have a stroke or in others context,mortality.That type of event outcome can be framed in two ways,

  • 11:06

    JONATHAN COOK [continued]: but what is important is if it's framed as following people upfor a period of time and then working if everyonein the study has or hasn't had the controlgroup or the reference level, is also important,as well as the target difference, which couldbe expressed in different ways.

  • 11:26

    JONATHAN COOK [continued]: You could say you're comparing 10% to 5%,so you,re trying to half a rate.You might talk about that way, or you might say, wellthat's a 5% reduction what we wouldcall an absolute reduction.There is also a way to use the time to the eventif you have that information and that also lowersthe potential of following up people

  • 11:48

    JONATHAN COOK [continued]: for a different amount that implies a different analysis,and it implies a different sample size calculation.And there's more options really in the waythat that's done because it dependson various factors about when you recruit peopleand how long you want to follow them up,and what pattern of events is likely to happen in the two

  • 12:11

    JONATHAN COOK [continued]: groups?And you can make various assumptions around that,but again you can specify and do needto specify what is either directlyor implicitly you're interested in terms of a target reference.I have mostly so far talked about sample size

  • 12:35

    JONATHAN COOK [continued]: as if it's just a numerical thing that you doand then you get a number.The truth is you'll do multiple calculations because you'reinterested in how sensitive the figure is to various inputs.And can be the case that maybe it doesn't seem intuitive

  • 12:55

    JONATHAN COOK [continued]: if you're not familiar with working in the areathat a small difference and one of the primariescan make a really big difference the sample size you needand that's hugely important because the differenceof recruiting say 60 people versus recruiting 6,000is massive and that has massive implicationsfor various things, a randomized controlled trial.

  • 13:15

    JONATHAN COOK [continued]: We're talking about individuals typicallywhen we see the sample size 2.You're talking about having more than 5,000more people in that example in the studywho'd have to be recruited to the study.Typically, they would go through a consent processthat have to have their data collectedand that involves many different things

  • 13:38

    JONATHAN COOK [continued]: and ultimately, it also involves cost in various senses.You could talk about that as a resource that's used.It's staff time that's needed to do that, the facilities areneeded to deliver the treatments and also typically there'sa funder backing a study, and that might be a companyor it might be some sort of government organization funding

  • 14:04

    JONATHAN COOK [continued]: body or it might be a charity, but whoever it is they wantgood value for their money.And so it's some much more complicated process than justa simple calculation and here's the final number.It's very simple to do say there is a clear ethical principle

  • 14:28

    JONATHAN COOK [continued]: at stake, which is, a randomized control trial isan experiment of a kind we do not tend to use our languagebecause it makes people unnecessarily nervous about it,but in scientifically, we are doing an experiment.We are comparing two options of some kind,and they can vary degrees and some thingsmay vary to a level that wouldn't bother people at all

  • 14:49

    JONATHAN COOK [continued]: and they wouldn't be interested or it doesn't reallyaffect them in any meaningful way.You could very, very subtle things about the processthat someone goes to see their doctor,for example that wouldn't bother them.Things like delay would bother thembut, actually maybe the person that theymeet at the desk first of all whointroduces the kind of treatment and sets up

  • 15:11

    JONATHAN COOK [continued]: that side of things, they might be less concerned.Most people, I think would be less concerned about that sortof change.But whatever you're doing, you are altering something that'shappening already, and usually, wewould think that goes through some consent process.And given that there are historical experimentation that

  • 15:34

    JONATHAN COOK [continued]: took place on humans, which were clearlyunethical and unacceptable, the process that we typicallyhave in most countries is we have an independent bodythat assesses a clinical trial and approves its conduct.There can be various bodies.Often that's universities or hospitals, where that body lies

  • 15:56

    JONATHAN COOK [continued]: but all behind that is the principle that we shouldn'tbe causing unnecessary harm to people,we shouldn't be burdening them unnecessarily,we shouldn't expose them to unfair risksthat they are unaware of.So there's various ways that comes in.What we do one is the study that will give us an answer as well.

  • 16:18

    JONATHAN COOK [continued]: So we need enough people, but what is enough?Can be debated and discussed.But what we don't want is a study which cann't deliveranything that doesn't provide any information.So for it to be ethical in that sense,we want to have some sort of reassurance.

  • 16:43

    JONATHAN COOK [continued]: Where it becomes more difficult to calculate a sample sizeis when there's very little informationto guide what you're doing.So if you're dealing with a new treatment that really doesn'thave much existing evidence related to it,or it's quite different from what's gone before,it's quite difficult to necessarily know

  • 17:05

    JONATHAN COOK [continued]: what to expect in terms of the outcomeand how something's going to occur or what sort of typicalsay mean level you don't anticipate.And so that's quite difficult.I've certainly been involved in a few projects wherethe existing scientific literature isinsufficient to give you a really strong reassurance.

  • 17:27

    JONATHAN COOK [continued]: There's various ways you can deal with that.Most approaches tend to be of or aredata driven or some sort of expert opinionor some sort of combination of If you really don't have anywhat you might call empirical data,then you may have to base on a judgment about what is similar

  • 17:50

    JONATHAN COOK [continued]: or what could be possible here or what we wouldbe interested in our outcome.Typically, our studies are probably too small,and that's probably the reality that we do a studyand it's probably not big enough.

  • 18:13

    JONATHAN COOK [continued]: The whole process of doing a sample like calculationat least makes more explicit what those risks are,and provides for someone independent to assesshow likely is the study's going to provide this answer,but a risk is if our studies go an insufficient numberof people, and then often you can lose data as you go along,

  • 18:34

    JONATHAN COOK [continued]: what you get is an answer that really doesn't see anythingand that unfortunately is quite possiblein statistical analysis.There's often a lot of uncertainty about measurementsand how they're done, and it's quite difficultto pick up subtle and small differences.A less likely risk, but is possibleas you recruited too many people,

  • 18:56

    JONATHAN COOK [continued]: and so your risk is not so much the scientific issue per say,it's more that you've wasted resource and time youcould be doing another trials.Maybe you've spent twice as long during one studyand you could have done another oneand got two answers instead of one.

  • 19:21

    JONATHAN COOK [continued]: There's various resources you could go to.There's a number of very good textbooks, which address issuein the scientific literature.There's a number of helpful papers addressing sample sizecalculations.I think, what I would first say is I wouldn't recommend someonedoing it--

  • 19:41

    JONATHAN COOK [continued]: learning by their self or someone to guide themand typically that would mean goingto start station who's got some experience in doingthese calculations because it is quite easy to go wrong,particularly if you're doing something that'snot that standard design.I do know of examples where quite experienced researchers

  • 20:04

    JONATHAN COOK [continued]: have gone wrong, unfortunately in that area.In terms of target difference, whichI would say for mostly face studiespossibly the most difficult input to address being involvedin a project recently called delta2, which follows on from project called delta,

  • 20:24

    JONATHAN COOK [continued]: and in that we provide some guidance for people whoare doing these late face studies about howto go about doing a sample size calculationand also how to report it, which is really important.When we do our publication after our studythat we are quite open and explicit about what we weretrying to achieve at the beginning and that helps people

  • 20:45

    JONATHAN COOK [continued]: assess the study afterwards.


Jonathan Cook, Associate Professor, Centre for Statistics in Medicine at University of Oxford, discusses sample size calculations for randomized-controlled trials, including main considerations, practical implications, ethical considerations, and challenges.

Looks like you do not have access to this content.

Introduction to Sample Size Calculations for Randomized Controlled Trials

Jonathan Cook, Associate Professor, Centre for Statistics in Medicine at University of Oxford, discusses sample size calculations for randomized-controlled trials, including main considerations, practical implications, ethical considerations, and challenges.

Copy and paste the following HTML into your website