Public Engagement for Improving Understanding of Randomized Control Trials

[Public Engagement for Improving Understandingof Randomized Control Trials]

DR. SANDRA GALVIN: My name is Dr. Sandra Galvinand I'm the Programme Manager for a national supportnetwork in Ireland called the Health Research Board TrialsMethodology Research Network.As I said, we're a national support networkand we operate across five funded universitiesin Ireland, in the NUIG, UCC, UL, TCD, and UCD.Our remit-- and we've been funded since 2015--

DR. SANDRA GALVIN [continued]: our remit is around strengthening the methodologiesthat are used within trials.So across the board, right from whenyou start to pick your research questionto when you decide what participantsare going to take part and conducting,analyzing, reporting, disseminating your findings.Were very broad remit, but we do a lotof work, a lot of good work across each of these stages

DR. SANDRA GALVIN [continued]: within randomized trials.And we operate within Ireland but we'revery lucky to have a lot of key collaborations within the UK.And we're very proud to say that we're nowa member of the MRC, NIHR, TMRP here in the UK,and we work with a lot of collaborators internationally.So we hope that our research will have both a nationaland an international impact.So as part of the work of the HRB-TMRN, and we're

DR. SANDRA GALVIN [continued]: very aware that there are significant issues with regardsto the area around recruitment and retentionto randomized trials.And as part of the work that we do across several projectswithin trials such as studies within trials,within core outcome sets, we often findand this is through feedback from the general public,that there is a lot of misconceptionsor misunderstandings around what it means to take part

DR. SANDRA GALVIN [continued]: in a randomized trial.Why we need randomized trials?How they can benefit healthcare?There seems to be a disjoint or a lackof understanding among the general publicas to why we need them.And why we should do them and why we should fund them.So with that in mind, we knew we wantedto do something with the TMRN around public engagement.So this pillar, we call it a pillar of activity,

DR. SANDRA GALVIN [continued]: grew very organically within the TMRNsince it was funded in 2015.So as part of this, we wanted to engage with the general publicin a very meaningful way around the conceptof a randomized trial.And quite frankly over a coffee one morning,we decided, let's do something a little bit different.Let's step outside our comfort zone

DR. SANDRA GALVIN [continued]: and let's engage with the youngermembers of our community.So it started with this concept of this initiative calledSTART, which is Schools Teaching Awareness of Randomised Trials.So what we thought initially was,gosh, our primary school childrenare going to be able to understand these concepts.I mean we have issues with trials every dayof the week around these concepts, and what we did was

DR. SANDRA GALVIN [continued]: we set the challenge to primary school childrenbetween the ages of 8 and 12 to conducttheir very own randomised trial within the classroomenvironment.So it's a competition, it's a national competitionthat happens every year.It's currently in its fifth year and of course,what happened with this was not only did the childrenrise to the challenge, they completely surpassed

DR. SANDRA GALVIN [continued]: our expectations and did an amazing job.And continue to do an amazing joband we see actually improvements and higher standards comingevery year of this competition.So what we did was we put togethersome resources on our website.We organized this competition, we opened itout to Irish primary schools nationallyto submit their very own randomised trialsand present their findings in their own way,

DR. SANDRA GALVIN [continued]: so whichever way they come through it.So some of the trials that we getpresented and led by the childrenare submitted in the form of a podcast,they're submitted in the form of a PowerPoint presentation,we have some beautiful scrapbook presentations put togethercovered in glitter and gold and all sorts of creative aspects.We've even had one school present their findings

DR. SANDRA GALVIN [continued]: of the randomised trial in the form of a drama,acting out scenes related to their experienceswithin the trial.So as I mentioned there's been 45 randomized trials conductedin Ireland that are led by children in the classroomenvironment.What's unique about this is the first timethat the concept of randomized trialshas been introduced in the Irish curriculum.

DR. SANDRA GALVIN [continued]: We've been told by teachers that they'redelighted to have this new teaching tooland it's funny we never thought of itas that when we started out.We never thought that this was going to be a teaching toolbut it very much aligns with the primary school curriculum.And I think that's why it's been so successful.Teachers see that it covers aspects of creativity.It covers aspects around language,around act, and around drama, but also about maths,

DR. SANDRA GALVIN [continued]: and about calculations, and analyzing their trials.So every step of the randomized trial is led by the children.So we start off just as we all dowithin the world of randomized trialsdeciding your research question.You don't do that in a vacuum, you do that amongst your peers,amongst the people that it's going to affect.How are you going to conduct your trial?

DR. SANDRA GALVIN [continued]: So what is it that you're analyzing?What is it that you're measuring?The children have actually succeeded reallywell in doing this.The issue around randomization.Why do you need randomization and indeed howdo you do randomization?I mean, we've had children come upwith fantastic methods such as using colored lollipop sticksto do the randomization.And what's interesting then is we have an award ceremony

DR. SANDRA GALVIN [continued]: every year.The top three schools that submitted an entry to STARTare invited to come to NUI Galwayand we coincide that with international clinical trialsday which as you know, is the 20th of May every year.So we have this award ceremony, we'revery lucky that we have a lot of people that contribute to that.A lot of volunteers, it's certainly not just one or twoof us in NUI Galway.

DR. SANDRA GALVIN [continued]: It's a lot of individuals that are behind START.And what we do in that day is we celebrate science more broadly.But we also celebrate the three trialsthat have been submitted.We have some fantastic judges thatare coming over last year and this year.And we've had Andy Oxman coming from Oslo as partof our judging panel last year.Next year we have Sean Treweek from Aberdeen Trial Forge.

DR. SANDRA GALVIN [continued]: We have Sarah Chapman from Cochrane UKand we have an Irish primary school teacher, Ashley Murraywho's also coming.She helps out with the drive and the initiativearound START within schools.So I suppose START has kind of captured our imaginationand it's something that we feel is really benefitingto this broader message with the publicaround why we need trials.

DR. SANDRA GALVIN [continued]: We know that the children are not justclosing this down within the classroom, they're going home,they're talking to their teachers about it.We had one school that engage with their local pharmacyaround their trial and spread the wordthroughout the community.We have parents getting involved,for example, some of the childrendid some of their trials within the classroombut they also took home a survey and got some feedbackfrom their parents on what they were doing.

DR. SANDRA GALVIN [continued]: And all of the steps within a real randomized trialare being replicated.So there's the patient information leafletthat you would have, there is informed consent forms.And so really, it's bringing all of those steps and processesto the primary school environmentbut we feel creating citizen scientists.It's creating future scientists, but alsopresent day critical thinkers.

DR. SANDRA GALVIN [continued]: These children are starting to think more criticallyabout health claims and we think that's really important.So when start has been such a success now,we're in our fifth year, we decidedthat we'd love to do something with itmore broadly with the general public,with the adult generation.We wanted to do something to engagewith the general public around why we need trials, why they're

DR. SANDRA GALVIN [continued]: important, the same processes.And we also wanted to give that understandingof the steps of the trial.So one misconception that came was very clear to usthrough our research in the TMRN was this ideathat doing randomization is because the doctor istoo lazy to assign me to the intervention of the controlgroup.But of course, randomization is the most important factor

DR. SANDRA GALVIN [continued]: of a randomized trial.It what gives it its gold standard,is what underpins the scientific rigor of a randomized trial.So we were very disheartened to think that people justthought we were being lazy by randomizing them to one or twogroups.So with that in mind and with what we've learned from START,The People's Trial was born.So The People's Trial is again another initiative

DR. SANDRA GALVIN [continued]: through the TMRN.And again led by Professor Declan Devaneand ourselves in NUI Galway.And it's funded by the health research board.And it is what I like to call, START for adults.So it's taken the concept of STARTbut it's opening it up really widely to the general public.So it's not a competition.It's an online platform.

DR. SANDRA GALVIN [continued]: And it's been delivered with a fantastic steering group,not just based in Ireland but our colleaguesagain across the UK have gotten involved as well.And it's not just an Irish initiative, it's open,it's very broadly open.So for example there's different stageswithin The People's Trial.It is a platform that allows for a randomized trialto be conducted completely out of our hands

DR. SANDRA GALVIN [continued]: and that's the exciting thing.We don't know what the question is going to be.We don't know how we're going to answer it.We don't know how we're going to measure it.The people that do know are the general public,they are making all the decisions in this.So we're just controlling the platform in the backgroundto making this possible.So there are different stages within The People's Trial.And they look at, for example, deciding the research question.

DR. SANDRA GALVIN [continued]: How do you do that.How do you pick the question.And as I mentioned earlier, you don't do that in a vacuum,you do that amongst your peers in deciding and deliberating.What is the most important questionthat this needs to answer.We are saying to the general publicthough to have a lot of fun and to makethis an inclusive question that loads of peoplecan take part in.The stages of the randomized trial are open and available,

DR. SANDRA GALVIN [continued]: and we're currently just after closing phasetwo of The People's Trial.So that means that we asked a heap of peopleglobally to submit a research question.And I believe we've had over 20 countriesget involved in this with 158 research questions beingpitched.The next phase is, OK, let's see how popular these questions areand let's get it down to a top 10,

DR. SANDRA GALVIN [continued]: then let's get it down to the top 3.Again, just like you do in a real randomized trial.So we've just looked at getting to the top research questionand that's going to launch very soon, it'll be phase three.And that's when we have to get down to the nitty gritty of howdo we do this.So some examples of the questionsthat have come through are very much like what the children aredoing in START, they relate to their own personal environments

DR. SANDRA GALVIN [continued]: and their own personal lives.So a lot of this relates to quality of sleep,making us work better and more efficient in our workplace.So the impact of social media are looking at your phone lateat night, does that impact your sleep quality.That's just more of the examples of the questionsthat are coming through.And another question will be around,does going for a 30 minute walk at lunchtime everydaymake you more efficient in the afternoon in your work

DR. SANDRA GALVIN [continued]: environment.Or does drinking a hot drink before bed againimproves sleep quality?A lot of it centered around sleep quality,I think it's a sign of the times.But again, the question just has to be fun, engagingand inclusive[What do researchers need to consider when formulatingtheir research question?]I think from our perspective within the TMRN,

DR. SANDRA GALVIN [continued]: what we're very, I suppose, what we driveor what recommendations we make to people when they'reconducting their research are to consider that question verycarefully.So what you need to do in that is to reach out to your peers,to reach out to the wider group that are considering thisor that are involved in this.

DR. SANDRA GALVIN [continued]: And one thing that we would do is, OK, look as the bestup to date evidence.Is your research question sufficientlyunderpinned by a lack of evidence?Has this already been answered?I mean, we talk every day about research wasteand the significance that can have.And we're certainly not in the businessof replication for the sake of replication,for the sake of putting your name on something.

DR. SANDRA GALVIN [continued]: And the research question needs to beunderpinned by a gap and a need and a drive for that need.And one aspect that I suppose has become very much moreaware of within Ireland and I'm sure the UK is the same,is the drive and the need behind PPI within your research.So the need to have the perspectiveof the patients and the general public in what you're doing.

DR. SANDRA GALVIN [continued]: That again, we're not just researchers in a voidor in a vacuum working away because wethink it's interesting.Do the funder really want us to do this?Is this going to make an impact on policy?Our patients and the public onboardwith what we were trying to do.So for example, you might have a really interesting researchquestion but you might be slightly goinga bit less left of center of what's actually needed.

DR. SANDRA GALVIN [continued]: And I think the best way to get advicearound that is to actually talk to membersof the public, our patients, or the people for whichthat research is relevant to.I think that's how we should be guiding our research.And one of the areas which we've made strides in thisis around an agenda setting processwithin the area of trial recruitment and trial

DR. SANDRA GALVIN [continued]: retention.And that's the prioritization setting exercise.So we've identified research questions,which are important for all stakeholder groupswithin the area of recruitment.And our colleagues in Aberdeen have done the samewith retention[Why should researchers register their trials?]

DR. SANDRA GALVIN [continued]: The purpose behind registering your trialis that, again, to avoid this area of duplication.But to also make sure that we're true to ourselvesand that we're true to the work that we've been conducting.So when you register your trial, it'slike you're holding your hands up and saying.Look this is actually what I'm going to do.And I'm not going to be tempted or swayedby positive or negative findings at the end of my trial.Because I saw somebody mentioned recently around,

DR. SANDRA GALVIN [continued]: to stop using the phrase, null findings are negative findings.Because their findings, their findings in themselves.It doesn't matter if your trial provedthat it was going to be effective in a certain area.If it wasn't effective, that's just as important.I think registering your trial issafeguarding against researchers going off on a tangent.And they probably mean well, but it

DR. SANDRA GALVIN [continued]: kind of safeguards against that temptationto stray left a field of what you were actuallydoing at the start.So again, this is something that wedo within our outreach programs, so the START initiativeand The People's Trial.The children in our initiative register their trials.And we explain that to them, we say,it's actually not that we're trying to get you to,

DR. SANDRA GALVIN [continued]: they actually thought it was because nobody else woulddo the trial if they picked that one.And I said no, sometimes we do need to repeat trials,it's usually good evidence underpinning that.But it just so, it just makes surethat you're declaring all your handsat the start of your research.You're saying, this is what I'm going to do.This is how I'm going to do it.So I think it opens it up for getting more guidance and morecritique on what you're doing.

DR. SANDRA GALVIN [continued]: So if you are declaring publicly whatyou're about to do in your trial,I think that can only give and strengthenthe work that you're going to do and give it more robustness.[What are the key considerations when selecting trial outcomes?]Selecting outcomes are very important when you'reconsidering a randomized trial.And this is a lot of work that we've

DR. SANDRA GALVIN [continued]: done within the HRB-TMRN is around coral conceptdevelopment.So our director, Professor Declan Devanewould be very involved within core outcomes at developmentand within our COMET, [Core OutcomeMeasures in Effectiveness Trials]the COMET initiative, which drives this globally.So we often find, I mean the needfor delivering on a good core outcome set,or having access to a good core outcome set

DR. SANDRA GALVIN [continued]: is because we have a lot of people doinga lot of research in trials.And they could be within a very specific areabut actually the outcomes don't match up.So building that evidence base and doingyour systematic review afterwards,or you're looking at your evidence synthesis,it's very hard to match up the outcomes of trialswhen you don't have a good core outcome set

DR. SANDRA GALVIN [continued]: there to underpin it.So it kind of gets people to streamline their thinking whenthey're coming to measurements.And a good core outcome set is justthe minimum amount of reporting that you need to do.That's not to say you can't add to that with your own research,but as with the initiative around with COMETand the work around core outcomesset development is to just look at the minimum set

DR. SANDRA GALVIN [continued]: of requirements that should be in there.And at least it means you can match up the researchafterwards, that you can look at a group of trialstogether and do a better analysis.And then again, that strengthens the evidence base that we havethat we're getting from these trials.It means we can make better and more informedhealth decisions because the powerbehind the evidence that's been provided is much stronger.

DR. SANDRA GALVIN [continued]: [How should participants be selected for trials?]I think it depends on the study that's been conductedbut I suppose more broadly speaking.And there are lots of considerationsthat people should be making when they think about whothey're going to consider.I mean, some of the research that weare hoping to get involved in the HRB-TMRN centers

DR. SANDRA GALVIN [continued]: around those from marginalized groups, or underreported,underrepresented groups within trials.I think when we're looking at ethicsand we're looking at what target populationwe want to look at in trials, it'sprobably easy for researchers to eliminate those certain groups.So for example, when you consider issues

DR. SANDRA GALVIN [continued]: around recruiting pregnant women or havingpregnant women involved in your trial,I think there is a lot of reasons why researcherspossibly shy away from making surethat these underrepresented groups are includedwithin research.So I think that when you're setting outto select your participants, again,I would go back to in this area and about inclusivity

DR. SANDRA GALVIN [continued]: and about considering having a good PPI panel involvedin your research.So again, members of the public, if it's a specific diseasearea, patients who have gone through this illnessthey bring a completely different perception to thisand can really help you shape who you should be consideringwhen you're involving the general public or patients

DR. SANDRA GALVIN [continued]: within a randomized trial.And we kind of steer you in the right direction when it comesto who should be involved.But I suppose, researchers maybe needto take ownership of the fact that sometimes we steer awayfrom including marginalized groupsor underrepresented groups.But they still need access to good health care,they still need the evidence to underpinwhy a particular treatment is going to be working for them

DR. SANDRA GALVIN [continued]: and they may bring a different dimension to it.So I definitely think it's an areathat we need to get better at, the trialsneed to consider more.And I think just a practical tip as Isaid is around engaging with public and the patients whohave walked that route before within that illnessor within that specific area.[Does the general public understand the principles

DR. SANDRA GALVIN [continued]: of randomization?]We've learned a lot from our public outreachand as I mentioned earlier, with the STARTand with The People's Trial initiativeis this lack of understanding around why we do this,so we have an intervention group and we have a control group.And there seems to be this idea within the general publicfrom what we're doing is that this idea of it's

DR. SANDRA GALVIN [continued]: laziness as to why we randomize.And that is a crucial misconceptionwithin the general public that is possiblyhindering people's participation in trials.Because as we know, less than 50% of randomized trialsreach their recruitment targets and there'sa lot of reasons behind that.But I believe one is around this idea with the general public

DR. SANDRA GALVIN [continued]: that actually randomization means less choice.When in fact, it actually means that you're strengtheningthe process of a randomized trial and you're giving itthat robustness that it needs.Because we need randomization to make sure we have equal groupsand that they're equally represented,that they're there just by chance so thenyou have two very similar groups that you're comparing.

DR. SANDRA GALVIN [continued]: And you're giving it the best chanceat being a fair test of treatments.[What are the key considerations for researchers conductinga trial?]Some of the key considerations for conducting the trial,overall, actually probably go backto when you're setting up your trialand when you're starting out.And that's around getting the right team, the right team

DR. SANDRA GALVIN [continued]: behind this trial.So for example, having a bio-statistician on board a lotof people would recognize that as being really important.But what we find unfortunately is sometimespeople think of that a little bit too late.And they come, when they've got the data collected for example,and they come looking for someone to analyze the dataand make what is going to come out to the trial.

DR. SANDRA GALVIN [continued]: A bio-statistician should actuallybe there from the very start, youneed somebody who has that expertise and thatguidance to shape the trial, knowing what findingsare going to come out of this.But to have the right design, the right and underpinningmethodology throughout the trial.So I think I would say it goes back to the teamthat you have behind the trial.And people that are involved at each stage

DR. SANDRA GALVIN [continued]: again, including if possible and if appropriate,a member of the public or a patientrepresentative to be involved in it.Making sure you have good statistical advicethroughout the trial on the design.I also think making sure that you'veconsidered other designs.So sometimes, we fall into a trap as trialswhere we look at what's been done before, how they did it.

DR. SANDRA GALVIN [continued]: The simple maybe design that they've usedand we think, Oh, we'll just stay within our safety netand we'll just continue to do that.And a lot of the work within the HRB-TMRNis around looking at novel designs.And looking how we can conduct the trialin a more novel and innovative manner.And maybe even answering a coupleof other research questions through usinga more complex design as well.

DR. SANDRA GALVIN [continued]: [How can researchers report their findings more widely?]Reporting our findings is actually an areathat the HRB-TMRN has been involved withand has done a bit of research.And so we're very concerned around the factthat when great trials are being conductedthe evidence is fantastic from them.But then what happens is they get published in a paper

DR. SANDRA GALVIN [continued]: and they fall into this mass of hundreds of thousandsand I think probably even millions of scientific papersthat are published every year.And they might be of great qualityand they might be something that should be shared with everyonebut how do you do that.So I think when you start presentingyour findings of a trial, I don't just think of we,the TMRN don't just think of it as a published paper.

DR. SANDRA GALVIN [continued]: But unfortunately that's the metricthat a lot of researchers go by, they justwant to get the papers published and that'sunderstandable because his reasons behind that.But one thing that we look at is aroundpresenting the findings in a much better way and how canyou get those complex findings back to the peoplethat need it most.And that's usually the conditions that are actuallymaking the decisions.

DR. SANDRA GALVIN [continued]: But also as well getting that information upto the general public for example,because they need to know about the reasonsthat their doctors are making these decisions.And I think they have an interestand they want to be involved in this process.So some of the work that we've looked atis using various methodology to present findings backto the clinicians.And some of the work that's been done in Galway by Aislinn

DR. SANDRA GALVIN [continued]: Conway and Professor Declan Devane looks at, for example,doing a grand rounds on the findings of research.So within the hospital having a meetingwhereby the research is being presentedto the clinicians in a very accessible and easy manner.And should we look at presenting research, for example,not just as a paper but for example presenting it

DR. SANDRA GALVIN [continued]: as an infographic.Should we be breaking down the findings much more easily.I think there's huge scope around this,this is a huge area for research.That we need to be a bit more creativeand we need to stand up and be accountable for the factthat while we're generating research findings,are we as good at getting that into practice and particularlyinto policy which of course, is the ultimate goal

DR. SANDRA GALVIN [continued]: for all researchers and trialist.So I think when you consider your findings,I would say don't stop when you just published the paper, tryyour best to go the extra mile.And it might just be something as simple as reaching outto, for example, national networkswere really actually interested in this area.I mean, the HRB-TMRN if anyone publishes a methodology paper

DR. SANDRA GALVIN [continued]: in Ireland, I mean we're jumping up and downto receive this into promoters.And we do a lot of promotion of research findings in trialmethodology in our end.But I know for example, there's a lot of other networkswhich are very keen to get those findingsand try and share them and present them.Another area for example, would beto just consider delivering a webinar or an online education

DR. SANDRA GALVIN [continued]: resource and your paper.It's much easier sometimes for researchersto sit down to a computer and watch maybe a one hour or 40minutes webinar than to sit down and have time to read a paper.I think we, probably a lot of researchers and trialist,I don't think it matters how senior you were.We always read the plain language summaryI think and take our findings from there.

DR. SANDRA GALVIN [continued]: So why not consider making this moreaccessible and presentable.And I think a lot of the work that we'vedone with for example, in the public engagements with STARTand through The People's Trial that the public areteaching us this.So the children are teaching us that for example, they own it,very few of them presented it in a paper.Some presented in a PowerPoint format,

DR. SANDRA GALVIN [continued]: the majority come back as podcasts, as websites,as dramas, and so interesting.And we should be learning from them,we should be taking a leaf out of their bookwhen it comes to reporting our findings.