Measuring Patient Experience in Clinical Trials: Feasibility Study

[MUSIC PLAYING]

PETER BOWER: Hi.I'm Peter Bower.I'm from the University of Manchester.I'm a professor of health services research.

AILSA DONNELLY: I'm Ailsa Donnelly.I'm a patient and public contributorat the University of Manchester, and Iwork with Peter and the team on this project.

PETER BOWER: I do quite a lot of clinical trials,looking at applied health researchand trying to find better ways of organizingservices for patients.And the work that we're doing at the momentis about how we can make those clinical trials moreacceptable to patients.

AILSA DONNELLY: I'm a patient who'staken part in a lot of trials-- varied sorts.And, although feedback is all the thing these daysand we're forever being asked to rate this and that--as a participant in a trial, I have neverbeen asked for my feedback.And I would love to be asked.Because I have quite a lot to say.

PETER BOWER: So the study we're talking about todayis called PACT, which stands for patient-centered clinicaltrials.It is funded by the National Institutefor Health Research, Research for Patient Benefit Program.And what we're trying to do within this study is,essentially, find ways in which wecan encourage people who are running clinical trialsto measure the experience of patients in those trials

PETER BOWER [continued]: in a way that's going to be helpful for designingnew trials in the future and making surethat patients who take part in trialsfeel it's a really good experience,want to take part again, and potentially tellother people that taking part in research is a good thing.So where-- it's a relatively small studyto test the feasibility of doing this.And it's a study which requires us, as a team,

PETER BOWER [continued]: to work with other study teams.So we need to encourage others to take part in this research,so we can get it done, which is whatmakes it challenging at times, in termsof the methods that we use.So at the moment, we're towards the end of the study.We've done quite a lot of the work, spenta lot of time talking to patients and people involvedin trials.

PETER BOWER [continued]: And we're coming to the end of the project,and trying to work out what we've learned,and thinking about how we can take the work forward.We run trials-- so Manchester runs a fair amount of trials.And I work in that world of people who work in trials.

PETER BOWER [continued]: And it's always interesting to methat we measure people in trials all the time--every aspect of them.So we'll measure aspects of their biology,the way they feel--all these issues.But it's very rare that we measuretheir experience of actually taking part in the trial.So we did some work a few years agolooking at how often people measured

PETER BOWER [continued]: the experience of patients.And it doesn't-- it happens, but not very often,and not in a routine fashion.So I've always thought that's a bit of a mistake.Because we want patients to take part in trials.We want to-- them to be recruited.We want to hold them in the trialsand retain them over time.And we really need to know whether the trial is working

PETER BOWER [continued]: for them-- whether it's burdensome,whether it's tiring, whether it'sannoying-- all the things that you'd think about a service.You think, well, actually, what's this like for patients?And at the moment we just don't know.So I was quite interested there was a gap there,and we could fill that gap by just doingsome work in that space, and seeing, actually,is there appetite to do this?Is it possible?Can we get patients engaged?

PETER BOWER [continued]: So that was what drew me to actually put the fundingbill in.And luckily, we were successful, because Ithink they saw that it was potentially quite important.It's a relatively big team for small projects.And they are-- modern studies like thisyou have to involve a lot of different disciplines

PETER BOWER [continued]: and a lot of different patients.So I'm the principal investigator.So I'm responsible for delivering the study.We've got academics in a number of other universitiesthat are taking part in this.We've got patient representatives,like Ailsa and a whole group of others, who have reallyhelped us with this study.And then we've got the research team who are actuallydelivering the study and dealing with sites on a day

PETER BOWER [continued]: to day basis.So modern science is team science,so there's a big team involved, but it makes it fun to do.And it means you get lots and lots of different expertiseto deal with the problems that occur in any research study.So it came out of that original idea that there was a gap.

PETER BOWER [continued]: So we thought, well, nobody's measuring patient experience.So is it possible to develop somethingthat people could use to measure patient experience,and then get people to adopt that in their trials?So those were the key questions that we wanted to answer.And it was-- we badged it as what was called a feasibilitystudy to try out--see what the responses of patients and professionals

PETER BOWER [continued]: were to the issues we were trying to deal with.And then the idea is, we can, at the end of that, say,is this feasible?Can we take this forward?And if we think this is a viable thing,to take forward in the future.We'll put in other funding bids, work with other stakeholdersto take it forward.So we're dipping our toes in this approach,seeing if it works-- partly because it's not

PETER BOWER [continued]: been done before, which suggests maybe there's difficulties,maybe there's particular challenges.So that's what we're trying to do with this researchand that's why it's set up the way it is.It's got staged design.So we spent a lot of time talking

PETER BOWER [continued]: to patients and professionals about their viewson this issue-- about measuring patients' experience.So we talked to patients who were taking part in trials.We talked to people who ran trials or ran trials unitsto get their experiences.Brought all that together.The second stage, then, was to try and develop a measure.So that would be a short questionnaire or surveythat could be used in trials to explore patients' experience.

PETER BOWER [continued]: So there was a process to follow with that.We very carefully worked with patientsto make sure the questions made sense,and that they would capture what was important.And at the end of that, we had the measure.And then we had the task of going out and tryingto get other trials to use that measure.So it's a slightly different type of research.You're not doing your research.You're helping others to do your research, which

PETER BOWER [continued]: raises lots of challenges.It's kind of embedded research or a study within a study.But the idea is, that by working with others,you get greater reach.So the idea was we would work with researchers acrossthe country to get them to adopt our measure-- try it out--and then we could look at the resultstogether and work out what was useful, what wasn't useful.

PETER BOWER [continued]: And use that to work out, actually, canwe do this in the future.Because what I'd love to do is, to do this at scale.So to get all trials in the country usinga patient experience measure, consistently and routinely,as part of the service they offer as a trials unit.We're a long way from that--a really long way from that.These things take time.You've got to build up relationships.

PETER BOWER [continued]: You've got to build up credibility.You've got to show people that this is possible.I mean, these are scientists.You need to have data that says this is possible.This is feasible.This is useful.So we're at the first stage of that process.And we can-- there's been lots of challenges.But I think it's the nature of this sort of work.It's a long term process, because you'retrying to change attitudes to doing something.

PETER BOWER [continued]: When people haven't measured experience in the past,and you want them to measure it all the time--that's quite a long journey.So it takes a while.We talk to patients and professionalsabout what they thought was important to measure.We look at other measures that hadbeen used-- they're not really used very often,

PETER BOWER [continued]: but there are some out there.The National Institute for Health Research ClinicalResearch Network does some of this typeof experience measurement-- in a very different way,but they use it.So we looked at all the questions that were available.And then we mapped those to the areasthat people thought were important.You always need to keep these things short.Nobody wants a 50 item or 100 item questionnaire.

PETER BOWER [continued]: So we developed a questionnaire.And then my colleague, Nick Small,did a very long process of working with patientsin a process of what's called cognitive interviewing--sitting down with patients, presenting themwith the questions, and trying to get them to articulatewhat that means to them and why they'reresponding in a certain way.So it's a very, very detailed process.You have to be very responsive to what patients are saying.

PETER BOWER [continued]: And you modify the text and modify the termsthat you use as you go through.So that was a long, drawn out process of getting--and you never get agreement.Everyone thinks it should be this way.It should be that way.But you have to come to consensus.And at the end, we've got a measure whichwe think is fit for purpose.And we're using that now.I'm sure it will change.I'm sure it will change.It will be added to or modified in the future.

PETER BOWER [continued]: It needs to be translated, et cetera.So there's all these other issues.But we went through a very detailed process for patientsto understand, does our measure capturewhat's important to you?And there were some items in therethat I didn't think would be included.I think there are items on things like trustand how proud they were to take part in a trial.I wouldn't have put those in, if you'd asked meat the start of the study.

PETER BOWER [continued]: But patients said this is important to us.So we've added items around those issues.So a very detailed, kind of qualitative process--but quite interesting to see how it develops and howpatient insights can impact on the final product.

PETER BOWER [continued]: We've done it in a variety of ways.The early stage work, we kind of reach outthrough various fora to get different people.So we'll-- in Manchester, we've got something called Researchfor the Future, which enables us to reach out to interestedpatients who've already signed up to take part in this sortof research.We tried a variety of methods to reach out to different groups.

PETER BOWER [continued]: Because we really wanted it to be quite a diverse groupof people.And that's really hard.And I don't think we were very successful at that,but we really tried to reach out to populationswho maybe wouldn't necessarily take part in research.Really difficult. We've got loadsto learn about how to do that well.But we've brought in a very broad group--or as broad as we could, and with a variety of experiences--

PETER BOWER [continued]: some who've taken part in trials, some who haven't--and drew on those.And then, the rest of the researchwas really reaching out to other people's trials.So we worked with other trials units, one in Aberdeenand one locally in Manchester.And they would essentially reach out to those populations.So though they're recruiting different populationsto those individual trials, we're

PETER BOWER [continued]: reaching out to those populationsthrough the other trials groups.So it's a mixed approach to recruiting people.And it's not easy.I've got to say, in terms of reaching outto get diverse populations, as a community of researchers,I don't think we're that good at reaching out and making surethat the people we bring in are representative.

PETER BOWER [continued]: But we're getting better.People are aware it's a problem, and Ithink there's more and more efforts to try and drawdifferent groups in.I mean, I'm from--Manchester's very diverse.And I think we've got a long way to goto reflect that diversity.But I think we're aware, and we'retrying to reach out more to a wider variety of people.

PETER BOWER [continued]: The start the project was fairly straightforward.We reached out to ethics and said, the sort of workwe're doing.We get approvals for that.Not that complicated.It's with volunteers.It's not invasive.You're just talking to people.So that's not that complicated.The complexities came when we weretrying to do our research through other groups.So they've got approvals.And then we need to add our study to their approvals.

PETER BOWER [continued]: And that's complicated because it's relatively new.So some ethics committees are not used to it.It raises some issues about how the two relate.So people have agreed to take part in one bit of research,and then you're kind of asking themto take part in something else.So it can be difficult to explain that.

PETER BOWER [continued]: Ethics committees can have concerns about whether patientsare going to understand those issues,whether they're going to be clear about what they're doing.And so, we have experience of doing this.And sometimes it works really welland sometimes it takes a long, long time.But you just have to be patient.You just have to be clear.I don't think there's any ethical issues raisedby what we're doing.

PETER BOWER [continued]: It's just an issue about explaining and being clearwith both the committees and the patients exactlywhat's happening, and who's taking part,and who's responsible for the research.So because we're doing research through another team,there were big discussions about, well,do you need to tell them it's beingdone by the University of Manchester?Or do you tell them it's by done by the University of Aberdeen?

PETER BOWER [continued]: Our view was, it's an Aberdeen project.They're adopting us.Ethics disagreed.And I still disagree with them.But they had the legitimate point, that actually, no.This is a Manchester project.And people need to know it's Manchesterworking with Aberdeen.And that was complicated.So we've come to an agreement about that.And the approvals are in place.

PETER BOWER [continued]: They just took a long time, a lot of discussion,a lot of back and forth.But it's a new approach.You do studies within studies--these issues get raised.And I think the advantage is, as theyget raised, ethics committees and the health researchauthority-- they're quite good at realizing there's a problemand then working with researchers to kind of dealwith that problem in the future, so it doesn't become something

PETER BOWER [continued]: that just every time becomes an issue.So we were a bit of a pathfinder in termsof how this would work.And we had our challenges.But I think we got somewhere.And I think we know what we neededto do in future to make it clearerfor patients and for the people involved in the approvals.

PETER BOWER [continued]: Essentially, what we wanted to dowas for them to adopt our questionnaireinto their routine measurement.So trials measure all the time.They're always sending out questionnairesor getting people into clinic to measure things.So we said, just add our measure to that.Now unfortunately, for various reasons,the trials we were engaged with-- theydidn't have a measurement planned within our time scale.

PETER BOWER [continued]: So they actually had to survey patients separately,which has advantages and disadvantages.It's complicated.And it's an additional burden for patients.One of the advantages is, it makes it different.So it's almost like asking for people's experienceof the trial outside of the main trial.So we're quite interested in how the advantagesand disadvantages-- and as we do more of this work,

PETER BOWER [continued]: I think we'll start to look at whether it's betterto do it separately, and measure experienceafter you've measured all the other things,or whether it's better to do it as part of the overall trialmeasurement.Our key desire was to reduce burdenfor the trials and patients.So they weren't having to ask-- get other lettersand so on and so forth.But we've only done this two or three times.So we don't know what's best for patients.

PETER BOWER [continued]: So again, that'll be something that we need to reflect on--talk to patients, talk to trialists,and actually work out, actually, what'sthe best way of doing this.And it'll be horses for courses, I'm sure.Some trials will do it one way.Some trials will do it in another way.And it's just about finding what'sbest for particular trials and particular patient groups.

PETER BOWER [continued]: What we're trying to do is analyze the data to producea report for the trials unit, essentially, to say,this is what patients are saying about your trial.And what we want to do is present that in a varietyof ways, and talk to trials unit and say, actually,what's helpful?Because it's, in some ways, saying, 50% of the people

PETER BOWER [continued]: in your trial didn't like what you were doing--might be helpful.We need to present it in a way that's practical.And what we--I guess what we described it is, is it actionable?They're not just going to feel, that's disappointing.And they're not going to think, oh, Yeah.That's fine.We don't have to do anything.That they were going to look at it, and think actually,there's interesting information here.We should think about this in the future.

PETER BOWER [continued]: And we're not suggesting trialists do thingson the basis of one set of results.But we're hoping if they do this all the time, they could say,well, actually, we've done five trials.These four or five-- but this trialseem to have some struggles with patient experience.So what was different?What can we learn from that?Also comparing between different units.So Aberdeen and Manchester and Bristol and Oxford--

PETER BOWER [continued]: how are they doing in terms of patient experience?And does that mean anything?Does it think-- does it mean that you're doing thingsslightly differently and there's things you could learn?There may be trials units that get really good patientexperience.And we could all think about, well,what are you doing that makes it so special.We had a talk yesterday at the conferenceabout sending Christmas cards to patients at the end--through trials-- to build up a feeling of community

PETER BOWER [continued]: and they were doing something worthwhile.And there may be things like that some trials units do--some don't--and that make a big difference to patients.So we're analyzing the data in a fairlystraightforward, descriptive way,and just giving percentages.We're thinking about how best to present that so trialistsfind it useful and interesting.And we'll sit down and talk to them about that.

PETER BOWER [continued]: And then the key thing is really whatdo they want to know that's goingto help them make things different in the futureabout how they design trials.I think it'll be-- just be a process of working throughwhat gets their interest, and what they thinkthey can do something about.Some things are really difficult.I mean, parking is a big problem for patients in trials.When they come into hospitals, theremay not be a lot they can do about that.

PETER BOWER [continued]: Issues such as burden, the amount of timeit takes, communication, results at the end of the trial--lots of these things are probably quite--not easy to deal with, but could be dealt with.And trials units can look down and say, actually,what do we want to make a difference?Because the key thing is, we want patientsto come back into trials.We want them to stay in the trialwhen they're in the trial.

PETER BOWER [continued]: We want them to come back to trials.We want them to talk to their families, their kids, and say,I've taken part in a trial.And it was really good.And if it wasn't really good, theyneed to feed that back so that the next person comingto the trial doesn't face the same sort of issues.So we're working out how to analyze it.But it's more about how we present it.

PETER BOWER [continued]: I don't think we did anything where I'd say we made a big--I think we may have made a couple of judgment errors.I think I was a little bit too confidentabout that what I'd done before was going to be acceptablethis time.So I think if there's an error, itwas me assuming the previous precedent was going to apply.And so that was probably the error.Apart from that, I don't think there's anything specific.

PETER BOWER [continued]: We-- doing research with other groups,working with other institutions always raises challenges.I think you always underestimate how long it's going to take.So how long different contracts departmentsare going to take to work things out.That was another issue.I'm not sure we could have done anything differently.Maybe I was just a little bit naive about how long

PETER BOWER [continued]: it would take.And we should have said instead-- itwas a relatively short study of 18 months.We probably should have said we'regoing to need two or three years to work this through.But it's just attention.You want to do it as efficiently as possible.You don't want to tell the funder that'sgoing to cost more than you think it should cost.So maybe I was a little bit naive in terms of the funding.But Yeah-- I think we--

PETER BOWER [continued]: a couple of errors maybe in terms of making assumptions,but I've certainly learnt from that.

AILSA DONNELLY: I do quite a bit of trainingwith people who are running trials.And my initial question is always, how many of youhave actually taken part in a trial as a participant?And fewer than a third--probably be much less than that.And I think, well, how can they really understandthe patient experience and the patient journeyunless they have actually felt it for themselves?Because talking to people who've done a trial--

AILSA DONNELLY [continued]: it's a huge learning curve for them.So I think if they haven't done it themselves--and then some of them won't--then it's my role to try and bring the patient perspectiveto them, so we can work together to make them betterfor everybody.Because clinical trials would benefit everybody.I'm a member of a patient and carergroup called PRIMER, which is allied to the central primarycare at Manchester.

AILSA DONNELLY [continued]: And researchers come to us and talk about their research,and we then give them input.And through that, I became involved in some initial trialwith Pete Bower and the team.And from that, having worked on that project,I then get involved in this one, which is called PACT.

AILSA DONNELLY [continued]: I was a public co-applicant on this project.And so, I did have a role, as such,rather than just being a contributorto the general forum.And my role, I think, was to workwith one of the researchers to facilitate the group--to make sure that things--to act as liaison, really, betweenthe public contributors, and patient contributors,and the research team.We had a group of five public participants.

AILSA DONNELLY [continued]: They were a mix of diverse backgrounds, although probablynot as diverse as we would like them to be.Diverse backgrounds.Some had taken part in clinical trials.Some hadn't.Some had a lot of experience of research.For some people, this was their first research project.So that was good, because we had both experienceand sort of fresh eyes.And we met regularly, and also hada lot of email communication in between meetings.

AILSA DONNELLY [continued]: They were sent a lot of information to digest.They did a wonderful job.They were very responsive.And our meetings, I think, were lively and sometimes quitechallenging to the researchers.We were involved at every stage of the project with the who,the how, the when, what to ask, when to ask it, how to ask it,who we should ask.We did part of the-- we looked through some

AILSA DONNELLY [continued]: of the cognitive interview transcriptsto give our feedback on that and makesure we were on the same lines as the researchers.Most recently, we were involved in co-designinga poster for this conference.And I think we were amazed at howmuch the public contributors actually got involved in that.And I was certainly amazed at the feedback they gave,and how much time they were willing to give to it.Because we had a short deadline.

AILSA DONNELLY [continued]: But there was a huge amount of input, which was fantastic.