We thought it would be just a really great next step just to have a little bit more open discussion. So we’re going to move to a panel just before the break, obviously with our two speakers, so with TJ and Nikunj, but also asking Ashley Slagle to join us. And so if you guys would like to just come up and take a seat, we can do that. What we’d like to do, we have some questions but also look to take some more general questions from you in the audience, and also take discussion as well. As we progress with this, let’s hear your viewpoints and your comments as well on the questions that we’re looking at. But I’m just thinking to hear first from TJ on the experience of being in a clinical trial, really pertinent points there that maybe help us to understand how we can do things a little better. And then obviously the guidance with the FDA. Both Ashley and Nikunj are ex-FDA, so I know they’re not representing the FDA here but they obviously have that background as well, which is really helpful. So hopefully we can have a good discussion here.
So let me just kick off with the first panel question. We didn’t discuss how we’re going to do this, did we. I don’t know, there’s a microphone there, and maybe Ashley if you—oh you’re wired up, perfect. And Nikunj is going to shout. There you go, look at that.
So I don’t know who we’re going to address this to first, but the first question is: It’s clear how the industry might seek the voice of the patient in development and selection of COA instruments, so I think this is kind of to the point, Nikunj, that you were bringing up around we need to understand what’s important to the patient, what their priorities are, and figure out what instruments then can measure those things more effectively. So, in addition to that, where else might the patient voice be heard more in the clinical trials themselves. Maybe we should start with you, TJ, I think you’d be the ideal place to start with that.
I feel like I’ve talked so much already. I will be brief so I can pass it over to Nik and Ashley. I mentioned earlier that the voice of the patient should be part of trial design. And I’m a continual advocate for someone out there, or the industry out there, leading by example and saying, we’re going to involve the patient in the life cycle of our product, from the conception of a trial, once there is some kind of scientific basis for it, we have a patient population we think we can address an issue they have, let’s bring them in. And let’s not just bring them in. I used the example before where a company brought in metastatic breast cancer patients, they had a two-day session with them, and that was great, they had all this input. And I said, what did you do with them after that? Well, we just did our trial. For the minimal amount of cost it would take, for the time and effort and money, why didn’t you continue to involve them? Why isn’t there a standing checkpoint? Are we doing this the right way? Are we communicating our trial the right way when we put it out there to the field, to the research physicians that are basically representing us? So I think the opportunity for the industry to have that patient voice is to have it consistently part of the trial, whereas if it’s not part of the trial, there should be a reason it’s not rather than the other way around.
To that point, TJ, in your experience, when you completed a trial, did anybody come and ask you about your experience or try to learn from you in any way or ask for feedback in terms of ways it could be done to make it easier for other patients coming through later?
Only because I already was giving them my feedback publicly. So yes, I will say that Merck has heard a lot from me. Not just that; as a voice, I’ve gotten to speak for thousands and thousands of patients. I don’t think their voice is heard, and I very minimally can represent their opinions, and I don’t think that a normal patient gets much of a chance to input what could have been done differently, what could have been done better, what means something to me, not just in a clinical outcomes setting, but just in the entire process of being involved in a clinical trial.
Thank you. And Ashley, from your perspective particularly in some of your consulting work you do, do you see how we’re involving the patient more perhaps, or any examples?
Yes, thanks Bill. When you first shared this question with me, I was reflecting on it, and I guess I always need to be a contrarian, so I’m going to disagree with something. The question starts with, “it’s clear that as an industry how we might seek the voice of a patient in developing and selecting COA measures.” I’m not sure that’s yet entirely clear. And so I agree with TJ’s point that the patient should be involved from start to finish across all aspects of drug development. And in this one piece, we have some really good methods for qualitative research where we interview patients and we talk to them, we do concept elicitation, cognitive debriefing. But then we take that information from them and the research team goes back and huddles and we make decisions. And I think that’s a place where we need to better engage patients to be involved as expert members of the research team, so when we’re developing coding frameworks, conceptual frameworks from those patient data, the qualitative research, patients are involved. When we’re setting up qualitative study protocols we’re engaging patients. When we’re making decisions about item reduction even, the patients should be active members of the research team. Beyond developing and selecting instruments, I agree that it’s critical to have patient input on clinical trial protocol design.
This is an anecdote. There was a study that was having a lot of trouble recruiting, and so they brought in some patients to help them look through the protocol. And it was as simple as these children were in school when there were clinic visits schedules and so they just couldn’t participate. And that seems so, sort of duh. But we’re all experts in our own pieces, and if you’re not specifically thinking about that sort of stuff, those things get overlooked. So I think having the patient voice to just point out where we’re making bad decisions, frankly, can be really helpful.
A couple more points I want to quickly make, and then I’m anxious to hear what Nikunj has to say. I think two-way education and communication is really important. So we’re trying to involve the patient voice increasingly, but I think we’re also doing a bad job of communicating back decisions and findings that we come to that have incorporated the patient voice. And so to the extent that we can do that, I think it’s really important. And I’ve been fortunate to work on some programs where patients are part of the research team, and they’re integrated fully into the research team. But before that, we did some workshops with them so that they could understand the regulatory setting, patient-reported outcome development, and that was really helpful to facilitate their participation on the research team.
And then the last point I want to make. I’m an advocate for involving the patient voice in everything that we do. But my pragmatic side says that sometimes that slows the process down a little bit. And so I think we really need to be careful that we have patients involved and we listen to the patient voice at important points, but we don’t just involve the patient voice to involve the patient voice without a purpose, to the extent that it could slow down our research and slow down getting good drugs to patients. So we just need to be careful that we don’t let that happen in our effort to engage patient voice.
I agree with Ashley by the way. I think Ashley brings up a really good point about communicating how a patient’s voice was incorporated back to patients and those are some of the things that we are trying to do internally. I can’t say we are great at it. I think we are working on it, I would say, and I think we really need to continue to do that to really show them the impact that they made in designing a trial or designing an endpoint for example, especially if you have a PRO endpoint, or any other clinical endpoint honestly.
And I think another example was, when do you involve patients, I guess. Do we need patients being involved in every single process of developing measure throughout the trial. Probably not every single but I agree, at critical points where we need patient input. And if you’re not doing that I think you need to do that. And it should not be just checking the box, we talked to a patient somewhere out there, and we check the box. It has to be well thought out, why do you need the input from the patient, what kind of input are you looking for from patients. And what are you going to do with that information. So I think it really has to be a thoughtful approach.
And I think to go into your question, Bill, about beyond just PROs, I think one of my personal goals when I joined AstraZeneca was really get involved earlier. How can they integrate patient voice early development. A lot of time your goals are getting PK sampling, getting clinical pharmacology data, maybe initial safety signal or efficacy signal for example. I think patients can really play an important part in helping understand the tolerability of particularly oncology agents. They are pretty toxic. And how do we decide what dose you want to go with. I know clinicians usually complete a CTC if you are familiar with that and get toxicity but I think patients may have a different perspective on it and I think we need to understand that. And if you are in the PRO world you probably know PROCTCAE. I think that can play a great role in an earlier setting to really understand that.
Another challenge I guess I have sometimes in using PRO measures, and going back to your question about what can we do about additional new things. Early trial, smaller sample size, so when you interpret the data you only have like 20 patients or 15 or 30. A PRO may not help you—it may get you some answers, but you’re not going to be able to make any conclusion on it. So I think patient interviews can play an important role to qualitatively actually really get an idea of how patients experience toxicities or if there is any benefit to the experience to really understand—I mean that’s not going to be submitted for drug approval, but it will help you design your future studies, future Phase II program or Phase III programs. I think that’s really important to get patients involved much earlier, in my opinion.
Thank you. TJ, just to build from that a little bit. How reassuring, do you think, it would be for a patient if they were approached by a physician and in learning about a clinical trial they discovered or were told that actually this has been designed with patient input and the things that we’re measuring in this trial have been determined as important to patients with your condition? Would that be reassuring or even an influence for a patient to think, actually that trial sounds like something I’d like to participate in?
It may be. It should be. I don’t know in terms of circumstances; it probably won’t matter too much. Where I think it becomes important so much that you decide to be on a trial or not is more your level of comfort, that you don’t feel that I am being subjected to a test. That there is somebody looking out for me, hey there is someone with melanoma that went and designed this protocol with us and helped us to clarify language of the informed consent and really gave input in what it’s going to be like to be a patient on this trial. I think the average patient would feel a little more at ease in a clinical setting, where you’re going in, especially in an oncology type world where this is potentially a life or death situation, or a very serious diagnosis, where someone is overwhelmed, just to have the reassurance that someone was sticking up for them and giving their voice before they got there would help make a patient feel more at ease.
Thanks you. Any comments anyone would like to make? Or additional questions at this point?
AUDIENCE MEMBER 1
When you presented about PFDD I was thinking about, is there any other type of drug development has to be patient-focused, right. And the manifestation of PFDD in terms of clinical research is in terms of decentralized trials or [unclear 14:34] trials, right. If you think about recruitment being a challenge and 70% of patients live more than two hours away from clinical sites, despite that being a no-brainer that you have to make trials for patients, if you look at the number of [unclear 14:47] trials right now, it’s like ten. So what do you think are the barriers in designing more decentralized trials, like why is that number so low? And then why is there only like—Science 37 is doing some work in that space but I’m sure CRF Health is also in that position. So can you talk a bit about that in terms of barriers that you see in trials becoming more [unclear 15:10] and decentralized and what AZ is doing in that space?
I think, going back to your question about the principles of PFDD are how do you reduce patient burden. And you can apply those more, what TJ was alluding to and what Bill was talking about, how do you reduce the number of assessments. When I am looking at study protocol and I’m trying to figure out what should be my PRO assessment strategy—is it going to be every cycle, every other cycle—and I look at everything else in the protocol, the sampling, I cringe, I’m like how is a patient going to be able to do all those things. It’s like a two-day job basically, you go one hour you go here, another hour you go to this. So it’s a lot to ask a patient to complete this. So going back to how do you ease the burden, and I’m thinking about that when I’m actually proposing assessment strategy. How am I adequately collecting the PRO data so I can still interpret it, but at the same time not burdening the patient by asking 40-50-60-70 item questions every day or every week. So what are we doing something about that.
I think part of your question about decentralized trials, I think that’s probably beyond the PFDD but the principles are there. Especially like a rare disease for example where you may have one center, like Hopkins for example, and patients are flying in from all over the country to come to the center because the specialist is there. Maybe that’s the only center that’s recruiting patients. How do we help patients minimize travel time and expense and things like that from Alabama. So I think that decentralized trials, we probably can’t do complete decentralization. I would say it’s going to be somewhere in the middle probably. There are some things the patient can do at home, like ePRO for example, I think we are already doing that, so they don’t have to come to the site. But there could be additional things that could be added on too, that patients could do at home. And I think some of the things we’re internally actively looking at that, how do we ease patient burden and leverage technology.
There’s a question here at the back.
AUDIENCE MEMBER 2
I saw a presentation, I think it was at [unclear 17:41] by Bayer about how before they used eCOA in a trial they gave a sample of their own employees the tablets to keep track. And they had terrible compliance over the course of a month or two. They even had someone who gave a different DNA sample than who the participant was and used it to design their trial. I sort of wonder, how often do you—or what are your thoughts on reaching out to maybe oncologists who had cancer and participated in a clinical trial, or someone who works in research design who got cancer and participated in a clinical trial. There’s got to be a fair number of these people. Or ask employees at your companies about their thoughts on design. It seems like it could be the most instructive person to ask about their experience.
I actually want to follow up on a point that you made about the employees who had terrible compliance. In part, that’s because they didn’t have an incentive probably to complete that instrument, that ePRO device. But I think one of the things that we need to do a better job of to minimize that occurring in clinical trials is communicating with patients about the importance of why we’re asking them to complete these 40-50-60-70 questions, whatever it is. Because I think what happens a lot of times is we spend a lot of time selecting the measures, we spend a lot of time developing good fit-for-purpose instruments, we put them onto an ePRO device, and then we just deploy them with nothing to the patients. We all get those surveys, request for surveys through emails. If they don’t hook me in the first sentence, I just delete the email. And I think we need to do that with patients where before they’re completing these, or we’re asking them to complete these assessments, as part of the training and getting up and running in the trial, we’re explaining to them why this is so important and how these data will help future patients and help them, and we’ll share the data back once the trial is over, really give the incentive to complete these instruments. So I don’t think that the example of the Bayer employees is directly applicable to the patients in trials, but I think it informs us that we need to help the patients understand better what those incentives are, why it’s important to complete the questionnaires in the study.
I have a question for TJ here. From your perspective in the study, and I’m not sure if you had to fill out questionnaires or not, what do you think is the motivating for you to fill those out? And what would be the demotivating factors?
As one of my TransCelerate projects we’re designing a patient experience questionnaire. And I kind of had to go through it like, why would you want to fill this out, I don’t get it, why would I want to do this? I’m just checking a box. Or you’re checking a box that I’m filling this out for you. Tell me what you’re going to measure and what we’re trying to learn from it. As a patient, my motivation, you saw in the presentation what my motivation was. But there’s also, I think for most patients who participate in trials, there is some sense of altruism. I’ve heard from other patients that this won’t help me. A good friend’s father participated in a trial where he passed away from side effects. And he said, I don’t think this is going to help me, but I think that it will help somebody else. And that sense, I believe, runs through a large majority of patients who participate in trials. So to engage them in a questionnaire, it shouldn’t have to be like a rote part of it. We’re asking these questions because we want to measure X, whatever that X is, there’s a reason we want to measure it, and this is what it is. Or it fits in with your trial endpoints because it will help us understand what your body—whatever that measurement is, it gives me as a trial participant an incentive to say, yeah I see why we’re doing this and why I’m inconvenienced, and I’m willing to help.
Thank you. That leads us quite nicely into the next question actually, which is more about, as we think we’ve got an audience here who are engaged in electronic solutions that capture data from patients. So if we think about those, it might be just collecting patient-reported outcomes on a smartphone, it might be collecting activity data using a wearable tracker. It could be patients doing blood pressure monitoring or collecting their body weight using sensor-based systems at home. As we think about all of those, what would be the relevant way to perhaps seek patient input into the way we do that, and maybe the way we can do it better? Are there particular things we should be looking at, do you think? I don’t know who to ask first, but—Ashley, okay.
I do. I think we still should be asking of course, even though we’re focused here on electronic data collection, it’s still important to be speaking with patients about what to measure, what questions to put on those devices. But how to measure. So we heard an example earlier about patients—Nikunj mentioned that some patients are uncomfortable using their own personal devices in a BYOD setting. But I’ve heard the flip side where there was a group of patients who were, I think teenaged boys, who were unwilling to use a provisioned device. They just absolutely refused to carry their own device and a provisioned device. And so, I think it’s critically important to understand from your specific patient population what’s their preference, what are they willing to do. Is the device annoying. I know we set alarms to help motivate and limit missing data, but some alarms, we’ve heard later from patients, are so annoying that it ends up causing them to turn the device off and that creates more missing data. So I think having engagement with patients to understand their preferences around the device selection, but then also some of the things that we kind of think of as rote now where we’re just, oh it’ll have an alarm, but how we implement that alarm we could benefit from learning from patients. I think what patients are willing to tolerate in terms of assessment frequency and length of the questionnaires, number of items, even though it’s on electronic platform and it’s pretty simple sometimes to scroll through and answer the questions, we still need to do a better job of gauging where is that threshold for the patients in terms of burden. And I don’t think it’s just the number of items or the interaction with the number of items on the device, whether it’s a simplistic or an annoying device. But I think it’s also, are the questions relevant, did the patient see a benefit in that, and so understanding the complex issues around the assessment and the questions and helping better understand burden and all of those different characteristics rather than just length and is the device functional.
Jump in real quick. I will say something that is going to be obvious to everybody. You will never make every patient happy. And as a patient voice, I can say that with a clear conscience that it’s going to be impossible to design the perfect trial, to implement the perfect wearable or electronic device that will measure everything that every patient wants. So to that level, if you don’t make—don’t chase perfection and give up greatness, or whatever the saying is. Go for the measures that are meaningful, make it as easy for the patient as possible without having to burden yourself with, well what about this patient, what about this patient, what about this patient. Because there is no one silver bullet that’s going to make every measurable outcome easy to access and easy to get from a teenager to a more elderly population. And give them the exact same thing that will enable them to give you what you want.
I was going to add to that, your question about how do we get patient input I guess. I think one of the things you can do is learn from your mistakes. At the end of the trial you could do an exit survey for example, where you could ask patients, what are the pain points, or what did you like the most about it. They’ll be honest and they’ll tell you, and how do we integrated it so that the next time we’re designing assessment strategy or eCOA strategy we’ll learn from it and maybe try to optimize it, come up with something better. And besides that, you can always do patient interviews prior to the trial probably to understand. But you may not get—remember it’s a small patient you’re going to ask right. And where I sit in, we have trials in China, Japan, Korea, Europe, US. So you’ve got to think about all those patients all around the world. And one size can’t fit all probably. So you have to consider those different scenarios to come up with something that will work for, hopefully, all patients. But it may not work for all patients, but really think about that and how do we get better at it.
That’s good. Any comments or observations or other questions from folks?
I’m ex-FDA and still very supportive of the work that FDA is doing. And I think they are helping to lead with industry and patient advocacy groups this patient-focused drug development. I think the phrase that TJ was referring to is, don’t let the perfect be the enemy of the good. And I learned that from Janet Woodcock who leads the Center for Drugs at FDA and she says that all the time in relation to patient-focused drug development, and I think it’s critical for all of us to remember that. Anyway, I just wanted to put a plug in for FDA, I do think they’re trying to do that. It’s a big shift. Different personalities, but generally the culture there is to not let the perfect be the enemy of the good.
That’s great. Those of you in the audience, just a quick show of hands. How many are working either in pharma companies or in CROs? Great, so we’ve got a good number. Of you, have you got any comments on this last point, really, in terms of what are you doing at the moment? Are there things that you’re doing within your research teams and your study teams to hear the patient voice as you design trials or maybe as you monitor trials? Has anyone got any comments or observations about the kind of things that you’re engaged with?
I guess if nobody has, we have to assume you’re not doing anything. Anybody brave enough to—okay we’ve got somebody at the back.
AUDIENCE MEMBER 3
I can’t actually give comment to specifically what pharma companies are doing. I know that they are doing things. I kind of hear the stories. I’m actually here myself to learn more about what we’re doing and how we can do better at things like this. My current role is actually on setting up some of that protocol design, implementing that design out to the sites and to actually see how that is effected on the patient so that we can get the data back and interpret it properly. So I know that there are things being done. I’ve heard of more of our trials implementing at the end of the trial something like these patient assessment surveys at the end. How good did we do running this trial, what was your experience. I know a lot of those things are things that we’re pushing towards doing, providing the lay summaries at the end of a summary so that a patient actually understands what they contributed to. So I know that there’s a lot out there but I personally don’t have that experience yet, and that’s why we’re here to learn a lot more about what we can do to make that experience better.
Great, thank you. Somebody else here as well.
AUDIENCE MEMBER 4
Thank you. I just want to let everyone know that I recently finished a trial, it was a field test trial. And what we did is we utilized about 200 of our employees to use on their personal phone or their work phone and test the digital platform, and then give feedback on that—was it easy, how did it work. Well we’re still waiting on the data for that, but I think overall it was pretty good. I mean initially there were issues, technical issues and bugs, so we had to halt and kind of do some fixes around it, and then as we rolled it out again, we gave instructions to them on a daily basis, almost daily basis of that they needed to do. So we did kind of spoon feed them with what they needed to do, but still we wanted to test how did the digital platform work, and how did they find the experience.
Thank you, that’s great. We hear this drive as well in the eCOA space. We talk about usability, so we always make sure that our devices or our solutions are usable to patients. But there’s a new word that we’re starting to hear a little bit more which is feasible and feasibility. And as we develop these digital solutions, are we doing feasibility studies, and we’re doing a quick proof-of-concept, perhaps in a group of patients where they’re able to use all the things that we’re wanting them to do in the protocol, just to see if that particular design and that particular solution is feasible, it’s going to be usable by them in the context of the clinical trial rather than just can I use this particular handheld device and answer the questions. It’s more than that, it’s about whether it fits with their lifestyles, with the way that they’re able to operate when they’re back at home doing these things in a trial situation.
I mean, I guess just maybe a final comment. Any comments on where feasibility testing might fit in this big picture of incorporating the patient voice into our trial designs? I know I’ve thrown that at you.
I think it’s really when you are implementing a global study that involves a hundred-plus sites and you got 25-30 countries maybe, and you’re doing an electronic administration, I think you better get your strategy right and you better figure out where are the problems that may—so I think patient input should be in there to, you know, do they actually understand the way that ePRO is designed. Will they be able to actually fill out the questionnaires. What would they do, how are they viewing the questions and are they scaled appropriately represented. So you need to get that right, otherwise what you’re going to run into, the problem comes to me basically in the middle of the trial and we’re getting calls from different sites and these things are not working or patient had a problem. And I have to figure out in the middle of the trial, what do you do about it. And then you’ve got to do a software update and if it goes right that’s great, if it doesn’t go right that’s not great. So I think you’ve got to get all those things figured out and I think initial input from patients, and I would say experts as well, because people who have been doing this day in and day out, they know how things work. So I would value their input as well to make sure that’s being taken care of.
I’ll agree with that. It’s not just the patient voice you’ll want to hear. It’s the site voice, it’s the trial admin, it’s the investigators where appropriate. It is one big huddle. And while patients appreciate being part of the huddle, we’re certainly not always the quarterback, for lack of a better word. Each participant in that trial will have input on the feasibility of making this work, and no voice should be that much louder than another one, because in the end it’s up to the sites to implement a solution that will work for the person who is filling it out and we’re all in this together so I think the co-design is an important part of feasibility.
Great. Well that just leaves me to thank our speakers and panelists. That’s been an absolutely exceptional first session. So thank you TJ, to Nikunj, and to Ashley. Let’s give them a round of applause.
[END AT 36:03]