Download Premium Content

First Name
Last Name
Country (Citizenship)
Opt-In to Future Emails
Thank you!
Error - something went wrong!

A Hybrid Online Trial of Lunasin in ALS/MND

June 13, 2017


Speaker: Paul Wicks, PatientsLikeMe


I’d like to introduce Paul Wicks, who is the VP of Innovation at PatientsLikeMe. Now, PatientsLikeMe have presented at these types of forums with us for—three times, is that right? And so we’re very excited to have Paul back again for his fourth time today. He’s been a regular presenter and a regular contribution to this forum, but also to the thought process behind everything that goes into how we treat the patients and how we start to engage these patients. So PatientsLikeMe, as I say, have been presenting for a while at this forum and are very much trying to empower patients and truly engage them throughout the clinical process. So without further ado, Paul.


Good morning everybody. Well it’s a great pleasure to be here again and to share some of our experiences. You know, I find often at these conferences, it’s great to hear all the interesting projects that people are working on and the case studies and what’s gone really well. I hope today as well to share some of the things that we’ve found to be a challenge, because really that’s the most important thing that we can share, so that if there’s a landmine we’ve stepped on, hopefully you don’t have to step on it yourself. So we’re going to be share some experiences and talking a lot today about a condition that is called motoneuron disease in the UK, or ALS or Lou Gehrig’s disease in the US. I’ll probably be calling it ALS throughout, but just to clarify, that’s what the condition it.

So when I say ALS, I think a lot of you now may think of this. So in 2014, my community—I have a PhD in ALS, I’m a caregiver to a family member with ALS—hit the jackpot. And every non-profit, every disease, every awareness organization has always wanted to go viral. And the Ice Bucket Challenge happened almost entirely by accident. It was sort of initiated as a bunch of golf buddies showing off to one another and all of a sudden it hit onto ALS and it really took off, and before we knew it we had Mark Zuckerberg dunking himself and I think Bill Gates made a robot to do it, and you know, it really sort of went everywhere.

But you know, behind the scenes, I’ve been doing this research for my entire adult career. Motoneuron disease is a really touch area. So prior to the Ice Bucket Challenge, which is a fond memory, most people think of Stephen Hawking. And if we look at films like The Theory of Everything, when that won the Oscar just after the Ice Bucket Challenge, again it was a key moment for awareness. But just to underline the severity of the condition, this is a disease with a 2-5 year survival. Most people are affected in the mid-fifties, so just as they’re hoping to come up to retirement and they’ve got kids and sometimes grandkids, and it causes a progressive disability where it might start in the limbs, or in the throat region, but people unfortunately lose muscle strength, and gradually that progresses to different areas until eventually over the course of the condition people are completely paralyzed. For the most part their mind is intact, and they’re sort of fully aware of everything that’s going on, so it’s a devastating condition. And unfortunately, the trials that we’ve done, there’s somewhere in the nature of 50-60 trials in the literature, there’s really only one drug so far that’s been used in clinical practice. That maybe slows the disease down by about 2-4 months. The FDA just approved another drug last week, which might have a sort of similar effect. So small things kind of building up on top of one another, rather than a silver bullet. But for so many of the trials that we’ve done, they’ve been ineffective. Or even worse, a number of them, maybe as many as a half, experimental treatments have actually ended up making patients progress faster than the placebo. So it’s an area where we have to be extremely careful and extremely cautious. But as you can imagine, the community wants more trials, more studies, they want to investigate things much much faster. So we at PatientsLikeMe, a company founded by a family affected by ALS, are really motivated to do this.

So PatientsLikeMe is an online community. We’ve been going for a long time now in the tech world, so we’ve been around since about 2006. And we started off in motoneuron disease. And what we offer to patients is this personal health record of a sort of a patient-reported outcome type of tracking mechanism. But instead of it just being an electronic data capture that goes to a database that just scientists can see, this is a tool that the patients themselves can see. And we aggregate up the data, because it’s all coded against ontologies and it’s being designed by scientists, so that we can aggregate up those reports and an individual patient can see how they’re doing relative to other people. So it’s a very powerful tool because all the patients’ data is shared with every other patient.


So by contrast on LinkedIn, if you and I want to look at each other’s profiles we have to sort of authenticate one another. On PatientsLikeMe, all 500,000 patients across 2700 conditions, they can all see all of each other’s data. Now, if you think of how that’s different to, say, your clinical trial data capture systems, that’s like blowing them wide open and, you know, putting them on Wikileaks.

So how do we do this? Well it’s pseudo-anonymous. All of our users sign a user agreement when they sign up, being clear about how this data is being used. And the types of conditions that we specialize in are these more serious conditions. So in terms of the risk-benefit tradeoff that people are making, this is not people with common cold. These are people with ALS, multiple sclerosis, HIV, mood disorders, IPF, that type of thing. So they really feel like there’s a lot of benefit in sharing that experience. So to date, we’ve generated tons of structured data, so symptom data, treatment data, and that’s gone off to various places like the FDA’s patient-focused drug development group. We work a lot with the pharmaceutical industry, and that’s a lot of how we’re funded as a for-profit company. And we’ve been able to do a little research with it. So I think when we first started, you know where I came out of academia, the idea of doing research over the internet as opposed to in a hospital seemed kind of outlandish, because on the internet no one knows you’re a dog, and what’s the data quality like and all that. So we’re going to get to that in a second. But I’m very proud to say that the research team at PatientsLikeMe has published nearly 100 publications now in peer-reviewed literature. Everything from some of the studies I’ll be describing to supporting the development of PROs much faster to describing the unmet needs of different conditions. So this talk will mostly just be talking about ALS and the attempts that we’ve been having to speed up research through what I call a sort of hybrid virtual trial.

So just curious, from show of hands, who’s heard of the remote trial, Pfizer’s overactive bladder trial from a couple years ago? Okay, so about a third of people.

So the history of virtual trials has been patchy. You know, everyone’s got an innovation department. Everyone knows the internet is going to be big. And as people have attempted to sort of take these big hundred-million-billion dollar behemoths of Phase III trials online, it’s been very difficult. It’s kind of a binary process. You can’t just put a little bit of it virtual, you have to try and figure it out. So when Craig Lipset talks about this from Pfizer, that they talk a lot about what they did with an overactive bladder trial where they tried to get all the actual visits to be done in people’s homes. And a lot of the difficulty was actually getting their regulatory people to sign off on this and then having a sort of multi-center research ethics committee in the US sign off on this. So a lot of that process of setting that up, and the electronic data capture and all the rest of it, was where the meat of it was. Unfortunately, when they actually went out to recruit patients, not enough of them were interested enough in the study to take part. And so, despite the cost and the exercise and all the rest of it, I think only a couple of dozen patients actually went through the process from a target of a few hundred. And Pfizer have actually been very open about this. And I think the risk is that that’s made people nervous, that you know, if the first time we tried something innovative it wasn’t an immediate home run, where is the ROI, that perhaps it might be too scary to attempt to do something like that. And I think that’s hampered innovation.

So where we’ve really gone is to say, well can we do a hybrid virtual trial. Can we take some of the best elements of a traditional RCT and combine it with the convenience of doing some more things at home and make a mesh of sort of a high-tech high-touch approach. But if you’re interested in the history, I’d go and recommend looking up Craig Lipset and REMOTE and OAB, overactive bladder, is the search term.

So to focus in a little bit then on ALS, I think one of the first questions if any of you have data mined this is to say, well how good is this community in terms of representativeness, so one thing that’s important to say, in terms of the demographics of who’s online. So usually people think of online systems as being, well maybe it’s very young people or very educated people, that type of thing. These are the demographics. In the first column of the registered user base we have on PatientsLikeMe. And the other columns here, this is basically the entire research literature of who are the types of people that take part in clinical trials. And what you’ll see is, you know, we’re certainly within comfortably about half a standard deviation in terms of age. We’re slightly more biased towards females but not that much. And in terms of the one licensed drug, really it’s all, you know, we actually have fairly representative systems here.

So with an online system, if the sample that you have is large enough and you actually recruit widely through Google AdWords and Facebook groups non-profit partnerships, it is possible to attain a group that is somewhat representative. But it depends on the condition. So for example, in a condition like lupus we may be under representative in African Americans. Or in a group like HIV, you know, you’re only going to get a certain type of user base on their insurance or what have you. So I think the case here is going to be specific to what condition. I think it can work in other places. It’s just important to say, it’s important to understand each condition and the nature of that.


In terms of the data, we’ve done a number of studies we’ve published that say that self-report patient-reported outcomes in this disease correlate above .9 with a clinician’s interpretation. And we’ve also done work to go and actually check through the IMS claims database. About 95% of patients can immediately match that if they say, you know, they are an ALS patient on PatientsLikeMe, we can find them very quickly in a claims database on the basis of their demographics. So I think we have some confidence that these people are who they say they are, they’re representative enough, the data is valid enough. So with that, we’ll sort of dive in a little bit.

So in terms of some of the data that people are reporting then that led us to this place, here is a set of the treatments that people say they’re taking. And we capture structured data using RxNorm and other databases, information on prescription drugs, as well as nutritional supplements. Lifestyle modifications, you know, people have listed things like dogs, cats, naps, prayer, range of motion exercises, one good treatment for spasticity appears to be yoga and red wine, you know, as an alternative to Baclofen, no side effects. Well, some side effects, depends on the dosage. And assistive equipment. And so you know, this is quite different from how we may think of people’s electronic medical records where you know, some people won’t tell their doctor they’re taking Coenzyme Q10 because they don’t want to be embarrassed or something like that. But the most frequently taken treatment is this drug up here, Riluzole which is the licensed treatment. And then if we say, well how effective do patients think these treatments are, so this is sort of amazon.com reviews if you like, of the treatments, and I’ve sort of compressed a bit of the information. But the blue colours, the dark blue, is to say this is the percentage of patients who think that it’s majorly effective for them and that’s just a perception, it’s not an answer to a real outcome. And the lower grades of blue are to say they have less confidence that it’s effective. And then grey means they can’t tell whether or not it’s working. So remember that Riluzole drug, it’s down here. Patients can’t tell that it’s working. And so a lot of people come off the drug, they can’t really detect. In fact it’s only a sort of C9ochrane meta-analysis that shows the 2-4 month survival improvement. But you can see her lots of off-label use of nutritional supplements and that type of thing. Actually lower down here is lithium which we’ll get to in a second. Or even things like creatine, so people are sort of going online and they say well creatine is used by bodybuilders, I’m losing muscle mass, so maybe I should order this stuff off the internet and try and bulk up. So you know, we’re seeing a lot of people self-experiment.

And in terms of how much data that we have from the engagement that we have, because people are able to find other patients like them in the site, they’re able to build community. If we compare the amount of data that you get in a typical trial, so this is ALS functional rating scale data. So 48 is me, 0 is Stephen Hawking, in terms of function. Most people lose about a point per month as they progress down. So in a typical clinical trial projectory, so PRO-ACT is a shared database of all the trials that have been done in the last few years that are being made available via consortium. And as you can see, most of the data only lasts 2-4 years in terms of follow up. This is the densest data we have on PatientsLikeMe and you can see, because we’ve been going for a long time, we’ve got a lot more longitudinal data. And that’s important, we’ll come back to this in a second, because it means that what we may be able to do—and this is the way we’ve chosen to do this—is in the case of the virtual hybrid study that we’re doing, is we’re not using a placebo one, it’s entire open label. And we’re using these  historical controls, closely matched to each of the treated patients, in order to try and figure out what’s going on.

Part of our motivation for this is we’ve been doing a lot of research about what patients find satisfying or dissatisfying about taking part in a clinical trial. So we’ve got a large rolling survey, it’s now up to about 10,000 completes across all different conditions, but this is nearly 350-ish patients with motoneuron disease saying what is the most important thing to them about taking part in a trial. And you would think with all I’ve said, that any time someone opened enrolment on a clinical trial, you know, it would fill up just like that. That’s not the case. So obviously these people are very motivated, they have a great deal of altruism. But also they’re sort of saying that it’s extremely important that they have an opportunity to improve their own health. And of course with all the disability that the condition can bring and the needs that it puts onto caregivers, asking people to come to clinic once a month, have MRI scans, and have lots of blood draws is very burdensome. So you know, they’re trying to work out a lot about how much distance they’d have to travel. If you live in Boston, great. If you live out in the sticks, you’re talking about a long journey to get to a major center. And you do see here, the possibility they might be given a placebo is extremely worrisome to some people, as well as the length, and even being reimbursed for their expenses.


So with that background in mind, I then want to tell you the story of what can happen when these types of conditions are not being met. So back in 2008, a very small study was published in Italy about a drug called lithium carbonate. So it’s the same drug used for a mood stabilizer in bipolar disorder. And this study said that in 16 treated patients and 28 controls, the patients taking lithium pretty much halted their progression. And the researchers that did this held a press conference, and the title was called Lithium Carbonate Halts the Progression of ALS. So of course, all the patients immediately read this, in fact they translated the Italian abstract using Google Translate and started showing up to their doctors and demanding lithium off label. And within six months, ten times the number of the original study, 160 patients, were on our website submitting the self-report version of the exact same measure that was used in the trial. And even before we as a science team had done anything, the patients were taking their data and putting it into a Google spreadsheet. And they were taking our historical control data and they were trying to figure out, is this working for me. So they were doing a participant-led research study, which I think is probably the first of its kind, and the whole thing was orchestrated by a quadriplegic Brazilian patient who didn’t speak English as a first language, working through Google Translate and using an EMG switch to coordinate 160 patients all around the world, with the assistance of a caregiver in America who had a PhD in geology. And so she was doing the statistics. And this was really a worldwide movement of people going up to the doctor, trying to get hold of lithium, and if the doctor said no, find a different doctor, you know. And so they were listing their side effects and the blood levels and what was going on. And what we were able to do is to actually upgrade our system and capture that data more systematically and actually to go back and look at controls. So we took all of the people who were self-reporting they were taking lithium. So an example patient might be the one in the light green. And we said okay, so for each of these patients, who would be the best match. So if you’re thinking of the demographic table of a trial you might thing well, if everyone’s 50 and half of them are male, maybe I should match them to people that are also 50 and half of them male. Yes, but within that group, you’ve got Stephen Hawking type people who are always going to be very flat, you know, or you’ve got people who are rapidly progressive. And if the mix of those is wrong, you won’t have a good match. So what we did is we built a matching algorithm that said okay, at the point that they’re taking the treatment, both orange line and the dark green line that they’re both similar to how the patient was at the start, what we need to do is match by reducing as much as possible the difference in the slopes. So for each of the patients that volunteered to take lithium, we matched between 3 and 5 historical controls by minimizing this sort of gap here. And what we found very quickly is that lithium doesn’t work. We found that out about six months after the patients had started trying it, and we actually shared this with the community at a big ALS symposium, and you know, the poster was sort of mocked.

So we actually shared this. This was written up in Nature Biotechnology. We uploaded the entire data set. So we de-identified it, we uploaded the algorithm, and we made everything open access. So I’m sure, you know, given that you’d all expect that the field took a look at this and said, okay lithium must be dead, we won’t bother wasting our time with it. Right? Wrong. The academic community ran four randomized control trials at a cost of about $20 million, and they came to the same conclusion three years after the patients did, by which time all the patients who had figured this out died. And during that time, we didn’t have another shot on goal, because we were so tied up with this. So I think the patients that were still around found that incredibly frustrating that this experience hadn’t really been used, and it was partly because it was too new, and there were all sorts of ethical questions. So right now around the world, there’s about a dozen bioethicists whose grant is doing nothing but studying this one case study to say, was this the right thing to do, who was the PI, was this ethical, is the Brazilian guy responsible, what happens when he died. Does that mean that now the responsibility passes to someone else. What if something had gone wrong, who could we sue. All sorts of big questions.

The plot thickened a couple of years later when actually a new cohort of patients—because if you think with the survival, you know, people come in, they could read all of this history and look at what happened there. They started using the exact same tools whilst they were within pharmaceutical company sponsored blinded Phase III RCTs. So now imagine you’ve got the classic several hundred million dollar study and 10% of the patients in your sample are adding their data to PatientsLikeMe, and they’re talking to one another and they’re tracking their data. And they’re saying things like, did your doctor tell you you have neutropenia? Oh yeah me too. Okay, so we’re not in the placebo group then, because if you look at the pattern for the drug, it says neutropenia is a side effect. So we’re going to go on Google Spreadsheet and we’re going to make a column that says neutropenia, yes or no. And we’ll put the yeses in one group and the noes in another group. And we’ll start running T-tests every day to try to figure out what’s going on.


So as you can expect, chief legal counsel at this company was unhappy about this. But you know, the question is what can one do. If it wasn’t on PatientsLikeMe, it could be on Twitter, it could be on Facebook. And I think it’s worth asking you know, going back to why people take part. Why might patients be doing that. In this case it’s a traditional trial. Patients weren’t told how they were doing. They would submit the data, it would be secret from them. They were blinded and so if they said, oh I’ve got neutropenia, does that mean I’m on the drug, the trial staff would have to go la la la la la, can’t hear you, pretend that you don’t know. Right? Having to treat people like they’re five-year-olds, it’s ridiculous. When the studies are published, most of the time they’re published closed-access. There’s not a lay summary. If there’s a presentation at a big symposium, it’s not being videoed like this talk is, so the people that took part can’t even hear what the results were. So to some extent, this is patients kind of sticking their fists in the air and saying okay, if you’re not going to partner with us, this is what we’re going to do. And it’s really a case of I think citizen power that we need to listen to.

Another study for a drug called NP001 by Neuraltis, a third of everybody in that Phase II was submitting their data on PatientsLikeMe. And that was a particularly difficult protocol. And I think what was interesting was, although that meant that the people who took part in the study were highly motivated, they were so motivated to take part that they were also willing to share their data. And in that case there were some patients who wanted to be in the trial but once the 50 slots were filled up, they actually tried to work out what the active ingredient was. They started looking through the patents, they incorrectly deduced that it might me sodium chlorite, and they started ordering that off the internet, drinking it, and injecting it. Sodium chlorite is industrial drain cleaner, which is bad to drink, on the whole. And what we were able to show when we published this in the BMJ is that sodium chlorite was actually killing patients faster than if they’d done nothing at all. And so immediately we published this, and use of it stopped immediately. But if you think of the vast sort of uncovered area of people trying stuff off label or taking weird wacky nutritional supplements or what have you, there’s a big iceberg here. And if we don’t address that and understand that, we’ll be in trouble. I was talking to somebody last night about actually adherence in clinical trials. And you know, what we often want is for people to adhere to the protocol. But if you think of even the way that’s set up—are you being compliant, you know—that’s setting up a power dynamic that if we’re not careful, patients are going to resent.

Okay, so that’s the bad news. So how do we improve all this? So I think we can be more proactive. You know, the reasons why we saw a lot of patients weren’t taking part in studies was they didn’t feel they had the time. A lot of them didn’t think the study would help them personally. And again, although we make a lot about altruism, really I would encourage everyone to think about conditional altruism. No one’s going to just, you know, voluntarily sacrifice everything in their lives and leave their kids in the basement and go off and have this part-time job called being a clinical trial participant. They still need to live their lives. And so what we wanted to do was to say well could we design a study that addresses some of these needs, is still scientific, and can make a useful contribution. So whereas a traditional study would say for many of these elements, you know, you have to take it on trust, this is the way we’ve always done things, the doctor is the guardian of all this type of information. You know, it’s going to be these very potent drugs, and in order to meet regulatory requirements, we’re going to have to do more and more procedures on you. And so it can get so as the number of procedures per protocol is growing by 6% year on year. So you know, even for a relatively mild condition now, you might have to come to clinic once a month, have CT scans, have blood draws, have all sorts of things. And it’s very inconvenient.

So what we wanted to do is try and say okay, well could we do a study that’s not participant led, that actually has a PI who is a physician. And we would try and take some of the elements that patients seem to be wanting, seem to be demanding, and see if we can sort of meet them halfway in partnership. So could we give those participants their results in real time, could we let them self track at home, you know, no one wants to drive for two hours just to have a nurse give you a questionnaire that you could have done at home. Can we constantly update them on how it’s doing. Could we maybe give them a drug that might not have nasty side effects, so we start off with a nutraceutical. And could we ensure that 100% of people are actually on the treatment, so there wasn’t this worry that maybe I’m on placebo and maybe I’m wasting my time.

So fortunately for us we found a great person to collaborate with. So this is Dr. Rick Bedlack, who is the best dressed neurologist you will ever see. And his philosophy in life is that he has a pretty grim and depressing job, and he intentionally dresses in a flamboyant way so that the patients have something to look forward to when they come to see him. And this has made a tremendous impact in a field where you can imagine, it’s mostly doom and gloom, right. So this is great. And he was just at a fundraiser in that outfit on the far righthand side, golfing with Bill Murray, so the two of them won the best dressed golfing contest.


And what he does is he investigates something called the X-files of ALS. So he’s got these Fox Mulder posters in his office. And so what he does is he publishes this work called ALS Untangled, it’s a consortium of about 100 clinicians and researchers from around the world, and I’m part of it. And basically what happens is patients say, does this nutraceutical work, does this off-label thing work, does this stem cell treatment work. And what we do is we go and investigate. We go and dig through all of our case studies, we go look at the pre-clinical mouse work, we look at data from PatientsLikeMe, we go and investigate the claims on people’s websites and we phone them up and if they say oh I had this patient and they were dying and now they’re great, we go and find them and we ask for their medical records, and we try and verify what’s going on here. And so we’ve published about 35-40 reports all open access, and they’re read, like 10-20-30,000 times, these things. So I think they’re doing a lot of good in terms of redirecting people away from things that are of no benefit.

So very unusually, one of these treatments is a nutritional supplement called Lunasin, it’s a soy peptide, originally developed at Stanford, and then it got out-licensed to this nutraceutical company. It came about because there was one patient called Mike McDuff, who’s been very public about this, and he’s been on TV. And he appeared to have ALS, and usually if someone gets better or has a reversal that’s more than just a sort of one- or two-point transient thing for a month or two, it means they’ve been misdiagnosed. It means they have multi-focal motoneuropathy or it means they have Kennedy’s disease or it means they have something weird. You know, even the best centers are only 95% accurate. In this case, he’s been diagnosed by three of the world’s leading experts as definitely having ALS. And he went from not being able to speak or swallow to now being able to give TV interviews and, you know, chow down on Thanksgiving dinner for the first time, which is highly unusual. And we published this study in Neurology, just tracking how many times these reversals happen. They don’t happen like this at all. This is a once in a generation type thing. And he attributes this to this supplement that he started taking called Lunasin. And so we thought, well what is a way that we can at least investigate this in a way that’s fast and maybe relatively cheap and that maybe a bit like the lithium study, could come up with an answer quite quickly. Because if we could induce another reverse of that magnitude in even one person, that would be incredible. And so we wanted to go about doing it. So the approach that we took, bearing in mind things like the remote study that I mentioned earlier and the lithium study and all the controversies, whatever, was to say well let’s split up responsibilities between the things for which a clinic is great and the things for which the internet can be great. So all the patients come through the Duke ALS Center in North Carolina, and they are screened and enrolled. So what that means is we know they definitely have ALS, because a neurologist has said so. And we know that it’s shown some degree of willingness to participate, because they’ve travelled down there and they know the doctor. And obviously if your doctor dressed as fun as that, you’d probably want to go and see him, right, so that’s probably a factor. And they have IRB, they did an FDA filing of an IND for the nutraceutical just in case it works. And they actually follow up quarterly by phone call with the patients. And they do their own assessment, and then we have the patients do their assessment, and we’re double checking. So even though a previous study said the correlation was high, we’re sort of triple checking if you like. And then there will be a follow-up study 12 months later. So there’s only two visits. Sorry, three. So the screening, an then there’s a baseline, and then there’s a follow up at 12 months later.

Everything else takes part on PatientsLikeMe. So every month, we’re asking patients to fill out this functional rating scale, their symptoms, their side effects, their evaluation, and their weight. And so they don’t have to come to clinic to do the rest of the stuff. So we try and give them a lot of stuff. We give them a nice bag, and we give them tubs of this stuff, it’s quite heavy, you know, so imagine you go to the GNC store and you saw this nutritional powder, you know, they’re going home with a good few tubs of that. And we give them lots of nice leaflets and cards to fill out and fridge reminders and that type of thing. And we have email campaigns that keep them engaged, and we also give them lots of feedback about how things are going.

So this actually worked out pretty well in terms of enrolment. So the average ALS trial enrols about two patients per site per month. This was enrolling five and a half patients per month. It’s the fastest enrolling ALS trial in history. So I think you could say there was a lot of untapped need to do this type of thing. The other thing that we did is we knew that we only had 50 spots on the trial. The total grant was about $300,000 which was .1% cost of the Phase III trial that was unblinded all those years earlier. And what we wanted to do was allow the people who couldn’t get into the trial to participate along at home. So we published the entire protocol up online, and you can get hold of this nutritional supplement off eBay or wherever you like. So if there were people who really believed that they would like to take part and they just couldn’t get to Carolina, we wanted to offer them that opportunity to do so. And that would avoid the oh is it really drain cleaner and I’ll try and get some. But also to ensure that we could get that data, so maybe potentially putting it into the analysis. But also, importantly, understand the difference, the delta between the level of data adherence you might get from a more traditional program versus the sort of play along at home group. We think that could be important for expanded access programs, compassionate use programs, so it might be a proxy of that.


So this generated a lot of high media interest, so Sanjay Gupta came around and interviewed Dr. Bedlack, we were on the podcast Reply All, if any you are internet nerds. It was even covered in The Atlantic and a couple of other places. So you know, as I say, ALS needs good news. And even the idea that we might be taking a different approach here was really exciting.

So how is it going so far? So in terms of data collection we’re about three-quarters of the way through. We’ll be announcing the results in December at the main ALS symposium. And it’s sort of too soon to say exactly what the final outcomes will be, but clearly if they’re good you’ll hear about it, don’t worry.

So in terms of data quality, what we want to share here was, for example, the extent of agreement between the clinicians, the nurses, and the patients. And what we see is in nearly every case there’s a very high agreement. So this is just the difference between the participant and the coordinator. So where there is a disagreement, it tends to be where the instrument itself is unclear. So a classic example is a question about can you get up and down the stairs. And there’s one rating if you do it with assistance, and one if you don’t. And most patients are interpreting assistance as meaning a person, but by the letter of the clinical trial law, assistance means using a handrail. Now I use the handrail when I go up and down stairs, and I’m perfectly mobile. So you know, perhaps this down to the definitional specificity of some of these instruments.

In terms of side effects, well look, this is a nutritional supplement, this is not chemo, this is not a stem cell transplant, this is not brain surgery. So the side effects are very mild. But that said, you know, just to be clear, twice a day you have to get a pint glass, and put in a cupful of this powder, sort of mix it up, and it’s a bit sludgy, you know. It’s not like a nice smoothie. It’s not great stuff. And so people are mixing it with Hershey’s chocolate syrup or ice cream or trying to make it more palatable. You’ve got to do that twice a day. And it’s quite filling. And so people are feeling fullness, they’re feeling constipation. And I think the thing that’s really interesting is, you know, I think I probably overstated already how awful this disease is, but you know, when we think that the drug that just got approved by the FDA—I can never get it right, it’s either Edaravone or Adavarone, I think it’s Edaravone, it’s going to be known as Radicava in the US—requires having infusion every day for 90 minutes for two weeks, then you’re off for two weeks, then you’re on for two weeks, then off for two weeks, for the rest of your life. So relative to two milkshakes a day, you know, that’s fairly burdensome. And what we find is, yeah, people don’t want to do as much as two milkshakes a day. So you go out to people and say what would you do. And people say oh yes I’ll definitely be 100% compliant with that, that seems fine. But in reality, real life kicks in. People feel constipated or, you know, they’re travelling and they don’t have a blender to mix in the stuff or what have you. So side effects are an issue, even for this relatively mild thing.

In terms of perceived efficacy, well again, most people can’t tell that it’s having an effect, and what’s interesting is, we have one or two people giving anecdotal reports of things that could be quite interesting. I didn’t dare try and bring a video to a conference, because as you know, that’s the recipe for disaster, but some of the patients are videoing themselves through their course of the treatment. There’s one patient who couldn’t move their legs at all, so they were completely bed bound. And since starting the drug—or the nutraceutical, sorry, I should say—they’ve been able to move their legs again against gravity, which almost never happens, and they’re able to turn in bed. Now that’s not getting back out of a wheelchair and doing cartwheels, but if you’re trying to turn that person and stop them from getting bed sores and you want the caregiver to stop having a bad back, that’s incredible for them. That doesn’t necessarily have to be part of the nutraceutical, it could be a placebo effect, it could be rewiring, it could be a fluke, whatever. So what we’re going to be doing is looking at, well does that really make meaningful improvement on the outcome. We took blood at the beginning, we’ll take blood at the end, we’re looking at histone acetylation, which is the theoretical mode of action of this. So we’ll try and understand the biology of that.

But if that’s true for even one person, and of course ALS could really be 50 diseases, that may be worth following up. And again, for 300 grand and a year’s work, that is not too bad. In terms of the burden, well you know, as I say, some people find this to be quite burdensome, as you can imagine. People with swallowing problems trying to get down this kind of claggy soupy stuff could be kind of a problem.

So in terms of data compliance, where are we. So six months in, and actually many of you will know this better than I can, of how good this is. But in terms of the degree to which people are being adherent to the protocol, at six months in we would say about 73% of people are actually compliant with the protocol or adherent to the protocol as we’d like. That’s not as high as we thought. We thought, given how convenient this is, that this actually might be a bit higher. One of the things that’s very striking though is that the play-along at home group, so the people who have opened the protocol and got hold of this stuff off the internet and are submitting their data voluntarily, their level of data adherence is 17%, which from an analysis point of view is basically unusable. And I think our concern for that is that when we see patients lobbying to have compassionate use access or expanded access or what have you, the notion that then those patients might voluntarily submit data that could go into an evidence package, unless it’s mandated in some way or there’s some way of leveraging that you don’t get the drug unless you enter your data, the data is not going to be useful enough. It’s going to be a pointless exercise and it’s not going to be helpful and—yeah, I think that things like Right to Try, while they seem like a good idea at the time, are actually not going to contribute to the scientific endeavour.


So in terms of lessons learned then, I mean I think that the hybrid virtual format is very appealing. Like I say, this mixture of high tech and high touch could be something that we can think about, and I think a lot of you guys who work in electronic data capture will be interested in this. The data quality does appear reasonable, and I think it would be easy to offer specific training in, say, how to fill out the instruments so that we don’t get some of the data quality issues we have. Regardless of the modality though, the format, we still have to contend with the actual treatment. I mean if this was a little white pill that you took once a day, I think people would find it easier to get along with. I think the retention is poorer than we hoped for actually. And again, some of the reason—we delved into it a little bit. So a couple of patients have sadly passed away, a couple of them progressed faster. What’s happening, a little bit like with the other study, is because these patients are so keen to take part and see benefits, if they don’t see a benefit pretty quickly, they drop out. And you know, that’s sort of the double edged sword of no placebo. I think that the people who follow along, we thought this was the right thing to do from an advocacy point of view and from an equality point of view to allow other people to sort of have a shot at this. But if you want to be a data purist about it, I’m afraid it’s not really working to the degree that we hoped. And in contrast to the previous participant-led research, this follow-along version doesn’t appear to have created as much value as we wanted. It’s important to go and trawl actively, so we found that video of the patient who got around on YouTube. We had to go find that. And we had to go and dig in, log in to ALS websites to see people’s experiences. But I think this is a very reasonable way. At the very least, in terms of regulatory hurdles and that sort of thing, to screen things out rapidly, you know. So if there were further leads that could arise from ALS Untangled, this would be a really nice way of screening them out.

But hopefully there’s some lessons learned in here that could apply to some of your practice, and we’ll be sharing some more details of that in December and we’ll be broadcasting that and making a webinar so that you can continue learning along with us and the patients to whom we’re very grateful for taking part in the study.

Thank you very much.

[Q&A Section begins at 37:52]


Thank you very much, Paul, for that incredible presentation. I must admit a single learning point from my point of view is really a realization that we as human beings are not good at doing what we’re told to do, even if we’re terminally ill. I mean, your graph about compliance dropping off over time with these guys that clearly have a horrible condition, and yet their compliance still drops. Fascinating. I would have thought they would have been the ones at the top. So I open it to the floor. Any questions, please?


Thank you. I’m wondering if according to you, is there anything you could have done differently in order to increase patient compliance?


Yeah. So just to be clear. When people haven’t filled out their data, we’re messaging them. We have a community moderator that’s actively engaging with them. If they haven’t responded at all to it three times, we’ll actually have the nurse at their clinic sort of phone them up and try and figure out what’s going on. So we are doing the human follow up that I think you would expect. But you know, I’ve met one or two of these participants, and they’ve just said, you know, it’s just difficult. It’s just very difficult to be compliant to the medicine. So I think what’s happening is sometimes people don’t want to feel like they’re letting us down, almost. And perhaps if they don’t think there’s been a change, it doesn’t feel meaningful for them to enter that data in. And maybe, again, that’s the double edged sword of seeing your progress. You know, it might be depressing for some people to see that it’s not working for them. And I suppose one advantage of the more blinded approach where the clinician sort of, you know, when you get up from the chair they’re going, you’re a seven. Maybe that means that you don’t have that information. So I think it’s very difficult to see, but one of the things we’ll be doing at the end of this, we’ll be going back to the patients. We’ll be asking them. And that’s really the best that we can do is to really ask them. And I think that’s something we would encourage everyone to do in their studies is to say well, you know, why wait till the end to sort of hear from the patients taking part, are there ways we can systematically learn and maybe make things more accessible even as we’re going through the study regardless of the mode that we’re doing it in.



One of the fascinating things about your approach is that it is what happens out in real lives as opposed to clinical trials. And I noticed in self-report data of what types of treatment, equipment, etc. that were used, I assume people are reporting multiple. How do you measure the attribution of the progress to one particular thing that they’re using, as opposed to some other of multiple?


Yeah, so it’s very messy. And I suppose one would have to be very cautious trying to do this in something like MS, where you’ve got some people taking Tysabri which is very potent, you’ve got some people taking Copaxone which is less so. So I would say that the saving grace, scientifically, is that nothing slows down ALS particularly. One other decision that we made, which is again probably worth noting is, the people that take part in RCTs are highly selected individuals. They’ve only had the symptoms for about two years, they’re generally in the middle in terms of progression, so they’re just fast enough not too slow and not too fast. We didn’t do that. So we’ve got patients who were much sicker. You know, but we were not looking for small effect. The program is called ALS reversals, so yeah, it was sort of snowball-chance-in-hell kind of Hail Mary effect. And so I think it was intended though with the idea that the chance of actually finding a reversal would be miniscule. But it’s to say what could we learn. And maybe the second time we do this or the third time we do this or the fourth time we do this, we’ll lock it down, be more rigorous, and then, knock wood, maybe we will find something one day. But I think the idea that answering a question like that costs tens of millions of dollars and takes years, that’s not a scalable approach. We need to look at what people like Elon Musk do to making things more replicable and scalable and what have you, and do that for clinical trials. I think it can be done.


So thanks, because while it’s a discipline of innovation, real world evidences, a lot of things that are coming on, well, I’m not sure we are prepared for that actually. Just a question. So you are really looking at patient-reported outcomes, and it’s mobility mostly, meaning that there are some devices now to try that kind of thing. So thinking about, well, accelerometry  basically but there are many things, including devices that are now FDA compliant that can use in trials. So where do you go with that? Do you try and show you tried something there? What’s coming?


Yeah, that’s a great question. So we’ve done a little bit of work in multiple sclerosis, say. We did the study with Biogen where we sent out 250 Fitbits to patients with MS and we sort of did the PRO alongside that. And as you would expect, you know, self-reported walking broadly correlates with how many steps you take. Not that controversial or exciting. I think the most interesting work is being done by the ALS Therapy Development Institute. They’re putting, as you say, medical grade accelerometers on each limb, and not just saying step count, they’re actually trying to see—because you know, there are unilateral presentations of the condition, and looking at spread. So yeah, I think that’s definitely where this is going. But again, this is burdensome too, right. I mean you’ve got to charge the things, if you scratch your head at night you can cut your face open, which we had a few people doing. And you feel weird, you know. People are already suffering, they might be drooling or using a walking stick, and now you’re going to truss them up like Ironman to go around in these things. So I think to the degree to which these things can be discreet and again fit into people’s lives is interesting. One thing I was impressed by on the announcement of Google’s new baseline study is they’re giving everybody a smartwatch and it looks like a normal watch, you know. It doesn’t look like I’ve got some weird nerd gadget attached to me. And I think that’s really what we need to think about, you know. I think for so many of us when we’re designing a research study we don’t actually think, well you know, would my gran want to be in this, would I want to be in this, would my cynical brother-in-law who doesn’t care about science, would they want to be in this. Could it fit into their lives. We spend so long thinking about the regulators and the protocol and what peer reviews will think in a journal, we often forget that people need to fit studies into their lives, not the other way around.


Now thank you, Paul, that was really really fascinating. And I think just to as well reiterate what Michelle and Alex were touching on in regards to, I think that, even if you weren’t able to use the data because of the compliance issues, that’s an important reminder of the ridiculously artificial environment that a clinical trial is and that it’s not really representative necessarily of what happens in the real world. Rather, I think that’s important that we always bear that in mind. My question was about a slide you had up around patients rating what helped them. And I think the third one was, patients were saying prayer was one of the things that helped them the most. And I’ve definitely never seen that measured in a protocol. But there is this obviously big focus on conmeds, additional things that patients are taking to try and maybe help their symptoms. And I was wondering your thoughts around these kind of additional behavioural support things that the patients are doing. Is that something we’re missing in clinical trials now, that we’re not asking about these kind of things?



So I think there’s a couple of things in there. So one of the things PatientsLikeMe has been doing a lot of is taking from this experience and saying can we work with sponsors to say, could we make your trial more patient centric, if we were to go and ask patients, here are the boiled-down elements of the protocol, what could we do to make this better. They do often talk about being very concerned about coming off their concomitant medication because they took years finding something that works, and then you’re saying yes but when you come into my clinical study, dear guinea pig, we want you to come off everything that’s been helpful, so that you get worse so that then my drug fixes you again. That’s a nice experiment, but that does not sound like a nice thing to be doing and it’s not something I would want to sign up for.

I think your point about the support is a great one. And you know, we need to find a way to deal with unblinding. But I think that the secret sauce out here is the patients. And we are treating patients like individual atoms that come to us and are treated in isolation. If there were some way that they could get together and share information and support one another and go, oh yeah I found that side effect too and you know what, if you mix it with peanut butter it’s much better, we can actually extract useful content from that and then bake it back into the protocol. But that requires a change in mindset, and I think most people will say, well the regulators won’t let us. Great, go lobby them. We lobby regulators about nearly everything else. Let’s find a way with patients to try and do this, because there’s a lot that we’re missing that could make it more powerful. And again, I think what this type of thing says is, listen, this is happening anyway. The real world is going on outside there, whether we like it or not. And the question is, can we again meet people halfway, and do experiments, let’s A-B test this stuff, let’s see how bad the impact is, but not just assume that the status quo and the gold standard or what have you is the only way of approaching things.


Any other questions for Paul at all?

Okay, Paul thank you once again very much for doing this fascinating presentation.

[END AT 47:25]

Previous Video
eCOA Insights and Trends: What is Burdensome?
eCOA Insights and Trends: What is Burdensome?

Next Video
eCOA Forum 2017 - Overview
eCOA Forum 2017 - Overview