Download Premium Content

First Name
Last Name
Country (Citizenship)
Opt-In to Future Emails
Thank you!
Error - something went wrong!

Electronic Implementation of the SF Tools

May 19, 2015

Optum's Michelle White and Martha Bayliss discuss the future of the SF-36 and new updates to the SF-60

Full Transcript



I am going to start off talking a bit about electronic implementation of the SF tools. Just to get us started—sorry I have to ask this—how many people have used or are familiar with the SF-36 or the SF-12? Okay, so I’ll go through this pretty quickly. I assumed that would be the case with this audience, but I will go through it just a little bit.

When we talk about the SF health surveys, we’re usually referring to the SF-36, the SF-12, or the SF-8, sometimes the SF-10, which is a caregiver-reported version. Usually people are using Version 2 now, hopefully. And people are using either the Standard, which is a four-week recall period, or the Acute, one-week recall period. We do have different versions available: a paper version, a single-item version that’s appropriate for a tablet or computer, and a different single-item version that’s appropriate for handheld, and I’m going to talk about why that is today. And that’s really one of the main focuses. The tool is standardized so that outcomes are comparable across groups, times, and populations. It incorporates eight health domains that are measured, which are summarized as a physical health component score and a mental health component score. I’m not going to go through that too much.

So one of the things we talked about earlier is how much of the implementation of PROs is electronic version versus paper version. And I’ve seen some data that goes across generally, and based on actually Paul’s urging about a year ago, we started looking at our licensing of our own tools and how that was going. So within the pharma part of the licensing, you see that although we didn’t really start licensing a single-item form specifically for electronic implementation until early 2010, so it took a little while for the word to get around. It doesn’t mean that it wasn’t used before then, it means that people just licensed the paper version and did whatever they did with it. So some of this increase in use is a bit misleading, probably, in that way. But you could see there’s sort of been a huge leap, where we’ve gone from 10-15% to, in 2014 about a third of all our licenses are for electronic implementation. We looked at it in this way—who was licensing the single-item form—but we thought perhaps it could be because this could be under-reporting because sometimes people still do try to license the paper version and switch and use something different. So we also looked at licenses that included our form review service, which we require, and the numbers were about the same. They were higher in the earlier years but they’re now about equal at 31%.

But it still doesn’t really show the picture, I think, because, sort of, as I talk and look at what types of licenses are using the electronic versions, this data is showing overall license. So you might have one license for a study with hundreds of thousands of administrations and another for 300 administrations. And this is treating them all equally. Anecdotally, I would say that our support group that helps people use our tools have noticed that there’s a huge uptick in electronic use of the tools for multinational studies and multisite studies within a single country. So the larger studies are tending to use electronic much much more than they had before.

I mentioned that we really didn’t start a program for electronic versions until about 2009-2010. We first developed this form in 2007. It displays every item on a screen with the instruction texts for context. And at that time, in 2007, smartphones were not quite yet popular, and that had already been in the planning for a couple years, so it was really more focused on computer screen and tablet, which was more popular in that time. Qualitative testing was conducted and supported the format.


And I’ll go through and show you. This is what the paper version looks like. So you can see how the items are formed in a grid. There’s sort of an instruction part that says, the following questions are about activities you might do during a typical day. Then there’s a stem at the top that says, does your health now limit you in these activities, if so how much. Then there’s the items that are all following without repeating that stem. Now if we go through and look at the tablet version, you’re going to see the top has that instruction on every single screen for each single item. The following question is about activities you might do during a typical day. And then the next part combines the stem and the question together. So instead of having the separate “does your health now limit you” and separated out, it puts it together. Does your health now limit you, for example, in moderate activities. Okay. I’m going to show you something slightly different when we go to the handheld.

So we developed that version for the tablet. It was developed to keep in mind to try to not lose any information, because we’re taking things out of that grid format and there is an assumption that maybe perhaps when respondents are filling it out in that way, they’re seeing things together in a certain way. So that’s why we tried to keep as much of that instruction text on every screen as possible. We did conduct some validation of it. We collected data in a 2009 norming study. It was not a crossover design. I think we’re probably going to talk more about equivalence studies tomorrow, I’m not going to get into that in great detail. But the way it was done for this study was that all the data was collected electronically. However, half of the data was collected with showing the paper version on screen exactly as the paper version is, and half of it was collected with that single-item form. There was a subgroup that answered the survey at a second time, so there was somewhat of a crossover, but there was no paper involved. The results were presented at ISPOR in 2011, showing that the scores and measurement properties were similar in both versions.

So we thought we were doing really great. And at the time, we had about 160 or 170 translations of the SF-36 paper version. And we were getting up to about 140 translations of the SF-36 in the tablet form. So it was being used greatly in lots of different types of studies. But people were having problems—this came out of feedback from different ePRO vendors—because they were starting to use handheld versions, and the text, especially when you translated some of the text, would not all fit on the screen, especially with repeating the instruction. So we had a landscape issue.

So you face a question then. Do you just change your form and make everyone use the new form—that’s the handheld, so all the people using tablets and computers need to then start using a different form—or do you maintain two forms? We decided to go ahead and maintain two forms because we had that large number of translations and it was so incorporated in a lot of pharmaceutical companies and their ePRO vendors wanted to keep using what they already had programmed and make it a little easier for them.

The main revision in our handheld version, I’ll show you here. So here’s the tablet version that we talked about. And here’s the handheld, which separates out. It has a screen at the beginning of the questions that has the instruction, and it has the stem, and this is showing both on one screen, but they come up one at a time. The item itself is alone on the next screen.

We designed this independent but then we got ahold of a copy of a similar version that was used in a validation study and it was very very similar, that had shown equivalence to paper. But we are currently doing a study to evaluate whether paper administration provides the same results as by handheld device and by smartphone.

So I’m going to skip through those. I’m going to skip through what is equivalence testing, because we’re going to talk about that tomorrow.

Is it really necessary? Yes it is. A lot of PROs were developed for paper, long 20 years before the thought of having it on computers was ever come about, so it is important. How much is important to do? Well, ISPOR had a good practice report that talked about it. And you know, we evaluated based on that paper what should be done. For the tablet form, the changes seemed to be considered very minor, so that it would only require cognitive debriefing and usability testing. But with the handheld, we thought the changes might be considered moderate, and thus equivalence testing and usability testing would be required, but not full psychometric testing.


Hasn’t this been done before? Yes, there’s been lots of studies that have implemented their own versions and tested it. There’s been of course Gwaltney et al.’s meta-analysis that showed overall there’s plenty of equivalence. However, the FDA still wants to know is there equivalence, and our study sponsors want to know. We want to see evidence that there’s equivalence.

So the study design. We have two sub-studies with 400 adults total, so 200 in each sub-study, being conducted in the Boston area. And there’s two pictures but it’s actually the same thing on the top and bottom—one’s for handheld and one’s for the app. After randomization, you have the first assessment. If you’re in the paper group you do paper, handheld you do handheld. We did distraction activities for about an hour in the middle. And then you switch, so the paper first group did the handheld, etc. So they did not come back the second day, it was all done in one sitting.

Inclusion criteria, the general ones that you have, and then we did it in four disease areas—COPD, osteoarthritis, Type 2 diabetes, and depression. We didn’t really have much exclusion criteria, it had mostly to do with the ability to actually sit and perform. The survey instruments included a screener, the SF-36v2 standard recall, single-item version handheld. The distraction task surveys included demographics, a chronic condition comorbidities checklist, a healthcare questionnaire, and 44 items on administration methods that included preference and effectiveness of different types of presentation of items, and that sort of thing. Then we had an exit survey afterward on the preference between electronic and paper. Then there was a second qualitative sub-study also beyond that, but I’m not talking about that today.

The findings are coming soon. Sorry that I can’t present them here, but we do have all data collected in all disease areas. The initial scoring is done, and the data review is complete. And we had an accepted abstract to present on the diabetes data at ISPOR 2015, so that’ll be forthcoming. And that will include more than the equivalence data, it will also include some of the distraction activity data as well and the preference data. We’ve also submitted an abstract to DIA for the equivalence data for all four disease areas, and if that doesn’t go we may try ISPOR Europe. We’ll also have a manuscript planned for later in the year.

So that’s sort of what’s up and coming with electronic use. I want to highlight the importance of doing quality reviews for our ePRO implementation. Changes to the forms could hurt the validity and reliability of the data that’s collected. There’s lots of ways that errors can happen. Often people are using forms they found on the internet and then they’re surprised when there’s problems later, that sort of thing. Or it could be incorrect programming. And just that we have heard from study sponsors that sometimes the requirement to have us conduct a form review—which is basically your ePRO vendor would send us screenshots of what they have programmed, and we verify that it looks like it should, and we return it—is a great service, is very minimal cost-wise, but it does add time. Even though we’re able to return them usually in a day to three days, if someone has 30 different languages coming in and they hit us all at once, and five different groups send 30 all at once, that’s obviously going to take some time. We have had some situations where people have had to submit four or five times back to us to get all the errors corrected. And that usually doesn’t happen when you’re working with one of the larger ePRO vendors, but sometimes you have a smaller startup, or someone who is an individual person doing it on their own that’s less familiar with our tools, and that can add a delay, which obviously drug developers don’t want to have. So we only really have been working with the groups that have been using our tools extensively and we have seen them over time do a great job of programming. But it’s basically like a library program where they’re able store the screens and the programming for different translations, so that when you’re going to implement an eCOA study using those forms, you would not have to do the form review for those languages. So it does save time and money to do that.


And part of the reason we’re able to do that is because, in the partnership, we work with the vendors to avoid drift, conduct re-reviews and make sure that things aren’t happening or getting misplaced.

And I’m going to just spend a few minutes on new research that’s underway on the SF-6D before I turn it over to Martha. This is a very very new area, so I don’t have as many slides on this.

What is the SF-6D? It’s a preference-based single-item measure of general health, widely used in economic evaluation and it enables calculation of QALYs. It’s scored 0.0-1.0 from worst to best health. And for the SF-6D, it uses utility weights from the UK, from the first version that they had.

You can see, it can be derived from either the SF-36 or the SF-12. And the rationale for developing an updated version comes from some questions that we got from the field over time. Were the best items selected? Should we revisit that? Are there some inconsistencies from ambiguous wording between the health states that were originally presented in the studies developing the SF-6D that may need to be more clarified in a new study? Are there any confusion in positive wording compared to negative wording on certain items? And how is the distribution? Are there floor effects and ceiling effects resulting from having used v1 initially in the development?

Another reason that we looked at doing a new one is whether or not the best approach was used in analysis, and so the standard gamble was the approach used in the original development. And that is a technique that links to the full, sort of, best health versus death scale. It involves presenting individuals with choices between alternatives, a health state that’s certain and a gamble with a better one and a worse one. And respondents are asked what probability of the better outcome would make them indifferent between remaining in a certain state or going to the other state.

There’s a lot of concerns about this approach. There’s concerns about every approach, right? But the main one is, it depends on the risk behavior of respondents. So is a person very risk-averse themselves. Usually there’s not testing done on who’s in these studies, but you try to assume that if you do a good job of recruitment that you’re going to get a mix, if you have a large enough sample of risk-averse versus not. But some respondents did have problems with the standard gamble technique in this and other studies. So the new strategy is to perform IRT and factor analysis to identify dimensions that would be best, perform IRT analysis to select the best items, do evaluation analyses using discrete choice experiments. And this is going to be an all-online study. And this is ongoing. It’s recently started. We are fully funded for work in the UK, Australia, Portugal, Canada, and the US, which our group Optum is actually funding specifically. It’s partially funded in Spain and Hong Kong and not yet funded in several other countries that we would like to do evaluations for. There’s lots of arguments whether having a single preference weight versus having for different countries is better. But we’re going to find out the answer by getting data in all the different countries and see how that goes. There is an international expert panel, this goes far beyond Optum’s work. In fact we’re not the ones that initiated it, we were brought in as expert. Jakob Bjorner, my colleague, is part of that. John Ware, Barb Gandek, Jordi Alonzo, Shunichi, Cindy Lam, several others are involved from a wide group. So we’re very excited to see what comes of that. We’re going to be doing some cognitive debriefing of the intended health states and survey. In the US we’re starting that in April-May time frame.

So that’s that. And I will turn it over to Martha.



Great. Thanks, Michelle, and all. I have a couple of different topics to talk about with you today, and I have to say usage statistics is probably one of the worst labels I could have picked. But I want you to, for a moment, get ready to hear about use of the SF-36 and other generic tools in areas beyond the life sciences, including and beyond the life sciences. And maybe this will help a little bit with the rage—apparently—that comes to mind when we mention the SF-36. So I’m going to try and dispel some of that.

One of the things that is probably not so well known in the industry is the extent of use of the SF tools in scholarly research. And in fact, we have a group within our company, within the legacy quality metric group called the Office of Grants and Scholarly Research, OGSR. And the company actually has a formal commitment and obligation actually to provide the tools at low or no cost for academic researchers. And I’ll show you some stats in a moment, but what we will show you is that the free licenses for scholarly research in 2014 were the highest of any year, so the use is growing in that context. And in the last year, we issued more than 400 licenses that were low or no cost for academic users. This is about 40% of all of our licensing activity. So it’s a significant amount of use of the instruments. The provider market, if you will, represents another three out of ten users. And then life sciences, right, where we all work and focus our attention and think the world revolves around us, well we’re actually only about 20% of the users of the SF tools.

So what you can see here graphically is just a representation of what I was talking about. You can also see sort of some trends over time—wish I could point at these—but the little orange bars are 2012, ’13 is yellow, and ’14 is the darkest bar. So we just wanted to leave you with a sense here that the life sciences market, while important, is actually not the predominant users of the SF tools.

So you know, sometimes we ask ourselves, well so we have to do this, but really what’s the payback, what’s the return for us and for all the researchers who use these instruments from having this really widespread use in academia. Well, one of the ways that we measure the return is by counting up all the articles that have been published based on these tools. And so we’re a little bit obsessive about this but we collect every single one, and we have a massive bibliographic database, and now it includes more than 23,000 publications based on the SF-36, -12, -8, and the other tools that we distribute. It’s actually an amazing asset for doing sort of meta-analyses and other kinds of deep-dives into the literature, based on this common metric.

So you can see that the bar way over to the right, the very tall one, is a count of the number of articles that were published just last year. So we’re really encouraged that there’s a lot of publication in the peer-reviewed literature, and this spans all kinds of users, commercial and non-commercial.

And this next slide shows again a picture of the publications by year, and way over to the right again is a nice tall bar representing the number of publications in 2014, and this is specific to publications on RCTs. So I look at this and I think—I’ve lost my years off the bottom, but if you count backwards, you can see there was kind of a drop off a number of years ago, and I tend to think that might coincide with the PRO guidance coming out and kind of a pullback on gee, what are we doing here, are these going to be useful after all, should we continue to invest, should be continue to collect data and publish. I think the answer is a resounding yes. I think we are all back in business thinking about PROs in clinical trials and getting the word out there in peer-reviewed publications.


Okay, I’m going to switch gears for a bit and talk to you about some research that Optum carried out in the last year or so. And I think it’s kind of interesting that Michelle spoke about an Optum-funded study about the instrument. I’m going to speak about another Optum-funded study, and I hope that you take away from this that we’re committed to the instruments and we’re committed to their use and their interpretation across a wide variety of applications.

We we took on this study because I work with a lot of pharma customers and I thought to myself, I need to know the mindset of my customers’ customers, right, so how are payers thinking about the data that are produced from PRO instruments. And so, just quickly, you know, we might ask ourselves about why payers would be interested in using PRO data. And we know that outside of the US, PROs are much more influential in making coverage decisions, just because of the way reimbursement and coverage happen in Europe. But in the US, payers typically haven’t been viewed as using PROs as an important aspect in determining patient access or coverage decisions. But if you—you know, as health reform is chugging along in the US, I think that perspective is changing, because there are numerous federal programs that require patient-based metrics as performance evaluation metrics. And there’s a few of them listed here, like the National Quality Forum the NCQAs, use of PROs, there’s a new thing that just popped into my phone earlier this morning about CMS requesting development of additional PROs for quality metrics. So this is coming along through the healthcare system, it’s coming along through drug development and I think now there’s really a convergence of use and maybe understanding of the value of these patient-based assessments.

One of the reasons that payers care about health status assessment is they’re starting to understand the usefulness of this data in predicting longer-term outcomes that perhaps haven’t been measured in a clinical trial. So there’s a couple of pictures here that show the relationship between scores on a generic SF-36 on the summary scores and the extent to which they are predictive of future healthcare expenditures. So on the left, you can see the relationship between physical summary scores and total monthly expenditures, with those in poor health, on the way left, having nine times the cost of those who had favorable health, way over on the right. There is a comparable story to be told related to mental health, in the orange bars, again with the very poorest health having a nine times higher likelihood of using mental health services, I think this is in the coming six months or so. So there’s a useful predictive application.

Okay, so let me tell you a bit about the study that we funded and carried out in the last couple of years. And our goal was to talk with you as payers and understand their current and future use of PROs in making access decisions, making coverage decisions for new pharmaceuticals. And we define PROs really broadly as the patient’s experience with the disease and its treatment. And I should tell you that these were interviews with payers, were conducted qualitatively by some colleagues of mine in a different part of Optum. So we didn’t tell the payers, oh we’re the SF-36 people, and we want to know what you think about PROs. So I was blinded as to who the payers are; the payers were blinded as to our specific involvement.

So I guess a couple of cautionary notes here. This was a qualitative study. I think what I will tell you about the findings are correct, directional in nature. This may be more hypothesis-generating than it is a confirmatory study. But I’m really interested to hear your thoughts about what we need to test, following these results. So in the box over here is a quick summary on the right. The methodology was one hour double-blind telephone interviews. And we made a point of talking to people, individuals who represent a wide range of payers. So we’ve got commercial payers, national commercial payers, like our parent company United Health Group, United Healthcare, more regional payers—this might be like a Blue Cross within a particular region, a couple of ACO organizations, and so on. You can see this here. I want to point out a couple of the specific respondents down lower on this list—the VA and the Department of Defence, because they have kind of a different relationship with their members than a commercial payer would.


So I’m going to give you a little bit of data from the study. And also, I guess, an invitation to come see our poster at the ISPOR meeting, because we’ll be presenting some of the really rich narrative stuff that came out from the interviews. But I’m going to summarize some rating scale data here today.

So we asked folks to use a ten-point scale to rate six different questions, I guess three pairs if you will. Two of them were about the relevance of PRO data, currently and in the future, five years from now. The second pair was about whether they would like to see more PRO data currently and whether they think that will change or be the same in five years. And the third set of questions was about whether pharma should be the source of investing in more PRO data around new treatment that are coming along. So what I want you to take away from this simple picture, I guess, is that the darker bars are longer to the right than the lighter bars every time. So my interpretation is, while some of these concepts are important now, especially the second two, they are expected to gain in importance in the coming five years. I was a little discouraged that the average score for how relevant now is only 3.7. Since I’ve been doing this for more than 20 years, you’d think it would be higher. But I’m encouraged that across the board the average score of future use was much higher. So six out of ten. I’m a measurement person, and it’s a little hard for me to talk about these numbers, because you know, it wasn't a validated instrument, so go with me on directionally correct on this one.

Okay, here’s an interesting different way to look at it. So remember I said we asked six questions, each on a ten-point rating scale. So the maximum anyone could get was 60, right, that’s my public math. The worst they could get was either 0 or a 6—I forget if the bottom was a 0 or 1. But anyway, what I take away from this picture is that there is a lot of variation, right, you have a couple of folks who answered up in the 50s, 55 and 58. You have a couple who were just way down in the basement, so they are saying I don’t care now and I don’t think I’m going to care. So who’s at the top? The very top is the Department of Defence. The second one is someone who is an ACO. There’s an actuary, which I can’t make sense of. The VA, another ACO. So these are entities that have a lot of risk, and a very long time horizon with their membership. Who is at the bottom? National commercial payers. Maybe it’s United, maybe it’s Aetna. They are having a lot of churn, they don’t have the long-term risk kind of relationship with their members. So they’re making short-term decisions about price, price, or price. And the notion of functional outcomes isn’t really in their decision process.

So let me just share a couple of headlines and opportunities that we conclude. And one of them is that not all payers are alike. We’ve seen some wide variation in their opinions about PRO data. And I think—this is just my own interpretation—that the difference may be due to the time horizon and the amount of risk in relation to their membership. When we asked about would they like more education, the answer is a resounding yes, they need to learn more about these tools and how they could help the payers make smarter decisions about how they’re going to spend their money.

So I don’t know if you guys watch Mythbusters, but I love it and my kids love it, so I thought we’d do some myth busting. So one of the myths that we came into was payers don’t care about PRO evidence, US payers. Well, I think the fact of the matter is, not all payers are alike, and some actually pay a lot of attention to PRO data, it’s very useful to them making their business decisions. Another myth that we poked at was whether PROs matter only if they make it into the label, and that came up already this morning. And the answer is no, the payers will consider all the data from the pivotal trials, whether or not it’s in the label, whether or not it was requested or mandated or blessed by the FDA. A third myth is, oh we’ll get that in Phase IV. And the fact is, it’s too late. The coverage decision will be made, the possibility of getting enough attention to revisit coverage or tiering is probably, that ship has probably sailed.


So what we see as opportunities for folks in the life sciences business is, again, thinking early on about how can we satisfy the needs of payers as well as regulators with relevant PRO education and evidence. And how do we make sure that the payer needs are incorporated into the overall PRO strategy from the get-go?

I’ll talk in a minute about the notion of focusing on standardized PROs with straightforward interpretation. And I hope this doesn’t incite rage, but I will talk about how a standard metric might be useful in appealing to a payer market, because quite practically we’re not going to teach every plan about every bespoke PRO that comes through the pharma drug development process. We think we can probably help them understand some standard metrics that they will be seeing over and over again.

So a couple of solutions that may come from within pharma, as I mentioned, will be to integrate the payer perspective when developing the PRO strategy. And then, sort of looking outside the walls of the sponsor organizations—and this comes to our own strategy as well as an organization—is we feel that we have an obligation to work with you as payers and help them learn more about these tools and how to interpret the results. We want to help them understand the relationship between self-rated health and future medical costs, because we think that’s where they care the most, is about understanding the implications of these data for their future risk within their membership. And you know, I’d love to talk with folks after this about opportunities to do that, what kind of venues would be meaningful.

So then the question really comes down to, well how do we do this? How do we satisfy the needs of payers and regulators. And I think really we’re a lot of the way there already. I think many of the PRO strategies have most of the components. But the place where the bridge is out is, we fail to bring the message home in a way that payers can consume it. We’re focused on the needs of the FDA and then we stop there. And Ari talked a bit earlier this morning about maybe some of the wasted opportunity, maybe some of the data that are in filing cabinets or maybe in trash cans that could have been brought to bear in the AMCP dossier to talk about burden of illness and treatment benefit using some of those metrics and, you know, these data that have already been collected at great expense.

So here’s a little graphic that suggests a way that PRO evidence could be combined or integrated for regulators and payers, and here’s an example of some work in Ankylosing Spondylitis. But what we’ve proposed up here is the notion of bringing these data into the formal process of submitting clinical trial data to the payers. And so in the AMCP dossier there’s a section where you can talk about the disease description. And when I talk to the folks within Optum who write dossiers, I say well do you talk about burden from a humanistic perspective? And they go, huh, no. So you know, if you establish the fact that burden can be measured using this metric, then you can come back and say hey look, we made it better, you know, we returned people to functioning at a level that’s normal for the population, after showing that it was impaired pre-treatment.

This is an approach in the clinical trial world that you’ll see in some areas, like rheumatoid arthritis. The recent article by Vibeke Strand et al. talked about comparing post-treatment scores to national norms, using the SF-36. So there’s some precedent, certainly, in certain therapeutic areas. And then I think the notion, over here in the orange box, is to take the interpretation a step farther, right, so it’s not just to say the treatment benefit was 3 points, it met the MID, blah blah blah. It’s to say, well so what, you know, what does this mean. And to use some of the relationships that we’ve studied really well that helped to show the relationship between a score change and the likelihood of return to work, or the likelihood of decreased annual expenditures.

So these are kinds of opportunities that are just sitting there, available to be brought forward in the communication.


Okay, just a couple of quick pictures, and then I’ll wrap up. I mentioned the bibliography with those 23,000 papers. So one of the ways that we like to dive into it and help to sort of characterize the disease burden—this could go into that section of the dossier—is to kind of look across all the articles that have published on the standard metric. And you start to get a kind of convincing presentation of what life is like for people who have this condition. And so here’s a quick summary of Ankylosing Spondylitis. It’s, you know a sort of casual meta-analysis, if you will, because we’re just looking at the average scores across, you know, 12 or so different published articles. But you start to believe the pattern, right, if it’s consistently seen across a lot of studies. So this, you can say in your dossier, this is the burden, this is the pattern of impairment that we’ve seen.

An interesting example, and this doesn’t come up very often, but this was also from Zelchan’s clinical trial data, was interpreting the generic PRO data in terms of the likely positive impact on medical expenditures over time, compared with placebo. So they’re taking these standardized metrics and they’re spinning it, if you will, they’re interpreting it into dollars. Into dollars that weren’t actually measured in the trial. So it’s maybe a novel way to wring some more juice out of that.

I won’t stay here very long, but we could talk afterwards about how do you put together your endpoint model to make sure you’ve considered what the regulators want to see and what the payers are going to want to see. You know, maybe just the addition of maybe another analysis—and we always suggest, you know, put it way down low so it’s not going to break your hierarchy—but if it comes out really strong, it may help in regulatory discussions and it may be really compelling for payers.

So let’s conclude, patient centricity is here to stay. I think I’m stealing that from a song somewhere. But what we found in our own study was that payers’ interest and reliance on PROs is going to increase—that’s job security for all of us in the room—particularly around symptomatic conditions. So to Ari’s point, hypercholesterolemia, no. Hypertension, probably not. Diabetes, really complicated. But lots of other conditions are very disabling, and people can tell you how they feel before and after treatment. There’s not a lot of literature on the payer’s uptake of PROs but there’s another study out of Xcenda that had some similar results. I’d be happy to share that poster with you. But the conclusion really was the importance of more collaboration between industry and regulators on the interpretation of these kinds of metrics.

So I think what should we all do? We should be ready with data that’s going to align with what all of our stakeholders want to see. And in particular, keep in mind that there are new novel interpretations of existing tools that can really take your data further into impressing this variety of stakeholders. And that can be done without adding burden or cost to the trials, because a lot of this is already happening.

Thank you very much.

[END at 43:20]

Previous Video
Panel Discussion: Jason Cole, Paul Margerison, Shimon Rura
Panel Discussion: Jason Cole, Paul Margerison, Shimon Rura

Next Video
Opting Out: Enforcing Completion of Questions on Electronic Systems
Opting Out: Enforcing Completion of Questions on Electronic Systems