Download Premium Content

First Name
Last Name
Country (Citizenship)
Opt-In to Future Emails
Thank you!
Error - something went wrong!

eCOA in Global Studies: Considerations and Best Practices for Localization

June 21, 2016

Presented by Brielle Sydor & Catherine Knoth, Transperfect & Megan Turner, GSK

Utilizing eCOA for data collection with non-English speaking patients requires planning for translating the content on the devices, as well as providing related services like rater training and help desk support in the native language(s). This presentation will outline best practices for linguistically validating a COA, migrating it to an electronic platform, performing post-localization testing, and translating associated materials, with special consideration given to completing the necessary steps in time for IRB submissions.

Full Transcript 


Brielle Sydor

Thank you all for joining today. We’ve broken our presentation into two parts. We’re going to focus primarily on the process for localization, going over everything from contracting all the way to IRB submissions and what the different considerations and recommendations for those are, and move into some common challenges and solutions we’ll see with the process.

Megan Turner

So when preparing licensing and translation in terms of eCOA, you’re going to need to know in advance, at the very least country feasibility so you can go ahead and get these pieces into place. When preparing licensing, you need to make sure the developer or the owner understand you want electronic implementation. You should get your source—usually it’s going to be English—request any translations and certificates. And next you’re going to do your contracting with your translation vendor so they can pair any new translations and adapt the ePRO. Simultaneously you want to get your eCOA contract in place, hopefully. And make sure if you’re going to do any in-country reviews, that you’ve alerted these people in advance so they’re prepared to do this in a few months when you’re ready for the reviews. Likewise, you need to know who the IRB coordinator is, they’re going to be a very important piece for you. Make sure to set up any of your kick-off meetings. I like to do smaller kick-off meetings versus one huge one, which can be a little overwhelming for all the parties involved, and all the information that each party needs is fairly appropriately conveyed in a larger kick-off, like an eight-hour, all-day one. And then, it’s important to determine who is going to be signing off on your spec document—that’s your specification document—and who is going to be involved in UAT for the devices, because whoever is going to be involved in UAT really needs to have been involved in the protocol pretty early on, so they understand and they’re not rehashing anything that’s already been sorted out in the spec phase as well.

I would actually like to call semi-realistic overview of the timeline instead of realistic. It’s sort of like a semi-realistic, semi-ideal. So as soon as you have your clinical study plan, that’s when you should know what your COAs are going to be, and you can begin licensing at that point, even with your robust country feasibility. Licensing, this is, of course, going to be on a sponsor-by-sponsor or CRO-to-CRO basis can take anywhere from say four to eight weeks, maybe longer. As that’s going on you can also begin getting our translation quotes in place and kick those off as soon as your licensing is ready. Preferably your eCOA process is also going on in tandem this whole time. And then, by the time you’ve hit ethics, you’ll have rolling deliveries of your translations to meet your ethics submission timelines.

Catherine Knoth

So I’m going to talk a little bit more about the localization and translation timelines aspect. My absolute favourite question is, how quickly can you get it to me. Because that’s constantly kind of the problem that we run into and we see, is by the time it gets to us, you guys just really need it in your hands right away, which makes perfect sense, because this is the last step. So basically, once the eCOA is contracted, and we’re contracted as the language provider, we kind of start working with the eCOA provider to ensure that we are going to meet all of the ethics deadlines that are needed. So this is kind of an overview of the standard process. We put up 8-20 weeks here, that’s kind of a broad overview of if we’re doing the translation from scratch, which includes the full linguistic validation process. This is a little bit longer of a timeline if we’re just migrating something from paper to ePRO or if we’re kind of doing similar things that were discusses earlier, like portal translations or things along those lines. That’s a little bit faster because it doesn’t need the full process that includes cognitive debriefing and different things like that in order to meet ISPOR regulations and things along those lines. So, from the kick-off meeting, we usually receive the source files—that’s what we call them, so anything we need to translate—from eCOA providers, in about two to three weeks. So it’s not quite right away. So even though everyone’s contracted there's still a little bit of time because the files need to be designed, things need to be sorted out, we need to be sure everything that's coming from that paper file or that needs to be included in the protocol is truly included on the device as well, so that nothing’s left out. So once we receive that, this is where the bulk of the timelines really take place. So this is where we’re doing the full translation or migrating or anything that’s needed to be done on our end. And this is for us the most important piece in the whole process, because if this portion is done perfectly, then the rest of the project should go very very smoothly, and it’ll just be a brief collaboration between the language service provider and also the eCOA provider.


By the way, that’s what LSP stands for, if anyone was curious, it stands for language service provider. They weren’t big enough to fit that whole thing on there each time, so sorry for the acronyms.

But this essentially is where, like I said, the bulk of it comes in. And if you’re working with a paper PRO, at this point that’s when you would receive your final documents ready to submit. But because we are working with eCOAs we need to kind of continue along the process to make sure it’s finalized to be on the device and perfect and ready for patients. So as you can see, it kind of goes back and forth between the eCOA provider and the language service provider. So it’s a really amazing collaboration between both parties and both vendors to ensure that everything is going to turn out perfectly on the device. And that’s kind of the wonderful thing that we’ve seen happen with this that you don’t necessarily see with paper PROs, is the actual collaboration between different groups. Otherwise

it would just be one vendor, one group receiving the files and kind of delivering it back to you.

In terms of timelines, basically after we provide the initial language in any sort of format that’s needed to be imported into the system or device, we are then provide with the screens in order to check the language as it would appear on the device. So this is a very important step that needs to be done by a linguist or someone that speaks the language, to ensure that what the patients in that site are going to see is actually accurate and something that they can read and is going to be appropriate for them. This is especially important if we’re doing the full linguistic validation process, which we’ll touch on a little bit later. And then at this point, so that’s usually about three to five days for us to receive that initial report and then an additional three to five days or so for us to provide back the first group of edits. So essentially, during this process what we’re doing is we’re having our linguists who are specialized in this material look at all of the documents that have been provided to them, and really ensure that all of the information appears properly, is showing up on all the screens appropriately, and that all the answer choices agree with the question at hand, and they’re really looking at it in detail to ensure that everything shows up properly and the way the patient would want to see it.

And then after that, we’re pretty much just going back and forth to ensure that the files are clean and everything is finished that way. And that includes simple things like formatting and just the things to make the file pretty too. And then at that point—so that includes kind of the third one on the right and then all of the middle ones there, and then once we just deemed the language perfect and ready to go, we will provide a language certificate that’s also provided to the sponsor and everyone involved so they can use that to submit. And one of the things that we have found that works the best in terms of timeline, especially when you’re working on projects that have up to 60 countries involved, is to really prioritize these based off of those ethics committee deadlines, or submission deadlines, because we have some room in this, and kind of what we’ve seen in the past that works really well—and speaking on that collaboration—is there is room built in kind of for us to manoeuvre and move things around to ensure that we’re hitting deadlines that are needed. So it’s always very important I think, from a timeline basis, at least from the language service provider aspect, to kind of know those final end dates, so ensure that we’re kind of prioritizing properly and that we’re able to kind of move things around or work together more efficiently to ensure that those are actually being met.

Brielle Sydor

And so we want to touch a bit more on linguistic validation. As Catherine mentioned, this can be the bulk of that translation time if you do need to do it at the beginning of the project. We generally say an average of 8-12 business weeks for that, give or take a few depending on the scope. So it’s really something you want to be aware of at project onset and plan for it so that you’re not surprised about missing submission dates because you thought this would go more quickly than might be anticipated.

So linguistic validation is recommended by ISPOR and the FDA for any COAs that are serving a primary or secondary endpoint or supporting a labeling claim. And what the intention of this rigorous process is, is to make sure that any translated PROs are not only linguistically equivalent to the English, but also conceptually equivalent. So you want patients who are reading though these items and questions in different countries to understand them in the same way that, if it’s a US English source file, the US English patient would understand it.


One example we give a lot is if the question said, are you feeling blue today. What does that idea of feeling blue really mean? Are you depressed? Disconnected? Apathetic? What are you trying to get at the patient’s feeling, and then how do you take that idea and concept and mirror it in the other languages, so that the patient’s understanding it the same way and answering the question appropriately. And this process it goes through is considered so essential to the data collection process that a sponsor can risk their data being rejected if the file doesn’t go through that type of process when it’s recommended.

So as we mentioned a little bit earlier with licensing, a lot of common COAs already exist and are validated in these languages, in which case you can either have a sponsor or CRO or oftentimes an ePRO or a localization service provider coordinate that licensing, get the existing validation measure, and from there it’s a simple migration process. If you’re using a homegrown questionnaire or if you have one that may be validated in English but isn’t validated in another language yet, you need to go through this process that’s outlined here. The most important part of that, I would say, is the cognitive debriefing, and that’s where we really get the patient perspective on the translations, on the questions, and make sure we’re achieving that conceptual equivalence. So what we’ll do is we’ll have a trained interviewer and then typically five to ten respondents who are native speakers, reside in that country that you’re rolling out to, and represent the patient population. And what the interviewer will do is ask some questions about the items in the PRO. So they’ll say, can you repeat this back to me in your own words, is there anything about this question you find offensive or misleading or hard to understand. And we’ll take that feedback and analyze it with the translation, and if there’s a few respondents that really struggle with one question or one thing being asked, we’ll adjust the translation to reflect that to ensure that once these PROs are being used in the study, they’re adequately understood.

Megan Turner

So with submissions, Catherine already mentioned that we need to know up front an idea of when submissions will occur. And there is that ability to batch them in waves so we can appropriately divvy up the work and still hit submission timelines. And there is some ability early on in the translation process that Brielle showed before to pivot your languages around a bit, but that’s only within the first few weeks of the translations. There does hit a point where you will begin either incurring greater costs or have timeline delays if you start trying to change up your translation too much when they will be delivered.

The standard eCOA submission package is going to include your screenshots or screen report, your user manual quick reference guide, your translation certificates for the screenshots, and your device label and sticker. Something that I have found in my work for the past nine years or so is that at this point, most of the ethics committees seem to not be willing to accept the English source as your final submission, you do have to provide translations and certificates at some point. There are various ways to do this, you may need to initially provide the paper translations if they exist and then provide the translated screenshots later, in order to keep moving things along. But you will have to provide translations and certificates at some point, probably 99.9% of the time.

Brielle Sydor

So a few other things we wanted to mention, that don’t necessarily become a part of the IRB submission package but that do support the eCOA localization roll-out and that you’ll want to have in place before you have your patients in, are listed here. So the first one is called post-localization testing, which is linguistic and functional testing on the ePRO device itself. As Megan mentioned, typically IRBs are just looking for screenshots of what’s on the ePRO device, but you want to make sure once that content is on the device itself, that when you’re pressing the next button in Russian it actually takes you to the next page. And this is something that many ePRO vendors and localization service providers will go through and do with native speaking testers, if possible, to make sure that you’re catching not only functional issues that might have come up as a result of importing the content, but also any linguistic issues of maybe an accent was dropped or maybe a word’s getting cut off and things you want to avoid.

The next thing is multi-lingual help desk support. So part of that submission package is having a device label on the device, which will often include a help desk phone number where the patient is—you know, their device screen froze and they can’t turn it on, they’re having some sort of issue having it work, they’ll have a help desk line they can call and you want to make sure that you’re offering that in their local language so that they can communicate with someone to help them solve the problem.

Third thing is site training, so when you’re rolling these out to your sites, are you making sure your personnel are trained in how to use the ePRO device, how to administer the questionnaire, and is that training content available in a localized format for whatever their native speaking language is So it could be e-learning modules, webinars, over the phone or WebX training sessions, making sure that you have those resources available to them so that they know how to administer these questionnaires.


And the fourth thing that most ePRO companies will help you with, but you definitely want to address when you’re planning, is customs for shipping the device to different countries. You want to make sure there’s no issues with getting them there on time and getting them back.

Catherine Knoth

So I’m just going to talk about some common challenges that I think all of us face, actually, it’s not just the localization vendors or the eCOA vendors either, it’s kind of everybody involved. So obviously we’ve already talked about it a little bit, like, timelines are very important. And knowing, kind of, how long everything will take on the front end and knowing, kind of those final submission deadlines, and having just an overall grasp of all of this before you’re kind of, you know, starting everything, is really important to have. So I think that it’s definitely been beneficial for us to discuss these things during kick-off calls, or even calls just when you know a study is coming. These are just really important things to have an idea of and make sure that, you know, especially if you have a study that has up to 60, 70, 80 languages—they’re fun—but they definitely take much longer than you may anticipate because of just the volume of everything you’re looking at. So definitely having an idea of how long that contracting, and you know, the actual overall process is going to take on the front end is really something we’ve found to be helpful. And then that way we can always reassess if, you know, certain languages aren’t going to meet the deadlines because they’re very aggressive, or you know, if we do see any risk we can kind of address it on the front end and kind of, you know, work together as a group to kind of find a solution to that and kind of reach out to different sites or kind of see if they’re willing to accept draft translations, kind of as we discussed earlier. We know that we’re on the front end, we can address it, rather than kind of that, oh my goodness, the night before, we’re all in panic mode. But that’s happened too, so that’s okay. 

And then also, speaking of kick-off calls, something I’ve probably mentioned four times already, so I apologize, but they’re important. It’s really important to have all of the parties that are actually going to involved on the kick-off call. They don’t necessarily need to be included, you know, in every single, the day to day or anything like that, but all the relevant parties should probably be included. Like, regularly we will do things with the eCOA vendor and then also the sponsor or the CRO, who’s going to be the most involved in that so that we’re kind of all on the same page and all expectations are set from the beginning. Everyone is aware of kind of the timelines and everyone’s aware of, you know, who is tracking what and who is speaking to what and who everyone is. I think that it’s really helpful and kind of, you know, improve visibility throughout the process and kind of make sure everything stays on track and on the same page and, you know, there’s less confusion of what is going on and who is handling what. And then a kind of more, what we've already talked about, our submissions, but in addition to kind of the timelines, it’s also really good to understand that kind of country-specific requirements they have, again, on the front end. It's very very important because if we know on the front end we can incorporate that into the process and ensure that, you know, that’s not causing any delays on the back end either. For example, France—we’re always speaking of France today—they generally need a privacy statement added to their devices, and you know, in the past that’s something we realized at the back end, whereas like now we can—now that we know that we can every time going in we can ask if that’s a requirement in the study, and if it is we can know that is going to be a thing that needs to be incorporated, and maybe a round one, a round two of screenshot review rather than, you know, scrambling at the back end to make sure that it’s put in place in the right spot, checked, and everything along those lines. So little things like that that can cause delays on the back end if we aren’t fully aware at the beginning. And a lot of different countries have very specific kind of interesting issues. Brazil sometimes doesn’t want any English in their screen reports for submissions at all. So what we’ll do is we’ll actually go through and remove all the spots where it would normally have that as well. So just like little things like that that can kind of cause problems, you know, for sponsors or for anyone that actually has to deal with submitting these things, that we can easily avoid if we kind of have a heads-up about that on the front end. And then eCOA projects have a little bit more back and forth, as you can tell, than a standard translation project that you might see. So sometimes it’s good to have an organizational plan in place at the beginning as well, to see who is going to be tracking all the different deliveries between the different vendors, things along those lines. You know, sometimes eCOA vendors will manage that whole tracking process, sometimes the language service provider will do it. It's just a very important thing to kind of keep in mind because—especially with larger projects—it’s easy to lose track of things. So it’s definitely good to have a plan in place for how everyone's going to be tracking them and making sure that you know each of those are getting hit. And some different eCOA and language service providers have different technologies that can help with that, so just kind of keeping that in mind to kind of see what will be the best option there.


And then, of course, the last one seems a little bit obvious, but I’m going to talk about it anyway. It’s best to always make sure you’re including a language service provider in any sort of screenshot review. So a lot of times in the past people—it’s been kind of confusing why a language service provider has been needed if you’re only migrating paper to eCOA. The main thing is that, if a native speaker isn’t looking at it, you don’t really know what could be missed. And it could be very simple, it could just be, you know, an accent missing or something minor like that, or that, you know, each answer choice has to change from masculine and feminine and they’re all actually the same. We’ve seen it all. So it’s definitely very important to always consider, and it seems, like I said, kind of an obvious and silly thing, but without it, it could really impact, you know, data and impact patient responses because maybe they don’t understand it just because of those simple issues that are arising. So just some little things that we’ve seen come up that, you know, are kind of easy things to avoid if you have the right information about them.

And that’s all we have for you. Any questions?

[END AT 21:30] 

Previous Video
The Integration of Patient Portals & eCOA Systems
The Integration of Patient Portals & eCOA Systems

<p>Patient centricity is ever growing. The field has expanded the involvement of patients and their voice i...

Next Video
Patient-Centered Outcomes in 2016: What's New, and What's Not Likely to Change
Patient-Centered Outcomes in 2016: What's New, and What's Not Likely to Change

<p>The landscape of Patient-Reported Outcomes measurement is changing. Learn more about the Mapi interpreta...