Afternoon, I'm Tracy Kimball, Senior Director with The Conference Forum, and I'm delighted to welcome you to our follow-up DPHARM conversation on modernizing clinical trials. We have an esteemed panel of CROs and sponsors to share their perspectives on realities and priorities. I'd like to introduce our panel moderator, Henry Wei, Head of Development Innovation at Regeneron. Henry, please take it away. Thanks. Thanks so much, Tracy. Hello, everyone. I'm Henry Wei, and by day, I'm your moderator for today, and by night, I am Executive Director of Development Innovation at Regeneron. We do have a great group of experts today, and I'm pleased to moderate this panel, which will discuss the realities and the priorities on our journey to modernize clinical trials and really get us some real-world experience examples data in many cases, so that you all can get an idea of how what's been actually going on in real life as we move concepts to reality. I'm pleased to introduce you to our three panelists today who will help us with their expert perspective on modernizing clinical trials. First up, we have Emily Mitchell, Executive Director at ICON for Decentralized Operations and longtime Executive Director of Biometrics at ICON PRA. I'll ask all of our panelists, like Emily, to introduce themselves further, as we roll into some of the questions. Next up, we have Rosamund Round, VP of the Patient Innovation Center and Decentralized Trials at Parexel, and a longtime leader of patient centricity at Parexel. Thank you, Ros, for joining us today. Finally, we also have Ronald Elash, Senior Director and Head of e-Telehealth and Patient Innovation at Regeneron, and also my colleague. Welcome, Ronald. Thank you for joining us. You're welcome to show your face on the camera if you wanna unmute your camera there. Thanks. All right. There we go. Hi, Ronald. All right. We'll get rolling here. We're gonna go through. I'll ask our panelists some questions. Don't worry, none of them are, you know, brain teasers per se, but they are relevant. We're gonna actually open it up to the audience for questions. As we go along, please, for those joining us in the audience, please enter your questions into the chat window, and then, if they actually are follow-ups to any of the topics we're discussing at the time, we'll be happy to actually try and stream them in as we go. We'll get started with the first question over to Emily. You know, we discussed prior to this, your team was a major, well, you know, set of co-authors and also contributors to a novel trial design in the CHIEF-HF heart failure study. Now, I understand this was a fully decentralized study in heart failure. Can you give us an idea of what sorts of key practical learnings you and ICON had from actually operating the study? For instance, what went into investigator training? What sort of technology platforms were nice to have versus must-have? Those sorts of things. Emily? Sure. Thanks, Henry. ICON had the privilege of actually starting up the CHIEF-HF heart failure study before we knew the pandemic was even gonna be a thing. From a conceptual standpoint, coming up with the idea to do a fully decentralized clinical trial was, you know, really innovative, I think ahead of the time of where we needed to be. Interestingly enough, we went through all the planning, including how we were gonna do the training of our investigators before the pandemic hit, and we actually had our first site activated that same week that the U.S. shut down due to the COVID-19 pandemic. Timing could not have been more ideal. One of the things that we did was, as part of our process, we used our site initiation visits as the opportunity for us to do that more hand-holding, engaging, training for those investigators. In hindsight worked really well, but was very limited in interaction between physicians, getting ideas of what has worked well for them in the past, and what hasn't, what challenges they face with technologies and being able to adapt the approach to what we were engaging with them on, from that perspective. Since then, we've started doing it in more of an interactive capacity, where we had the physicians all together in a virtual training or in a virtual investigator meeting. That's worked out really well. I think that that's, you know, a key lesson learned is making sure we try and coordinate and get the group together. To answer your second question about the technology piece, really making sure that we're picking platforms that are intuitive to use, that's a must-have. By that, I don't mean simplifying it to the point where we're losing the integrity or the quality of the data, but really, having it so that the platform is helping the participant and the site walk through the experience of the clinical trial. Because these were known patients to us, all of the patients were mined out of the site's EMRs, there was not necessarily the need for the telehealth. However, I think we've also learned from the pandemic, having that telehealth as a requirement is helpful, it's beneficial, it keeps that human element into the trial, which I think a lot of times when we were stuck in homes, not actually required to go anywhere, not allowed to go anywhere, people felt disconnected, and then they would disengage. That's definitely a nice to have, I think moving forward and something that we learned from the CHIEF study wasn't necessarily needed because these were known patients. They had that relationship with the physician. If you're thinking about virtual investigators and mega site models and, you know, digital recruitment, that human element should be added back into the studies. Excellent. Those are some great learnings, Emily. Again, to kind of recap a bit on the investigator training as a group, and then sort of the actual practical role of televisit sounds like nice to have, but not always needed if for known patients. As we shift our views to other types of technologies, and Renald heads up, I'm gonna ask you about this too. We've heard a lot of hype, to be quite honest, but also now increasing reality about wearables and digital endpoints in particular. I guess as a follow-up, Emily, in your experience, particularly from CHIEF-HF, what do you see as the role of purpose-built research devices, and then the role for consumer wearables, in clinical trials, especially when the trial is for regulatory decision-making? What do you make of this, Emily? Specifically for CHIEF, the wearable sensor data was not a primary endpoint. It was a nice to have. From that perspective, the commercial-grade approach worked just fine. People were more familiar with it. They were more comfortable with it. However, when we look at it being used for regulatory decision-making, because they were able to see how many steps they took, how active they were, the flights, all of that information, it could have biased how they were actually interacting, pushing themselves. Mm. Having that motivation factor, where a lot of the more medical grade keeps it more, looking just a normal watch so that they're not biased, they're not influencing the way that they're engaging with that sensor or that wearable. That ends up becoming more of an important factor when we're looking at that from a digital endpoint perspective. Excellent. Thanks, Emily. Now, Renald, turning to you. You've also led some extensive and impressive work at Regeneron and with TransCelerate and BIO on digital endpoints and wearables, also home-based sensor devices. What's been your experience so far with your group and team? For instance, what are the implications that sponsors should broadly understand? For instance, you and I think previously discussed the implications for sample sizes. Yeah. Thank you, Henry, and hello, everyone. Renald Elash, Head of Digital Health Technology here at Regeneron. Before we get into sort of the question, Henry, I just want to touch base based on Emily's point. The selection of the digital biomarker, and I think that's an important key for us to be successful. Every digital biomarker selection has to start with scientific and clinical need. What is the scientific need? What is the endpoint? Once you identify sort of that unmet need, we go through the concepts of interest and incorporate the validation process to be utilized within the study. In this stage, actually, that's where it goes kind of to how we can utilize the great work that TransCelerate and BIO has published around different frameworks in validating these digital biomarkers. With regards to the experience so far, I mean, well, at Regeneron, I think we had really great adoption around digital biomarker, and this is mostly due to the fact that how we approach the adoption. We start with why. Why we should adopt digital biomarkers. What are sort of the opportunities that biomarkers bring that traditional methods are not able to detect? Then, you know, what are the opportunities sort of for the clinical programs? As an organization as well, as we try to adopt these digital biomarkers, we have to have an understanding that these quantitative measures do provide us with an opportunity to really better detect pharmacologic effects of investigational product in early phase clinical studies. They also do provide us an understanding of overall patients' physiological sort of processes and treatment responses. What's even better, if we think of the wearable devices, passive monitoring gives us better insight in real sort of real-world context setting. Closing with the implication sort of question in the sample size, to that end, I mean, it's... If you think of it, more frequent assessment, larger effect size, lower variability could, but I truly believe that it will reduce sample size required in the clinical trials to detect the treatment effect. Just in general, I think taking this approach, science-focused approach of treating it as a tool, as a measurement versus a technology, will be successful. The lesson learned components, going back, it's important that we incorporate user experience research as we adopt these digital wearable technologies. Ideally have someone in-house and make it part of the process. You can have the best tool, but if it's not used by patients, you get no data. Secondly, what we have seen here at Regeneron, which was very successful, is having the verification process, early stages of validation in-house. You can speed the process and increase the adoption and kind of ensure that what you're trying to measure the tool is measuring. Before you get into a clinical trial, you have a clear understanding of the tool from the usability perspective, verification, and some stages of analytical validation as well. The key message, just want to keep it simple. Don't start with a device. Start with an endpoint in mind. Don't innovate for sake of innovating. Start with a device and then very, you know, start the stages of verification. That's, you know, and keep user in mind. Thanks so much, Ronald. Shifting to Rosamund Round, you know, you all had the Parexel Sensor Solutions group, who are, I understand, a very broad range of sensors and wearables. Considering what your colleagues and the other panelists have said, what have your experiences been, from where you sit? I think touching on what Ronald just mentioned, that real-world experience of patients. As an example, we had a device on a study that one of the sponsors wanted to use, and it was for a women-only study, and they needed to wear a patch for a week. All the validation had been done perfectly. Then when we did the testing with patients prior to going into a trial, we had these huge gaps in data. We spoke to these people that we've been working with, and most people want to shower at some point in seven days. They've been told not to keep it wet, and they were all taking it off to go in the shower. There was nothing wrong with the device, but it didn't work for patients in the real world. I think having that patient perspective and really making sure that we are thinking of what it means for someone day to day. Also sometimes people get really enthusiastic about sensors and want to put loads of them on a trial because they want to get loads of exciting data. You need to think for a patient about how they're going to understand, how they're going to interact with it, different charges that you might need for the devices, and just really think through that patient experience and how they receive and how they're interact. I think also what we've seen with regards to sensors is lower adoption than the other areas of DCT. I think so much change for so many people so quickly at the start of the pandemic that they embrace the things that they have to. For some people, it feels like almost too much to start thinking about this pie-piece as well. I think that as we get clearer regulatory guidance and people feel more comfortable with DCT, which I do think they're starting to now, I think we'll start to see that really increase in terms of usage over the next few years. Excellent. Thanks, Ros. I'm gonna actually follow up and shift a little bit here. You touched upon it, and I think, Emily, you touched on it at the very beginning also with investigator training. You know, Ros, let me focus on you. Your organization has seen the build-out of newer types of sites and networks to deliver this different type of modernized trial. Can you share with us a bit about these sort of new research models? You had mentioned something called the Community Alliance Network. Yes. What have been the key learnings as your organization has built those out and actually begun operating? Any surprises, you know, like nice expectations met? Thank you for asking. The Community Alliance Network is about taking the trial near to the patient, not necessarily in their home. One of the big learnings we had through the pandemic was not everyone wants a nurse in their house. Not everyone feels comfortable with technology, and I think optionality is really important. A lot of people told us they'd like something near their home or near their work rather than it actually where they live. In order to improve access to those patients and also the broad community, we know that there's a large percentage of underrepresentation in clinical research that we need to address. We built out the Community Alliance Network. One of the pieces to it is DCT adjacent/DCT evolution, where we have a partnership with CVS. Anyone online who's not in the US, they are everywhere. They're pharmacies, and they feel like they're on every street corner. We work with CVS. They have, they're actually building out at the moment. They've got about 40 sites at the moment that sit within their pharmacies, and they're building more. It means that someone can go if they're, you know, in a trial, but they want to go to a pharmacy near their work, then they'll have the ability to do that, usually for more of those chronic indications. We're partnering with an organization called Javara, and they are a community hospital support network. Moving away from those big academic hospitals that are in a city which is so critical to research, but some of them are overextended or don't have access to some of the diverse populations. We work with Javara, and what they've done is they're wrapping experience around inexperience. They're going to some of these, hospitals that are out in the community that maybe don't have extensive experience with clinical research, and they're providing the training and site coordinators, and they're organizing the ethics submissions and all of those things. We're able to go out further and broader, and again, people can go to a hospital near their home that they feel comfortable with rather than having to travel into a city. I think the patients appreciate that optionality that I mentioned before. We know that trust in pharmacy is actually really high. Pharmacists are often a real trusted advisor to people, and people like their local hospital. I think it's supporting our diversity goals as well. What we're aiming for is to have a study population that's reflective of the patient population. We know across the UK historically, we've not been achieving that by any stretch of the imagination. This is something that's really helping us move toward that goal. Excellent. Thanks. I'm hearing a lot of converging themes here. One of them is, you know, DCTs might not be panning out the way they were originally hyped, but there's still plenty of opportunity. Then also a theme on really getting at what I would probably term user experience and getting at the heart of what people really think. Renald, I know you and I have talked about, you know, this difference between aggressive interest in DCT, but from a sponsor perspective, what have you seen from where you sit in terms of site readiness, investigator readiness, almost pre that sort of investigator training that Emily discussed earlier. I understand your team, actually, you have the user experience researcher. What were the key lessons from what you've been able to look at? Yeah. No, thank you, Henry. The theme here seems to be the user experience research incorporates sort of user feedback as we design our studies. I think just number one, as an industry, we have to invest and make sure that we, as we modernize clinical trials, we consider and involve those that we intend to serve and those actually helping conduct our trials, and that's patients and sites. That's sort of where the user experience research is an important step for us to zero in and focus on the feedback from all lens to make sure that, one, the technologies that we use in trials are intuitive, easy to use, and also again, going back to making sure that it includes and considers the real-world context perspective. An example of that is, Henry Wei is showing here, the DCT site research that our team, Roland Bart, our user experience researcher, has conducted. The goal here was just we wanted to determine sort of the existing mental models and perception, site perception towards DCT, advancing technology adoption, but also understand the change in expectation from both ends, sites and users, as we reportedly, you know the adoption of DCTs have been increasing as part of the COVID. The survey was conducted, it's U.S.-based, conducted with 141 principal investigators and research coordinators. Some of the key findings, I mean, if you look at on the left side, only 25% of the sites have facilitated DCT within the last six months. This is quite different from overall perception with the industry around the adoption of DCTs. On the positive side, if you look at, the second part of the question, is 60% of those surveyed, they do predict having capabilities of delivering DCT within the next six months. That's a good sign. We also looked at, thematic analysis from overall site surveys. Some of the key findings, like breaks down the perceived risks from the site. The top perceived risks were reduction of both patient and staff compliance to the protocol, safety and loss of oversight. The top sort of reported concerns from the sites were site burden, which is an important aspect for us to consider and see how we work with the sites to help them through that. Trust security, and also making sure that they get the right support. You know, in closing here, as a sponsor, I think we have to make sure that we engage directly with our site as we modernize clinical trials and help them ensure, you know, making sure that their needs are heard, express their risks and concerns. Also we have to work with, you know, our partners, CROs and vendors in making sure that the sites are getting the support required in order for us to modernize clinical trials. The most important thing, I think in closing, a one-off survey and one-off usability research, it's not a solution. This has to be incorporated within our processes. We have to gain direct user feedback from both sites and patients as we design our clinical trials, as we modernize clinical trials. That's the only way for us to, you know, to have an advancement in modernizing clinical trials. Thanks, Ronald. That's great insight. You know, I think sites definitely appreciate their perspective being represented. Now, let's shift gears yet again. Still on the themes and the main avenues of modernization. There's one thing that's occurred has been this democratization of marketing, advertising, and awareness channels and methods. Digital media campaigns in particular are accessible to, well, pretty much the entire public now. You don't need a marketing department to run ads on YouTube or Facebook or Google search. But on top of that, interested patients can go directly into online experiences to learn more about a study, and often potentially even pre-screen and pre-enroll right there and then, without the drop off that you might face from traditional non-digital recruitment. Ros and also Emily, I'm gonna ask you, starting with Ros, you know, reflecting on the new world of digital media, what's been your experience and insight with digital media campaigns in modernized trials? I think one of the key things to think about is to understand where patients are looking for information and also think about their preferred language. We did a big piece of research a couple of years ago at Parexel for discussions on diversity, where we speak to patients, or we spoke to patients and physicians from those traditionally underrepresented communities from a race and ethnicity perspective. We've got this amazing member who was part of it, who's on our patient advisory council, called Iris, who lives in Queens, and she said, "There are like 20 common languages just in my neighborhood. If you're providing information out there online, and it's in English or Latin American Spanish, you are missing a massive proportion of the population just by not even bothering to, A, provide the right translations and sufficient translations. Be also go where people are looking for trusted information. I see Trishna has put a comment in, the chat. She's one of the patient advisors to Parexel as well and was part of our research, and she said, you know, in the UK, her parents, for example, listen to BBC Asian Network, and that would be a trusted place that they would go for information. Just going with Google and Facebook and assuming that that's gonna do the job and providing it in one or two languages probably isn't. I think with this democratization and also often a reduction in cost because we don't have to use some of these expensive agencies, we need to be channeling those funds instead to make sure that we're really giving patients what it is that they need, where they're seeking for information in a trusted place in order for them to click through. I think also in the experience, as Emily mentioned before, having that ability to talk to a real person. Yes, it's great to enroll on a trial, but as that very famous first DCT led Craig Lipset that ran found that if people don't get to speak to anyone through the whole experience, then actually the engagement can wane. Making sure that there are still those touch points along the way once a patient's on the trial. Excellent. Thank you, Ros. Emily, thoughts about that, especially the human component of this? I completely agree, Ros. You know, from our experience, we've done fantastic work within ICON as far as the digital media outreach goes. Looking at collaboration and partnership with advocacy groups as well. Standing up Facebook communities so those who are in a certain disease state can connect with others and see what care options are out there for them. There's so much that we can do from that digital perspective, but that's just one click. How do we turn that one click into one patient is really the important part, because what we've seen is just going with a complete pure digital approach ends up having tons of clicks and tons of interest. When you actually funnel it down to having that click convert into a human who is enrolling into a clinical trial. The funnel does tend to drop off. You'll have folks, for instance, who are very interested in clicking on something because they wanna learn about what is new and available for them within their disease. However, when they go and they click on it, now they're getting an outreach saying, "Hey, you wanna participate in a clinical trial?" All of a sudden there's a stigma associated with it that maybe they weren't sure that that's what they were getting themselves into in the initial advertisement that was out there. Really making sure that there is that human element, that persona that is developing that person from a click on a digital media outreach to bringing them in, giving them that human to speak with. Like Roz said, we've learned so much from the trials that have been done to date. You've got to ensure that we're doing the appropriate handoff and not just pushing it through an electronic funnel and having it be completely digital virtual throughout the process. It turns out people do matter. Thanks for that, Emily. Let's stay on this for a little bit, go a little bit deeper on this one. You know, I think we think of modernization doesn't have to equal digital, and I think that's one of the things that we're seeing here. Yes, Facebook has online communities, but, Ros, you touched upon this earlier also. What are you seeing in terms of, if not Facebook, modernization of patient engagement, community engagement? Any notable examples or just little learnings that you might have? Absolutely. I think what we've seen over the last couple of years for sure is that the patient voice is getting louder, and rightly so. Communities, whether that's online communities or whether they're out in physical communities, they don't want us parachuting in when they need something. They want us to partner with them and see them as equitable partners. That is absolutely critically important. Having, for example, long-term, meaningful, mutually beneficial relationships with advocacy groups. Making sure that we get the patient voice into trial planning, so we have those insights, and we understand the potential barriers, and we can think proactively about ways to overcome them. Often trying to reduce the volume of tests and procedures in a study so that we're not just asking things of patients 'cause it might be interesting, but we're making sure that we're kind of minimizing the burden while maximizing what we need to understand from a scientific and a medical perspective. I think those advocacy groups are critical. Those real-life people are critical. We have a patient ambassador who's actually a member of staff at Parexel. She's wonderful. She's called Stacy Hirt. If you're interested, look her up online. She's amazing. She's a working pharma, so she understands clinical research. She's a patient herself and a caregiver of a child with a rare disease. She is just. Her voice is so loud in our organization. She is at all the meetings saying, "But what about the patient? Why are you doing that procedure for them?" There was a meeting we had recently for an oncology trial, and it required a liver biopsy. Everyone was like, "Oh, a liver biopsy." She went, "Hang on. Has everyone been through a liver biopsy to know what that actually means?" No one had. She was like, "Well, actually, it's pretty terrible." Took everyone through it, and everyone's wowed, like, "Oh, okay." It really brings things off the page, and it makes you think about the frequency of the biopsies and the necessity of some of the different elements. Yeah, that loud patient voice and that long-term engagement, I think is just something that we're seeing and I think will continue, and rightly so. Thanks, Ros. It's good insight. I think back to Emily. You know, we think again, this theme of, well, there's modernization in digital, but then on the human element, any additional learnings or surprises for that matter? I think boots on the ground is not something we should forget about, right? Decentralization doesn't mean that you don't actually have to go out there and do the work still. Why not host a small community pop-up? You can then do ad hoc visits with a virtual investigator. It doesn't mean the physician has to be right there, right then. If you're looking to get large numbers and you know where the community base is, there's nothing that says that you can't take it to a community event, and have it be part of that community for them to have access to that care. I think that's one of our key learnings is, you know, digital doesn't mean that's the only approach. We still need to think about the communities, the people, where they are, how we can go into that community and help support them with access to care, access to clinical trials. Instead of making them take the onus on themselves to be the ones that have to go out and find it, let's bring it to them. Good advice. Okay. I feel bad 'cause we're almost dunking on digital here, but I think we're getting to a level of maturity, frankly, from the experience. There's some good comments, by the way, coming through here. I'm just gonna read them out loud for those not able to access the chat. You know, the blend of high tech and high touch from Jane Miles. It's not all or nothing. Jean Vincent writes, this is something we talk about a lot, understanding the PT burden and how patient burden and how that relates to the patient journey. A lot of good themes here. I'm gonna push this, maybe to an uncomfortable area that I think, one of our audience, Trishna, Varadha from Teva brought up, that, an example where we might have seen some unintended consequences from, you know, good intentions, with the drive, towards decentralization, but maybe some surprises we had not anticipated. Many vendors in the DCT space tout potential advantages of DCT in trial diversity. Just this week, as Trishna had pointed out, and we've also observed, CenterWatch has covered a survey by Velocity Clinical Research, from well over 1,000 trial-experienced patients. According to that survey. Some groups were more against home visits than others, to reflect refinement, if you will, of what the themes that we've heard from Emily, Roz, and Renald earlier. In this particular case, for example, no Black women picked home visits over virtual visits or on-site visits. In terms of those who wanted to go to a traditional in-person clinical site, I'm broadly summarizing here, but 70% of young Black women under 35 and 90% of older Black women aged 55-65 want to go to a physical clinical trial site to take part in a study. Just 46% of African American men said they would want to use a wearable or wearables in a clinical trial for anything from, you know, tracking blood pressure or heart rate. You know, with these new, more, I would argue, more mature learnings of what's actually needed to solve some of our big problems here, 'cause you've all had practical experience, what's your perspective on these findings? You know, not everyone is into home visits and maybe actually preferring in-person visits. Emily, I'll turn to you first. Any thoughts? I think this gives us a great opportunity to work with our sponsors and our regulatory agencies to give the patients a voice. Let them pick the pathway of how they want to engage in the clinical research. Just because we're talking about modernization and decentralization doesn't mean it has to be a one-size-fits-all, even within one protocol. That means that we can give them the ability to look at more than maybe just the racial diversity, but the socioeconomic diversity and the geographic diversity of where these patients are. There are definitely areas and regions where it's going to make more sense to have things like a nurse go into a home in a rural community where it's not easy for somebody with a rare disease to necessarily have to travel hours and hours and hours. do we offer them support around logistics of getting them to the site? These are ways that we can bolster and give them options for how they can engage with the trial versus mandating that it has to be done a given way. Yeah. It's the theme of optionality, right, Emily? That's very, very timely and good insight. Ros, from your perspective, any thoughts? Absolutely, yes. I mean, I think DCT in general, I could probably write a textbook of everything I've learned in the last three years. From a diversity perspective, we've learned a lot. As you kind of touched on earlier, I think we're fixing some problems and almost creating or discovering new ones as we go. This Perspectives on Diversity report that I mentioned earlier, which is available on our website for free if anyone wants to download it. We spoke to nearly 2,000 patients and physicians around the world, predominantly from the Latino and the African American communities, to find out really what they felt some of the barriers were to research access and get advice on ways to overcome them. We just learned so many things. One thing, for example, is in some cultures there is a degree of secrecy around disease. Providing a home nurse actually can uncover and reveal to those around them that they're sick, and some people may think that they're sick because of karma, or it could impact marriage prospects, for example. Actually, although you think you're helping by improving convenience, it can have these other challenges that can be kind of put onto patients. We actually had one of the first DCTs that we worked on. We had a site in Miami, and it was predominantly serving the Latino community and predominantly patients of lower socioeconomic status. A couple of months into recruitment, that site said, "Can we go back to being a traditional site? We can't recruit anybody. People don't want a nurse in their homes. Some people, you know, they're in multi-family homes and maybe they're embarrassed or they may be, you know, people may not be in the country legally, for example, and it's actually scaring them. That was something that I'd never experienced. We had another patient in Latin America, and they lived in a favela, and they chose to keep their study device at the site and go to the site every day to fill out their diary 'cause they didn't feel comfortable taking an expensive piece of equipment home because it could impact their safety. Just on a more simple level, you know, not everyone has a great cell phone that they could do a telehealth visit on. They may not have Wi-Fi or very good Wi-Fi. They may not have a great data plan. You're putting this kind of financial and practical burden on people. When you sign for it up front, brilliant, you can provision devices and all of those things, but it's thinking about it from the beginning and making sure that we're taking these things into consideration and doing the absolute best that we can. Yeah, we've learned lots, and we continue to iterate and improve and build, but I think by asking patients what they want early on, that's really important. I think another thing as well is we did another study thinking not about diversity just in terms of race and ethnicity, but we had a study in rheumatoid arthritis, and it was a really heavy visit schedule. It was visits once a week for a whole year, and we thought it would be perfect for DCT. We surveyed patients, and almost three-quarters of them said, "No, I love going to the site. I like chatting to people in the waiting room. I like seeing the doctor. Why would I wanna do it at home?" Whereas we thought, "Oh my goodness, they're gonna have to go to the hospital every week." Yeah, I think diversity means a lot of things. We're also doing a lot of work with the transgender and non-binary community, people with disabilities, and learning a lot from them. I think what Emily said as well about actually getting to the site. When we think about hybrid trials, we've been thinking about reimbursement for travel. One patient in our research said to us, "I can't afford the bus fare to get to the clinic, so reimbursing me is absolutely pointless because I can't afford it in the first place." We've been working with IRB to be able to provide those upfront stipends for patients so that they're not financially disadvantaged and we're not creating these barriers. I could go on and on. There's so many examples. We will never get everything perfectly right for everyone. Just like I can't represent all blonde ladies in England. You know, the people that we speak to, their kind of examples are not necessarily representative of everyone in their communities. I think making sure we listen and continue to build and learn and do better as we continue on this DC journey. You know, we evolved so quickly in a way no one would have anticipated a few years ago. I think having the time to settle down and get those learnings and embed and continue to iterate is going to be really critical over the next few years. Thank you for that, Ros. A lot of good insight here on patient insight gathering. Renald, what about from your perspective? Anything come to mind? Yeah, I mean, a lot covered, and I was quite frank, I wasn't surprised. Just to summarize, it really, it goes back to designing the protocol with the user in mind. I think it's important that as we design the protocol, we understand present disease prevalence by race and ethnicity. We get the right patient feedback and what works best for the disease population that we're serving. We cannot bring technology for sake of innovation. It will backfire, and we have a lot of examples, e-consent and so on. And I think the lesson learned here is when it comes to healthcare, patient want the human interaction. This is my belief as well. I think the reason why patients prefer a hybrid approach, and I think this is going to be sort of the way forward, the hybrid approach with options for patients to choose. Yeah. All right. The voice of experience from our three panelists coming through here. Hopefully with this good insight that you all from the audience can take away and translate to lots and lots of studies now and then going forward. We're gonna pivot to another interesting opportunity in the world of modernized trials that's really come about with methods in encryption and big data that have actually been around for some time in other domains. Now especially accelerated with the FDA having done something called privacy-preserving record linkage linking to real-world data in that particular case for studies for COVID vaccine follow-up, particularly both the CDC and the FDA having advanced those out of urgent necessity. There's a new era here of real-world data and linkage to real-world data. Sometimes you may hear this called tokenization, that term. Personally, I think we could describe it better, record linkage. Terminology aside, let's go to Emily. Emily, you know, thinking of, again, about the study that we were talking about earlier, but also in general, what's been your experience so far with these opportunities to link and pull in real-world data? I wanna kind of dispel the common myth out there, which is if you are token-based linked, it means that now anybody at, for example, Regeneron, has access to all of your medical records. That's not how it works. Yes, you have to sign a consent to have that linkage hooked up. You are de-identified, and the data is looked at in a longitudinal capacity, in a manner that we are putting together algorithms to pull that data for areas of given interest. This has really been a fantastic use for us that we did in the CHIEF-HF study, so we could passively follow these participants and see were they having increased incidences of hospitalization due to heart failure? Were they having a decrease? Were there changes in their medications because of the underlying condition of heart failure or diabetes or other anomalies that may have popped up that we could have gone and looked at? There were areas of special interest from an adverse event perspective and looking at claims of hospitalization and other serious adverse events that we were able to pull in and utilize in that longitudinal capacity to say, you know, X percentage of the patient population continued with this line of therapy for this many months post the study. Being able to look at it in that capacity really shows where you are as far as the research that's being done on that particular product, and if it is continuing to show effective utilization after the fact, helping them to identify target market segments of where they can go and identify patients, populations, and really focus in and hone in on the best approach for delivering the clinical trial and getting the data back in in a real-world capacity. Excellent. Thanks for that, real-world experience. No pun intended, I suppose. As a follow-up to that, Emily, I mean, so taking in perspective that example, like, does this change the opportunities that we have with decentralized trials? If so, how? I think this probably gives us a larger opportunity for our patients to maybe not necessarily feel like they have to report everything, especially if it may be an event that they aren't sure exactly what it means. Maybe they went to the hospital, but they don't know what the technical diagnosis was. By utilizing their linkage, their record linkage, we're able to go back and see what was that physician's assessment of what happened to that patient and what was the technical diagnosis instead of a, "Well, I had some chest pain, and I went to the hospital, and they gave me some meds." Well, now we can actually go and see, okay, they had a diagnosis of angina, and they were given this medication, and this is the follow-up course of treatment that the patient was given. Especially in a decentralized capacity where you may be having somebody who has an underlying condition using a virtual investigator who may not be their treating investigator, who also may not be the hospitalist that they saw. This is our way of being able to tie all that data together now. Excellent. Looks like a more robust future ahead with this capability. Okay. Before we turn to some of the question answers, I would invite other folks to add some other folks, please put in the chat any Q&A you might have for our panelists here, and then also you specifically, you might want to answer some of these. I am gonna kind of give some commentary here, even though I'm a moderator on that, the real-world data piece. You know, there is actually policy implication to this. For those in the audience who might be on the policy side, I've worked in you know U.S. federal government for the White House at one point, and one of the key elements, at least of U.S. policy, was that patients have a right to their data. There's different ways of encoding that, certainly in the HIPAA covered entities. This was actually poorly understood by the community. In 2013, the regulators went out of their way to clarify you have a right to not only your data, but to direct it to whoever you darn well please. Some of these properties we're now encountering in other ways to actually link through to real-world data that Emily just covered in terms of these token-based linkages that are possible. This will most likely continue to evolve across really the world as other countries continue to refine their privacy and also right to access type policies to health data. I would encourage folks to continue entering questions here. I'm gonna pull up one of the first ones here, from, you know, jump off for all three panelists here. Cynthia Balles from, I think the, BRANY, the Biomedical Research Alliance of New York. She asks, "Fantastic discussion panel. Thank you. Can you address how you witness the role of the IRB change as decentralized trials have gained in popularity? We as, BRANY, B-R-A-N-Y, have increased our board level of expertise and also have learned that added expertise in social media for recruitment materials has been essential." Emily, Ros, Rinal, any thoughts on that? The IRB. I think we need to think about how much experience and how quickly we've had it and then support the IRB. I think what we found actually from pretty early on is that as long as we were providing really clear, simple information, so that kind of explaining initially even what a DCT was in the early days, but, you know, showing what the patient journey would be and where their data would be held, and making sure that it was clear that we were doing this with the patient in mind. As Rinal was saying earlier, not innovation for sake of innovating. We've been doing it to do the best that we could for patients. I think as long as the information is really clear, then IRBs have been really supportive. You know, if the information is not clear or if there's something that maybe isn't optimal for a patient, then rightly continuing to do that job that they do of making sure that we're doing the right thing. I think to add to that, Ros, the thing that we've seen is the amount of patient-facing material that an IRB now has to review, the package in general, it has become much larger. The other thing we see is there's a lot more on-the-fly submissions. Okay, maybe this strategy, this approach didn't work. We're gonna pivot and take this strategy or this approach. Refine and tune up the message that is going out to the patient. Again, having to have the IRB do another ad hoc review and another ad hoc review, because we can go in fully vetted with how we're going to do our outreach and our communication and our interaction with participants, and it may completely miss the mark. Now you have to pivot and shift and refocus in on how we're engaging with them. I think being able to work closely with the IRBs from our perspective has been really helpful, that we're not driving a study on because we've added decentralization, but rather, thinking quick on our feet to adopt and adapt, to what the patient's needs are. I'll add another implication is with regard especially to the outcome measures that are used as a primary, secondary endpoint. It is important that as we conduct the translations, we include minimum tone forward, 1 backward, and also localize those translations and conduct linguistic validation in order for those to be approved as an endpoint. That, that's one implication to keep in mind as well. Excellent. Yeah. However, I have to note that all three of you did not mention that IRBs are generally unfamiliar with TikTok, but that might be evolving, especially if they post protocols on TikTok. It will soon. Look forward to seeing someday soon. Kidding aside, thank you for all three of your kind of perspective on that. I do think that IRBs continue to evolve and learn. Another question that came up here, Billy Yarr, who I believe is from Lyft, you know, the with a Y, the ride sharing company. He asks, "What are some of the operational tools that sponsors and CROs can implement to help increase access and increase diversity? Is offering transportation a key component?" I wonder why somebody from Lyft would ask that. I jump off to all three of you. Is transportation a key component here? What have you seen in practice? We definitely see transportation as being a key component within ICON's operational approach. That choice, that pathway, letting the patient pick which way to go, by offering coordination logistics around how they are going to get to the site, if that is the way that they choose to participate in the clinical trial, has been key for us. It helps reduce a lot of that burden. I think to Ros's point, the "Well, I don't have bus fare," if we can just pick up the tab for the Lyft or the Uber or the rideshare, and get it to the patient earlier instead of after the fact. It really makes it sort of a, you're now taking that burden away. Absolutely. In the scheme of the cost of a trial, it's so minimal. It's crazy to me that that's not just implemented on every trial. Another thing that we've implemented as well that's gone down really well is childcare stipends. Particularly if it's kind of women child raisers or their partners who have childcare responsibilities, paying a stipend so that they can go to site and know that their child will be well looked after. They don't have to worry about the cost of that. I think that's been really beneficial, too. So many other practical things as well. I'm quite food-oriented and, you know, if I was in a hospital, I want to know I was getting fed or if I need to bring a packed lunch or, you know. Just something as simple as having a token to be able to go to the hospital cafeteria, that can be, you know, it can take a worry away for a patient. I think the more you know in advance, the better you can prepare and the better you can do for patients. We run a patient advisory council, just after COVID hit, when lots of the hospitals were closed, and we said, "You know, what is it that you want to think about or know before you go?" It was all like, "What's your biggest fear of COVID?" I think that was the question we asked. It was all practical things like, "Do I need to take my own mask? Will they take my equipment when I leave? Will there be like one way in a corridor? Will I have to go and lift? Will I have to touch a button, sorry. Where is the hand sanitizer?" It was all of those practical things. I think as much of that practical burden during an intervention trial that we can take away to make a patient feel comfortable, to make them feel heard and looked after, I think all of that comfort of patient is key, but there are so many other things that we can do besides, for sure. Okay, we're gonna do one more question from the audience here before we have to. Oh, maybe a couple more. Okay, let's see if we can get to both of them here. Aidan Gannon, who is either, I think according to Google, you're either the lacrosse athletic manager for Yale or else worked at Monvo Clinical. I'm guessing the second. He asked, regardless of whether a traditional hybrid or DCT, protocols are getting more complex with more and more procedures year-on-year, directly impacting what we ask of sites, patients, caregivers, and study teams. What do the panel think can or should be done to reduce the number of procedures, regulatory and commercial issues notwithstanding? What do you think? Sorry, go on, Renald. No, no. Please speak. Thank you. I think the number one thing is to ask patients, but also sites. We have a nurse advisory panel that we use regularly where we do a patient burden analysis as soon as we get a draft synopsis in. It's very simple. We give a list of the tests and procedures and the length of time each one takes, and they tick a box, and then we map it onto the schedule of assessments to look at how long each visit is going to take. The first time I ever did it, visit one was going to take 17 hours with everything that we were asking of a patient. Actually just quantifying that burden from a patient's perspective, but also from a site perspective is critical because it's not like every patient's going to come in at 9 A.M. They might have their visit start at 2:00 P.M., and then they're going to be there through the night. Maybe they'll run and they'll go in, they'll come back. There's so many things you need to think about. I think it really empowers the study team if you can provide that with that quantitative data to be able to go back and maybe speak to the clinical scientists and others in the organization and say, "It's not just tricky, it's 17 hours. This is wild. Like, what are we doing?" Then I think people can start to unpick things and remove some of these things that are in there because it's interesting rather than it being absolutely necessary. That is honestly, that simple little thing that we do has been the number one way that we've encouraged people to reduce the burden by actually giving them that hard data and, you know, encouraging them to think about what that would mean for them if they were a patient. You know, what if it was a really long visit and you got discharged late at night? Like, how would you get, again, things about travel. Would you have a taxi service set up for people? Or would you be calling your husband to get the kids out of bed and come and collect you in the night? I think attaching it to kind of real-world human experiences and that this is the hours that you're talking about, really starts people being, you know, unpeeling the onion of things that they can take away. another point just to add there is how we can reduce sort of the complexity is by introducing more objective measures through sensors and wearable devices. It's not just introduced in DCT, but can we eliminate actually some of the visits required for patients to travel by incorporating these objective measures so we can kind of better understand? it goes back to the initial point, can we reduce the sample size required for us to detect treatment effects? these are methods that we have to start looking in and really honing in reducing the sample size, but also eliminating the need for patients to travel for a site visit because we're already collecting these data through passive monitoring. Roz, ICON does the same thing. We call it time and motion. We do it from a site perspective and a patient perspective. I'm glad that Parexel's doing it as well. Everybody in the industry needs to be doing this. Even, let's put the onus back to who's writing the protocols. Let's, you know, if we can empower them to have that same kind of a tool, then when we get the very first look at this, it's not too late. The protocol hasn't already been submitted to the FDA for initial review, and now we're not having to go back and amend it. You know, doing that upfront engagement and really thinking about it critically from a site burden, a patient burden, and ultimately, is it nice to have data or is it must-have data? Excellent. All right, so there's one more question then we're all out of time, unfortunately. Brad Norton had asked, how do you walk the line between supporting patients in their study participation and not being seen as providing inducements to patients? But we'll kind of maybe answer that in the chat or in your closing remarks here. We've covered a ton of ground today. Thank you so much to all of our panelists. As we close this out, I want you to reflect on everything we've talked about and let us know, you know, again, closing this out, if there's one thing that you know now today or maybe two things that you wish you knew starting out in our journey to modernize clinical trials, what would that be? I'll start off with you, Renel. I think keep things simple. You know, try to break into smaller pieces, complex problems. Solve for an unmet need. Always focus on solving for a problem that exists. Ask why we're doing this. As a leader, I think also you have to create a culture within your team that you celebrate failure at times, but also you value the effort versus the results. Thanks, Renel. Emily? I think if I had one thing, it would be that technology should not be driving the clinical research. It should be enabling the clinical research. I think very early on, we were very focused on how we could put tech into everything versus figuring out what it is that we needed to collect and then what could we use to collect that. In hindsight, we probably could have saved ourselves some headaches if we had done it a little bit on the backward approach. That's good stuff. Thank you, Emily. Roz. I think that people are a lot better at change and innovation than they maybe think when there's a critical need. We were talking about DCT for so long before COVID hit, and there was so much resistance out there. Out of necessity, everyone embraced it. What probably would have taken five, 10 years before, we pivoted in months. With everyone on board, we were able to charge forward, support patients, and have this huge, spectacular change in the industry. It's been so exciting, and there are amazing people on this panel and others that we've worked with across the industry to make it happen. I'm so proud. I am too. Thank you, thank you, thank you to our panelists, Emily Mitchell, Rosamond Round and Renel Laaj today. We've covered some fantastic spectrum of issues. Hopefully, you all in the audience have gotten a good idea of real data and experience. Thank you again so much to our three panelists. As a final note to everyone, if you'd like to get involved in how all three of these organizations modernize clinical trials, let us know. Post in the chat or links to the career websites for the three organizations for Regeneron, ICON, and Parexel. I know that folks would love to hear from you there. With that, thank you to, again, one more time to all of our panelists. Thank you to the organizers at DPHARM for this wonderful opportunity to hear from the wisdom from these experts. Thank you to you all in the audience for joining us as well. Thank you so much. Bye now. Thank you. Thank you.