Sometimes one trial can start smoothly and plug along nicely, while another stalls, stutters and even stops before the finish line. Comparing two trials can provide an interesting education in what factors contribute to the success of a study, and what can drag research down.
Outsourcing-Pharma (OSP) recently spoke with Gen Li (GL) president of clinical services firm Phesi, about the company’s recent analysis of two similar COVID-19 vaccine firms, and what led to the studies’ notably different site effectiveness index (SEI) scores.
OSP: Could you please explain what factors go into the SEI metric and what a favorable/unfavorable SEI means?
GL: The SEI metric was developed to answer questions involved in the clinical development process that are often overlooked. For example, how should we measure site activation and what is a good site activation plan? How do we define the relationship between site activation and other variables in clinical trial planning and execution?
SEI defines utilization of enrollment capacity within a group of investigator sites allocated to one clinical trial.
The variables include first subject first visit (FSFV) and last subject first visit (LSFV) at trial level; date of site opening for enrollment and date of site closing for enrollment, targeted/actual number of patients enrolled; maximum number of investigator sites activated and opened for enrollment between FSFV and LSFV. The higher the measure the better.
For a trial with 50 investigator sites and a 12-month enrollment cycle time, activating all 50 sites on the first day of the trial would yield an SEI of 100%. On the flip side, if we activated all 50 sites on the last day of the 12-month period, the SEI would be 0.
OSP: Why is attaining a favorable SEI important?
GL: The importance of SEI is in its potential for optimizing site activation. In simple terms, improving SEI means improving the execution of a trial – it can translate to reducing enrollment cycle time and needing fewer investigator sites. This means reducing costs and reducing the time it takes a trial to be completed.
Using a SEI measure rather than just looking at historical performance, or looking at an isolated data point on the number of patients enrolled, means we can identify areas for tangible improvement in trial planning and execution.
OSP: Similarly, why might it be especially important when COVID-19 vaccines and therapies are at the center?
GL: There is global pressure on the search for COVID-19 vaccines and therapies, and our data show that currently there is varying success in activating investigator sites and enrolling patients. This must be tackled to accelerate development.
The difference of a few months in a global pandemic is huge. But even post COVID-19, the need for improvement will remain. Bringing innovative and life-saving medicines to patients is complicated and capital intensive. Sponsors must improve SEI through a deep understanding of real-time, dynamically refreshed data and a forensic examination of the causes of obstacles.
OSP: How can Phesi work with clients to improve things like keeping protocol amendments to a minimum, to boost that SEI number?
GL: Phesi has an integrated approach to assessing variables that impact clinical trial planning and implementation. We can define quantitative relationships among key variables such as investigator site performance, business processes (especially site activation), and protocol design; this means clients get actionable insights that can help them mitigate problems and manage difficult areas – like CRO selection and oversight, protocol design to minimize amendments, and developing performance benchmarks.
This improves SEI, and improves the chance of a study achieving a successful outcome. Phesi’s SEI metric allows it to carry out this comprehensive analysis of all the moving parts involved in a trial. This kind of data-driven analysis is not only important now for COVID-19 related trials, but important for all future clinical development.
OSP: What else would you like to tell us about this SEI comparison regarding the two C19 vaccine trials, or about SEI measurement/results?
GL: The industry knows site activation is important but tends to look at it in isolation without knowing how to meaningfully measure it. SEI provides a quantitative measure of site activation in the context of other variables. For instance, trial B in this scenario used an adaptive design. This means starting a trial from early phase then continuously advancing the trial to later stage by adjusting the design based on patient data collected.
This has the potential of being quicker, but it also increases the financial risk. The trial could fail at any point, and all the resources and money invested to date would be wasted. Moreover, if we were to measure the trial from a late stage perspective, the outputs would be poor – SEI is low and enrollment cycle time is longer.
So an adaptive design offers the possibility of allowing patients to get innovative medicines sooner, but there is financial risk. The key for sponsors, including with adaptive design, is to mitigate the risks by making use of the data available.