As a project manager and subject matter expert for Bioclinica's Compass RBM software, I've seen more and more companies turning to intelligent monitoring tools to track site performance and put their risk-based monitoring strategy into practice. When setting up a tool such as Compass, it's essential to choose the most important aspects of your trial to track in order to monitor site performance and identify areas on which to focus the most attention. This is done using Key Risk Indicators (KRIs), which are used to monitor identified risk exposures over time. These indicators help you monitor the data that is most critical and should be focused on quality, safety and compliance. In my experience partnering with our Compass customers, I've identified a few things that you should consider when selecting KRIs to ensure you're tracking the right data for your studies.
1. Risk Identification and Tolerance
Assessing risk is at the heart of RBM, so make sure you're choosing KRIs that are applicable to the areas of your particular study or program with the highest level of risk—the areas on which you'd like to focus most of your attention. There are tools available to help you determine the right level of risk, such as the Risk Assessment and Classification Tool (RACT) developed by Transcelerate.
This ultimately depends on how comfortable you are with risk (risk tolerance) as well as the types of studies you're running. For example, if you are running a trial using a novel chemotherapy drug and a very sick subject population, you may want to be more conservative with how you monitor your data from a safety perspective. On the other hand, if you're doing a post-market surveillance study, you may have a higher tolerance for risk and would therefore be less conservative in your approach to monitoring.
2. Your Protocol
Make sure you choose the KRIs that make the most sense for your protocol. For example, one KRI you could choose is average days to case report form (CRF) completion, which is necessary for staying on top of how quickly data is being entered into the database. If you have a study with a short timeline, you'll want to set your threshold (or "trigger level") for this KRI quite low so you'll get an alert if sites aren't getting their data submitted quickly, ensuring you're sticking to your timeline.
Another important consideration related to your protocol is your patient population. Common KRIs to track are adverse event rates and serious adverse event rates. In some cases, like an oncology study with a very sick population, you may not want to include a KRI to track adverse events (or you may want to set a much higher threshold for this KRI), because with this population, you are expecting to see a large amount of adverse events during your study. In this situation, it could be better to choose a KRI focused on only serious adverse events (versus all adverse events).
3. The Number of KRIs You Choose
It's important to find a balance in the number of KRIs you choose. You need to choose enough KRIs to ensure you're tracking the most important aspects of your study; however, you don't want to choose so many KRIs that they become overwhelming and create unnecessary white noise.
Ultimately, the "right number" of KRIs will depend on your study and your needs. Some companies are comfortable taking a less conservative approach and only track seven or eight KRIs, while other companies prefer a larger number of KRIs in order to take a more detailed approach. In my experience, most companies choose between 15 and 30 KRIs per study.
4. Cross-Functional Input
Who are you involving in your KRI selection decision? Make sure that you decide upon KRIs cross-functionally so that you get perspectives from multiple stakeholders. You should consult with representatives not only from your clinical operations department, but also from the biostatistics department, the quality department, etc. Getting input from people in different roles and departments will help you take a more holistic approach to KRI selection.
Let's say your team is looking at a KRI related to the number of new subjects enrolled at a site within the last 30 days. Clinical operations personnel may focus on making sure the site has enough clinical study staff to handle the number of new subjects enrolled. Therefore, when setting the threshold, they may not believe there is a risk to the site's performance until 10 subjects are enrolled in a 30-day period, because at that point the site staff may not be able to handle the workload.
On the other hand, a biostatistician may bring a different perspective. They could be concerned if there are more than five patients enrolled at a site, as this may introduce bias in the data if one site has a majority of the enrollments. The benefit of cross-functional input is that you can identify these different viewpoints, discuss the consequences of either scenario and align on the most appropriate threshold for your study.
If you have any questions about RBM, Compass or how to select KRIs, I'd be happy to talk with you. Feel free to reach out to me in an email at firstname.lastname@example.org.