Therapeutic Benefit Ratings For FDA Breakthrough-Designated Devices

Stakeholders including health regulators and clinicians are discussing whether products that have been awarded breakthrough designation should receive a therapeutic benefit rating that clearly conveys to patients the level of clinical benefit that they can expect from such products. Medical technologies classified as breakthrough-designated, high-risk and/or novel devices are undergoing a faster and less traditional pathway to regulatory approval and subsequent clearance for marketing, and thus it may be appropriate to rate these products against major patient benefit against products that do not offer such benefits.

Why It Matters

The quest for faster development and clearance of medical technologies for patients’ immediate needs introduces uncertainty in an already complex system. To expedite the development and review for possible approval of foods and drugs for the diagnosis and treatment of diseases, Congress granted a breakthrough designation to speed up the development and review of technologies for serious diseases. Faster development and clearance typically means that a host of decision makers, from clinicians who will be faced with the challenging task of comparing the clinical performance of the new technology against other existing treatment options, to patients who may misinterpret FDA clearance of a technology for meaning it is best or superior, to payers who will be under intense pressure to cover the newer, and quite possibly costlier technology on stronger than usual evidence of benefits.

In addition to rating whether a device is likely to be “better,” it might also be rated for its “therapeutic benefit” – rating “how much better” that device is, in terms of survival benefit, symptom benefit, benefit to activities of daily living, benefit in decreasing complications, and benefit to health-related quality of life. Such a therapeutic benefit rating would help patients and health care providers have a brief, standardized way of understanding the degree to which a particular medical device could benefit a patient. The rating would not replace the clinical judgment of a doctor, but could help to inform and improve decision-making in the face of often confusing, incomplete, or biased information.

Why this debate is growing now

Products purporting to “transform” healthcare are getting more creative in their claims. While many have emerged in different categories such as digital health, implantables, robotics, neurostimulation devices and structural heart devices, each with the potential to deliver innovative benefits to patients and providers, there are very different quality profiles of evidence supporting these products at time of market entry. While a short list of positive data points, even those that include studies of only short duration or that rely on indirect clinical endpoints or limited follow-up, can enable highly informed stakeholders to market and price products to successful launch, this information typically does not enable clinicians to determine whether any benefits will be sustainable for patients long-term, generalizable to patients for whom the product is not indicated, or outweigh tradeoffs associated with the device’s risks and costs.

With decreasing healthcare budgets, the concept of “value” comes into play earlier in the product adoption cycle. A hospital may consider the cost of a device and consumables, surgeon training, service contracts and even expansion of the surgical suite. A payer may need to decide whether to reimburse a technology broadly, for specific indications or only after collection of certain data to better understand benefits and risks to patients. Patients will also weigh benefits and risks of surgery, post-operative recovery and out-of-pocket costs for the procedure. A benefit rating can serve as a useful guide to these critical decisions, based on the available evidence.

What a rating could look like in practice

The blog series on rating uncertainty continues with attempts to find a simple rating system that is realistic and honest. Here are a few categories that might correspond to the same clinical impact:

• Substantial expected benefit: meaningful improvement in patient-important outcomes compared with standard care.
• Moderate expected benefit: improvement likely, but smaller in magnitude or limited to certain subgroups.
• Incremental benefit: modest improvement, often in process measures or narrower endpoints.
• Uncertain benefit: promising signals, but evidence is incomplete, indirect, or not yet confirmed.

Transparency about rating of evidence strength and gaps is critical. The summary of the evidence rating should clearly articulate what evidence exists and what the rating means in terms of the strength of that evidence. In addition, the summary should explicitly state whether the evidence is derived from randomized trials, single-arm data, registries, other real-world data, or early feasibility studies, as well as whether clinical outcomes or surrogates are used as endpoints. This is crucial to distinguish a possible breakthrough from a breakthrough, and not stifle innovation by misrepresenting promising, but incomplete, research as ready for widespread clinical application.

Why a rating could improve trust

Explaining benefit ratings. A big part of patient education is demystifying the “breakthrough” label that the FDA gives to certain drugs. Many people have an unrealistic belief that “breakthrough-designated” equals “superior treatment.” In reality, “breakthrough” means that there is a reasonable expectation that the new drug will be serious therapy for patients with a particular disease, and that it will provide clinical benefit compared to existing therapies. The “breakthrough” designation can be granted earlier in the development process, in contrast to the traditional fast-tracks which require evidence of superiority to existing options.

While evaluating the benefits and risks of a new technology relative to current standard of care is critical, clinicians also would benefit from an assessment of the benefits and risks of a new technology relative to other technologies currently available. Initially, both clinicians and patients are drawn to the novelty of emerging technologies and it is crucial to separate new features from true advances in outcomes. A benefit rating will serve to distinguish enhanced outcomes from novelty and will provide a rating that can be used as part of informed consent processes as well as a metric for programmatic evaluation across sites and over time. The benefit rating will help to standardize early adopters’ perceptions of similar evidence as opposed to those of more conservative practitioners.

Who It Affects

Clearer benefit communication will primarily benefit patients and their families. Therapeutic benefit rating clearly communicates the benefits supported by clinical data to patients and health care professionals. A patient needing for instance an implantable device will want to know not only that such a device has an authorization but also in what way it will enable him to live his life (activities of daily living) to what extent his treatment will reduce need for hospitalization and/or increase life duration in comparison to existing alternatives. Patients and health care professionals will have the right to ask the following questions: What gain? How large is the gain? In which patients?

Use the ratings more as a conversation aid than a determinant of management. A high rating could be used to argue for earlier deployment of the technology in indicated patients where time is of essence. A lower rating could be used to argue for staged introduction, referral to centres of excellence, or registration of patients in active post-marketing clinical databases. The ratings provide a quick read on whether or not a technology appears to be potentially highly impactful versus providing incremental improvement in outcome for patients in busy practices.

Specialists, generalists, and care teams

A single rating might be used by different clinical groups for different clinical decisions. The rating could be used by specialists to weigh the advantages and disadvantages of different device-based intervention options within a class and to counsel patients accordingly. Generalists might use the rating to determine potential impacts on follow-up interactions, the frequency and nature of follow-up monitoring and testing, and the timing of referrals to other medical or ancillary services. Supporting staff (e.g., nurses, technologists, rehabilitation staff) might use the rating to predict clients’ training needs to various levels and to plan follow-up in the hospital, clinic or community settings.

The way a health benefit is rated could have significant implications for the day-to-day functioning of health systems and hospitals. A high benefit rating could be used to justify investment in new technology, staff training and development, staff credentialing and new service lines. A lower or uncertain rating could be used to support a “pilot” with clear criteria for scaling up, a limited rollout with specific outcomes pursued. Additionally, procurement teams may wish to incorporate benefit ratings into their purchasing framework alongside safety signals, maintenance and repair costs and impact on workflow.

Payers, policymakers, and employers

Developing benefit ratings could help ensure consistent benefit ratings by serving as a framework from which to begin benefit determinations. While payers have begun to undertake evidence reviews to inform their coverage decisions, the methodologies and timeframes differ greatly. A rating framework for benefits could help standardize and streamline Payers’ coverage policies, reducing redundant efforts to generate evidence for coverage decisions and helping manufacturers understand the scope and quality of evidence required for broad-based reimbursement.

In addition to determining whether promising but incomplete evidence warrants development into full evidence for a conditional coverage approach, Ratings could be used to assess evidence that a conditional coverage model exists for payers to cover patients’ use of a medical device at different stages of its development.

Ratings also could be used by policymakers to prioritize future postmarket studies for devices that present the greatest risk, highest cost, or fastest growing volume of devices entering the market.

What Changes

The first practical change that should occur in how medical devices are evaluated and developed is greater, clear communication between patients and clinicians. A rating would assist patients and clinicians in forming realistic expectations regarding the overall benefit of a particular technology. It would allow clinicians and industry to portray a realistic expectation to the market versus over hyping a technology with only incremental, uncertain, or mixed benefits to patients.

The second key area for change is Smarter adoption and monitoring. The ratings provide a direction of travel for smarter adoption for the technology within a hospital, highlight areas for training of clinicians and users and indicate where devices require registry participation or monitoring. But whereas previously there was argument over whether or not to adopt technology, now the conversation is about smarter adoption.

A third change is the increasing emphasis on payment linked to value. Assessment of value for patients could be a new criterion for decision makers when determining whether to provide reimbursement for a new medical technology. Technologies determined to offer high value for patients would be allowed a faster pathway for adoption and broader coverage while technologies determined to offer uncertain or less value would be granted conditional coverage with additional requirements such as collection of certain data, specific patient selection criteria, or experience with the technology at certain centers.

This initiative, known as the regulatory follow-up on certain devices, was the fourth major set of modifications to the premarket approval process for medical devices. Additionally, the law targets devices of uncertain benefit, those that receive little postmarket follow-up, or devices that were cleared using surrogate endpoints and can be studied postmarket and placed on active surveillance.

Why implementation is hard

Many outcomes are difficult to measure and therefore to score. Some can be scored as a single number but others cannot. For example, being able to go for a walk with the grandchildren for an afternoon each week can have different value to different people. While it may be very valuable to some, it may be of little value to others. As a result, the benefit framework should avoid spending too much time on trying to convert all of these outcomes into numbers and instead focus on setting out a clear prioritisation of the outcomes and how these relate to the decision at hand, to help guide clinical and other decision makers in judging whether a particular device will be good or poor for a particular patient.

There may be little evidence at the point of market entry. Utilisation of faster authorisation procedures can increase the chances that overall authorisation will be granted before full late term/very long term outcomes have been generated to demonstrate full benefit. An optimistic rating at early time points that proves to be consistent with later performance will be correct and helpful. An overly pessimistic rating at early time points will be unhelpful. However, a worked able approach could also include an ‘evidence maturity’ component to the rating to give the user an indication as to whether benefit has been established or whether evidence is emerging.

Manufacturers want to rate their products to bring credibility to value frameworks and support their use. Payers want to rate products to ensure that they are using their limited health care budgets cost effectively. However, the payer’s role in controlling costs to achieve affordability is likely to compromise their objectivity. Thus, it is possible that regulators may cede this role to an independent panel of multi-stakeholders, employing transparent and consistent methods while appropriately managing conflicts of interest, and making all relevant aspects of their processes and decisions publicly disclosed.

How ratings could be designed to stay fair

1. Expected clinical benefit level (substantial, moderate, incremental, uncertain).
2. Evidence confidence (high, medium, low) based on study design quality, follow-up length, and generalizability.

For a full solution, it might be desirable to separate the concept of clinical benefit from that of evidence strength. A intermediate solution might be to present information about treatment efficacy in two parts.

Any ratings developed for digital medical devices should be reusable and updateable. Over time devices and technology will evolve as will the experience of clinicians and the accrual of real world evidence. As a result any initial ratings developed should not be set in stone, but rather be a “living rating” updated periodically or as new evidence becomes available.

Clinical decision-making and patient conversations

You would rate the device in relation to best usual care initially and then use that as a conversation starter with the patient. The rating would give an estimate of the relative improvement for the patient, and then you could individualise that to suit the patient’s needs – e.g. presence of comorbidities, degree of frailty, life expectancy, personal priorities and home support.

Clarifying both benefit and uncertainty for patients allows for informed decision making. Patients and surgeons want to know: What percent improvement can I expect? What are the chances of serious complications versus minor complications? What is the typical recovery period? How long will the device last? How often will I need follow-up? What is the total out-of-pocket cost for this treatment? These ratings can be used to improve the quality of informed consent by anchoring patient and surgeon expectations to the best available evidence.

System-level trade-offs and equity

Technology ratings derived from these frameworks should inform, but not dictate, decisions about third party reimbursement to ensure that expenditure is consistent with the goals of health spending equity. Payers might want to withhold coverage of reasonable options for some patients if they only choose to cover technologies with very high ratings. Conversely, resources allocated to provide uncertain-benefit technologies could be siphoned off from more effective health services.

The generalizability of trial results should be rated explicitly. Although a trial may have enrolled 1000 patients, these patients do not necessarily represent all patients for whom the technology could be used. Performance may differ significantly between middle-class young patients and less affluent older patients in resource-poor settings. The strong rating process seeks to assess evidence that a technology would perform well on a variety of patients in different real-world settings and identifies additional needed postmarket data collection in inclusive populations.

What comes next

It might be better to roll this out in phases, rather than developing all of the ratings at once. I think an initial pilot could focus on the Breakthrough Devices with the biggest decisions and highest costs for patients and clinicians. The goal of the pilot would then be to test whether the ratings help patients, clinicians, and payers understand key information and achieve consistency without impairing access in unintended ways.

Ratings can be most useful to patients and families when incorporated into the electronic health record or into clinical decision support software at the point of care (e.g. during ordering, referral, or consent). Ratings can also be included in patient education documents (e.g. summaries of medical and surgical options) that list risks, alternatives and follow-up care.

Transparency is by far the most important aspect for effective ratings. The rating methodology must clearly explain what is rated and why, what evidence has been used for the rating and what uncertainties are involved. Transparency is achieved by public disclosure of the rating methodology, documentation of the methods and independent audits.

Bottom line

A benefit rating for breakthrough devices could help fill the divide between speed and clarity for devices designated as breakthrough. Done well, these ratings can enhance decision-making for these products, inform patients and their physicians about clinical benefits and uncertainties and guide reimbursement and postmarket surveillance decisions. Done poorly, however, such ratings could result in oversimplification of complex trade-offs, exacerbate inequities, or worse. The next best step, therefore, would be to design a benefit-rating approach and establish multi-stakeholder governance for the process. As real-world evidence unfolds, the ratings also must be refined to ensure they remain relevant and reliable.

References:

https://www.fda.gov/medical-devices/how-study-and-market-your-device/breakthrough-devices-program https://pmc.ncbi.nlm.nih.gov/articles/PMC12758617/