Regulation of SaMD in the U.S.: Webinar Summary

Randy Horton

Published on: August 3rd, 2022

Are you mystified by the regulatory process for SaMD in the U.S.? Unsure of how to begin your regulatory journey? Or worried about securing essential clearance from the FDA? Orthogonal and MATTER Chicago have you covered.

On August 24th, 2022, we hosted a webinar with 200+ attendees featuring a discussion between two MedTech experts who have helped bring SaMD to market for over a dozen firms: Clay Anselmo, Principal Quality and Regulatory Consultant at Shriner and Associates, and Bernhard Kappe, Orthogonal’s CEO and Co-Founder.

Clay and Bernhard drew on their wealth of experience to answer hot-button questions related to the regulatory challenges of SaMD in the U.S. They touched on the current state of SaMD regulation, existing regulatory pathways and the ongoing regulatory innovation at the FDA. They also shared some best practices to speed up the development, launch and evolution of SaMD under today’s regulatory frameworks and guidance.

Webinar Recording

This webinar was part of our monthly series of webinars titled the Move Faster and Break Nothing Webinar Series. Join us throughout the rest of 2022 as we host webinars featuring industry leaders and pioneers sharing cutting-edge Agile insights to collectively improve patient outcomes and drive business results.

The remainder of this blog starts with a summary of the six key takeaway points from the webinar, and then features an edited summary of Bernhard and Clay’s full conversation.

Six Key Takeaways from the Webinar

The FDA’s existing regulations for SaMD and other medical device software are based on older techniques for software development. The FDA is currently working with the MedTech industry to draft and publish updated guidance that addresses more contemporary topics, such as SaMD developed with Agile methodologies that incorporate emerging technologies like Artificial Intelligence (AI) and Machine Learning (ML) algorithms.
While there are some differences in regulatory pathways, it’s important to note that SaMD goes through the same paths as other medical devices. There is no special SaMD pathway.
On average, about 10% of 510(k) require clinical studies. Because SaMD are more likely to be considered novel, they are significantly more likely to have this requirement.
For SaMD that have no predicate device, just like any other medical device without a predicate, it is highly likely that the medical device manufacturer will need to collect human clinical data in advance of their regulatory submission.
Once your SaMD is in use, you will be able to continue to develop and evolve your SaMD using qualitative and quantitative testing and post-market surveillance. However, you will likely need to submit another 510(k) before making any changes to the software that will impact patient safety or advance claims.
The FDA has published draft guidance on Good Machine Learning Practices and a position paper on continuous learning algorithms, but formal regulations are still in development. To date, no continuously learning (i.e., unlocked) algorithms have passed through the FDA clearance process.

Detailed Webinar Summary

The following summary has been edited from the original recording for brevity and comprehension.

Why is SaMD so challenging to get through the regulatory process, especially when compared to traditional medical devices?

Bernhard: The regulations and standards that govern SaMD were created before the invention of Agile software development, and at a time when most medical devices did not have software in them. Now we see software everywhere. Software isn’t only in medical devices, it’s become a medical device in itself, encompassing a physical device, a smartphone app and a space on the cloud.

With this expansion of medical device software, we’re also now facing much more significant challenges of interoperability. For example, how do we integrate your medical device’s mobile app that sits inside someone else’s telemedicine application? How do we deal with AI and Machine Learning (ML) algorithms that can adaptively learn? How do we ensure our devices are effective but also able to adapt to change happening around us? These questions pose new challenges for the regulations and standards that we as an industry must follow to keep patients safe.

Clay: The definition for SaMD, in a regulatory context, is drawn around the software itself. That’s not to say SaMD doesn’t run on hardware – obviously, it does – but that hardware is a commercial computing platform, not a custom-built piece of hardware with special features that are necessary for the software to run.

That being said, we do need to manage the interfaces that are integral to the operation of SaMD, whether that be hardware, software or cloud computing. This idea of interconnectivity and interoperability is key to ensuring that SaMD continues to operate the way it’s intended to, and safely, over its life cycles. To do that, you need to ask questions like: What systems does your SaMD interface with, and how do those interfaces affect the product in terms of proving safety, effectiveness or substantial equivalency?

What are the current FDA regulatory pathways for medical devices and how long do they take to go through?

Clay: According to the FDA, devices fit into three main buckets:

Not a device: Healthcare-related applications that the FDA does not consider a regulated medical device. If you are in this category, FDA regulations and guidance documents do not apply.

Enforcement discretion: The FDA believes they have the authority to regulate these devices as medical devices, but is choosing not to because they perceive them to be low-risk. The FDA may offer recommendations, but not requirements.

Regulated medical devices: Devices that are fully regulated by the FDA, which are further broken down into these categories:

Class I devices: Lowest-risk regulated devices. Many do not require a 510(k) or premarket notification; some may be exempt from Good Manufacturing Practices or quality systems regulation; some might be exempt from both.

Class II devices: Moderate-risk regulated devices. Almost all require a 510(k); you can think of these as their review and clearance process. The time from submission of the 510(k) to clearance generally runs about 90 to 270 days. To get your device designated as Class II, you have to be able to point to a predicate device that is substantially similar to yours in intended use and technology, and demonstrate that your device is equivalent in function.

Class II De Novo: This is a type of 510(k) but without a predicate device. These devices would normally have been automatically assigned Class III, but due to their intended use, the FDA considers them Class II. De Novo devices have “special controls”, or an outline of the types of testing and data that you have to supply in your 510(k) in order to use this pathway. Once one product goes through De Novo, anyone after is able to do a traditional 510(k) using that first De Novo as their predicate device. However, they still have to follow the special controls guidance document used in the original De Novo.

Class III PMA: These are the highest-risk regulated devices. Devices without a predicate go into Class III by default. This lengthy pathway requires you to prove safety and effectiveness through clinical studies. The entire process, which includes the FDA review period, can take anywhere from one to five years.

What are the misconceptions about SaMD, in terms of regulations?

Clay: First of all, the new guidance documents don’t drastically deviate from established regulatory principles and practices. They generally map new technologies onto existing processes. Until we have a major change in regulations (from the FDA) or the law (from Congress), we should expect the SaMD pathways to follow the same paradigm that they always have.

Second, the architecture of your product dictates whether or not you can separate the hardware from the software. When hardware becomes customized in a way that is necessary for your device to fulfill its intended purpose, it becomes impossible to separate the two.

Third, just because your SaMD is an app doesn’t mean you absolve yourself of managing the platform it operates on. If your SaMD runs on Apple iOS, you need to know that iOS changes a lot and that you’ll need to react accordingly. For example, how do you make sure that your device will always run on the hardware that it’s intended to run on?

The last misconception is that taking a SaMD pathway will substantially cut down your time to market. Though there are some differences in the regulatory pathway, SaMD submissions still flow through the same long premarket approval and clearance processes we’ve had for all these years.

Bernhard: One of the promises of the FDA Pre-Cert program was streamlining the clearance process for lower-risk SaMD, which really excited a lot of us in the industry. Unfortunately, the FDA doesn’t have the regulatory authority to implement the Pre-Cert program as they originally envisioned it. It’s up to Congress to grant that regulatory authority to the FDA in a law, and I think that the likelihood of that happening over the next few years is low.

Clay: There are some promising future programs that could improve the regulatory process for SaMD, but none of them are a magic bullet. Even with them, you could still have a lengthy path to market.

How did the 21st Century Cures Act affect SaMD regulatory pathways like De Novo or 510(k)?

Clay: The 21st Century Cures Act reduced the regulatory burden for certain software products. It altered device classification rule definitions, causing some products to fall under certain exemptions. If you looked at a particular product and its intended use a while ago, it might be worth taking another look now, as it may have been moved to “not a device” or enforcement discretion.

If your product requires pre-market clearance, the standard 510(k), De Novo or PMA pathways still apply. While the FDA has done a good job of introducing pilot programs for shortened pathways, it takes time for them to roll out to the general public. I urge you to be cautiously optimistic to what the future may hold, but understand that SaMD generally flow through the common regulatory pathways in the U.S. Your expectations for timing and preparing the necessary submission content are the same or even increased for new technologies.

Devices incorporating new technologies like AI/ML will generally take the De Novo pathway, a classification process for where there is no predicate device. It’s an involved, interactive process with the FDA. You will need to discuss what the special controls need to be, how to structure your pre-market submission, if you need clinical studies, etc. The total time to market, from when you start engaging with a regulator until clearance, is longer than the traditional 510(k) process. But clinical data requirements for 510(k) products continue to grow, especially for new technologies, so keep that in mind.

What FDA guidance exists to help guide developers creating SaMD with emerging technologies?

Clay: The FDA has done a good job of accommodating these emergent technologies within their existing regulatory framework. They created the Digital Health Center of Excellence as a resource containing their current understanding of SaMD. The Pre-Cert pilot proposal was another step forward. There’s also a bevy of new guidance documents, especially in the International Medical Device Regulators Forum, which I recommend you review.

Bernhard: The FDA has created or drafted guidance documents to address new technologies specific to SaMD. For example, their guidance documents on interoperability or multiple function devices provide a good illustration of how they want developers to treat those cases. Currently, there is no AI-specific guidance; there is, however, a position paper, as well as a draft guidance on Good Machine Learning Practices.

Similarly, there are standards and guidance documents produced by standards bodies that the FDA has formally recognized: ISO 13485, which is also referenced by the EU; IEC 62304, which covers software lifecycle processes; ISO AAMI TIR45 on the application of Agile practices within 62304. These are all documents that the FDA has accepted that we rely on as we think about building out medical device products.

Orthogonal’s blog “Climbing the Mountain of Regulatory Documents” is a great place to start when building a background on regulatory guidance for SaMD.

What percentage of products require clinical evidence/data for pre-submission?

Bernhard: I’ve seen a statistic that, on average, around 10% of 510(k) require some kind of clinical trial or clinical data validation, as opposed to bench validation. Clay, what are you seeing in terms of the percentage of products requiring more clinical evidence?

Clay: To be clear, clinical data isn’t always a prospective randomized control arm study. It can be data gathered on human subjects: in retrospective, in prospective, in a confirmatory study or a control study. We lump all of them into this bucket of “clinical data.”

A very high percentage of De Novo products require human clinical data. The newer the technology, the higher percentage of those devices that will require clinical data. My perception is biased, because I don’t often work on standard 510(k) submissions, but more than 50% of submissions I do work on require clinical data.

What are your strategies for choosing regulatory pathways for SaMD?

Bernhard: First, just because you can choose an easier path, doesn’t mean you should. You want to be careful to not treat something that is a medical device as a non-medical device. But if you aren’t making clinical claims, and thus have no associated patient risks, you’ll have a lot of freedom to iterate and improve on a non-regulated service, and often at a lower cost, than for a medical device.

On the other hand, it’s easier to get reimbursement, and as a startup to get a high valuation, when your product has gone through the clearance process. An example of that is Akili. They chose to put their low-risk Digital Therapeutic through a clinical trial, and used the clinical trial evidence as part of their FDA clearance.

Second, it’s likely your first regulatory filing won’t be your last, especially for software. You might have a roadmap that says that, down the line, you’re going to be able to make fantastic claims, but that won’t be part of your first FDA clearance. Starting with lesser claims is a faster and cheaper way to market.

What are your best practices for developing SaMD and medical device software compliant with FDA regulations?

Bernhard: When creating SaMD, you have to deal with the constraints of the software and of the medical device. Software moves quickly and is not wholly under our control. A medical device must ensure patient safety, demonstrate effectiveness and show compliance, which involves documentary evidence that takes time to generate.

My recommendations center on how you optimize for all of these constraints at the same time:

Adopt the mindset of product evolution as an ongoing process, in terms of speed and efficiency. Be ready to respond to change quickly, whether that’s to new data, a changing ecosystem, altered reimbursement models, new mobile data from other devices, cloud computing or software platforms. You can start small and simple, and extend to something new down the road.

Capture real-world data. The FDA actively promotes the use of real-world data over clinical trial data, which you can use to reinforce your claims or potentially make different ones. Efficient data capture and case management will help you evolve in a more streamlined fashion, could let you skip expensive clinical trials, and will enable you to protect yourself in terms of post-market surveillance.

Employ modular architecture. IEC 62304 contains guidance around software segregation, cybersecurity and interoperability that you can use to help architect your system to be easier to modify over time. Instrumenting modular architecture and automating it makes it easier to assess the impact of change, especially for things outside of your control.

Take a quality DevOps approach. Include processes outside of the software development life cycle into your end-to-end product lifecycle, like user experience (UX), human factors, risk analysis and design controls. Integrating and automating these into your DevOps process increases safety and compliance while shortening your timeframe.

Draw on techniques outside of the medical device space to aid in UX and human factors, like Lean UX and Three User Thursdays. Extending fast feedback loops to use errors and human factors will get you a higher quality product faster.

Integrate product analytics into your SaMD. Supplement your qualitative testing with users with quantitative data on how the product is used in the market. You can use this data for post-market surveillance and as a way to improve your product.

How do you apply risk management to continuous learning algorithms?

Clay: I’m unaware of any approved or cleared continuous learning algorithm on the market today. The ones I’ve dealt with do not continuously learn at time of release. You may retrain it with additional real-world data and re-release another version, but that’s treated as a software change and likely requires a new 510(k). Therefore, the application of risk management doesn’t vary between a non-continuous evolving algorithm and a continuously evolving one. The challenge of managing risk for a continuously learning algorithm is something we will have to face and solve in the future.

Bernhard: The FDA’s position paper on regulatory framework for AI/ML-based SaMD responds to potential changes to regulations for these devices and provides a framework for those changes, but I haven’t seen it in practice yet. This framework could be useful for offline learning that requires periodic updates.

If you were to have a continuous learning algorithm, you could come to an agreement with the FDA on how the software is going to change and what kind of updates you could make to it without filing a new 510(k). As far as risk management, you’d need to bound where an AI algorithm might be operating inside a device and how that is impacting the patient. That’s easier said than done, and we haven’t actually gotten there yet.

The FDA is still figuring out how to deal with AI changes, and it will take a while to get clarity and consistency from them. Just because someone else was able to employ a particular approach doesn’t mean that will be available to you.

Is there any special treatment required for AI/ML-based SaMD which changes over time due to changes in ML models? What kind of special documentation is required for such products?

Clay: This is similar to the previous question, but I want to comment that there is no such thing as a small change to a ML model. If you’re contemplating retraining a ML model with a new data set in a multifold cross-validation approach, what you get is more than an incremental iteration of what you did before. That makes it hard to make the case that some subset of your testing can be repeated, and would be enough to support another approval or clearance depending on the class of product.

The FDA has contemplated the use of a “standalone performance assessment”, an in-between bench and clinical trial where you gather real-world data and clinical data and apply it to both the old and new algorithm to compare their performance. If you can demonstrate that you had an improvement on a representative data set of your intended use population, you may be able to avoid doing a prospective clinical study. But I don’t know of any framework that says retraining an AI/ML algorithm does not require a 510(k), or that it can be done purely with bench training.

A summary of the laws, standards and guidance referenced in this blog

From the FDA:

From IMDRF:

From ISO/IEC: