Article

Agile in FDA Regulated Medical Software

Michael Walkden

Published on: May 10th, 2011

This post was previously on the Pathfinder Software site. Pathfinder Software changed its name to Orthogonal in 2016. Read more.

Testing is a crucial part of all software development. If you don’t believe this you are likely just starting your career or haven’t had to support an application in production with users. It becomes doubly important in regulated environments like the ones for medical devices. This is because testing for regulated software serves multiple purposes. There are the usual software product development reasons to consider (i.e. professional integrity and no one wants a bug in production). But it is also because of the FDA, in particular, wants to make sure you are performing validation and verification of the software. Verification (Was the product built right?) Validation (Was the right product built?)

While I acknowledge there are a number of useful and important ways to describe software testing techniques, I strive to keep things simple, and understandable, by the whole team. To that end, I split testing into two tracks: Functional and Technical. Functional is about bring business value to the stakeholders – the validation part. On an Agile project, we are testing the acceptance criteria of each story. This validates we have realized the benefit to the business and end users. If some thing cannot be tested by an end user, it doesn’t exist or has significant value. This is of course untrue. However, to simplify things and keep the team laser focused on only working on tasks that provide business benefit I start with this idea. All other testing activities are testing Technical requirements. Or, verify that the software is correct. Some examples of these technical requirements are things like performance criteria, common libraries to the backend and data retention rules. Those who do testing for a living are looking at the screen sideways by now I suspect. I realize that these things must be done correctly or the system will not realize the business value to an end user. Both are needed. That’s why the FDA requires proof of verification and validation.

The common flow of testing in an Agile project is to create acceptance criteria for each story then do regression testing after each iteration:

On the other hand, tradition testing documents and verifies each step before it is performed. The goal is for planning to be exhaustive and cover all possible scenarios.

Traditional testing is expensive and often makes validation hard to do. On the other hand, the common Agile testing approach is going to make documenting verification activities difficult. Once you acknowledge both are needed you can find a way that works. The best tools we’ve found leverage and extend the automated testing you are already doing. Look at the software testing as having two “tracks”.

#1 Manual Functional Testing of acceptance criteria. All of the speed and simplicity of user stories. An efficient communication mechanism for stakeholders.

#2 Test everything else via automated tools. Ensures all get tested. In our case, we use unit testing and 100% code coverage.

Here is a graphic of how we can document that we are testing against pre-approved requirements – both functional (validation) and technical (verification).