Article

10 Myths About Agile and the FDA

Bernhard Kappe

Published on: October 6th, 2015

Myth: Verification and Validation should only start after all development is complete.

Truth: In traditional waterfall, verification and validation activities often commence at the tail end of a project, after all, development is complete. This diverges from the FDA’s recommendations:

“The FDA does recommend that software validation and verification activities be conducted throughout the entire software lifecycle.”

In Agile, verification and validation start at the user story level, providing traceability from user story to code, code to unit test, and user story to acceptance test. Using a combination of test-driven development, coding conventions and ALM software, much of this process can be automated. Formal verification and validation activities are also conducted at increment and release checkpoints.

In agile, validation becomes integral to development, rather than an afterthought. By conducting verification and validation activities throughout the software lifecycle, Agile catches defects earlier, reducing surprises, rework and project slippage.

2. Myth: The only medical device companies that have implemented Agile are startups.

Truth: A growing number of large medical device and diagnostics firms have implemented Agile and demonstrating success, including Abbott Labs (https://www.computer.org/csdl/proceedings/agile/2009/3768/00/3768a151-abs.html), GE Healthcare, Medtronic, Roche, St. Jude Medical, Cochlear, Boston Scientific and Toshiba.(AAMI TIR45:2012 Guidance on the use of AGILE practices in the development of medical device software, Association for the Advancement of Medical Instrumentation, August 2012. )

3. Myth: The FDA prescribes waterfall development

Truth: While many industry professionals believe that FDA regulations require waterfall, neither the FDA’s 21 CFR Part 820 Quality System Regulation nor other regulations derived from it prescribe a particular development methodology. In fact, the FDA explicitly cautions against using waterfall for complex devices:

“The Waterfall model’s usefulness in practice is limited, for more complex devices. a concurrent engineering model is more representative.”

In addition, the US FDA explicitly recognized agile as an acceptable standard in January of 2013, referring to AAMI TIR45: 2012, Guidance on the use of AGILE practices in the development of medical device software.

4. Myth: Agile teams don’t do risk management

Truth: Similar to other aspects of a software development process, not all risks can be known at the beginning of a project, and we discover most hazards as a system evolves.

This is acknowledged in ISO 14971, the international standard for risk management systems for medical devices:

“Risk can be introduced throughout the product lifecycle, and risks that become apparent at one point in the life-cycle can be managed by action taken at a completely different point in the life cycle.”

In Agile, smaller sets of functionality (designated by a story) are taken all the way from story level requirements through development, verification and validation. Activities such as risk analysis, evaluation and risk control are done at a story level, as well as an increment/feature and system level. Our experience has been that this is best done by embedding risk management at each level of the lifecycle, with inputs and outputs from each activity within that level.

5. Myth: International Standards like IEC 62304 are incompatible with Agile

Truth: IEC 62304 is compatible with both waterfall and agile methodology:

“It is easiest to describe the processes in this standard in a sequence, implying a ‘waterfall’ or ‘once-through’ life cycle model. However, other life cycles can also be used. Some development (model) strategies as defined at ISO/IEC 12207 [9] include (see also Table B.1): … Evolutionary: The ‘evolutionary’ strategy also develops a SYSTEM in builds but differs from the incremental strategy in acknowledging that the user need is not fully understood and all requirements cannot be defined up front. In this strategy, customer needs and SYSTEM requirements are partially defined up front, then are refined in each succeeding build.” – IEC 62304 – Medical device software – Software life cycle processes, 2006. Annex B, pp. 75-76.

6. Myth: Agile is just a Fad

Truth: Agile methods evolved in the early 1990s and became mainstream among large organizations in many industries in the early to mid-2000s. The Standish group CHAOS report, one of the most respected studies of the state of software projects has consistently shown a high correlation between agile and success since enterprise adoption started taking hold in the mid-2000s.

7. Myth: User stories are the only requirements in agile

Truth: User stories are the beginning of requirements definition, not the end. They form a small enough unit and are easily understandable by end-users and customers. Functional requirements are further defined through task flows, annotated wireframes and acceptance criteria, and nonfunctional requirements are captured through techniques such as FURPS+ matrices. (FURPS+ is a model for classifying requirements. FURPS is an acronym that stands for Functionality, Usability, Reliability, Performance and Supportability. The + in FURPS+ stands for design constraints, implementation requirements, interface requirements and physical requirements.)

8. Myth: Agile sacrifices quality for speed

Truth: Quality is one of the main aims of the agile methodology, and many of the core agile practices are designed to improve software quality while maintaining project velocity.

While traditional software development teams often spend 1/3 to 1/2 their time on defect troubleshooting and rework, agile practices are designed to catch and fix defects early, when they are easiest and cheapest to fix.

Agile practices such as user stories, unit tests, test driven development, continuous integration and zero bug tolerance are designed to improve and maintain software quality. This allows teams to move much faster with fewer defects than in traditional waterfall.

9. Myth: Agile is undisciplined and incompatible with quality management

Truth: The myth of agile as “cowboy coding” or “write software, ask questions later” is always popular, but couldn’t be further from the truth. Planning, requirements definition, design, testing and validation are all part of the agile process, with the difference that agile repeats this process on smaller increments. It’s designed to get feedback early and often during the course of product design and use this feedback to continuously improve the product. This approach allows for greater transparency and control of the development process than the traditional waterfall. The agile feedback loop can be adapted to incorporate risk management, human factors and verification and validation that meet the FDA’s quality system regulation.

10. Myth: Agile can only be applied to software

Truth: While agile methodology initially evolved in software development, the core principles that not all requirements can be known upfront, that design is an iterative process that benefits from splitting features into smaller, more modular chunks that can be delivered and validated are valuable in hardware projects as well. Methods such as Commitment-Based Project Management (CBPM) use these core principles for hardware design and development. Using these methodologies together can work especially well for combined hardware and software projects, allowing design, development, testing and validation to happen in parallel, saving time and surfacing defects much earlier, when they are cheaper to fix.