FDA’s Draft Computer Software Assurance Guidance Changes the Game for Quality System and Production Software Validation

On September 13, 2022 the Center for Devices and Radiological Health (CDRH) and the Center for Biologics Evaluation and Research (CBER) jointly released a new draft guidance document titled “Computer Software Assurance for Production and Quality System Software” (the “CSA Draft Guidance”).¹FDA, “Computer Software Assurance for Production and Quality System Software: Draft Guidance for Industry and Food and Drug Administration Staff,” Sept. 13, 2022, https://www.fda.gov/media/161521/download [CSA Draft Guidance].  The CSA Draft Guidance is intended to provide clarity regarding FDA’s expectations for the validation of software that is used in medical device production and/or device quality systems.  FDA is accepting comments on the CSA Draft Guidance through November 14, 2022, docket number FDA-2022-D-0795.

Getting to the Current State

For decades, FDA enforced regulatory requirements for computer software used to automate compliance to quality system / CGMP (Current Good Manufacturing Practices) requirements or the manufacturing production process under broad regulatory provisions.

• 21 CFR § 820.70(i) regulates the use of computer systems used to support production of or the Quality System for medical devices.
  
  
• 21 CFR Part 11 addresses the use of electronic records and electronic signatures across all industries regulated by the FDA.

Elaborating on these regulations, FDA has authored, or participated in the development of, several guidance documents and technical reports.

• CDRH and CBER authored the “General Principles of Software Validation” (GPSV) guidance in 1997 and revised it in 2002.
  
  
• CDER, CBER, and FDA’s Center for Veterinary Medicine (CVM) collaborated in 2016 to draft a guidance document on “Data Integrity and Compliance with CGMP”, which was finalized in 2018. A footnote included in the guidance references the GPSV.
  
  
• FDA collaborated with the International Society for Pharmaceutical Engineering (ISPE) in the development of “A Risk-Based Approach to Compliant GxP Computerized Systems” (GAMP), which saw its latest revision released this year.
  
  
• FDA served as co-chair of the Association for the Advancement of Medical Instrumentation’s (AAMI) AAMI TIR36:2007 (“Validation of software for regulated purposes”), which was revised and harmonized as an international guidance in AAMI/ISO TIR80002-2:2017 (Medical device software – Part 2: Validation of software for medical device quality systems”).

The common theme across this regulatory literature over the last 20 years has been the establishment of a standardized set of software lifecycle activities and deliverables for systems subject to software validation requirements.

Systems subject to validation requirements can be as small as a spreadsheet used to track manufacturing nonconformances and as large as an Enterprise Resource Planning (ERP) system costing millions of dollars and driving large segments of a manufacturer’s business.  They can be as diverse as software that automates manufacturing equipment (managed by the production team) and that serves as a complaint management system (managed by corporate IT).  The level of risk inherent in software subject to validation requirements may vary from directly impacting safety to simply supporting a compliance activity, with no impact to the product.

Regardless of size, complexity, impact, and risk, the same expected activities and deliverables drive validation planning and approaches.

Computer Software Assurance (CSA) to the Rescue

After approximately six years of deliberation, collaborations between industry and FDA, and unanticipated delays (namely, the COVID-19 pandemic), CDRH and CBER released the CSA Draft Guidance on 13 September 2022.

The goal of CSA initiative was to advance a risk-based approach to ensure proper performance of software impacting device production or the quality system.  Effective implementation of software validation under the CSA Draft Guidance is hoped to reduce the activity and record generation burden across a device manufacturer’s overall inventory of regulated software systems without compromising rigor, where product safety or efficacy is at stake.

The introduction to the CSA Draft Guidance makes clear that it supplements the existing GPSV guidance and does not replace it.  The GPSV guidance is scoped to include:

• Software used as a component, part, or accessory of a medical device;
  
  
• Software that is itself a medical device (e.g., blood establishment software);
  
  
• Software used in the production of a device (e.g., programmable logic controllers in manufacturing equipment); and
  
  
• Software used in implementation of the device manufacturer's quality system (e.g., software that records and maintains the device history record).

The GPSV guidance addresses the last two items in Section 6 (Validation of Automated Process Equipment and Quality System Software), which directs production and quality system software to leverage the rest of the guidance, while adding some particulars regarding off-the-shelf (OTS) software.  The CSA Draft Guidance (once finalized) will replace Section 6 of the GPSV only.  The remainder of the GPSV guidance will remain in force.

One clear break from the GPSV guidance is how the CSA Draft Guidance focuses its language on software features and functions, where the GPSV guidance (and most other regulatory literature in this space) speaks in terms of entire systems.  The new approach allows for more targeted assurance activities versus validating an entire system, when only a subset of its functionality may be subject to validation regulatory requirements.

Notably, the CSA Draft Guidance does not refer to AAMI/ISO TIR80002-2:2017, which the FDA co-authored and which provides an oft-used toolkit for software validation process and records.

How the CSA Draft Guidance Breaks Down

1. Categorizing the Level of Risk

To facilitate the new risk-based approach, the CSA Draft Guidance makes a distinction between software “used directly as part of production or the quality system” and software “used to support production or the quality system” noting that both categories are subject to the requirement for software validation in 21 CFR § 820.70(i).²CSA Draft Guidance at 7.  The latter category includes development and test tools, as applied to production or quality system software, and general record-keeping not associated with quality records.  Supporting infrastructure not specific to production or the quality system (including networking) is clarified as being outside the scope of § 820.70(i) and not subject to validation requirements.

Using the “direct” and “support” distinctions, the CSA Draft Guidance encourages firms to complete a risk analysis focused on the risk of the software failing to perform as intended (distinct from the safety risk analysis performed by medical devices using ISO 14971, which focuses on the risk of the device causing harm).  Although a manufacturer may apply many levels of risk based on this analysis, FDA will only see two tiers: (1) “high process risk” and (2) “not high process risk.”³Id. at 10.

Process risk is high “when its failure to perform as intended may result in a quality problem that foreseeably compromises safety, meaning an increased medical device risk” (e.g., maintaining process parameters essential to device safety or quality, determining product acceptability with little or no additional human awareness or review, production of device instructions for use).⁴Id.

Process risk is not high “when its failure to perform as intended would not result in a quality problem that foreseeably compromises safety.”⁵Id.  This includes situations “where failure to perform as intended would not result in a quality problem, as well as situations where failure to perform as intended may result in a quality problem that does not foreseeably lead to compromised safety” (e.g., QMS software such as CAPA, complaint management, change control management, procedure management; collects and records process data with no direct impact on production or process performance).⁶Id.  Software functionality that “supports production or the quality system” falls into the “not high risk” category.

Key Takeaway: “FDA is primarily concerned with the review and assurance for those software features, functions, and operations that are high process risk because a failure also poses a medical device risk.”⁷Id. at 11.

2. Determining the Software Assurance Activities and Required Documentation

Once the risk level of a software feature or function is determined, the CSA Draft Guidance encourages manufacturers to tailor assurance activities commensurate with the two categories of risk defined in the CSA Draft Guidance, as follows.

• High process risk
  
  
  • Identify activities commensurate with medical device
    
    
  • Require more documentation.
    
    
  • Think in terms of assurance activities that are scripted (e.g., documented test cases with detailed steps, expected results, actual results, and detailed setup).
    
    
  • Note that this section relegates high process risk to those that may pose severe harm, which may seem inconsistent with how the term is defined elsewhere in the draft guidance.
    
    
• Not high process risk
  
  
  • Identify activities commensurate with process
    
    
  • Require less documentation.
    
    
  • Think in terms of assurance activities that may not be scripted (e.g., ad-hoc testing, error-guessing, exploratory testing)
    
    
  • Note that this section allows low process risk to include harms that are not severe, which may seem inconsistent with how the term is defined elsewhere in the draft guidance.

When determining what documentation may be appropriate for the chosen assurance activities based on the level of risk, the CSA Draft Guidance establishes a set of general content requirements such as:

• the intended use of the software feature, function, or operation;
  
  
• the determination of risk of the software feature, function, or operation; and
  
  
• documentation of the assurance activities conducted, including:
  
  
  • description of the testing conducted based on the assurance activity;
    
    
  • issues found (e.g., deviations, failures) and the disposition;
    
    
  • conclusion statement declaring acceptability of the results;
    
    
  • the date of testing/assessment and the name of the person who conducted the testing/assessment; and
    
    
  • established review and approval when appropriate (e.g., when necessary, a signature and date of an individual with signatory authority).

Depending on the assurance activity, a varying degree of other record content may be necessary, and the CSA Draft Guidance provides examples for each of the five defined scripted and unscripted assurance methods.

The CSA Draft Guidance concludes with one example of record content and three general CSA examples.

The Bottom Line

The risk-based approach defined in the CSA Draft Guidance is both more flexible and less clear than the GPSV in terms of process requirements and regulatory expectations related to validating software that is used to automate device production or in the quality system.  Manufacturers must be prepared to use and articulate critical thinking to establish and support determined levels of risk for a given software program, to determine and implement appropriate assurance activities, and to document associated records.

Although the level of staffing and process burden that device firms have applied to non-device software validation varies widely, the percentage of FDA inspection observations associated with this quality system requirement relative to the full set of inspection observations has remained in the low single-digits for years.  This guidance, once finalized, may provide FDA with a tool that brings more attention to software validation / assurance and can lead to increased attention during FDA inspections.

As manufacturers begin implementing the principles laid out in the CSA Draft Guidance and then support and explain their chosen approach to FDA, the definition of “good” will eventually stabilize.  In the meantime, both industry and regulators can anticipate a period of challenges and uncertainty.