FDA: Software Assurance guidance for Production and Quality System Software

This draft guidance is intended to:

• Computer Software Assurance is a risk-based approach to establish confidence in the automation used for production or quality systems, that software is fit for its intended use.
• CSA focus on preventing the introduction of defects into the software development process, identify & fit suitable additional rigor where it is required.
• Describe various methods and testing activities that may be applied to establish CSA and provide objective evidence to fulfill regulatory requirements (i.e Computer Software Validation requirements in 21CFR part 820).

Scope:

• This draft guidance to provide recommendations on Computer Software Assurance for computers and automated data processing systems used as part of medical device production or the quality system.
• This guidance provides additionally discusses specific risk considerations, acceptable testing methods, and efficient generation of objective evidence for production or quality system software.

Steps in CSA Approach:

The CSA risk framework focuses on four following steps:

• Identifying the intended use (use of the software feature or function)
• Determining the risk-based approach
• Determining the appropriate assurance activities
• Establishing the appropriate record

Identifying the Intended Use:

• 21 CFR 820.70(i) requires any software that is used as part of the production or quality system needs to be validated for its intended use
• In general, software can be categorized as:
  1. Direct Impact: Software being used for a production or quality system process. Ex: Software intended for automating production/quality processes, inspection, testing, or the Collection and processing of production/quality system data or maintaining a quality record under the regulation.
  2. Indirect Impact: software that supports a production or quality system process. Ex: Software intended for use as development tools that test or monitor software systems or that automate testing activities for the software used as part of production or the quality system, such as those used for developing and running scripts. And used for automating general record i.e not part of the quality record.
• Any category of Software may have one or more intended uses depending on the individual features, functions, and operations of that software. However, it is essential to thoroughly understand the features, functions, and operations of a software when determining the intended use.
• Based on complexity of software, and operations have different roles, they may produce different risks to product quality & Patient safety.
• FDA recommends that manufacturers examine the intended uses of the Individual features, functions, and operations to facilitate development of a risk-based assurance strategy with different assurance activities for individual features, functions, or operations. Ex: A commercial off-the-shelf (COTS) software is used to document time and temperature data for a curing process. Manufacturer may not need to perform additional assurance Activities beyond. These below mentioned are Sufficient to establish that the software is fit for its intended use.
• What the COTS software developer has already conducted in terms of validation
• Conducting a vendor assessment
• Software installation and configuration

Determining the Risk-based Approach:

• After determines that software intended use as part of production or the quality system, FDA recommends using a risk-based analysis to determine appropriate assurance activities.
• The CSA’s risk-based approach suggests classifying software as either “high process risk” and “not high process risk”.
  • High process risk:
    • When software does not perform as intended, it may result in a quality problem that may foreseeably compromise safety. meaning an increased medical device risk. Ex:Analyze acceptability of product without manual intervention.A software that can monitor and maintain temperature, pressure, or humidity in a sterile room and failure of which can impact product quality drastically.
  • Not high process risk:
    • Where software doesn’t perform as intended, would not result in a quality problem that foreseeably compromises safety. As well as situations where failure to perform as intended may result in a quality problem that may not foreseeably to compromise safety. Ex:
      • A software that is used as part the quality system for CAPA routing, automated logging/ tracking of complaints, automated change control management / automated procedure management.
      • Collect and record data from the process for monitoring and review purposes that do not have a direct impact on production or process performance and software used to support production or the quality system.
• Manufacturers can also choose to further categorize these software as “moderate”, “intermediate”, or “low” to determine assurance activities; however, FDA will consider them under “Not high process risk” category only.

Determining the Appropriate Assurance Activities:

• For high process risk software, FDA suggests perform scripted testing or limited scripted testing as appropriate.
  1. Scripted testing: In which the tester’s actions are prescribed by written instructions in a test case step by step with evidence. Scripted testing includes both robust and limited scripted testing.
  2. Robust scripted testing: Scripted testing efforts in which the risk of the computer System or automation includes evidence of repeatability, traceability to requirements, and auditability.
  3. Limited scripted testing: A hybrid approach of scripted and unscripted testing that is appropriately scaled according to the risk of the computer system or automation. This approach may apply scripted testing for high-risk features or operations and un scripted testing for low to medium-risk items as part of the same assurance effort.
• For not high-risk software, FDA suggests perform un-scripted testing. Testing methods such as ad-hoc testing, error-guessing, exploratory testing, or a combination of methods that is suitable for the risk of the intended use.
  1. Unscripted testing: Dynamic testing in which the tester’s actions are not prescribed by Written instructions step by step and no need to collect evidence. It includes:
  2. Ad-hoc testing: A concept derived from un scripted practice that focuses primarily on performing testing that does not follow on large amounts of documentation to execute (e.g., test Protocol) or documenting evidence.
  3. Error-guessing: A test design technique in which test cases are derived on the basis of the tester’s knowledge of past failures or general knowledge of failure modes.
  4. Exploratory testing: Experience- based testing in which the tester spontaneously designs and executes tests based on the tester’s existing relevant knowledge, prior exploration of the test item (including results from previous tests), and heuristic “rules of thumb” regarding common software behaviors and types of failure.
  5. Exploratory testing looks for hidden properties, including hidden, un anticipated user behaviors, or accidental use situations that could interfere with other software properties being tested and could pose a risk of software failure (towards patient safety).

Establishing the Appropriate Record:

• Sufficient document evidence required to demonstrate that the software feature, function, or operation was assessed and performs as intended. In general, the record should include the following:
  1. The intended use of the software feature, function, or operation;
  2. Risk-Based Analysis: The determination of risk of the software feature, function, or operation;
  3. Documentation of the assurance activities conducted: Includes description of the testing conducted based on the assurance activity (test steps, test results, tester details);
  4. Issues found (e.g., deviations, failures) and the disposition
  5. Conclusion statement declaring acceptability of the results
  6. The date of testing/assessment and the name of the person who conducted the testing/assessment
  7. Established review and approval when appropriate (e.g., when necessary, a signature and date of an individual with signatory authority)
• Documentation of assurance activities need not include more evidence than necessary to show that the software feature, function, or operation performs as intended for the risk identified. And FDA encourages the industry to use system- generated objective evidence such as system logs, audit trails rather than the traditional paper-based test scripts and screenshot evidence.