Summary
On September 2022, the U.S. FDA released a draft guidance to provide a proposed framework on computer software assurance for computers and automated data processing systems used as part of a medical device’s production or quality system [1]. This draft guidance focuses primarily on FDA’s thinking regarding the following:
- Computer software assurance as a risk-based approach to establish confidence in the automation used for production or quality systems, and identify where additional rigor may be appropriate
- Various methods and testing activities that may be applied to establish computer software assurance and provide objective evidence to fulfill regulatory requirements.
The draft guidance outlines the following steps, detailed below, for applying the proposed framework:
- Identifying the Intended Use
- Determining the Risk-Based Approach
- Determining the Appropriate Assurance Activities
- Establishing the Appropriate Record
Identifying the Intended Use
Manufacturers should first determine whether a software is intended for use as part of production or the quality system. Software used as part of production or the quality system generally falls into one of two categories:
- Software that is used directly as part of production or the quality system. This includes, for example, software for automation of production or quality system process, inspection, testing, or collection of production/quality data.
- Software that supports production or the quality system. This can include scripts to test or monitor software systems or to automate a record-keeping process that is not directly tied to a quality record.
In cases where software is not considered to be used as part of production or the quality system, the software would not fall within the scope of the guidance. For example, email applications, accounting applications, or network programs would generally not be considered as part of the production or quality system.
Ultimately, it is up to the manufacturer to define the category of their software. FDA expects this decision-making process and rationale to be documented within applicable SOPs.
Determining the Risk-Based Approach
After identifying the intended use of the software and whether it falls within the scope of the guidance, manufacturers should apply a risk-based approach to determine appropriate assurance activities. This approach should be specific to the software and should consider reasonably foreseeable failures that may impact the software’s performance or prevent it from performing as intended. For example, improper configuration of the software, compromised security, or errors in data storage. The potential risks resulting from these software failures should be assessed. The assessment could be similar to what is described in ISO 14971 but that may more burdensome than necessary. A simpler risk assessment methodology is encouraged.
The guidance makes a distinction between risks in a binary manner, defined below:
- High process risk: when the software’s failure to perform as intended may result in a quality problem that foreseeably compromises safety, meaning an increased medical device risk.
- Not high process risk: when a software’s failure to perform as intended would not result in a quality problem that foreseeably compromises safety.
The FDA is primarily concerned with assurance activities for software that qualify as a high process risk because failure of their features, functions, and/or operations are more likely to pose a medical device risk. Some examples of high process risks provided by the guidance include:
- Software that maintains process parameters that affect the physical properties of a device;
- Software meant to measure, inspect, analyze, or determine acceptability of a product or process with limited or no human review;
- Software used to produce labeling (such as instructions for use) that is provided to patients and that is necessary for safe operation of the device.
Determining the Appropriate Assurance Activities
Whether the risk is considered high or low, FDA expects manufacturers to put in place assurance activities that are commensurate with the risk level. The higher the risk, the more rigorous the assurance activities. The guidance provides the following examples of assurance activities which can be applied according to the risk level determined by the manufacturer:
- Unscripted testing: dynamic testing in which the tester’s actions are not prescribed by written instructions in a test case. This includes ad-hoc testing, error-guessing, and exploratory testing.
- Scripted testing: dynamic testing in which the tester’s actions are prescribed by written instructions in a test case. This includes both robust and limited scripted testing.
It is the responsibility of the manufacturer to determine appropriate assurance activities and their level of rigor for their software. The guidance also suggests that manufacturers consider and leverage activities that might already be in place in order to further support the level of assurance activities put in place for their software.
Establishing the Appropriate Record
Finally, manufacturers should expect to document and record evidence of their assurance activities. Records should capture sufficient evidence to demonstrate that the production or quality system software features, functions, and/or operations were assessed and that they performed as intended. The guidance provides examples of assurance activity records which, at a minimum, should include the following information:
- The intended use of the software feature, function, or operation;
- The determination of risk of the software feature, function, or operation;
- Documentation of the assurance activities conducted, including:
- description of the testing conducted based on the assurance activity;
- issues found and the disposition;
- conclusion statement declaring acceptability of the results;
- the date of testing/assessment and the name of the person who conducted these activities;
- established review and approval when appropriate (for example, signature and date of an individual with signatory authority)
An example of a record is provided below. In this example, a manufacturer is recording assurance activities for a hypothetical spreadsheet that the manufacturer has developed to collect and graph nonconformance data for monitoring purposes:
Conclusion
With this draft guidance, the FDA attempts to provide clarity around their expectations for software validation for computers and automated data processing systems used as part of production of the quality system. By following the framework proposed in this guidance, the FDA hopes to promote product quality and patient safety through manufacturing practices that affect the overall quality of a medical device. The guidance is still in draft form, with the comment period ending on November 14, 2022. Once final, it will supplement the FDA Guidance on General Principles of Software Validation [2].
References
