When building a product such as an anti-lock braking system for an automobile, or a railroad, or process control safety system, making sure that the product works as specified is a big part of functional safety. When it is time for the system to engage to prevent an accident, you need to know that the system is going to work almost all of the time. One area that is often overlooked in the effort to ensure that the system works properly is the effects of software tools used in the development process. Examples of such tools include:

  • Compilers
  • Static checkers
  • Verification tools
  • Test automation tools
Since these tools are usually purchased from a third party, they are normally treated as something beyond our control and therefore not considered in the safety analysis. However doing this would surely be a mistake as these tools can be very complex themselves and subject to the same errors that we can prevent, find, and fix in our own code. By eliminating analysis of these tools, you are essentially assuming that these tools are infallible. This is obviously not true, but the fact that you made this assumption is not always that obvious.

So if we can agree that these tools must be analyzed, what can be done? At exida, we have been facing this issue for many years and have come up with a standard methodology that has worked well. Additionally the methodology is compliant with multiple safety standards including IEC 61508 (General), ISO 26262 (Automotive), and EN 50128 (Rail) and to some extent IEC 60880 (Nuclear). The methodology includes the following six steps:

Step 1: Description of the assumed workflow(s), tool functions, and properties used in the workflow

  • Define exactly how to tool will be used including its relationship to other related tools.
Step 2: HAZOP based tool confidence level determination technique (criticality per function)
  • Determine the criticality of each tool function. This analysis is based on the effect a tool failure could have on the final product. Steps (3) through (6) only need to be implemented for the tools that have been classified as critical tools that can affect the functional safety of the product.
Step 3: Analysis of tool qualification requirements and related tailoring of software safety lifecycle requirements for the tool
  • Tool qualification is the step that shows the tool conforms to its specification or user manual. Some of the requirements for how to do this require some analysis and tailoring of other standards. For example, ISO 26262 states that one technique that can be used to show tool qualification is conformance to a safety standard. However, ISO 26262 readily admits that not all of the requirements of most safety standards will apply to tools, so these standards must be analyzed and tailored to the requirements that are applicable.
Step 4: Assessment of their implementation
  • This analyzes how well the tool development conformed to the tailored requirements. As we are focusing on the suitability and safety of the tools for use in functional safety projects, this “standard approach” is extended by (5) and (6).
Step 5: Tool failure analysis in the specified tool safety application environment and workflow(s)
  • This is analysis of potential failure modes of the tool along with a definition of mitigating measures that make it likely that such failures will be discovered so that they can be corrected.
Step 6: Execution of additional safety validation tests (specifically fault insertions derived from Step 5)

Five of the six steps defined in this methodology can be done by the user of the tool. However, Step 4 requires that the tool manufacturer participate. This is not always easy, but it is a necessary step for critical tools. There are some alternatives for vendor participation, such as confidence from use that can be used in some cases. Confidence from use can serve as validation of tools that have been successfully used on many previous projects. However, this approach is not applicable to tools that have not been used before, or for the higher Automotive Safety Integrity Levels (ASIL). In these cases the tool development process must be analyzed.

Tagged as:     software safety lifecycle     software     michael medoff     iso 26262     IEC 61508     iec 60880     hazop     en 50128     automotive safety integrity levels     asil  

Other Blog Posts By Michael Medoff