Most engineers who design and verify safety instrumented functions (SIFs) understand how hard it is to design a manual proof test with high effectiveness (also called high proof test coverage). Those folks who understand that a proof test is not likely to detect all failures never use simplified equations that assume perfect proof testing like those from IEC 61508 Part 6.  Those equations are there for information only.

Given that proof testing is not 100%, what percentage is detected during a proof test?  That is a number needed for realistic PFDavg verification calculations. At exida we use the FMEDA technique that we developed to determine failure rates, failure modes, and automatic diagnostic coverage. That process requires that we examine every part in a product and for each part answer the following questions:

• How does this component failure impact the product?
• Is this failure detected by automatic diagnostics?
• If not, is this failure detected by the manual proof test?

This procedure is clearly detailed, systematic, and accurate, however beyond the scope of anyone not intimately familiar with a product. That is why exida publishes proof test coverage predictions for one or more example proof tests in FMEDA reports. 

Proof test coverage is a measure of how many undetected dangerous failure are detected by the proof test.  Imagine a product with 100 FITS of dangerous failures. Automatic diagnostics are poor and detect only 10 FITS. That means Lambda DD is 10 FITS and Lambda DU is 90 FITS.  Imagine that a manual proof test can be done during operation that can detect 72 of these 90 FITS. The proof test coverage is 72/90 = 80%.  There are 18 FITS of DU never detected!

What about a similar product with good automatic diagnostics that can detect 90 of the 100 FITS?  In this case the Lambda DD is 90 FITS and the Lambda DU is 10 FITS.  Imagine the same proof test is used, but the automatic diagnostics have already detected 70 of the 72 FITS. So the proof test now detects 2 of the 10 FITS. Proof test coverage is 20%. That does not sound so impressive, but the bottom line is that in the first case the automatic diagnostics, combined with the proof test detect all but 18 FITS.  In the second case automatic diagnostics combined with proof test detect all but 8 FITS. This is a much better situation.

So never use proof test coverage as a measure of quality for a product.  Almost the opposite is true.