Have you ever wondered if you tested your product enough? Either in enough ways or for a long enough time?  This assumes that you’d prefer to discover all the problems before your customers do. If you follow a 61508 compliant development process, you should be able to answer that question with a comfortable “yes”.  A good development process will lay out the activities needed to achieve the desired systematic capability and identify the three major test phases of validation, integration and unit testing.  A good development process will have verification steps for review of requirements, design, and test plans.  A good development process will provide traceability between requirements and test plans.  Unfortunately, testing is not 100% effective; don’t fool yourself into believing it is.  There are only so many variables you can control during the test so you want to carefully identify the ones that are critical for safety functions. 

Remember that if you can’t precisely define the product behaviors in the requirements, testing will be more of a hit-or-miss exercise.  (I wish I had a dollar for every time I said “It all starts with requirements”.)  But let’s assume you have a good set of reviewed and approved requirements.  Now you can establish the basic validation test plan while the rest of product development continues.  If any confusion or discrepancies are uncovered in the validation test plan, go back to review the requirements to be sure they are specific and clear then adjust as needed.  After the architecture design is reviewed and approved, the integration test plan can begin.  This test phase focuses on all the interfaces of major functionality.  These may be product features that a customer interacts with (like configuration), but most often these are internal features between hardware of software functions (like communications between multiple processors). 

If the development process is mature, almost no software code has been written by this point.  Many software experts cite test-driven-development as a best practice.  (you can find more info at this link)  By doing this, the unit test plan is developed in coordination with the implementation.  Alternatively, you could use pair-programming techniques, where 2 people have shared responsibility for implementation and testing.  (you can find more info at this link)  Even the old style single programmer/tester is workable, but at least one other person should be involved in the testing… two heads are better than one.  Many test tools are available today for unit testing, and most provide test coverage metrics to show where the testing may fall short of 100% coverage.  These areas can be augmented with manual test cases.  All these techniques can help to make unit testing more effective.

Some testing (like EMC test) is based on specific standards and the pass/fail criteria are pretty objective.  Other testing (like integration test) takes more work to develop a good test plan.  Regardless of the test phase, the key is to clearly define the objectives and pass/fail criteria so you can create the proper test plan.  The test procedures (test equipment, test commands, test tools) can be established later when the design has been determined.

To help ensure that testing will be effective at finding problems, follow these steps for all test phases:

• conduct test plan reviews;
• conduct test results reviews;
• use equivalence class and boundary value techniques to reduce the test cases;
• provide some test overlap so that complex features can be tested in different ways at different test stages;
• test at different levels and at different times.

Don’t let your customer be the one to say you haven’t tested your product enough!