Free Combinatorial Testing Tool
Wednesday, 17 November 2010

Testing just six parameter combinations is as good as exhaustive testing - NIST has a (free) tool for that.

Banner

 

National Institute of Standards and Technology (NIST) has released the latest version of its free testing tool, complete with a new tutorial on how to use it.

The tool is a fairly specialised combinatorial testing rig called Advanced Combinatorial Testing System (ACTS). It can process tests for 2-way through 6-way interactions.  However, based on studies of crashes, it is an empirical finding that most failures (70 to 95%) are triggered by only two-variables interacting and close to 100% are triggered by no more than six. Thus the NIST concludes that combinatorial testing with 6-way interactions is as good in practice as exhaustive testing. The program also claims to implement testing efficiently by grouping tests into covering sets. 

 

 

It is important to realise that the software provided only computes the test schedules - you have to actually implement the tests. For example, if you have a code library which works on n mobile phones each using m browsers with p language engines the program will tell you which combinations need testing - you then have to create the configurations and perform the test. This means that the ACTS program can be used in a wide variety of testing situations and be built into your own larger testing framework.

The new tutorial, Practical Combinatorial Testing, introduces key concepts and methods along with explaining the use of software tools for generating combinatorial tests.  The tutorial is designed to be accessible to undergraduate students in computer science or engineering it is available from http://csrc.nist.gov/groups/SNS/acts/documents/SP800-142-101006.pdf

More information and download from:http://csrc.nist.gov/groups/SNS/acts

Banner


Jakarta EE 11 Modernises Test Kits
26/06/2025

The Eclipse Foundation has announced the general availability of the Jakarta EE 11 Platform. This release has modernised Test Compatibility Kits (TCKs), and introduces the Jakarta Data speci [ ... ]



Fei-Fei Li On Spatial Intelligence As The Next Frontier In AI
11/07/2025

Last month in front of an enthusiastic audience at Y Combinator, Fei-Fei Li, often called the godmother of AI, talked to Diana Hu about spatial intelligence and why she considers it the next crit [ ... ]


More News

Last Updated ( Wednesday, 17 November 2010 )