Difference between revisions of "Testing"

From Net-SNMP Wiki
Jump to: navigation, search
(Available Test Groups)
(Available Test Groups)
Line 71: Line 71:
 
|-
 
|-
 
|}
 
|}
 +
 +
== Architecture of the Testing System ==
 +
 +
The Net-SNMP testing infrastructure is divided into parts:
 +
 +
* The RUNFULLTESTS program, which is a harness for knowing how to execute individual tests and groups of tests.
 +
* Directories of tests that form ''groups''
 +
* Individual tests that are written in the form of a particular ''test style''
 +
* Supporting files that know how to build and run particular ''test style''s.
 +
 +
=== RUNFULLTESTS ===
 +
 +
Internally to RUNFULLTESTS it:
 +
 +
# examines the list of arguments for which test groups should be run
 +
# optionally filters that list further because of regexps passed on the command line via the -r switch
 +
# for each test:
 +
## determines the ''test style'' of the test
 +
## looks for a supporting ''_build'' rule and executes it (if found) with the name of the test file.  It also records the last line of the output as the new test file to pass to the _run routine.
 +
## looks for a supporting ''_run'' rule and executes it with the name of the test to run and collects it output as TAP output
 +
## If no ''_run'' system was found, it executes the file itself and collects the output as TAP output (ie, it must a stand alone script)
 +
# Displays a summary of the results from all the tests

Revision as of 17:11, 8 December 2010

Net-SNMP comes with a testing suite that is simple to use, and fairly simple to write new tests for. The instructions below describe both how to the suite and how to write new tests for it.

Using the Test System

The simplest way to run the tests is to simply:

# cd testing
# make test

And watch the output.

Notes for version 5.5 and below

make test really just invokes the "RUNTESTS" script. See the RUNTESTS -h output for help

Everything below this point applies primarily to version 5.6 and above. Although the "simple" tests described below still work and thus it's still possible to write new tests using that format for version prior to 5.6.

Notes for version 5.6 and above

In version 5.6 the testing suite was rewritten to provide:

make test is still the simple way to run the default set of tests. This should work on every platform, although platforms with newer versions of perl will get increased levels of support. Platforms without perl at all can't take advantage of the TAP collection process and the output will thus be very verbose.

make test will still run the default set of tests for you, now using the RUNFULLTESTS script. However, there are more tests available that can be run in addition to the complete set. The test groups are located in individual directories in the testing/fulltests directory. The one exception is the support directory which merely contains supporting files for the other directories.

Running a particular test

To run a particular group use:

 # cd testing
 # ./RUNFULLTESTS -g groupname

Where groupname is the directory name you wish to execute the tests from. For example:

 # ./RUNFULLTESTS -g unit-tests

will run the unit-tests for the libraries.

Available Test Groups

Requires Net-SNMP 5.6

Note: Features discussed on this page require Net-SNMP version 5.6 or higher.

The following list describes the test groups available in the latest version of Net-SNMP. These will only be available from 5.6 onward, and as Net-SNMP development continues some may only be available in the more recent versions:

Test Group Description
all A special keyword to run every available test group
default The default list of tests to run. This group is expected to work even with older versions of perl. The other groups frequently require Newer versions of perl (5.8+).
unit-tests Runs basic unit tests for the libraries. These tests generally involve compiling a simple c-application that execercises various aspects of the Net-SNMP C Libraries.
transports Tests each available transport type for its ability to send and receive data. Simple SNMP messages are sent from a client to an agent and back using each available transport type. Will likely require a properly setup network (eg, a working IPv6 interface if you compiled in IPv6 support).
tls Extensive tests of the TLS and DTLS transports to ensure they're functioning and all the features are working
perl Simple tests of the perl module and it's ability to send and receive data

Architecture of the Testing System

The Net-SNMP testing infrastructure is divided into parts:

  • The RUNFULLTESTS program, which is a harness for knowing how to execute individual tests and groups of tests.
  • Directories of tests that form groups
  • Individual tests that are written in the form of a particular test style
  • Supporting files that know how to build and run particular test styles.

RUNFULLTESTS

Internally to RUNFULLTESTS it:

  1. examines the list of arguments for which test groups should be run
  2. optionally filters that list further because of regexps passed on the command line via the -r switch
  3. for each test:
    1. determines the test style of the test
    2. looks for a supporting _build rule and executes it (if found) with the name of the test file. It also records the last line of the output as the new test file to pass to the _run routine.
    3. looks for a supporting _run rule and executes it with the name of the test to run and collects it output as TAP output
    4. If no _run system was found, it executes the file itself and collects the output as TAP output (ie, it must a stand alone script)
  4. Displays a summary of the results from all the tests