198752Sjmallett# $FreeBSD: releng/11.0/usr.bin/tests/regress.m4 263227 2014-03-16 08:04:06Z jmmv $ 298752Sjmallett 398755Sjmallettdnl A library of routines for doing regression tests for userland utilities. 498755Sjmallett 598755Sjmallettdnl Start up. We initialise the exit status to 0 (no failure) and change 698755Sjmallettdnl into the directory specified by our first argument, which is the 798755Sjmallettdnl directory to run the tests inside. 898752Sjmallettdefine(`REGRESSION_START', 998752SjmallettTESTDIR=$1 1098752Sjmallettif [ -z "$TESTDIR" ]; then 1198752Sjmallett TESTDIR=. 1298752Sjmallettfi 1398752Sjmallettcd $TESTDIR 1498752Sjmallett 1598752SjmallettSTATUS=0) 1698752Sjmallett 1798812Sjmallettdnl Check $? to see if we passed or failed. The first parameter is the test 1898812Sjmallettdnl which passed or failed. It may be nil. 1998812Sjmallettdefine(`REGRESSION_PASSFAIL', 2098812Sjmallettif [ $? -eq 0 ]; then 21137587Snik echo "ok - $1 # Test detected no regression. (in $TESTDIR)" 2298812Sjmallettelse 2398812Sjmallett STATUS=$? 24137587Snik echo "not ok - $1 # Test failed: regression detected. See above. (in $TESTDIR)" 2598812Sjmallettfi) 2698812Sjmallett 2798755Sjmallettdnl An actual test. The first parameter is the test name. The second is the 2898755Sjmallettdnl command/commands to execute for the actual test. Their exit status is 2998755Sjmallettdnl checked. It is assumed that the test will output to stdout, and that the 3098755Sjmallettdnl output to be used to check for regression will be in regress.TESTNAME.out. 3198752Sjmallettdefine(`REGRESSION_TEST', 32263227Sjmmv$2 | diff -u ${SRCDIR:-.}/regress.$1.out - 3398812SjmallettREGRESSION_PASSFAIL($1)) 3498752Sjmallett 3598758Sjmallettdnl A freeform regression test. Only exit status is checked. 3698758Sjmallettdefine(`REGRESSION_TEST_FREEFORM', 3798758Sjmallett$2 3898812SjmallettREGRESSION_PASSFAIL($1)) 3998758Sjmallett 4098758Sjmallettdnl A regression test like REGRESSION_TEST, except only regress.out is used 4198758Sjmallettdnl for checking output differences. The first argument is the command, the 4298758Sjmallettdnl second argument (which may be empty) is the test name. 4398758Sjmallettdefine(`REGRESSION_TEST_ONE', 44263227Sjmmv$1 | diff -u ${SRCDIR:-.}/regress.out - 4598812SjmallettREGRESSION_PASSFAIL($2)) 4698758Sjmallett 4798758Sjmallettdnl A fatal error. This will exit with the given status (first argument) and 4898758Sjmallettdnl print the message (second argument) prefixed with the string "FATAL :" to 4998758Sjmallettdnl the error stream. 5098758Sjmallettdefine(`REGRESSION_FATAL', 51137587Snikecho "Bail out! $2 (in $TESTDIR)" > /dev/stderr 5298758Sjmallettexit $1) 5398758Sjmallett 5498755Sjmallettdnl Cleanup. Exit with the status code of the last failure. Should probably 5598755Sjmallettdnl be the number of failed tests, but hey presto, this is what it does. This 5698755Sjmallettdnl could also clean up potential droppings, if some forms of regression tests 5798755Sjmallettdnl end up using mktemp(1) or such. 5898752Sjmallettdefine(`REGRESSION_END', 5998752Sjmallettexit $STATUS) 60