README revision 146822
1$FreeBSD: head/tools/regression/usr.bin/make/README 146822 2005-05-31 14:13:07Z harti $ 2 3This directory contains regression tests for make(1). 4 5To invoke the tests install prove(1) from ports/devel/p5-Test-Harness and 6run 'prove -r'. Alternatively one can use 'sh ./all.sh test' and scan the 7output for '^not ok'. 8 9---------------------------------------------------------------------------- 10 11The rest of this file is intented for developers. 12 13The tests are invoked via the test.sh script or prove(1) from p5-Test-Harness. 14Tests are normally executed in a special test directory that is built under 15/tmp. The reason for this is, that make tests are generally influenced by 16all file in a directory, by files in one of make's obscure object directories 17as well as in other directories make happens to look into. Therefor the 18test scripts build a clean environment in the temp directory and the 19tests are executed by cd-ing into that directory and invoking make. The 20output of the make run (standard output, standard error and the exit status) 21are written into files that are created in another directory. So the layout 22for the shell/builtin test looks like: 23 24 ./shell/builtin/ - directory with test stuff 25 /tmp/make.${USER}/shell/builtin - actual test directory 26 /tmp/make.${USER}/shell/builtin.OUTPUT - output files 27 28So a full test consists of the following steps: 29 30 setup - Set up the test environment by creating the test directory 31 and populating it with the needed files. If the test 32 directory already exists an error is printed. 33 34 run - Run the test and produce the output into the output 35 directory. 36 37 show - Show the result files on the screen. 38 39 compare - Compare the results in the output directory with those 40 in the test source directory. This just prints whether 41 the test was ok or not in the format used by prove(1). 42 43 diff - Diff the output files and the expected output files. 44 45 reset - Reset the test to its initial state. 46 47 clean - Remove both the test directory and the output directory. 48 49Each of these steps can independently be invoked with the test script 50contained in each directory. These test scripts are called test.t. 51Additionally the scripts understand the following commands: 52 53 test - Run setup, run and compare. 54 55 prove - Run setup, run, compare and clean. This is identically 56 to invoking the script without an argument. 57 58 desc - Print a short test description. 59 60 update - Update the expected results from the actual results. 61 62The test script has the following syntax: 63 64 % test.t [-v] [-m path_to_make_binary] command 65 66To invoke it via prove(1) use: 67 68 % [MAKE_PROG=path_to_make_binary] prove [options] [files/directories] 69 70Example: 71 % sh test.t -m `pwd`/../obj/make run 72 % MAKE_PROG=/usr/obj/usr/src/usr.bin/make/make prove -r 73 74The test scripts use the following environment variables that can be set 75by the user in the test script's environment: 76 77 WORK_BASE 78 - Base directory for working files. If not set 79 /tmp/make.${USER} is used. 80 81 MAKE_PROG 82 - Path to the make program to test. If not set 83 /usr/bin/make is used. 84 85The following variables are available to test scripts: 86 87 SRC_BASE 88 - test source base directory. This is determined by 89 repeatedly doing cd .. and checking for common.sh. 90 Therefor this can fail if a test source directory is 91 actually a symbolic link and is physically not located 92 below the directory containing common.sh. 93 94 SUBDIR 95 - subdirectory below WORK_BASE and SRC_BASE for current test 96 97 WORK_DIR 98 - ${WORK_BASE}/${SUBDIR} 99 100 SRC_DIR 101 - ${SRC_BASE}/${SUBDIR} 102 103The following variables and functions may be defined by the test script. 104This also lists special filenames. 105 106 DESC 107 A one-line description of the test. 108 109 TEST_MAKE_DIRS 110 A list of pairs of directory names and modes. These 111 directories are created during setup and reset. When 112 the directory already exists (during reset) only the 113 mode change is applied. 114 115 TEST_MAKE_DIRS="subdir 775 subdir/sub 555" 116 117 TEST_COPY_FILES 118 A list of pairs of file names and modes. These files 119 are copied from the source to the working directory 120 during setup and reset. When the file already exists 121 (during reset) only the mode change is applied. Files 122 may be copied from/to sub-directories. The sub-directory 123 in the working directory must already exists (see 124 TEST_MAKE_DIRS). 125 126 TEST_COPY_FILES="libtest.a 444 subdir/libfoo.a 444" 127 128 TEST_TOUCH 129 List of pairs of file names and arguments to touch(1). 130 During setup and reset for each list element touch(1) 131 is executed. 132 133 TEST_TOUCH="file1 '-t 200501011257'" 134 135 TEST_LINK 136 List of pairs of filenames. Each pair is passed to ln(1). 137 All names are prefixed with the working directory. 138 139 Makefile 140 If a file with this name exists in the source directory 141 it is automatically copied to the working directory. 142 143 setup_test() 144 If this function exists it is executed at the end of the 145 setup. 146 147 reset_test() 148 If this function exists it is executed at the end of the 149 reset. 150 151 TEST_CLEAN_FILES 152 A list of file to be deleted when resetting. 153 154 TEST_N 155 Number of tests in this script. If not set this is assumed 156 to be 1. 157 158 TEST_<number> 159 Arguments to make for test number <number>. If not set 160 the default argument of test<number> is used. To run a test 161 without argument to make, set TEST_<number> to the empty string. 162 163 TEST_<number>_SKIP 164 To skip a test (for whatever reason) this should be set 165 to a string explaining the reason for skipping the test. 166 167 run_test() 168 Function to run a test. This function gets a single argument 169 which is the number of the test to executed. The default 170 function evaluates the variable TEST_<number> and calls 171 make with the arguments in this variable. 172 173