1\input texinfo @c -*- texinfo -*- 2 3@settitle FATE Automated Testing Environment 4@titlepage 5@center @titlefont{FATE Automated Testing Environment} 6@end titlepage 7 8@top 9 10@contents 11 12@chapter Introduction 13 14FATE provides a regression testsuite embedded within the Libav build system. 15It can be run locally and optionally configured to send reports to a web 16aggregator and viewer @url{http://fate.libav.org}. 17 18It is advised to run FATE before submitting patches to the current codebase 19and provide new tests when submitting patches to add additional features. 20 21@chapter Running FATE 22 23@section Samples and References 24In order to run, FATE needs a large amount of data (samples and references) 25that is provided separately from the actual source distribution. 26 27To inform the build system about the testsuite location, pass 28@option{--samples=<path to the samples>} to @command{configure} or set the 29@var{SAMPLES} Make variable or the @var{FATE_SAMPLES} environment variable 30to a suitable value. 31 32The dataset is available through @command{rsync}, is possible to fetch 33the current sample using the straight rsync command or through a specific 34@ref{Makefile target}. 35 36@example 37# rsync -aL rsync://fate-suite.libav.org/fate-suite/ fate-suite 38@end example 39 40@example 41# make fate-rsync SAMPLES=fate-suite 42@end example 43 44 45@chapter Manual Run 46FATE regression test can be run through @command{make}. 47Specific Makefile targets and Makefile variables are available: 48 49@anchor{Makefile target} 50@section FATE Makefile targets 51@table @option 52@item fate-list 53List all fate/regression test targets. 54@item fate-rsync 55Shortcut to download the fate test samples to the specified testsuite location. 56@item fate 57Run the FATE test suite (requires the fate-suite dataset). 58@end table 59 60@section Fate Makefile variables 61@table @option 62@item V 63Verbosity level, can be set to 0, 1 or 2. 64@table @option 65 @item 0 66 show just the test arguments 67 @item 1 68 show just the command used in the test 69 @item 2 70 show everything 71@end table 72@item SAMPLES 73Specify or override the path to the FATE samples at make time, it has a 74meaning only while running the regression tests. 75@item THREADS 76Specify how many threads to use while running regression tests, it is 77quite useful to detect thread-related regressions. 78@end table 79 80@example 81 make V=1 SAMPLES=/var/fate/samples THREADS=2 fate 82@end example 83 84@chapter Automated Tests 85In order to automatically testing specific configurations, e.g. multiple 86compilers, @command{tests/fate.sh} is provided. 87 88This shell script builds Libav, runs the regression tests and prepares a 89report that can be sent to @url{fate.libav.org} or directly examined locally. 90 91@section Testing Profiles 92The configuration file passed to @command{fate.sh} is shell scripts as well. 93 94It must provide at least a @var{slot} identifier, the @var{repo} from 95which fetch the sources, the @var{samples} directory, a @var{workdir} with 96enough space to build and run all the tests. 97Optional submit command @var{fate_recv} and a @var{comment} to describe 98the testing profile are available. 99 100Additional optional parameter to tune the Libav building and reporting process 101can be passed. 102 103@example 104slot= # some unique identifier 105repo=git://git.libav.org/libav.git # the source repository 106samples=/path/to/fate/samples 107workdir= # directory in which to do all the work 108fate_recv="ssh -T fate@@fate.libav.org" # command to submit report 109comment= # optional description 110 111# the following are optional and map to configure options 112arch= 113cpu= 114cross_prefix= 115cc= 116target_os= 117sysroot= 118target_exec= 119target_path= 120extra_cflags= 121extra_ldflags= 122extra_libs= 123extra_conf= # extra configure options not covered above 124 125#make= # name of GNU make if not 'make' 126makeopts= # extra options passed to 'make' 127#tar= # command to create a tar archive from its arguments on 128 # stdout, defaults to 'tar c' 129@end example 130 131@section Submitting Reports 132In order to send reports you need to create an @command{ssh} key and send it 133to @email{root@@libav.org}. 134The current server fingerprint is @var{a4:99:d7:d3:1c:92:0d:56:d6:d5:61:be:01:ae:7d:e6} 135 136