NEWS revision 261897
11541SrgrimesMajor changes between releases Automated Testing Framework 21541Srgrimes=========================================================================== 31541Srgrimes 41541Srgrimes 51541SrgrimesChanges in version 0.20 61541Srgrimes*********************** 71541Srgrimes 81541SrgrimesExperimental version released on February 7th, 2014. 91541Srgrimes 101541SrgrimesThis is the first release without the code for the deprecated tools. If 111541Srgrimesyou require such code, please fetch a copy of the 0.19 release and extract 121541Srgrimesthe 'tools' directory for your own consumption. 131541Srgrimes 141541Srgrimes* Removed the deprecated tools. This includes atf-config, atf-report, 151541Srgrimes atf-run and atf-version. 161541Srgrimes 171541Srgrimes 181541SrgrimesChanges in version 0.19 191541Srgrimes*********************** 201541Srgrimes 211541SrgrimesExperimental version released on February 7th, 2014. 221541Srgrimes 231541SrgrimesThis is the last release to bundle the code for the deprecated tools. 241541SrgrimesThe next release will drop their code and will stop worrying about 251541Srgrimesbackwards compatibility between the ATF libraries and what the old tools 261541Srgrimesmay or may not support. 271541Srgrimes 281541SrgrimesIf you still require the old tools for some reason, grab a copy of the 291541Srgrimes'tools' directory now. The code in this directory is standalone and 301541Srgrimesdoes not depend on any internal details of atf-c++ any longer. 311541Srgrimes 321541Srgrimes* Various fixes and improvements to support running as part of the FreeBSD 331541Srgrimes test suite. 341541Srgrimes 351541Srgrimes* Project hosting moved from Google Code (as a subproject of Kyua) to 361541Srgrimes GitHub (as a first-class project). The main reason for the change is 371541Srgrimes the suppression of binary downloads in Google Code on Jan 15th, 2014. 381541Srgrimes See https://github.com/jmmv/atf/ 391541Srgrimes 401541Srgrimes* Removed builtin help from atf-sh(1) and atf-check(1) for simplicity 411541Srgrimes reasons. In other words, their -h option is gone. 421541Srgrimes 431541Srgrimes* Moved the code of the deprecated tools into a 'tools' directory and 441541Srgrimes completely decoupled their code from the internals of atf-c++. The 451541Srgrimes reason for this is to painlessly allow a third-party to maintain a 461541Srgrimes copy of these tools after we delete them because upcoming changes to 471541Srgrimes atf-c++ would break the stale tools. 481541Srgrimes 491541Srgrimes 501541SrgrimesChanges in version 0.18 511541Srgrimes*********************** 521541Srgrimes 531541SrgrimesExperimental version released on November 16th, 2013. 541541Srgrimes 551541Srgrimes* Issue 45: Added require.memory support in atf-run for FreeBSD. 561541Srgrimes 571541Srgrimes* Fixed an issue with the handling of cin with libc++. 581541Srgrimes 591541Srgrimes* Issue 64: Fixed various mandoc formatting warnings. 601541Srgrimes 611541Srgrimes* NetBSD PR bin/48284: Made atf-check flush its progress message to 621541Srgrimes stdout so that an interrupted test case always shows the last message 631541Srgrimes being executed. 641541Srgrimes 651541Srgrimes* NetBSD PR bin/48285: Fixed atf_check examples in atf-sh-api(3). 661541Srgrimes 671541Srgrimes 681541SrgrimesChanges in version 0.17 691541Srgrimes*********************** 701541Srgrimes 711541SrgrimesExperimental version released on February 14th, 2013. 721541Srgrimes 731541Srgrimes* Added the atf_utils_cat_file, atf_utils_compare_file, 741541Srgrimes atf_utils_copy_file, atf_utils_create_file, atf_utils_file_exists, 751541Srgrimes atf_utils_fork, atf_utils_grep_file, atf_utils_grep_string, 761541Srgrimes atf_utils_readline, atf_utils_redirect and atf_utils_wait utility 771541Srgrimes functions to atf-c-api. Documented the already-public 781541Srgrimes atf_utils_free_charpp function. 791541Srgrimes 801541Srgrimes* Added the cat_file, compare_file, copy_file, create_file, file_exists, 811541Srgrimes fork, grep_collection, grep_file, grep_string, redirect and wait 821541Srgrimes functions to the atf::utils namespace of atf-c++-api. These are 831541Srgrimes wrappers around the same functions added to the atf-c-api library. 841541Srgrimes 851541Srgrimes* Added the ATF_CHECK_MATCH, ATF_CHECK_MATCH_MSG, ATF_REQUIRE_MATCH and 861541Srgrimes ATF_REQUIRE_MATCH_MSG macros to atf-c to simplify the validation of a 871541Srgrimes string against a regular expression. 881541Srgrimes 891541Srgrimes* Miscellaneous fixes for manpage typos and compilation problems with 901541Srgrimes clang. 911541Srgrimes 921541Srgrimes* Added caching of the results of those configure tests that rely on 931541Srgrimes executing a test program. This should help crossbuild systems by 941541Srgrimes providing a mechanism to pre-specify what the results should be. 951541Srgrimes 961541Srgrimes* PR bin/45690: Make atf-report convert any non-printable characters to 971541Srgrimes a plain-text representation (matching their corresponding hexadecimal 981541Srgrimes entities) in XML output files. This is to prevent the output of test 991541Srgrimes cases from breaking xsltproc later. 1001541Srgrimes 1011541Srgrimes 1021541SrgrimesChanges in version 0.16 1031541Srgrimes*********************** 1041541Srgrimes 1051541SrgrimesExperimental version released on July 10th, 2012. 1061541Srgrimes 1071541Srgrimes* Added a --enable-tools flag to configure to request the build of the 1081541Srgrimes deprecated ATF tools, whose build is now disabled by default. In order 1091541Srgrimes to continue running tests, you should migrate to Kyua instead of enabling 1101541Srgrimes the build of the deprecated tools. The kyua-atf-compat package provides 1111541Srgrimes transitional compatibility versions of atf-run and atf-report built on 1121541Srgrimes top of Kyua. 1131541Srgrimes 1141541Srgrimes* Tweaked the ATF_TEST_CASE macro of atf-c++ so that the compiler can 1151541Srgrimes detect defined but unused test cases. 1161541Srgrimes 1171541Srgrimes* PR bin/45859: Fixed some XSLT bugs that resulted in the tc-time and 1181541Srgrimes tp-time XML tags leaking into the generated HTML file. Also improved 1191541Srgrimes the CSS file slightly to correct alignment and color issues with the 1201541Srgrimes timestamps column. 1211541Srgrimes 1221541Srgrimes* Optimized atf-c++/macros.hpp so that GNU G++ consumes less memory during 1231541Srgrimes compilation with GNU G++. 1241541Srgrimes 1251541Srgrimes* Flipped the default to building shared libraries for atf-c and atf-c++, 1261541Srgrimes and started versioning them. As a side-effect, this removes the 1271541Srgrimes --enable-unstable-shared flag from configure that appears to not work any 1281541Srgrimes more (under NetBSD). Additionally, some distributions require the use of 1291541Srgrimes shared libraries for proper dependency tracking (e.g. Fedora), so it is 1301541Srgrimes better if we do the right versioning upstream. 1311541Srgrimes 1321541Srgrimes* Project hosting moved from an adhoc solution (custom web site and 1331541Srgrimes Monotone repository) to Google Code (standard wiki and Git). ATF now 1341541Srgrimes lives in a subcomponent of the Kyua project. 1351541Srgrimes 1361541Srgrimes 1371541SrgrimesChanges in version 0.15 1381541Srgrimes*********************** 1391541Srgrimes 1401541SrgrimesExperimental version released on January 16th, 2012. 1411541Srgrimes 1421541Srgrimes* Respect stdin in atf-check. The previous release silenced stdin for any 1431541Srgrimes processes spawned by atf, not only test programs, which caused breakage 1441541Srgrimes in tests that pipe data through atf-check. 1451541Srgrimes 1461541Srgrimes* Performance improvements to atf-sh. 1471541Srgrimes 1481541Srgrimes* Enabled detection of unused parameters and variables in the code and 1491541Srgrimes fixed all warnings. 1501541Srgrimes 1511541Srgrimes* Changed the behavior of "developer mode". Compiler warnings are now 1521541Srgrimes enabled unconditionally regardless of whether we are in developer mode or 1531541Srgrimes not; developer mode is now only used to perform strict warning checks and 1541541Srgrimes to enable assertions. Additionally, developer mode is now only 1551541Srgrimes automatically enabled when building from the repository, not for formal 1561541Srgrimes releases. 1571541Srgrimes 1581541Srgrimes* Added new Autoconf M4 macros (ATF_ARG_WITH, ATF_CHECK_C and 1591541Srgrimes ATF_CHECK_CXX) to provide a consistent way of defining a --with-arg flag 1601541Srgrimes in configure scripts and detecting the presence of any of the ATF 1611541Srgrimes bindings. Note that ATF_CHECK_SH was already introduced in 0.14, but it 1621541Srgrimes has now been modified to also honor --with-atf if instantiated. 1631541Srgrimes 1641541Srgrimes* Added timing support to atf-run / atf-report. 1651541Srgrimes 1661541Srgrimes* Added support for a 'require.memory' property, to specify the minimum 1671541Srgrimes amount of physical memory needed by the test case to yield valid results. 1681541Srgrimes 1691541Srgrimes* PR bin/45690: Force an ISO-8859-1 encoding in the XML files generated by 1701541Srgrimes atf-report so that invalid data in the output of test cases does not 1711541Srgrimes mangle our report. 1721541Srgrimes 1731541Srgrimes 1741541SrgrimesChanges in version 0.14 1751541Srgrimes*********************** 1761541Srgrimes 1771541SrgrimesExperimental version released on June 14th, 2011. 1781541Srgrimes 1791541Srgrimes* Added a pkg-config file for atf-sh and an aclocal file to ease the 1801541Srgrimes detection of atf-sh from autoconf scripts. 1811541Srgrimes 1821541Srgrimes* Made the default test case body defined by atf_sh fail. This is to 1831541Srgrimes ensure that test cases are properly defined in test programs and helps 1841541Srgrimes in catching typos in the names of the body functions. 1851541Srgrimes 1861541Srgrimes* PR bin/44882: Made atf-run connect the stdin of test cases to /dev/zero. 1871541Srgrimes This provides more consistent results with "normal" execution (in 1881541Srgrimes particular, when tests are executed detached from a terminal). 1891541Srgrimes 1901541Srgrimes* Made atf-run hardcode TZ=UTC for test cases. It used to undefine TZ, but 1911541Srgrimes that does not take into account that libc determines the current timezone 1921541Srgrimes from a configuration file. 1931541Srgrimes 1941541Srgrimes* All test programs will now print a warning when they are not run through 1951541Srgrimes atf-run(1) stating that this is unsupported and may deliver incorrect 1961541Srgrimes results. 1971541Srgrimes 1981541Srgrimes* Added support for the 'require.files' test-case property. This allows 1991541Srgrimes test cases to specify installed files that must be present for the test 2001541Srgrimes case to run. 2011541Srgrimes 2021541Srgrimes 2031541SrgrimesChanges in version 0.13 2041541Srgrimes*********************** 2051541Srgrimes 2061541SrgrimesExperimental version released on March 31st, 2011. 2071541Srgrimes 2081541SrgrimesThis is the first release after the creation of the Kyua project, a more 2091541Srgrimesmodular and reliable replacement for ATF. From now on, ATF will change to 2101541Srgrimesaccomodate the transition to this new codebase, but ATF will still continue 2111541Srgrimesto see development in the short/medium term. Check out the project page at 2121541Srgrimeshttp://code.google.com/p/kyua/ for more details. 2131541Srgrimes 2141541SrgrimesThe changes in this release are: 2151541Srgrimes 2161541Srgrimes* Added support to run the tests with the Kyua runtime engine (kyua-cli), a 2171541Srgrimes new package that aims to replace atf-run and atf-report. The ATF tests 2181541Srgrimes can be run with the new system by issuing a 'make installcheck-kyua' from 2191541Srgrimes the top-level directory of the project (assuming the 'kyua' binary is 2201541Srgrimes available during the configuration stage of ATF). 2211541Srgrimes 2221541Srgrimes* atf-run and atf-report are now in maintenance mode (but *not* deprecated 2231541Srgrimes yet!). Kyua already implements a new, much more reliable runtime engine 2241541Srgrimes that provides similar features to these tools. That said, it is not 2251541Srgrimes complete yet so all development efforts should go towards it. 2261541Srgrimes 2271541Srgrimes* If GDB is installed, atf-run dumps the stack trace of crashing test 2281541Srgrimes programs in an attempt to aid debugging. Contributed by Antti Kantee. 2291541Srgrimes 2301541Srgrimes* Reverted default timeout change in previous release and reset its value 2311541Srgrimes to 5 minutes. This was causing several issues, specially when running 2321541Srgrimes the existing NetBSD test suite in qemu. 2331541Srgrimes 2341541Srgrimes* Fixed the 'match' output checker in atf-check to properly validate the 2351541Srgrimes last line of a file even if it does not have a newline. 2361541Srgrimes 2371541Srgrimes* Added the ATF_REQUIRE_IN and ATF_REQUIRE_NOT_IN macros to atf-c++ to 2381541Srgrimes check for the presence (or lack thereof) of an element in a collection. 2391541Srgrimes 2401541Srgrimes* PR bin/44176: Fixed a race condition in atf-run that would crash atf-run 2411541Srgrimes when the cleanup of a test case triggered asynchronous modifications to 2421541Srgrimes its work directory (e.g. killing a daemon process that cleans up a pid 2431541Srgrimes file in the work directory). 2441541Srgrimes 2451541Srgrimes* PR bin/44301: Fixed the sample XSLT file to report bogus test programs 2461541Srgrimes instead of just listing them as having 0 test cases. 2471541Srgrimes 2481541Srgrimes 2491541SrgrimesChanges in version 0.12 2501541Srgrimes*********************** 2511541Srgrimes 2521541SrgrimesExperimental version released on November 7th, 2010. 2531541Srgrimes 2541541Srgrimes* Added the ATF_REQUIRE_THROW_RE to atf-c++, which is the same as 2551541Srgrimes ATF_REQUIRE_THROW but allows checking for the validity of the exception's 2561541Srgrimes error message by means of a regular expression. 2571541Srgrimes 2581541Srgrimes* Added the ATF_REQUIRE_MATCH to atf-c++, which allows checking for a 2591541Srgrimes regular expression match in a string. 2601541Srgrimes 2611541Srgrimes* Changed the default timeout for test cases from 5 minutes to 30 seconds. 2621541Srgrimes 30 seconds is long enough for virtually all tests to complete, and 5 2631541Srgrimes minutes is a way too long pause in a test suite where a single test case 2641541Srgrimes stalls. 2651541Srgrimes 2661541Srgrimes* Deprecated the use.fs property. While this seemed like a good idea in 2671541Srgrimes the first place to impose more control on what test cases can do, it 2681541Srgrimes turns out to be bad. First, use.fs=false prevents bogus test cases 2691541Srgrimes from dumping core so after-the-fact debugging is harder. Second, 2701541Srgrimes supporting use.fs adds a lot of unnecessary complexity. atf-run will 2711541Srgrimes now ignore any value provided to use.fs and will allow test cases to 2721541Srgrimes freely access the file system if they wish to. 2731541Srgrimes 2741541Srgrimes* Added the atf_tc_get_config_var_as_{bool,long}{,_wd} functions to the atf-c 2751541Srgrimes library. The 'text' module became private in 0.11 but was being used 2761541Srgrimes externally to simplify the parsing of configuration variables. 2771541Srgrimes 2781541Srgrimes* Made atf-run recognize the 'unprivileged-user' configuration variable 2791541Srgrimes and automatically drop root privileges when a test case sets 2801541Srgrimes require.user=unprivileged. Note that this is, by no means, done for 2811541Srgrimes security purposes; this is just for user convenience; tests should, in 2821541Srgrimes general, not be blindly run as root in the first place. 2831541Srgrimes 2841541Srgrimes 2851541SrgrimesChanges in version 0.11 2861541Srgrimes*********************** 2871541Srgrimes 2881541SrgrimesExperimental version released on October 20th, 2010. 2891541Srgrimes 2901541Srgrimes* The ATF_CHECK* macros in atf-c++ were renamed to ATF_REQUIRE* to match 2911541Srgrimes their counterparts in atf-c. 2921541Srgrimes 2931541Srgrimes* Clearly separated the modules in atf-c that are supposed to be public 2941541Srgrimes from those that are implementation details. The header files for the 2951541Srgrimes internal modules are not installed any more. 2961541Srgrimes 2971541Srgrimes* Made the atf-check tool private. It is only required by atf-sh and being 2981541Srgrimes public has the danger of causing confusion. Also, making it private 2991541Srgrimes simplifies the public API of atf. 3001541Srgrimes 3011541Srgrimes* Changed atf-sh to enable per-command error checking (set -e) by default. 3021541Srgrimes This catches many cases in which a test case is broken but it is not 3031541Srgrimes reported as such because execution continues. 3041541Srgrimes 3051541Srgrimes* Fixed the XSTL and CSS stylesheets to support expected failures. 3061541Srgrimes 3071541Srgrimes 3081541SrgrimesChanges in version 0.10 3091541Srgrimes*********************** 3101541Srgrimes 3111541SrgrimesExperimental version released on July 2nd, 2010. 3121541Srgrimes 3131541SrgrimesMiscellaneous features 3141541Srgrimes 3151541Srgrimes* Added expected failures support to test cases and atf-run. These 3161541Srgrimes include, for example, expected clean exits, expected reception of fatal 3171541Srgrimes signals, expected timeouts and expected errors in condition checks. 3181541Srgrimes These statuses can be used to denote test cases that are known to fail 3191541Srgrimes due to a bug in the code they are testing. atf-report reports these 3201541Srgrimes tests separately but they do not count towards the failed test cases 3211541Srgrimes amount. 3221541Srgrimes 3231541Srgrimes* Added the ATF_CHECK_ERRNO and ATF_REQUIRE_ERRNO to the C library to 3241541Srgrimes allow easy checking of call failures that update errno. 3251541Srgrimes 3261541Srgrimes* Added the has.cleanup meta-data property to test caes that specifies 3271541Srgrimes whether the test case has a cleanup routine or not; its value is 3281541Srgrimes automatically set. This property is read by atf-run to know if it has to 3291541Srgrimes run the cleanup routine; skipping this run for every test case 3301541Srgrimes significantly speeds up the run time of test suites. 3311541Srgrimes 3321541Srgrimes* Reversed the order of the ATF_CHECK_THROW macro in the C++ binding to 3331541Srgrimes take the expected exception as the first argument and the statement to 3341541Srgrimes execute as the second argument. 3351541Srgrimes 3361541SrgrimesChanges in atf-check 3371541Srgrimes 3381541Srgrimes* Changed atf-check to support negating the status and output checks by 3391541Srgrimes prefixing them with not- and added support to specify multiple checkers 3401541Srgrimes for stdout and stderr, not only one. 3411541Srgrimes 3421541Srgrimes* Added the match output checker to atf-check to look for regular 3431541Srgrimes expressions in the stdout and stderr of commands. 3441541Srgrimes 3451541Srgrimes* Modified the exit checks in atf-check to support checking for the 3461541Srgrimes reception of signals. 3471541Srgrimes 3481541SrgrimesCode simplifications and cleanups 3491541Srgrimes 3501541Srgrimes* Removed usage messages from test programs to simplify the 3511541Srgrimes implementation of every binding by a significant amount. They just now 3521541Srgrimes refer the user to the appropriate manual page and do not attempt to wrap 3531541Srgrimes lines on terminal boundaries. Test programs are not supposed to be run 3541541Srgrimes by users directly so this minor interface regression is not important. 3551541Srgrimes 3561541Srgrimes* Removed the atf-format internal utility, which is unused after the 3571541Srgrimes change documented above. 3581541Srgrimes 3591541Srgrimes* Removed the atf-cleanup internal utility. It has been unused since the 3601541Srgrimes test case isolation was moved to atf-run in 0.8 3611541Srgrimes 3621541Srgrimes* Splitted the Makefile.am into smaller files for easier maintenance and 3631541Srgrimes dropped the use of M4. Only affects users building from the repository 3641541Srgrimes sources. 3651541Srgrimes 3661541Srgrimes* Intermixed tests with the source files in the source tree to provide 3671541Srgrimes them more visibility and easier access. The tests directory is gone from 3681541Srgrimes the source tree and tests are now suffixed by _test, not prefixed by t_. 3691541Srgrimes 3701541Srgrimes* Simplifications to the atf-c library: removed the io, tcr and ui 3711541Srgrimes modules as they had become unnecessary after all simplifications 3721541Srgrimes introduced since the 0.8 release. 3731541Srgrimes 3741541Srgrimes* Removed the application/X-atf-tcr format introduced in 0.8 release. 3751541Srgrimes Tests now print a much simplified format that is easy to parse and nicer 3761541Srgrimes to read by end users. As a side effect, the default for test cases is 3771541Srgrimes now to print their results to stdout unless otherwise stated by providing 3781541Srgrimes the -r flag. 3791541Srgrimes 3801541Srgrimes* Removed XML distribution documents and replaced them with plain-text 3811541Srgrimes documents. They provided little value and introduced a lot of complexity 3821541Srgrimes to the build system. 3831541Srgrimes 3841541Srgrimes* Simplified the output of atf-version by not attempting to print a 3851541Srgrimes revision number when building form a distfile. Makes the build system 3861541Srgrimes easier to maintain. 3871541Srgrimes 3881541Srgrimes 3891541SrgrimesChanges in version 0.9 3901541Srgrimes********************** 3911541Srgrimes 3921541SrgrimesExperimental version released on June 3rd, 2010. 3931541Srgrimes 3941541Srgrimes* Added atf-sh, an interpreter to process test programs written using 3951541Srgrimes the shell API. This is not really a shell interpreter by itself though: 3961541Srgrimes it is just a wrapper around the system shell that eases the loading of 3971541Srgrimes the necessary ATF libraries. 3981541Srgrimes 3991541Srgrimes* Removed atf-compile in favour of atf-sh. 4001541Srgrimes 4011541Srgrimes* Added the use.fs metadata property to test case, which is used to 4021541Srgrimes specify which test cases require file system access. This is to 4031541Srgrimes highlight dependencies on external resources more clearly and to speed up 4041541Srgrimes the execution of test suites by skipping the creation of many unnecessary 4051541Srgrimes work directories. 4061541Srgrimes 4071541Srgrimes* Fixed test programs to get a sane default value for their source 4081541Srgrimes directory. This means that it should not be necessary any more to pass 4091541Srgrimes -s when running test programs that do not live in the current directory. 4101541Srgrimes 4111541Srgrimes* Defining test case headers became optional. This is trivial to achieve 4121541Srgrimes in shell-based tests but a bit ugly in C and C++. In C, use the new 4131541Srgrimes ATF_TC_WITHOUT_HEAD macro to define the test case, and in C++ use 4141541Srgrimes ATF_TEST_CASE_WITHOUT_HEAD. 4151541Srgrimes 4161541Srgrimes 4171541SrgrimesChanges in version 0.8 4181541Srgrimes********************** 4191541Srgrimes 4201541SrgrimesExperimental version released on May 7th, 2010. 4211541Srgrimes 4221541Srgrimes* Test programs no longer run several test cases in a row. The execution 4231541Srgrimes of a test program now requires a test case name, and that single test 4241541Srgrimes case is executed. To execute several test cases, use the atf-run utility 4251541Srgrimes as usual. 4261541Srgrimes 4271541Srgrimes* Test programs no longer fork a subprocess to isolate the execution of 4281541Srgrimes test cases. They run the test case code in-process, and a crash of the 4291541Srgrimes test case will result in a crash of the test program. This is to ease 4301541Srgrimes debugging of faulty test cases. 4311541Srgrimes 4321541Srgrimes* Test programs no longer isolate their test cases. This means that they 4331541Srgrimes will not create temporary directories nor sanitize the environment any 4341541Srgrimes more. Yes: running a test case that depends on system state by hand will 4351541Srgrimes most likely yield different results depending on where (machine, 4361541Srgrimes directory, user environment, etc.) it is run. Isolation has been moved 4371541Srgrimes to atf-run. 4381541Srgrimes 4391541Srgrimes* Test programs no longer print a cryptic format (application/X-atf-tcs) 4401541Srgrimes on a special file channel. They can now print whatever they want on the 4411541Srgrimes screen. Because test programs can now only run one test case every time, 4421541Srgrimes providing controlled output is not necessary any more. 4431541Srgrimes 4441541Srgrimes* Test programs no longer write their status into a special file 4451541Srgrimes descriptor. Instead, they create a file with the results, which is later 4461541Srgrimes parsed by atf-run. This changes the semantics of the -r flag. 4471541Srgrimes 4481541Srgrimes* atf-run has been adjusted to perform the test case isolation. As a 4491541Srgrimes result, there is now a single canonical place that implements the 4501541Srgrimes isolation of test caes. In previous releases, the three language 451 bindings (C, C++ and shell) had to be kept in sync with each other (read: 452 not a nice thing to do at all). As a side effect of this change, writing 453 bindings for other languages will be much, much easier from now on. 454 455* atf-run forks test programs on a test case basis, instead of on a test 456 program basis as it did before. This is to provide the test case 457 isolation that was before implemented by the test programs themselves. 458 459* Removed the atf-exec tool. This was used to implement test case 460 isolation in atf-sh, but it is now unnecessary. 461 462* It is now optional to define the descr meta-data property. It has been 463 proven to be mostly useless, because test cases often carry a descriptive 464 name of their own. 465 466 467Changes in version 0.7 468********************** 469 470Experimental version released on December 22nd, 2009. 471 472* Added build-time checks to atf-c and atf-c++. A binding for atf-sh 473 will come later. 474 475* Migrated all build-time checks for header files to proper ATF tests. 476 This demonstrates the use of the new feature described above. 477 478* Added an internal API for child process management. 479 480* Converted all plain-text distribution documents to a Docbook canonical 481 version, and include pre-generated plain text and HTML copies in the 482 distribution file. 483 484* Simplified the contents of the Makefile.am by regenerating it from a 485 canonical Makefile.am.m4 source. As a side-effect, some dependency 486 specifications were fixed. 487 488* Migrated all checks from the check target to installcheck, as these 489 require ATF to be installed. 490 491* Fixed sign comparison mismatches triggered by the now-enabled 492 -Wsign-compare. 493 494* Fixed many memory and object leaks. 495 496 497Changes in version 0.6 498********************** 499 500Experimental version released on January 18th, 2009. 501 502* Make atf-exec be able to kill its child process after a certain period 503 of time; this is controlled through the new -t option. 504 505* Change atf-sh to use atf-exec's -t option to control the test case's 506 timeouts, instead of doing it internally. Same behavior as before, but 507 noticeably faster. 508 509* atf-exec's -g option and atf-killpg are gone due to the previous 510 change. 511 512* Added the atf-check(1) tool, a program that executes a given command 513 and checks its exit code against a known value and allows the management 514 of stdout and stderr in multiple ways. This replaces the previous 515 atf_check function in the atf-sh library and exposes this functionality 516 to both atf-c and atf-c++. 517 518* Added the ATF_REQUIRE family of macros to the C interface. These help 519 in checking for fatal test conditions. The old ATF_CHECK macros now 520 perform non-fatal checks only. I.e. by using ATF_CHECK, the test case 521 can now continue its execution and the failures will not be reported 522 until the end of the whole run. 523 524* Extended the amount of ATF_CHECK_* C macros with new ones to provide 525 more features to the developer. These also have their corresponding 526 counterparts in the ATF_REQUIRE_* family. The new macros (listing the 527 suffixes only) are: _EQ (replaces _EQUAL), _EQ_MSG, _STREQ and 528 _STREQ_MSG. 529 530 531Changes in version 0.5 532********************** 533 534Experimental version released on May 1st, 2008. 535 536* Clauses 3 and 4 of the BSD license used by the project were dropped. 537 All the code is now under a 2-clause BSD license compatible with the GNU 538 General Public License (GPL). 539 540* Added a C-only binding so that binary test programs do not need to be 541 tied to C++ at all. This binding is now known as the atf-c library. 542 543* Renamed the C++ binding to atf-c++ for consistency with the new atf-c. 544 545* Renamed the POSIX shell binding to atf-sh for consistency with the new 546 atf-c and atf-c++. 547 548* Added a -w flag to test programs through which it is possible to 549 specify the work directory to be used. This was possible in prior 550 releases by defining the workdir configuration variable (-v workdir=...), 551 but was a conceptually incorrect mechanism. 552 553* Test programs now preserve the execution order of test cases when they 554 are given in the command line. Even those mentioned more than once are 555 executed multiple times to comply with the user's requests. 556 557 558Changes in version 0.4 559********************** 560 561Experimental version released on February 4th, 2008. 562 563* Added two new manual pages, atf-c++-api and atf-sh-api, describing the 564 C++ and POSIX shell interfaces used to write test programs. 565 566* Added a pkg-config file, useful to get the flags to build against the 567 C++ library or to easily detect the presence of ATF. 568 569* Added a way for test cases to require a specific architecture and/or 570 machine type through the new 'require.arch' and 'require.machine' 571 meta-data properties, respectively. 572 573* Added the 'timeout' property to test cases, useful to set an 574 upper-bound limit for the test's run time and thus prevent global test 575 program stalls due to the test case's misbehavior. 576 577* Added the atf-exec(1) internal utility, used to execute a command 578 after changing the process group it belongs to. 579 580* Added the atf-killpg(1) internal utility, used to kill process groups. 581 582* Multiple portability fixes. Of special interest, full support for 583 SunOS (Solaris Express Developer Edition 2007/09) using the Sun Studio 12 584 C++ compiler. 585 586* Fixed a serious bug that prevented atf-run(1) from working at all 587 under Fedora 8 x86_64. Due to the nature of the bug, other platforms 588 were likely affected too. 589 590 591Changes in version 0.3 592********************** 593 594Experimental version released on November 11th, 2007. 595 596* Added XML output support to atf-report. This is accompanied by a DTD 597 for the format's structure and sample XSLT/CSS files to post-process this 598 output and convert it to a plain HTML report. 599 600* Changed atf-run to add system information to the report it generates. 601 This is currently used by atf-report's XML output only, and is later 602 printed in the HTML reports in a nice and useful summary table. The user 603 and system administrator are allowed to tune this feature by means of 604 hooks. 605 606* Removed the test cases' 'isolated' property. This was intended to 607 avoid touching the file system at all when running the related test case, 608 but this has not been true for a long while: some control files are 609 unconditionally required for several purposes, and we cannot easily get 610 rid of them. This way we remove several critical and delicate pieces of 611 code. 612 613* Improved atf-report's CSV output format to include information about 614 test programs too. 615 616* Fixed the tests that used atf-compile to not require this tool as a 617 helper. Avoids systems without build-time utilities to skip many tests 618 that could otherwise be run. (E.g. NetBSD without the comp.tgz set 619 installed.) 620 621* Many general cleanups: Fixed many pieces of code marked as ugly and/or 622 incomplete. 623 624 625Changes in version 0.2 626********************** 627 628Experimental version released on September 20th, 2007. 629 630* Test cases now get a known umask on entry. 631 632* atf-run now detects many unexpected failures caused by test programs and 633 reports them as bogus tests. atf-report is able to handle these new 634 errors and nicely reports them to the user. 635 636* All the data formats read and written by the tools have been 637 documented and cleaned up. These include those grammars that define how 638 the different components communicate with each other as well as the 639 format of files written by the developers and users: the Atffiles and the 640 configuration files. 641 642* Added the atf-version tool, a utility that displays information about 643 the currently installed version of ATF. 644 645* Test cases can now define an optional cleanup routine to undo their 646 actions regardless of their exit status. 647 648* atf-report now summarizes the list of failed (bogus) test programs 649 when using the ticker output format. 650 651* Test programs now capture some termination signals and clean up any 652 temporary files before exiting the program. 653 654* Multiple bug fixes and improvements all around. 655 656 657Changes in version 0.1 658********************** 659 660Experimental version released on August 20th, 2007. 661 662* First public version. This was released coinciding with the end of the 663 Google Summer of Code 2007 program. 664 665 666=========================================================================== 667vim: filetype=text:textwidth=75:expandtab:shiftwidth=2:softtabstop=2 668