NEWS revision 258289
1240116SmarcelMajor changes between releases Automated Testing Framework 2240116Smarcel=========================================================================== 3240116Smarcel 4240116Smarcel 5258289SjmmvChanges in version 0.18 6258289Sjmmv*********************** 7258289Sjmmv 8258289SjmmvExperimental version released on November 16th, 2013. 9258289Sjmmv 10258289Sjmmv* Issue 45: Added require.memory support in atf-run for FreeBSD. 11258289Sjmmv 12258289Sjmmv* Fixed an issue with the handling of cin with libc++. 13258289Sjmmv 14258289Sjmmv* Issue 64: Fixed various mandoc formatting warnings. 15258289Sjmmv 16258289Sjmmv* NetBSD PR bin/48284: Made atf-check flush its progress message to 17258289Sjmmv stdout so that an interrupted test case always shows the last message 18258289Sjmmv being executed. 19258289Sjmmv 20258289Sjmmv* NetBSD PR bin/48285: Fixed atf_check examples in atf-sh-api(3). 21258289Sjmmv 22258289Sjmmv 23258289SjmmvChanges in version 0.17 24258289Sjmmv*********************** 25258289Sjmmv 26258289SjmmvExperimental version released on February 14th, 2013. 27258289Sjmmv 28258289Sjmmv* Added the atf_utils_cat_file, atf_utils_compare_file, 29258289Sjmmv atf_utils_copy_file, atf_utils_create_file, atf_utils_file_exists, 30258289Sjmmv atf_utils_fork, atf_utils_grep_file, atf_utils_grep_string, 31258289Sjmmv atf_utils_readline, atf_utils_redirect and atf_utils_wait utility 32258289Sjmmv functions to atf-c-api. Documented the already-public 33258289Sjmmv atf_utils_free_charpp function. 34258289Sjmmv 35258289Sjmmv* Added the cat_file, compare_file, copy_file, create_file, file_exists, 36258289Sjmmv fork, grep_collection, grep_file, grep_string, redirect and wait 37258289Sjmmv functions to the atf::utils namespace of atf-c++-api. These are 38258289Sjmmv wrappers around the same functions added to the atf-c-api library. 39258289Sjmmv 40258289Sjmmv* Added the ATF_CHECK_MATCH, ATF_CHECK_MATCH_MSG, ATF_REQUIRE_MATCH and 41258289Sjmmv ATF_REQUIRE_MATCH_MSG macros to atf-c to simplify the validation of a 42258289Sjmmv string against a regular expression. 43258289Sjmmv 44258289Sjmmv* Miscellaneous fixes for manpage typos and compilation problems with 45258289Sjmmv clang. 46258289Sjmmv 47258289Sjmmv* Added caching of the results of those configure tests that rely on 48258289Sjmmv executing a test program. This should help crossbuild systems by 49258289Sjmmv providing a mechanism to pre-specify what the results should be. 50258289Sjmmv 51258289Sjmmv* PR bin/45690: Make atf-report convert any non-printable characters to 52258289Sjmmv a plain-text representation (matching their corresponding hexadecimal 53258289Sjmmv entities) in XML output files. This is to prevent the output of test 54258289Sjmmv cases from breaking xsltproc later. 55258289Sjmmv 56258289Sjmmv 57240116SmarcelChanges in version 0.16 58240116Smarcel*********************** 59240116Smarcel 60240116SmarcelExperimental version released on July 10th, 2012. 61240116Smarcel 62240116Smarcel* Added a --enable-tools flag to configure to request the build of the 63240116Smarcel deprecated ATF tools, whose build is now disabled by default. In order 64240116Smarcel to continue running tests, you should migrate to Kyua instead of enabling 65240116Smarcel the build of the deprecated tools. The kyua-atf-compat package provides 66240116Smarcel transitional compatibility versions of atf-run and atf-report built on 67240116Smarcel top of Kyua. 68240116Smarcel 69240116Smarcel* Tweaked the ATF_TEST_CASE macro of atf-c++ so that the compiler can 70240116Smarcel detect defined but unused test cases. 71240116Smarcel 72240116Smarcel* PR bin/45859: Fixed some XSLT bugs that resulted in the tc-time and 73240116Smarcel tp-time XML tags leaking into the generated HTML file. Also improved 74240116Smarcel the CSS file slightly to correct alignment and color issues with the 75240116Smarcel timestamps column. 76240116Smarcel 77240116Smarcel* Optimized atf-c++/macros.hpp so that GNU G++ consumes less memory during 78240116Smarcel compilation with GNU G++. 79240116Smarcel 80240116Smarcel* Flipped the default to building shared libraries for atf-c and atf-c++, 81240116Smarcel and started versioning them. As a side-effect, this removes the 82240116Smarcel --enable-unstable-shared flag from configure that appears to not work any 83240116Smarcel more (under NetBSD). Additionally, some distributions require the use of 84240116Smarcel shared libraries for proper dependency tracking (e.g. Fedora), so it is 85240116Smarcel better if we do the right versioning upstream. 86240116Smarcel 87240116Smarcel* Project hosting moved from an adhoc solution (custom web site and 88240116Smarcel Monotone repository) to Google Code (standard wiki and Git). ATF now 89240116Smarcel lives in a subcomponent of the Kyua project. 90240116Smarcel 91240116Smarcel 92240116SmarcelChanges in version 0.15 93240116Smarcel*********************** 94240116Smarcel 95240116SmarcelExperimental version released on January 16th, 2012. 96240116Smarcel 97240116Smarcel* Respect stdin in atf-check. The previous release silenced stdin for any 98240116Smarcel processes spawned by atf, not only test programs, which caused breakage 99240116Smarcel in tests that pipe data through atf-check. 100240116Smarcel 101240116Smarcel* Performance improvements to atf-sh. 102240116Smarcel 103240116Smarcel* Enabled detection of unused parameters and variables in the code and 104240116Smarcel fixed all warnings. 105240116Smarcel 106240116Smarcel* Changed the behavior of "developer mode". Compiler warnings are now 107240116Smarcel enabled unconditionally regardless of whether we are in developer mode or 108240116Smarcel not; developer mode is now only used to perform strict warning checks and 109240116Smarcel to enable assertions. Additionally, developer mode is now only 110240116Smarcel automatically enabled when building from the repository, not for formal 111240116Smarcel releases. 112240116Smarcel 113240116Smarcel* Added new Autoconf M4 macros (ATF_ARG_WITH, ATF_CHECK_C and 114240116Smarcel ATF_CHECK_CXX) to provide a consistent way of defining a --with-arg flag 115240116Smarcel in configure scripts and detecting the presence of any of the ATF 116240116Smarcel bindings. Note that ATF_CHECK_SH was already introduced in 0.14, but it 117240116Smarcel has now been modified to also honor --with-atf if instantiated. 118240116Smarcel 119240116Smarcel* Added timing support to atf-run / atf-report. 120240116Smarcel 121240116Smarcel* Added support for a 'require.memory' property, to specify the minimum 122240116Smarcel amount of physical memory needed by the test case to yield valid results. 123240116Smarcel 124240116Smarcel* PR bin/45690: Force an ISO-8859-1 encoding in the XML files generated by 125240116Smarcel atf-report so that invalid data in the output of test cases does not 126240116Smarcel mangle our report. 127240116Smarcel 128240116Smarcel 129240116SmarcelChanges in version 0.14 130240116Smarcel*********************** 131240116Smarcel 132240116SmarcelExperimental version released on June 14th, 2011. 133240116Smarcel 134240116Smarcel* Added a pkg-config file for atf-sh and an aclocal file to ease the 135240116Smarcel detection of atf-sh from autoconf scripts. 136240116Smarcel 137240116Smarcel* Made the default test case body defined by atf_sh fail. This is to 138240116Smarcel ensure that test cases are properly defined in test programs and helps 139240116Smarcel in catching typos in the names of the body functions. 140240116Smarcel 141240116Smarcel* PR bin/44882: Made atf-run connect the stdin of test cases to /dev/zero. 142240116Smarcel This provides more consistent results with "normal" execution (in 143240116Smarcel particular, when tests are executed detached from a terminal). 144240116Smarcel 145240116Smarcel* Made atf-run hardcode TZ=UTC for test cases. It used to undefine TZ, but 146240116Smarcel that does not take into account that libc determines the current timezone 147240116Smarcel from a configuration file. 148240116Smarcel 149240116Smarcel* All test programs will now print a warning when they are not run through 150240116Smarcel atf-run(1) stating that this is unsupported and may deliver incorrect 151240116Smarcel results. 152240116Smarcel 153240116Smarcel* Added support for the 'require.files' test-case property. This allows 154240116Smarcel test cases to specify installed files that must be present for the test 155240116Smarcel case to run. 156240116Smarcel 157240116Smarcel 158240116SmarcelChanges in version 0.13 159240116Smarcel*********************** 160240116Smarcel 161240116SmarcelExperimental version released on March 31st, 2011. 162240116Smarcel 163240116SmarcelThis is the first release after the creation of the Kyua project, a more 164240116Smarcelmodular and reliable replacement for ATF. From now on, ATF will change to 165240116Smarcelaccomodate the transition to this new codebase, but ATF will still continue 166240116Smarcelto see development in the short/medium term. Check out the project page at 167240116Smarcelhttp://code.google.com/p/kyua/ for more details. 168240116Smarcel 169240116SmarcelThe changes in this release are: 170240116Smarcel 171240116Smarcel* Added support to run the tests with the Kyua runtime engine (kyua-cli), a 172240116Smarcel new package that aims to replace atf-run and atf-report. The ATF tests 173240116Smarcel can be run with the new system by issuing a 'make installcheck-kyua' from 174240116Smarcel the top-level directory of the project (assuming the 'kyua' binary is 175240116Smarcel available during the configuration stage of ATF). 176240116Smarcel 177240116Smarcel* atf-run and atf-report are now in maintenance mode (but *not* deprecated 178240116Smarcel yet!). Kyua already implements a new, much more reliable runtime engine 179240116Smarcel that provides similar features to these tools. That said, it is not 180240116Smarcel complete yet so all development efforts should go towards it. 181240116Smarcel 182240116Smarcel* If GDB is installed, atf-run dumps the stack trace of crashing test 183240116Smarcel programs in an attempt to aid debugging. Contributed by Antti Kantee. 184240116Smarcel 185240116Smarcel* Reverted default timeout change in previous release and reset its value 186240116Smarcel to 5 minutes. This was causing several issues, specially when running 187240116Smarcel the existing NetBSD test suite in qemu. 188240116Smarcel 189240116Smarcel* Fixed the 'match' output checker in atf-check to properly validate the 190240116Smarcel last line of a file even if it does not have a newline. 191240116Smarcel 192240116Smarcel* Added the ATF_REQUIRE_IN and ATF_REQUIRE_NOT_IN macros to atf-c++ to 193240116Smarcel check for the presence (or lack thereof) of an element in a collection. 194240116Smarcel 195240116Smarcel* PR bin/44176: Fixed a race condition in atf-run that would crash atf-run 196240116Smarcel when the cleanup of a test case triggered asynchronous modifications to 197240116Smarcel its work directory (e.g. killing a daemon process that cleans up a pid 198240116Smarcel file in the work directory). 199240116Smarcel 200240116Smarcel* PR bin/44301: Fixed the sample XSLT file to report bogus test programs 201240116Smarcel instead of just listing them as having 0 test cases. 202240116Smarcel 203240116Smarcel 204240116SmarcelChanges in version 0.12 205240116Smarcel*********************** 206240116Smarcel 207240116SmarcelExperimental version released on November 7th, 2010. 208240116Smarcel 209240116Smarcel* Added the ATF_REQUIRE_THROW_RE to atf-c++, which is the same as 210240116Smarcel ATF_REQUIRE_THROW but allows checking for the validity of the exception's 211240116Smarcel error message by means of a regular expression. 212240116Smarcel 213240116Smarcel* Added the ATF_REQUIRE_MATCH to atf-c++, which allows checking for a 214240116Smarcel regular expression match in a string. 215240116Smarcel 216240116Smarcel* Changed the default timeout for test cases from 5 minutes to 30 seconds. 217240116Smarcel 30 seconds is long enough for virtually all tests to complete, and 5 218240116Smarcel minutes is a way too long pause in a test suite where a single test case 219240116Smarcel stalls. 220240116Smarcel 221240116Smarcel* Deprecated the use.fs property. While this seemed like a good idea in 222240116Smarcel the first place to impose more control on what test cases can do, it 223240116Smarcel turns out to be bad. First, use.fs=false prevents bogus test cases 224240116Smarcel from dumping core so after-the-fact debugging is harder. Second, 225240116Smarcel supporting use.fs adds a lot of unnecessary complexity. atf-run will 226240116Smarcel now ignore any value provided to use.fs and will allow test cases to 227240116Smarcel freely access the file system if they wish to. 228240116Smarcel 229240116Smarcel* Added the atf_tc_get_config_var_as_{bool,long}{,_wd} functions to the atf-c 230240116Smarcel library. The 'text' module became private in 0.11 but was being used 231240116Smarcel externally to simplify the parsing of configuration variables. 232240116Smarcel 233240116Smarcel* Made atf-run recognize the 'unprivileged-user' configuration variable 234240116Smarcel and automatically drop root privileges when a test case sets 235240116Smarcel require.user=unprivileged. Note that this is, by no means, done for 236240116Smarcel security purposes; this is just for user convenience; tests should, in 237240116Smarcel general, not be blindly run as root in the first place. 238240116Smarcel 239240116Smarcel 240240116SmarcelChanges in version 0.11 241240116Smarcel*********************** 242240116Smarcel 243240116SmarcelExperimental version released on October 20th, 2010. 244240116Smarcel 245240116Smarcel* The ATF_CHECK* macros in atf-c++ were renamed to ATF_REQUIRE* to match 246240116Smarcel their counterparts in atf-c. 247240116Smarcel 248240116Smarcel* Clearly separated the modules in atf-c that are supposed to be public 249240116Smarcel from those that are implementation details. The header files for the 250240116Smarcel internal modules are not installed any more. 251240116Smarcel 252240116Smarcel* Made the atf-check tool private. It is only required by atf-sh and being 253240116Smarcel public has the danger of causing confusion. Also, making it private 254240116Smarcel simplifies the public API of atf. 255240116Smarcel 256240116Smarcel* Changed atf-sh to enable per-command error checking (set -e) by default. 257240116Smarcel This catches many cases in which a test case is broken but it is not 258240116Smarcel reported as such because execution continues. 259240116Smarcel 260240116Smarcel* Fixed the XSTL and CSS stylesheets to support expected failures. 261240116Smarcel 262240116Smarcel 263240116SmarcelChanges in version 0.10 264240116Smarcel*********************** 265240116Smarcel 266240116SmarcelExperimental version released on July 2nd, 2010. 267240116Smarcel 268240116SmarcelMiscellaneous features 269240116Smarcel 270240116Smarcel* Added expected failures support to test cases and atf-run. These 271240116Smarcel include, for example, expected clean exits, expected reception of fatal 272240116Smarcel signals, expected timeouts and expected errors in condition checks. 273240116Smarcel These statuses can be used to denote test cases that are known to fail 274240116Smarcel due to a bug in the code they are testing. atf-report reports these 275240116Smarcel tests separately but they do not count towards the failed test cases 276240116Smarcel amount. 277240116Smarcel 278240116Smarcel* Added the ATF_CHECK_ERRNO and ATF_REQUIRE_ERRNO to the C library to 279240116Smarcel allow easy checking of call failures that update errno. 280240116Smarcel 281240116Smarcel* Added the has.cleanup meta-data property to test caes that specifies 282240116Smarcel whether the test case has a cleanup routine or not; its value is 283240116Smarcel automatically set. This property is read by atf-run to know if it has to 284240116Smarcel run the cleanup routine; skipping this run for every test case 285240116Smarcel significantly speeds up the run time of test suites. 286240116Smarcel 287240116Smarcel* Reversed the order of the ATF_CHECK_THROW macro in the C++ binding to 288240116Smarcel take the expected exception as the first argument and the statement to 289240116Smarcel execute as the second argument. 290240116Smarcel 291240116SmarcelChanges in atf-check 292240116Smarcel 293240116Smarcel* Changed atf-check to support negating the status and output checks by 294240116Smarcel prefixing them with not- and added support to specify multiple checkers 295240116Smarcel for stdout and stderr, not only one. 296240116Smarcel 297240116Smarcel* Added the match output checker to atf-check to look for regular 298240116Smarcel expressions in the stdout and stderr of commands. 299240116Smarcel 300240116Smarcel* Modified the exit checks in atf-check to support checking for the 301240116Smarcel reception of signals. 302240116Smarcel 303240116SmarcelCode simplifications and cleanups 304240116Smarcel 305240116Smarcel* Removed usage messages from test programs to simplify the 306240116Smarcel implementation of every binding by a significant amount. They just now 307240116Smarcel refer the user to the appropriate manual page and do not attempt to wrap 308240116Smarcel lines on terminal boundaries. Test programs are not supposed to be run 309240116Smarcel by users directly so this minor interface regression is not important. 310240116Smarcel 311240116Smarcel* Removed the atf-format internal utility, which is unused after the 312240116Smarcel change documented above. 313240116Smarcel 314240116Smarcel* Removed the atf-cleanup internal utility. It has been unused since the 315240116Smarcel test case isolation was moved to atf-run in 0.8 316240116Smarcel 317240116Smarcel* Splitted the Makefile.am into smaller files for easier maintenance and 318240116Smarcel dropped the use of M4. Only affects users building from the repository 319240116Smarcel sources. 320240116Smarcel 321240116Smarcel* Intermixed tests with the source files in the source tree to provide 322240116Smarcel them more visibility and easier access. The tests directory is gone from 323240116Smarcel the source tree and tests are now suffixed by _test, not prefixed by t_. 324240116Smarcel 325240116Smarcel* Simplifications to the atf-c library: removed the io, tcr and ui 326240116Smarcel modules as they had become unnecessary after all simplifications 327240116Smarcel introduced since the 0.8 release. 328240116Smarcel 329240116Smarcel* Removed the application/X-atf-tcr format introduced in 0.8 release. 330240116Smarcel Tests now print a much simplified format that is easy to parse and nicer 331240116Smarcel to read by end users. As a side effect, the default for test cases is 332240116Smarcel now to print their results to stdout unless otherwise stated by providing 333240116Smarcel the -r flag. 334240116Smarcel 335240116Smarcel* Removed XML distribution documents and replaced them with plain-text 336240116Smarcel documents. They provided little value and introduced a lot of complexity 337240116Smarcel to the build system. 338240116Smarcel 339240116Smarcel* Simplified the output of atf-version by not attempting to print a 340240116Smarcel revision number when building form a distfile. Makes the build system 341240116Smarcel easier to maintain. 342240116Smarcel 343240116Smarcel 344240116SmarcelChanges in version 0.9 345240116Smarcel********************** 346240116Smarcel 347240116SmarcelExperimental version released on June 3rd, 2010. 348240116Smarcel 349240116Smarcel* Added atf-sh, an interpreter to process test programs written using 350240116Smarcel the shell API. This is not really a shell interpreter by itself though: 351240116Smarcel it is just a wrapper around the system shell that eases the loading of 352240116Smarcel the necessary ATF libraries. 353240116Smarcel 354240116Smarcel* Removed atf-compile in favour of atf-sh. 355240116Smarcel 356240116Smarcel* Added the use.fs metadata property to test case, which is used to 357240116Smarcel specify which test cases require file system access. This is to 358240116Smarcel highlight dependencies on external resources more clearly and to speed up 359240116Smarcel the execution of test suites by skipping the creation of many unnecessary 360240116Smarcel work directories. 361240116Smarcel 362240116Smarcel* Fixed test programs to get a sane default value for their source 363240116Smarcel directory. This means that it should not be necessary any more to pass 364240116Smarcel -s when running test programs that do not live in the current directory. 365240116Smarcel 366240116Smarcel* Defining test case headers became optional. This is trivial to achieve 367240116Smarcel in shell-based tests but a bit ugly in C and C++. In C, use the new 368240116Smarcel ATF_TC_WITHOUT_HEAD macro to define the test case, and in C++ use 369240116Smarcel ATF_TEST_CASE_WITHOUT_HEAD. 370240116Smarcel 371240116Smarcel 372240116SmarcelChanges in version 0.8 373240116Smarcel********************** 374240116Smarcel 375240116SmarcelExperimental version released on May 7th, 2010. 376240116Smarcel 377240116Smarcel* Test programs no longer run several test cases in a row. The execution 378240116Smarcel of a test program now requires a test case name, and that single test 379240116Smarcel case is executed. To execute several test cases, use the atf-run utility 380240116Smarcel as usual. 381240116Smarcel 382240116Smarcel* Test programs no longer fork a subprocess to isolate the execution of 383240116Smarcel test cases. They run the test case code in-process, and a crash of the 384240116Smarcel test case will result in a crash of the test program. This is to ease 385240116Smarcel debugging of faulty test cases. 386240116Smarcel 387240116Smarcel* Test programs no longer isolate their test cases. This means that they 388240116Smarcel will not create temporary directories nor sanitize the environment any 389240116Smarcel more. Yes: running a test case that depends on system state by hand will 390240116Smarcel most likely yield different results depending on where (machine, 391240116Smarcel directory, user environment, etc.) it is run. Isolation has been moved 392240116Smarcel to atf-run. 393240116Smarcel 394240116Smarcel* Test programs no longer print a cryptic format (application/X-atf-tcs) 395240116Smarcel on a special file channel. They can now print whatever they want on the 396240116Smarcel screen. Because test programs can now only run one test case every time, 397240116Smarcel providing controlled output is not necessary any more. 398240116Smarcel 399240116Smarcel* Test programs no longer write their status into a special file 400240116Smarcel descriptor. Instead, they create a file with the results, which is later 401240116Smarcel parsed by atf-run. This changes the semantics of the -r flag. 402240116Smarcel 403240116Smarcel* atf-run has been adjusted to perform the test case isolation. As a 404240116Smarcel result, there is now a single canonical place that implements the 405240116Smarcel isolation of test caes. In previous releases, the three language 406240116Smarcel bindings (C, C++ and shell) had to be kept in sync with each other (read: 407240116Smarcel not a nice thing to do at all). As a side effect of this change, writing 408240116Smarcel bindings for other languages will be much, much easier from now on. 409240116Smarcel 410240116Smarcel* atf-run forks test programs on a test case basis, instead of on a test 411240116Smarcel program basis as it did before. This is to provide the test case 412240116Smarcel isolation that was before implemented by the test programs themselves. 413240116Smarcel 414240116Smarcel* Removed the atf-exec tool. This was used to implement test case 415240116Smarcel isolation in atf-sh, but it is now unnecessary. 416240116Smarcel 417240116Smarcel* It is now optional to define the descr meta-data property. It has been 418240116Smarcel proven to be mostly useless, because test cases often carry a descriptive 419240116Smarcel name of their own. 420240116Smarcel 421240116Smarcel 422240116SmarcelChanges in version 0.7 423240116Smarcel********************** 424240116Smarcel 425240116SmarcelExperimental version released on December 22nd, 2009. 426240116Smarcel 427240116Smarcel* Added build-time checks to atf-c and atf-c++. A binding for atf-sh 428240116Smarcel will come later. 429240116Smarcel 430240116Smarcel* Migrated all build-time checks for header files to proper ATF tests. 431240116Smarcel This demonstrates the use of the new feature described above. 432240116Smarcel 433240116Smarcel* Added an internal API for child process management. 434240116Smarcel 435240116Smarcel* Converted all plain-text distribution documents to a Docbook canonical 436240116Smarcel version, and include pre-generated plain text and HTML copies in the 437240116Smarcel distribution file. 438240116Smarcel 439240116Smarcel* Simplified the contents of the Makefile.am by regenerating it from a 440240116Smarcel canonical Makefile.am.m4 source. As a side-effect, some dependency 441240116Smarcel specifications were fixed. 442240116Smarcel 443240116Smarcel* Migrated all checks from the check target to installcheck, as these 444240116Smarcel require ATF to be installed. 445240116Smarcel 446240116Smarcel* Fixed sign comparison mismatches triggered by the now-enabled 447240116Smarcel -Wsign-compare. 448240116Smarcel 449240116Smarcel* Fixed many memory and object leaks. 450240116Smarcel 451240116Smarcel 452240116SmarcelChanges in version 0.6 453240116Smarcel********************** 454240116Smarcel 455240116SmarcelExperimental version released on January 18th, 2009. 456240116Smarcel 457240116Smarcel* Make atf-exec be able to kill its child process after a certain period 458240116Smarcel of time; this is controlled through the new -t option. 459240116Smarcel 460240116Smarcel* Change atf-sh to use atf-exec's -t option to control the test case's 461240116Smarcel timeouts, instead of doing it internally. Same behavior as before, but 462240116Smarcel noticeably faster. 463240116Smarcel 464240116Smarcel* atf-exec's -g option and atf-killpg are gone due to the previous 465240116Smarcel change. 466240116Smarcel 467240116Smarcel* Added the atf-check(1) tool, a program that executes a given command 468240116Smarcel and checks its exit code against a known value and allows the management 469240116Smarcel of stdout and stderr in multiple ways. This replaces the previous 470240116Smarcel atf_check function in the atf-sh library and exposes this functionality 471240116Smarcel to both atf-c and atf-c++. 472240116Smarcel 473240116Smarcel* Added the ATF_REQUIRE family of macros to the C interface. These help 474240116Smarcel in checking for fatal test conditions. The old ATF_CHECK macros now 475240116Smarcel perform non-fatal checks only. I.e. by using ATF_CHECK, the test case 476240116Smarcel can now continue its execution and the failures will not be reported 477240116Smarcel until the end of the whole run. 478240116Smarcel 479240116Smarcel* Extended the amount of ATF_CHECK_* C macros with new ones to provide 480240116Smarcel more features to the developer. These also have their corresponding 481240116Smarcel counterparts in the ATF_REQUIRE_* family. The new macros (listing the 482240116Smarcel suffixes only) are: _EQ (replaces _EQUAL), _EQ_MSG, _STREQ and 483240116Smarcel _STREQ_MSG. 484240116Smarcel 485240116Smarcel 486240116SmarcelChanges in version 0.5 487240116Smarcel********************** 488240116Smarcel 489240116SmarcelExperimental version released on May 1st, 2008. 490240116Smarcel 491240116Smarcel* Clauses 3 and 4 of the BSD license used by the project were dropped. 492240116Smarcel All the code is now under a 2-clause BSD license compatible with the GNU 493240116Smarcel General Public License (GPL). 494240116Smarcel 495240116Smarcel* Added a C-only binding so that binary test programs do not need to be 496240116Smarcel tied to C++ at all. This binding is now known as the atf-c library. 497240116Smarcel 498240116Smarcel* Renamed the C++ binding to atf-c++ for consistency with the new atf-c. 499240116Smarcel 500240116Smarcel* Renamed the POSIX shell binding to atf-sh for consistency with the new 501240116Smarcel atf-c and atf-c++. 502240116Smarcel 503240116Smarcel* Added a -w flag to test programs through which it is possible to 504240116Smarcel specify the work directory to be used. This was possible in prior 505240116Smarcel releases by defining the workdir configuration variable (-v workdir=...), 506240116Smarcel but was a conceptually incorrect mechanism. 507240116Smarcel 508240116Smarcel* Test programs now preserve the execution order of test cases when they 509240116Smarcel are given in the command line. Even those mentioned more than once are 510240116Smarcel executed multiple times to comply with the user's requests. 511240116Smarcel 512240116Smarcel 513240116SmarcelChanges in version 0.4 514240116Smarcel********************** 515240116Smarcel 516240116SmarcelExperimental version released on February 4th, 2008. 517240116Smarcel 518240116Smarcel* Added two new manual pages, atf-c++-api and atf-sh-api, describing the 519240116Smarcel C++ and POSIX shell interfaces used to write test programs. 520240116Smarcel 521240116Smarcel* Added a pkg-config file, useful to get the flags to build against the 522240116Smarcel C++ library or to easily detect the presence of ATF. 523240116Smarcel 524240116Smarcel* Added a way for test cases to require a specific architecture and/or 525240116Smarcel machine type through the new 'require.arch' and 'require.machine' 526240116Smarcel meta-data properties, respectively. 527240116Smarcel 528240116Smarcel* Added the 'timeout' property to test cases, useful to set an 529240116Smarcel upper-bound limit for the test's run time and thus prevent global test 530240116Smarcel program stalls due to the test case's misbehavior. 531240116Smarcel 532240116Smarcel* Added the atf-exec(1) internal utility, used to execute a command 533240116Smarcel after changing the process group it belongs to. 534240116Smarcel 535240116Smarcel* Added the atf-killpg(1) internal utility, used to kill process groups. 536240116Smarcel 537240116Smarcel* Multiple portability fixes. Of special interest, full support for 538240116Smarcel SunOS (Solaris Express Developer Edition 2007/09) using the Sun Studio 12 539240116Smarcel C++ compiler. 540240116Smarcel 541240116Smarcel* Fixed a serious bug that prevented atf-run(1) from working at all 542240116Smarcel under Fedora 8 x86_64. Due to the nature of the bug, other platforms 543240116Smarcel were likely affected too. 544240116Smarcel 545240116Smarcel 546240116SmarcelChanges in version 0.3 547240116Smarcel********************** 548240116Smarcel 549240116SmarcelExperimental version released on November 11th, 2007. 550240116Smarcel 551240116Smarcel* Added XML output support to atf-report. This is accompanied by a DTD 552240116Smarcel for the format's structure and sample XSLT/CSS files to post-process this 553240116Smarcel output and convert it to a plain HTML report. 554240116Smarcel 555240116Smarcel* Changed atf-run to add system information to the report it generates. 556240116Smarcel This is currently used by atf-report's XML output only, and is later 557240116Smarcel printed in the HTML reports in a nice and useful summary table. The user 558240116Smarcel and system administrator are allowed to tune this feature by means of 559240116Smarcel hooks. 560240116Smarcel 561240116Smarcel* Removed the test cases' 'isolated' property. This was intended to 562240116Smarcel avoid touching the file system at all when running the related test case, 563240116Smarcel but this has not been true for a long while: some control files are 564240116Smarcel unconditionally required for several purposes, and we cannot easily get 565240116Smarcel rid of them. This way we remove several critical and delicate pieces of 566240116Smarcel code. 567240116Smarcel 568240116Smarcel* Improved atf-report's CSV output format to include information about 569240116Smarcel test programs too. 570240116Smarcel 571240116Smarcel* Fixed the tests that used atf-compile to not require this tool as a 572240116Smarcel helper. Avoids systems without build-time utilities to skip many tests 573240116Smarcel that could otherwise be run. (E.g. NetBSD without the comp.tgz set 574240116Smarcel installed.) 575240116Smarcel 576240116Smarcel* Many general cleanups: Fixed many pieces of code marked as ugly and/or 577240116Smarcel incomplete. 578240116Smarcel 579240116Smarcel 580240116SmarcelChanges in version 0.2 581240116Smarcel********************** 582240116Smarcel 583240116SmarcelExperimental version released on September 20th, 2007. 584240116Smarcel 585240116Smarcel* Test cases now get a known umask on entry. 586240116Smarcel 587240116Smarcel* atf-run now detects many unexpected failures caused by test programs and 588240116Smarcel reports them as bogus tests. atf-report is able to handle these new 589240116Smarcel errors and nicely reports them to the user. 590240116Smarcel 591240116Smarcel* All the data formats read and written by the tools have been 592240116Smarcel documented and cleaned up. These include those grammars that define how 593240116Smarcel the different components communicate with each other as well as the 594240116Smarcel format of files written by the developers and users: the Atffiles and the 595240116Smarcel configuration files. 596240116Smarcel 597240116Smarcel* Added the atf-version tool, a utility that displays information about 598240116Smarcel the currently installed version of ATF. 599240116Smarcel 600240116Smarcel* Test cases can now define an optional cleanup routine to undo their 601240116Smarcel actions regardless of their exit status. 602240116Smarcel 603240116Smarcel* atf-report now summarizes the list of failed (bogus) test programs 604240116Smarcel when using the ticker output format. 605240116Smarcel 606240116Smarcel* Test programs now capture some termination signals and clean up any 607240116Smarcel temporary files before exiting the program. 608240116Smarcel 609240116Smarcel* Multiple bug fixes and improvements all around. 610240116Smarcel 611240116Smarcel 612240116SmarcelChanges in version 0.1 613240116Smarcel********************** 614240116Smarcel 615240116SmarcelExperimental version released on August 20th, 2007. 616240116Smarcel 617240116Smarcel* First public version. This was released coinciding with the end of the 618240116Smarcel Google Summer of Code 2007 program. 619240116Smarcel 620240116Smarcel 621240116Smarcel=========================================================================== 622240116Smarcelvim: filetype=text:textwidth=75:expandtab:shiftwidth=2:softtabstop=2 623