xnu_quick_test - this tool will do a quick test of every (well, to be honest most) system calls we support in xnu. WARNING - this is not meant to be a full regression test of all the system calls. The intent is to have a quick test of each system call that can be run very easily and quickly when a new kernel is built. This tool is meant to grow as we find xnu problems that could have be caught before we submit to a build train. So please add more tests and make the existing ones better. Some of the original tests are nothing more than place holders and quite lame. Just keep in mind that the tool should run as fast as possible. If it gets too slow then most people will stop running it. LP64 testing tip - when adding or modifying tests, keep in mind the variants in the LP64 world. If xnu gets passed a structure the varies in size between 32 and 64-bit processes, try to test that a field in the structure contains valid data. For example, if we know foo structure looks like: struct foo { int an_int; long a_long; int another_int; } And if we know what another_int should contain then test for the known value since it's offset will vary depending on whether the calling process is 32 or 64 bits. NOTE - we have several workarounds and test exceptions for some outstanding bugs in xnu. All the workarounds are marked with "todo" and some comments noting the radar number of the offending bug. Do a search for "todo" in the source files for this project to locate which tests have known failures. And please tag any new exceptions you find with "todo" in the comment and the radar number of the bug. To build a fat binary, export ARCH="i386 x86_64". This will work for any architectures that Apple gcc recognizes. Added four defines which you can use at the compile line to build variants. DEBUG turn on additional printfs CONFORMANCE_TESTS_IN_XNU when conformance tests are in xnu, set this to 1 TEST_SYSTEM_CALLS test system calls (doesn't compile; a different bug) RUN_UNDER_TESTBOTS when running under testbots, set this to 1 by default, all four are set to 0, i.e. disabled. To build, export MORECFLAGS with the values you want set, e.g. export MORECFLAGS="-D DEBUG=1 -D CONFORMANCE_TESTS_IN_XNU=1" todo: -- have folks with area expertise fix lame tests (most of the networking related tests are pretty lame) -- mach system calls support USAGE: xnu_quick_test -target TARGET_PATH -f[ailures] MAX_FAILS_ALLOWED # number of test cases that may fail before we give up. defaults to 0 -l[ist] # list all the tests this tool performs -r[un] 1, 3, 10 - 19 # run specific tests. enter individual test numbers and/or range of numbers. use -list to list tests. -s[kip] # skip setuid tests -t[arget] TARGET_PATH # path to directory where tool will create test files. defaults to "/tmp/" -x[ilog] # To run the xnu_quick_test with xilog reporting enabled. examples: --- Place all test files and directories at the root of volume "test_vol" --- xnu_quick_test -t /Volumes/test_vol/ --- Run the tool for tests 10 thru 15, test 18 and test 20 --- xnu_quick_test -r 10-15, 18, 20 --- example of running the tool to list all the tests it currently supports --- xnu_quick_test -l List of all tests this tool performs... 1 syscall 2 fork, wait4, exit 3 fsync, ftruncate, lseek, pread, pwrite, read, readv, truncate, write, writev 4 close, fpathconf, fstat, open, pathconf 5 link, stat, unlink 6 chdir, fchdir 7 access, chmod, fchmod 8 chown, fchown, lchown, lstat, readlink, symlink 9 fstatfs, getattrlist, getfsstat, statfs 10 getpid, getppid, pipe 11 getauid, gettid, getuid, geteuid, issetugid, setaudit_addr, seteuid, settid, settid_with_pid, setuid 12 mkdir, rmdir, umask 13 mknod, sync 14 fsync, getsockopt, poll, select, setsockopt, socketpair 15 accept, bind, connect, getpeername, getsockname, listen, socket, recvmsg, sendmsg, sendto, sendfile 16 chflags, fchflags 17 kill, vfork, execve 18 getegid, getgid, getgroups, setegid, setgid, setgroups 19 dup, dup2, getdtablesize 20 getrusage, profil 21 getitimer, setitimer, sigaction, sigpending, sigprocmask, sigsuspend, sigwait 22 acct 23 ioctl 24 chroot 25 madvise, mincore, minherit, mlock, mlock, mmap, mprotect, msync, munmap 26 getpgrp, getpgid, getsid, setpgid, setpgrp, setsid 27 fcntl 28 getlogin, setlogin 29 getpriority, setpriority 30 futimes, gettimeofday, settimeofday, utimes 31 rename, stat 32 flock 33 mkfifo, read, write 34 quotactl 35 getrlimit, setrlimit 36 getattrlist, getdirentries, getdirentriesattr, setattrlist 37 exchangedata 38 searchfs 39 sem_close, sem_open, sem_post, sem_trywait, sem_unlink, sem_wait 40 semctl, semget, semop 41 shm_open, shm_unlink 42 shmat, shmctl, shmdt, shmget 43 fgetxattr, flistxattr, fremovexattr, fsetxattr, getxattr, listxattr, removexattr, setxattr 44 aio_cancel, aio_error, aio_read, aio_return, aio_suspend, aio_write, fcntl, lio_listio 45 kevent, kqueue 46 msgctl, msgget, msgrcv, msgsnd --- example of a full test run --- cottje% ./BUILD/dst/xnu_quick_test created test directory at "/tmp/xnu_quick_test-1660251855" Will allow 0 failures before testing is aborted Begin testing - Tue Dec 13 15:56:50 2005 test #1 - syscall test #2 - fork, wait4, exit test #3 - fsync, ftruncate, lseek, pread, pwrite, read, readv, truncate, write, writev test #4 - close, fpathconf, fstat, open, pathconf test #5 - link, stat, unlink test #6 - chdir, fchdir test #7 - access, chmod, fchmod test #8 - chown, fchown, lchown, lstat, readlink, symlink test #9 - fstatfs, getattrlist, getfsstat, statfs test #10 - getpid, getppid, pipe test #11 - getauid, gettid, getuid, geteuid, issetugid, setauid, seteuid, settid, settid_with_pid, setuid test #12 - mkdir, rmdir, umask test #13 - mknod, sync test #14 - fsync, getsockopt, poll, select, setsockopt, socketpair test #15 - accept, bind, connect, getpeername, getsockname, listen, socket, recvmsg, sendmsg, sendto test #16 - chflags, fchflags test #17 - kill, vfork, execve test #18 - getegid, getgid, getgroups, setegid, setgid, setgroups test #19 - dup, dup2, getdtablesize test #20 - getrusage, profil test #21 - ktrace test #22 - getitimer, setitimer, sigaction, sigpending, sigprocmask, sigsuspend, sigwait test #23 - acct test #24 - ioctl test #25 - chroot test #26 - madvise, mincore, minherit, mlock, mlock, mmap, mprotect, msync, munmap test #27 - getpgrp, getpgid, getsid, setpgid, setpgrp, setsid test #28 - fcntl test #29 - getlogin, setlogin test #30 - getpriority, setpriority test #31 - futimes, gettimeofday, settimeofday, utimes test #32 - rename, stat test #33 - flock test #34 - mkfifo, read, write test #35 - quotactl test #36 - getrlimit, setrlimit test #37 - getattrlist, getdirentries, getdirentriesattr, setattrlist test #38 - exchangedata test #39 - searchfs test #40 - sem_close, sem_open, sem_post, sem_trywait, sem_unlink, sem_wait test #41 - semctl, semget, semop test #42 - shm_open, shm_unlink test #43 - shmat, shmctl, shmdt, shmget test #44 - fgetxattr, flistxattr, fremovexattr, fsetxattr, getxattr, listxattr, removexattr, setxattr test #45 - aio_cancel, aio_error, aio_read, aio_return, aio_suspend, aio_write, fcntl, lio_listio test #46 - kevent, kqueue test #47 - msgctl, msgget, msgrcv, msgsnd End testing - Tue Dec 13 15:57:08 2005