1Mac OS X specific notes 2 3*** Instructions before Starting libMicro *** 4 5# Disable Open directory and LDAP using Directory Utility app 6# Turn off airport 7# Turn off spotlight. In terminal, execute the following: 8 launchctl unload /System/Library/LaunchDaemons/com.apple.metadata.mds.plist 9# Turn off Time Machine in System Preferences 10# Wait at least 2 minutes after boot to desktop for boot cache to settle down 11 12*** Make and run quickstart *** 13 14 make 15 ./bench >output.txt 16runs the libMicro test suite excluding the lmbench tests and gives you a text file named output.txt with the results of one run. 17 18 19 ./multiview output1.txt output2.txt >compare.html 20gives you a html file comparing two runs. 21 22*** To run libMicro testsuite with stepper disabled *** 23*** For Desktop use coreos_bench script*** 24 25To get a more consistent result of libMicro benchmark run, we need to disable the 26stepper to prevent it from causing wide variations in results. See rdar://6243819 27for details. 28 29So to run libMicro test suite with stepper disabled, use 'coreos_bench' script 30instead of 'bench' script. 31 32For example: 33./coreos_bench > output.txt 34runs the libMicro test suite excluding the lmbench tests and gives you a text file named output.txt with the results of one run, with stepper disabled. 35 36Note: 371) We need '/usr/local/bin/pstates' to disable the stepper. Install AppleInternal package 38which provides '/usr/local/bin/pstates'. 39 402) 'coreos_bench' script is used exactly like the 'bench' script. All the usage examples for 41'bench' script in this readme file also holds true for 'coreos_bench' script. 42 43 44 45 46 47*** Makefile *** 48 49The Makefile invokes Makefile.Darwin which invokes Makefile.com.Darwin. 50Just invoke make, with options if necessary, and everything should 51build correctly. The binaries are placed in a directory called 52bin-ARCH where ARCH is the default or specified when building via 53the ARCH flag. 54 55Note: 561) The binaries of apple added tests are placed in a directory called 57 apple/bin-ARCH 58 592) All the binaries under bin-ARCH and apple/bin-ARCH are code signed 60 during build. 61 62options for invoking Makefile are: 63ARCH defaults to i386 64 65 to build fat/multi architecture, specify 66 make ARCH=fat 67 the makefile will automatically build with ARCH_FLAG="-arch i386 -arch x86_64" and put the results in bin-fat 68 69 to build for ARM architecture, 70 first set an environment variable 'SDKROOT' to point to iPhone internal sdk 71 For example: 72 $export SDKROOT="/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.Internal.sdk/" 73 to build use: 74 make ARCH=ARM_ARCH where ARM_ARCH can be for e.g. armv6, armv7 75 this will put the results in bin-ARM_ARCH 76 77 to build with only two of the architectures see below 78 79ARCH_FLAG defaults to -arch $(ARCH) 80 to build fat/multi architecture, specify 81 make ARCH_FLAG="-arch i386" ARCH=fat 82 this will put the results in bin-fat 83 84OPT_FLAG defaults to -g 85 to build optimized, specify make OPT_FLAG=-Os 86 87SEMOP_FLAG defaults to -DUSE_SEMOP 88 to eliminate SEMOP usage, specify make SEMOP_FLAG= 89 this is needed on some lower-end systems (e.g. M63) 90 91These can be combined, e.g. 92 make ARCH=i386 SEMOP_FLAG= 93 94*** Before running benchmarks *** 95 96The shell script create_stuff should be run before any benchmarking 97 98this script takes care of raising the process limits which would 99otherwise cause several of the tests to fail - if not you will see: 100 Running: pipe_pst1 101 fork: Resource temporarily unavailable 102in your stderr during the runs. After you run create_stuff, the 103system then needs to be rebooted. 104 105*** running the benchmarks *** 106 107The shell script "bench" will run all the benchmarks, or you can 108pass it a parameter to run a single benchmark, e.g. 109 110*** To run libMicro on the embedded platform, use "embd_bench" script. 'embd_bench' script is used exactly like the 'bench' script. All the usage examples for 111'bench' script in this readme file also holds true for 'embd_bench' script. *** 112 113 bench lmbench_bw_unix 114 115By default the script will run only the libMicro testsuite excluding the lmbench tests. 116To run the libmicro testsuite with the lmbench tests included, just pass the -l parameter. e.g, 117 118 bench -l 119To run only the lmbench testsuite 120 121 bench lmbench 122 123To display the usage, just do 124 bench -h 125 126Watch for: 127 # WARNINGS 128 # Quantization error likely;increase batch size (-B option) 4X to avoid. 129in the output 130To see an example run the supplied testbench script 131 132Add or adjust the -B parameter for any benchmark that fails. The 133Quantization error will refer to the benchmark preceding the error, 134not the one following... 135 136A typical run: 137 $ make clean 138 $ make 139 $ ./create_stuff 140 $ ./bench > output1 141 Running: getpid 142 for 0.13353 seconds 143 Running: getppid 144 for 3.65609 seconds 145 Running: getenv 146 for 0.20924 seconds 147 Running: getenvT2 148 for 0.37437 seconds 149 Running: gettimeofday 150 for 0.58077 seconds 151 etc... 152 153Use the supplied multiview script to compare runs like: 154 155multiview output1 output2 > compare.html 156open compare.html (safari launches) 157will show output2 results as a percentage change from the output1 results 158 159*** Adding additional benchmark tests *** 160 161Look at the sample file trivial.c. This demonstrates how to do 162argument passing, the flow of control of a benchmark, etc. for the 163trivial case. The tests starting with "lmbench_" were ported from 164the lmbench suite, so they might be good examples as well. 165 166*** A note regarding future changes in bench.sh script *** 167coreos_bench.sh script is almost identical to bench.sh script, except that it 168has additional code to disable the stepper during libmicro benchmark run. 169 170In future, if bench.sh script is modified, make sure the changes reflect 171in coreos_bench.sh script also. 172 173*** Things to do *** 174 175* port the rest of the lmbench benchmarks into this framework 176 177* create website that will allow easy ability to compare many builds 178across many machines with historical repository of runs 179 180* document better how to write a benchmark for this framework 181(started in trivial.c) 182 183* check this into xnu/test 184 185* create new benchmarks 186 187*** Leopard notes *** 188 189Due to rdar://4654956 and its original, rdar://2588252 you cannot 190run these tests on Leopard without removing the cascade_lockf test. 191There may be other tests which panic a Leopard system. 192 193*** benchDS notes *** 194 195From rdar://problem/7468995 add the ability to benchmark the key APIs 196for server daemons. In particular, a test binary is added for each of: 197 198 ODQueryCreateWithNode() (standard User, Groups, and Hosts records) 199 getaddrinfo() (hosts and ports) 200 mbr_check_service_membership() 201 mbr_check_membership() 202 getpwnam() 203 getpwuid() 204 getgrgid() 205 getpwent() 206 getgrent() 207 getgrnam() 208 209The script benchDS is provided to run a standard set of tests presuming 210that the tests are run by root on a system configured with an OD binding. 211The OD server (local or remote) must have a set of accounts created with 212od_acount_create shell script. This script must also be run as root, 213and passed a single argument of the number of users to create. It creates 214od_test_{1..N}, and all belong to a ds_test_group1(gid 1211). In addition, 215ds_test_group2(gid 1212) is created which has no users as members. User ids are 216set sequentially from 5000. In order to administer the OD server, it assumes 217user 'diradmin' and password 'admin' are the OD admin. 218 219Also, these tests consult the APIs listed, which can be run against the local 220account info, or even Active Directory. 221 222Thus, the quick recipe is: 223 Install X Server 224 Enable OD, and create directory admin user 'diradmin' with password 'admin' 225 As root run: od_account_create 1000 226 Now run the test, as root: ./benchDS 1000 > output-file 227 228 229In addition, od_account_delete 1000 will delete the 1000 users created with od_account_create. 230 231 232