1Contents: Changes to Jam 2.5rc3. 2Author: Ingo Weinhold (bonefish@users.sf.net) 3 4This version of Jam is NOT the original one distributed by Perforce 5(www.perforce.com). This file lists its differences to the original version 62.5rc3. The patches have originally been applied to version 2.4 and had to be 7adjusted more or less to work with 2.5rc3. 8 9* Jamfile tree processing 10 11 Changes to Jambase, compile.{c,h}, scan.c, jamgram.yy. Jam does now always 12 read the whole project Jamfile tree, not only the subtree starting in the 13 subdirectory it has been invoked from. If not supplied with a target on the 14 command line, however, only the targets in that subtree are built and those 15 the former ones depend on. 16 17 18* Header Caching 19 20 Taken from `//guest/matt_armstrong/jam/patched_version/...' from the 21 public Perforce depot (`public.perforce.com:1666'). Originally implemented 22 by Craig McPheeters, and improved by Matt Armstrong. The following text 23 stems from the file LOCAL_DIFFERENCES.txt in Matt's version. 24 25 <quote> 26 This code is taken from //guest/craig_mcpheeters/jam/src/ on the 27 Perforce public depot. Many thanks to Craig McPheeters for making his 28 code available. It is delimited by the OPT_HEADER_CACHE_EXT #define 29 within the code. 30 31 Jam has a facility to scan source files for other files they might 32 include. This code implements a cache of these scans, so the entire 33 source tree need not be scanned each time jam is run. This brings the 34 following benefits: 35 36 - If a file would otherwise be scanned multiple times in a 37 single jam run (because the same file is represented by 38 multiple targets, perhaps each with a different grist), it 39 will now be scanned only once. In this way, things are 40 faster even if the cache file is not present when Jam is 41 run. 42 43 - If a cache entry is present in the cache file when Jam 44 starts, and the file has not changed since the last time it 45 was scanned, Jam will not bother to re-scan it. This 46 markedly increaces Jam startup times for large projects. 47 48 This code has improvements over Craig McPheeters' original 49 version. I've described all of these changes to Craig and he 50 intends to incorporate them back into his version. The changes 51 are: 52 53 - The actual name of the cache file is controlled by the 54 HCACHEFILE Jam variable. If HCACHEFILE is left unset (the 55 default), reading and writing of a cache file is not 56 performed. The cache is always used internally regardless 57 of HCACHEFILE, which helps when HDRGRIST causes the same 58 file to be scanned multiple times. 59 60 Setting LOCATE and SEARCH on the the HCACHEFILE works as 61 well, so you can place anywhere on disk you like or even 62 search for it in several directories. You may also set it 63 in your environment to share it amongst all your projects. 64 65 - The .jamdeps file is in a new format that allows binary data 66 to be in any of the fields, in particular the file names. 67 The original code would break if a file name contained the 68 '@' or '\n' characters. The format is also versioned, 69 allowing upgrades to automatically ignore old .jamdeps 70 files. The format remains human readable. In addition, 71 care has been taken to not add the entry into the header 72 cache until the entire record has been successfully read from 73 the file. 74 75 - The cache stores the value of HDRPATTERN with each cache 76 entry, and it is compared along with the file's date to 77 determine if there is a cache hit. If the HDRPATTERN does 78 not match, it is treated as a cache miss. This allows 79 HDRPATTERN to change without worrying about stale cache 80 entries. It also allows the same file to be scanned 81 multiple times with different HDRPATTERN values. 82 83 - Each cache entry is given an "age" which is the maximum 84 number of times a given header cache entry can go unused 85 before it is purged from the cache. This helps clean up old 86 entries in the .jamdeps file when files move around or are 87 removed from your project. 88 89 You control the maximum age with the HCACHEMAXAGE variable. 90 If set to 0, no cache aging is performed. Otherwise it is 91 the number of times a jam must be run before an unused cache 92 entry is purged. The default for HCACHEMAXAGE if left unset 93 is 100. 94 95 - Jambase itself is changed. 96 97 SubDir now always sets HDRGRIST to $(SOURCE_GRIST) so header 98 scanning can deal with multiple header files of the same 99 name in different directories. With the header cache, this 100 does no longer incurs a performance penalty -- a given file 101 will still only be scanned once. 102 103 The FGristSourceFiles rule is now just an alias for 104 FGristFiles. Header files do not necessarily have global 105 visibility, and the header cache eliminates any performance 106 penalty this might otherwise incur. 107 108 Because of all these improvements, the following claims can be 109 made about this header cache implementation that can not be made 110 about Craig McPheeters' original version. 111 112 - The semantics of a Jam run will never be different because of 113 the header cache (the HDRPATTERN check ensures this). 114 115 - It will never be necessary to delete .jamdeps to fix obscure 116 jam problems or purge old entries. 117 </quote> 118 119 120* Jamfile Caching 121 122 As large build systems may consist of a huge number of Jamfiles, the 123 mere reading of these files may take considerable time. This version 124 implements a cache for them. Since the time stamps of the files still 125 need to retrieved to check whether the cached entries are still up to 126 date, the benefits to be expected are not that big though. 127 128 The name of the cache file is controlled by the JCACHEFILE Jam variable. 129 If JCACHEFILE is left unset (the default), reading and writing of a cache 130 file is not performed. Setting the SEARCH and LOCATE variables does work 131 as expected. 132 133 134* Stat Data and Directory Caching Server (BeOS only) 135 136 Also an optimization for large build systems. Since the BeOS FS cache 137 is terrible, stat()ing targets to get their timestamp or see if they exist 138 at all, and reading directories usually happens on disk, since the data 139 from the previous run are already out of the cache, if the build system 140 is large enough. 141 142 This change externalizes all stat()ing and directory reading into a 143 dedicated server process which caches the data, so that they can be 144 served from memory the next time they are requested. The server uses 145 the BeOS node monitoring to keep the data up to date. 146 147 The feature particularly leverages the header and jamfile caching, since 148 after the first run the timestamps of the jamfiles and headers are 149 cached too, so that reading the jamfiles and performing the header 150 scanning doesn't require any disk accesses at all (besides reading the 151 cache files, of course). 152 153 Drawbacks are that the first run of jam will be slower, mainly due to 154 the communication overhead with the server, and that the server consumes 155 memory to store the cached data. The server's memory footprint is quite 156 reasonable, though. 157 158* Disabled the "..skipped x for lack of y..." message 159 Disabled as it is not very useful information and hides the interesting 160 info in noise (why it failed). It should probably be a command line option 161 as it might be interesting in some cases. Also added a "Build Failure" at 162 the end if there were failed targets. (Fredrik Holmqvist) 163* Only write "...patience..." every 10000th target instead of 1000th. 164 (Fredrik Holmqvist) 165