1Contents: Changes to Jam 2.5rc3.
2Author:   Ingo Weinhold (bonefish@users.sf.net)
3
4This version of Jam is NOT the original one distributed by Perforce
5(www.perforce.com). This file lists its differences to the original version
62.5rc3. The patches have originally been applied to version 2.4 and had to be
7adjusted more or less to work with 2.5rc3.
8
9* Jamfile tree processing
10
11  Changes to Jambase, compile.{c,h}, scan.c, jamgram.yy. Jam does now always
12  read the whole project Jamfile tree, not only the subtree starting in the
13  subdirectory it has been invoked from. If not supplied with a target on the
14  command line, however, only the targets in that subtree are built and those
15  the former ones depend on.
16
17
18* Header Caching
19
20  Taken from `//guest/matt_armstrong/jam/patched_version/...' from the
21  public Perforce depot (`public.perforce.com:1666'). Originally implemented
22  by Craig McPheeters, and improved by Matt Armstrong. The following text
23  stems from the file LOCAL_DIFFERENCES.txt in Matt's version.
24
25  <quote>
26    This code is taken from //guest/craig_mcpheeters/jam/src/ on the
27    Perforce public depot.  Many thanks to Craig McPheeters for making his
28    code available.  It is delimited by the OPT_HEADER_CACHE_EXT #define
29    within the code.
30
31    Jam has a facility to scan source files for other files they might
32    include.  This code implements a cache of these scans, so the entire
33    source tree need not be scanned each time jam is run.  This brings the
34    following benefits:
35
36	- If a file would otherwise be scanned multiple times in a
37	  single jam run (because the same file is represented by
38	  multiple targets, perhaps each with a different grist), it
39	  will now be scanned only once.  In this way, things are
40	  faster even if the cache file is not present when Jam is
41	  run.
42
43	- If a cache entry is present in the cache file when Jam
44	  starts, and the file has not changed since the last time it
45	  was scanned, Jam will not bother to re-scan it.  This
46	  markedly increaces Jam startup times for large projects.
47
48    This code has improvements over Craig McPheeters' original
49    version.  I've described all of these changes to Craig and he
50    intends to incorporate them back into his version.  The changes
51    are:
52
53	- The actual name of the cache file is controlled by the
54	  HCACHEFILE Jam variable.  If HCACHEFILE is left unset (the
55	  default), reading and writing of a cache file is not
56	  performed.  The cache is always used internally regardless
57	  of HCACHEFILE, which helps when HDRGRIST causes the same
58	  file to be scanned multiple times.
59
60	  Setting LOCATE and SEARCH on the the HCACHEFILE works as
61	  well, so you can place anywhere on disk you like or even
62	  search for it in several directories.  You may also set it
63	  in your environment to share it amongst all your projects.
64
65	- The .jamdeps file is in a new format that allows binary data
66	  to be in any of the fields, in particular the file names.
67	  The original code would break if a file name contained the
68	  '@' or '\n' characters.  The format is also versioned,
69	  allowing upgrades to automatically ignore old .jamdeps
70	  files.  The format remains human readable.  In addition,
71	  care has been taken to not add the entry into the header
72	  cache until the entire record has been successfully read from
73	  the file.
74
75	- The cache stores the value of HDRPATTERN with each cache
76	  entry, and it is compared along with the file's date to
77	  determine if there is a cache hit.  If the HDRPATTERN does
78	  not match, it is treated as a cache miss.  This allows
79	  HDRPATTERN to change without worrying about stale cache
80	  entries.  It also allows the same file to be scanned
81	  multiple times with different HDRPATTERN values.
82
83	- Each cache entry is given an "age" which is the maximum
84	  number of times a given header cache entry can go unused
85	  before it is purged from the cache.  This helps clean up old
86	  entries in the .jamdeps file when files move around or are
87	  removed from your project.
88
89	  You control the maximum age with the HCACHEMAXAGE variable.
90	  If set to 0, no cache aging is performed.  Otherwise it is
91	  the number of times a jam must be run before an unused cache
92	  entry is purged.  The default for HCACHEMAXAGE if left unset
93	  is 100.
94
95	- Jambase itself is changed.
96
97	  SubDir now always sets HDRGRIST to $(SOURCE_GRIST) so header
98	  scanning can deal with multiple header files of the same
99	  name in different directories.  With the header cache, this
100	  does no longer incurs a performance penalty -- a given file
101	  will still only be scanned once.
102
103	  The FGristSourceFiles rule is now just an alias for
104	  FGristFiles.  Header files do not necessarily have global
105	  visibility, and the header cache eliminates any performance
106	  penalty this might otherwise incur.
107
108    Because of all these improvements, the following claims can be
109    made about this header cache implementation that can not be made
110    about Craig McPheeters' original version.
111
112	- The semantics of a Jam run will never be different because of
113	  the header cache (the HDRPATTERN check ensures this).
114
115	- It will never be necessary to delete .jamdeps to fix obscure
116	  jam problems or purge old entries.
117  </quote>
118
119
120* Jamfile Caching
121
122  As large build systems may consist of a huge number of Jamfiles, the
123  mere reading of these files may take considerable time. This version
124  implements a cache for them. Since the time stamps of the files still
125  need to retrieved to check whether the cached entries are still up to
126  date, the benefits to be expected are not that big though.
127
128  The name of the cache file is controlled by the JCACHEFILE Jam variable.
129  If JCACHEFILE is left unset (the default), reading and writing of a cache
130  file is not performed. Setting the SEARCH and LOCATE variables does work
131  as expected.
132
133
134* Stat Data and Directory Caching Server (BeOS only)
135
136  Also an optimization for large build systems. Since the BeOS FS cache
137  is terrible, stat()ing targets to get their timestamp or see if they exist
138  at all, and reading directories usually happens on disk, since the data
139  from the previous run are already out of the cache, if the build system
140  is large enough.
141
142  This change externalizes all stat()ing and directory reading into a
143  dedicated server process which caches the data, so that they can be
144  served from memory the next time they are requested. The server uses
145  the BeOS node monitoring to keep the data up to date.
146
147  The feature particularly leverages the header and jamfile caching, since
148  after the first run the timestamps of the jamfiles and headers are
149  cached too, so that reading the jamfiles and performing the header
150  scanning doesn't require any disk accesses at all (besides reading the
151  cache files, of course).
152
153  Drawbacks are that the first run of jam will be slower, mainly due to
154  the communication overhead with the server, and that the server consumes
155  memory to store the cached data. The server's memory footprint is quite
156  reasonable, though.
157
158* Disabled the "..skipped x for lack of y..." message
159  Disabled as it is not very useful information and hides the interesting
160  info in noise (why it failed). It should probably be a command line option
161  as it might be interesting in some cases. Also added a "Build Failure" at
162  the end if there were failed targets. (Fredrik Holmqvist)
163* Only write "...patience..." every 10000th target instead of 1000th.
164  (Fredrik Holmqvist)
165