#
303975 |
|
11-Aug-2016 |
gjb |
Copy stable/11@r303970 to releng/11.0 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, and rename it to RC1.
Update __FreeBSD_version.
Use the quarterly branch for the default FreeBSD.conf pkg(8) repo and the dvd1.iso packages population.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation |
#
302408 |
|
08-Jul-2016 |
gjb |
Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle. Prune svn:mergeinfo from the new branch, as nothing has been merged here.
Additional commits post-branch will follow.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation
|
#
301286 |
|
03-Jun-2016 |
bdrewery |
WITH_CCACHE_BUILD + WITH_META_MODE: Ignore ccache changes.
Ccache will not affect the output of the objects, so just ignore it for meta mode handling. This avoids having everything rebuild if ccache is updated.
Sponsored by: EMC / Isilon Storage Division
|
#
300943 |
|
29-May-2016 |
bdrewery |
GCC External: Revert r300886, r300904, r300917, r300918
The fix in r300873 is mostly enough. A fix for lib32 will be committed.separately.
|
#
300904 |
|
28-May-2016 |
bdrewery |
Always export X_* vars.
This fixes CROSS_TOOLCHAIN builds after r300886 since it relies on X_COMPILER_TYPE being set.
The X_* vars will only represent the external compiler being used.
Sponsored by: EMC / Isilon Storage Division
|
#
300525 |
|
23-May-2016 |
bdrewery |
Use sed(1) to determine cc version rather than tail(1) since it is already in ITOOLS.
This fixes 'tail: not found' warnings in installworld after r300351.
Sponsored by: EMC / Isilon Storage Division
|
#
300351 |
|
21-May-2016 |
bdrewery |
Fetch the __FreeBSD_cc_version as COMPILER_FREEBSD_VERSION.
Reviewed by: brooks, bapt, dim Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D6356
|
#
300350 |
|
21-May-2016 |
bdrewery |
Auto determine X_COMPILER_TYPE/X_COMPILER_VERSION if XCC is set.
Reviewed by: brooks, bapt Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D6355
|
#
296282 |
|
01-Mar-2016 |
bdrewery |
CCACHE_BUILD+META_MODE: Lookup and canonicalize the default CCACHE_DIR.
This is done to prevent not having CCACHE_DIR causing meta mode with filemon to see stat changes in the ccache dir and cause rebuilds.
Sponsored by: EMC / Isilon Storage Division
|
#
293014 |
|
31-Dec-2015 |
dim |
Merge r293006 from clang380-import branch:
For determining the compiler version, quote the string to be echo'd, otherwise the command might fail. This is because clang -v now results in the following:
FreeBSD clang version 3.8.0 (trunk 256633) (based on LLVM 3.8.0svn)
The second "3.8.8svn)" string tripped up the shell command.
MFC after: 3 days
|
#
292002 |
|
08-Dec-2015 |
bdrewery |
CCACHE_BUILD: Only export CCACHE_PATH= if it was already set with a value.
Older ccache don't work with an empty CCACHE_PATH value. They will error with: ccache: FATAL: Could not find compiler "cc" in PATH make: "/mnt/bdrewery/git/onefs/src/share/mk/bsd.compiler.mk" line 134: Unable to determine compiler type for /usr/local/bin/ccache cc. Consider setting COMPILER_TYPE.
Sponsored by: EMC / Isilon Storage Division
|
#
291995 |
|
08-Dec-2015 |
bdrewery |
Fix some makeman issues.
- Don't bother looking up REVISION/BRANCH/etc from release/, or the CPUTYPE check, as these are not used for makeman and wastes time. The also invokes auto.obj.mk after I reverted auto.obj.mk ignoring -V in r291312. - Don't modify CC or PATH when WITH_CCACHE_BUILD or WITH_META_MODE is enabled as it leads to bsd.compiler.mk errors.
Sponsored by: EMC / Isilon Storage Division
|
#
290526 |
|
08-Nov-2015 |
bdrewery |
Add built-in ccache build support via WITH_CCACHE_BUILD option.
ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not used to achieve a safe pseudo-incremental build. This is explained in more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the cache, but with a full cache saves 30-50% in build times. When combined with the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the resulting dependency file, which it does not do when using mkdep(1)/'CC -E'. Stats are provided at the end of this message.
This removes the need to modify /etc/make.conf with the CC:= and CXX:= lines which conflicted with external compiler support [3] (causing the bootstrap compiler to not be built which lead to obscure failures [4]), incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid Clang errors with parenthesis, and did not work with META_MODE.
The option name was picked to match the existing option in ports. This feature is available for both in-src and out-of-src builds that use /usr/share/mk.
Linking, assembly compiles, and pre-processing avoid using ccache since it is only overhead. ccache does nothing special in these modes, although there is no harm in calling it for them.
CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap compiler to hash the content of the compiler binary to determine if it should be a cache miss. For external compilers the 'mtime' option is used as it is more efficient and likely to be correct. Future work may optimize the 'content' check using the same checks as whether a bootstrap compiler is needed to be built.
The CCACHE_CPP2 pessimization is currently default in our devel/ccache port due to Clang requiring it. Clang's -Wparentheses-equality, -Wtautological-compare, and -Wself-assign warnings do not mix well with compiling already-pre-processed code that may have expanded macros that trigger the warnings. GCC has so far not had this issue so it is allowed to disable the CCACHE_CPP2 default in our port.
Sharing a cache between multiple checkouts, or systems, is explained in the ccache manual. Sharing a cache over NFS would likely not be worth it, but syncing cache directories between systems may be useful for an organization. There is also a memcached backend available [5]. Due to using an object directory outside of the source directory though you will need to ensure that both are in the same prefix and all users use the same layout. A possible working layout is as follows: Source: /some/prefix/src1 Source: /some/prefix/src2 Source: /some/prefix/src3 Objdir: /some/prefix/obj Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj' This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace all absolute paths to be relative. Using something like this is required due to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache adds into its hash for the object without CCACHE_BASEDIR.
distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc. I have not personally tested this and assume it will not mix well with using the bootstrap compiler.
The cache from buildworld can be reused in a subdir by first running 'make buildenv' (from r290424).
Note that the cache is currently different depending on whether -j is used or not due to ccache enabling -fdiagnostics-color automatically if stderr is a TTY, which bmake only does if not using -j.
The system I used for testing was: WITNESS Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log. The arc was fully populated with src tree files and ccache objects. RAM: 76GiB CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz 2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16
The WITH_FAST_DEPEND feature was used for comparison here as well to show the dramatic time savings with a full cache.
buildworld: x buildworld-before + buildworld-ccache-empty * buildworld-ccache-full % buildworld-ccache-full-fastdep # buildworld-fastdep +-------------------------------------------------------------------------------+ |% * # +| |% * # +| |% * # xxx +| | |A | | A| | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 3744.13 3794.31 3752.25 3763.5633 26.935139 + 3 4519 4525.04 4520.73 4521.59 3.1104823 Difference at 95.0% confidence 758.027 +/- 43.4565 20.1412% +/- 1.15466% (Student's t, pooled s = 19.1726) * 3 1823.08 1827.2 1825.62 1825.3 2.0785572 Difference at 95.0% confidence -1938.26 +/- 43.298 -51.5007% +/- 1.15045% (Student's t, pooled s = 19.1026) % 3 1266.96 1279.37 1270.47 1272.2667 6.3971113 Difference at 95.0% confidence -2491.3 +/- 44.3704 -66.1952% +/- 1.17895% (Student's t, pooled s = 19.5758) # 3 3153.34 3155.16 3154.2 3154.2333 0.91045776 Difference at 95.0% confidence -609.33 +/- 43.1943 -16.1902% +/- 1.1477% (Student's t, pooled s = 19.0569)
buildkernel: x buildkernel-before + buildkernel-ccache-empty * buildkernel-ccache-empty-fastdep % buildkernel-ccache-full # buildkernel-ccache-full-fastdep @ buildkernel-fastdep +-------------------------------------------------------------------------------+ |# @ % * | |# @ % * x + | |# @ % * xx ++| | MA | | MA| | A | | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 571.57 573.94 571.79 572.43333 1.3094401 + 3 727.97 731.91 728.06 729.31333 2.2492295 Difference at 95.0% confidence 156.88 +/- 4.17129 27.4058% +/- 0.728695% (Student's t, pooled s = 1.84034) * 3 527.1 528.29 528.08 527.82333 0.63516402 Difference at 95.0% confidence -44.61 +/- 2.33254 -7.79305% +/- 0.407478% (Student's t, pooled s = 1.02909) % 3 400.4 401.05 400.62 400.69 0.3306055 Difference at 95.0% confidence -171.743 +/- 2.16453 -30.0023% +/- 0.378128% (Student's t, pooled s = 0.954969) # 3 201.94 203.34 202.28 202.52 0.73020545 Difference at 95.0% confidence -369.913 +/- 2.40293 -64.6212% +/- 0.419774% (Student's t, pooled s = 1.06015) @ 3 369.12 370.57 369.3 369.66333 0.79033748 Difference at 95.0% confidence -202.77 +/- 2.45131 -35.4225% +/- 0.428227% (Student's t, pooled s = 1.0815)
[1] https://ccache.samba.org/performance.html [2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html [3] https://reviews.freebsd.org/D3484 [5] https://github.com/jrosdahl/ccache/pull/30
PR: 182944 [4] MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division Relnotes: yes
|
#
290220 |
|
31-Oct-2015 |
bdrewery |
Don't hide stderr when checking ${CC} --version.
This can have important debugging information such as 'cc: not found' or 'ccache: error: Could not find compiler "cc" in PATH'.
MFC after: 2 weeks Sponsored by: EMC / Isilon Storage Division
|
#
289663 |
|
20-Oct-2015 |
bdrewery |
Improve safety of caching from r289659 by only importing of none of the variables are already set. This should cover odd cases such as the COMPILER_TYPE override in lib/csu/powerpc64.
X-MFC-With: r289659 MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division
|
#
289659 |
|
20-Oct-2015 |
bdrewery |
Pass COMPILER_TYPE and COMPILER_VERSION to sub-makes to avoid redundant lookups.
This uses a special variable name based on a hash of ${CC}, ${PATH}, and ${MACHINE} to ensure that a cached value is not used if any of these values changes to use a new compiler.
Before this there were 34,620 fork/exec from bsd.compiler.mk during a buildworld. After this there are 608. More improvement is needed to cache a value from the top-level before descending into subdirs in the various build phases.
Reviewed by: brooks (earlier version) MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3898
|
#
284345 |
|
13-Jun-2015 |
sjg |
Add META_MODE support.
Off by default, build behaves normally. WITH_META_MODE we get auto objdir creation, the ability to start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796 Reviewed by: brooks imp
|
#
275589 |
|
08-Dec-2014 |
ngie |
The previous commit should have been a logical or not a logical "and"
Pointyhat to: me
|
#
275588 |
|
08-Dec-2014 |
ngie |
${CC} --version doesn't need to be queried if both COMPILER_TYPE and COMPILER_VERSION are known
|
#
275557 |
|
06-Dec-2014 |
ngie |
Fix typos in comments and wrap to <80 columns
MFC after: 3 days
|
#
273405 |
|
21-Oct-2014 |
bapt |
When using an external toolchain note that gcc 4.8+ supports C++11
Submitted by: imp
|
#
266587 |
|
23-May-2014 |
imp |
Allow CC to not actually exist. During the ports INDEX run, all the Makefiles are evaluated without building things. In a normal build, the prerequisites would be built, and CC would be an actual thing. In an INDEX build, though, they don't exists. Redirect stderr to get rid of annoying messages, and assume that the compiler version is 0 if the actual compiler can't tell us. Do this in preference to guessing based on numbers because gcc410 might be 4.10, or 4.1.0 and without carefully crafted special knowledge we differentiate between them easily (also ming-gcc has no clues at all). Elsewhere, don't trust the compiler version if it is 0.
|
#
265829 |
|
10-May-2014 |
imp |
Support, to the extent we generate proper command lines, compiling with clang 3.3. Useful for test building -current on a -stable system in individual directories. Potentially useful if we ever want to support, say, gcc 4.8 or 4.9's new warnings when building with an external toolchain (but such support not yet committed). Document the bsd.compiler.mk interface.
|
#
265581 |
|
07-May-2014 |
imp |
bsd.compiler.mk was implicitly included by bsd.own.mk in historical versions. With its movement to src.opts.mk, bsd.prog.mk was testing COMPILER_TYPE without including the bsd.compiler.mk anymore. In the source tree, this caused no problems, for reasons that aren't clear, but does cause problems outside of the source tree. Allow bsd.compiler.mk to be included multiple times safely, and always include bsd.compiler.mk at the top of bsd.prog.mk. Resist the urge to put it in bsd.init.mk, since that would reintroduce the implicit include.
|
#
247527 |
|
01-Mar-2013 |
brooks |
Provide slightly more helpful feedback when we can't figure out what compiler the user is using.
PR: misc/173914
|
#
242658 |
|
06-Nov-2012 |
dim |
Fix typo in bsd.compiler.mk.
|
#
242633 |
|
05-Nov-2012 |
brooks |
Allow CC to be gcc46 or similar by matching gcc*.
Don't test the value of the :M matches, either the string matches or does not.
|
#
240966 |
|
26-Sep-2012 |
brooks |
Add a COMPILER_FEATURES variable that is set in bsd.compiler.mk.
When the compiler is clang add c++11 to COMPILER_FEATURES.
When the compiler supports c++11, default to building libc++. This will not impact the compliation of programs, but will make it easier for users with clang compiled systems to test libc++ by passing -stdlib=libc++ to their c++ compliations.
|
#
240468 |
|
13-Sep-2012 |
brooks |
Introduce a new make variable COMPILER_TYPE that specifies what type of compiler is being used (currently clang or gcc). COMPILER_TYPE is set in the new bsd.compiler.mk file based on the value of the CC variable or, should it prove informative, by running ${CC} --version and examining the output.
To avoid negative performance impacts in the default case and correct value for COMPILER_TYPE type is determined and passed in the environment of submake instances while building world.
Replace adhoc attempts at determining the compiler type by examining CC or MK_CLANG_IS_CC with checks of COMPILER_TYPE. This eliminates bootstrapping complications when first setting WITH_CLANG_IS_CC.
Sponsored by: DARPA, AFRL Reviewed by: Yamaya Takashi <yamayan@kbh.biglobe.ne.jp>, imp, linimon (with some modifications post review) MFC after: 2 weeks
|