build_hacking.xml revision 1.8
1<section xmlns="http://docbook.org/ns/docbook" version="5.0" 2 xml:id="appendix.porting.build_hacking" xreflabel="Build Hacking"> 3<?dbhtml filename="build_hacking.html"?> 4 5<info><title>Configure and Build Hacking</title> 6 <keywordset> 7 <keyword>C++</keyword> 8 <keyword>build</keyword> 9 <keyword>configure</keyword> 10 <keyword>hacking</keyword> 11 <keyword>version</keyword> 12 <keyword>dynamic</keyword> 13 <keyword>shared</keyword> 14 </keywordset> 15</info> 16 17<section xml:id="build_hacking.prereq"><info><title>Prerequisites</title></info> 18 19 <para> 20 As noted <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/install/prerequisites.html">previously</link>, 21 certain other tools are necessary for hacking on files that 22 control configure (<code>configure.ac</code>, 23 <code>acinclude.m4</code>) and make 24 (<code>Makefile.am</code>). These additional tools 25 (<code>automake</code>, and <code>autoconf</code>) are further 26 described in detail in their respective manuals. All the libraries 27 in GCC try to stay in sync with each other in terms of versions of 28 the auto-tools used, so please try to play nicely with the 29 neighbors. 30 </para> 31</section> 32 33<section xml:id="build_hacking.overview"> 34<info><title>Overview</title></info> 35 36<section xml:id="build_hacking.overview.basic"> 37<info><title>General Process</title></info> 38 39<para> 40 The configure process begins the act of building libstdc++, and is 41 started via: 42</para> 43 44<screen> 45<computeroutput> 46configure 47</computeroutput> 48</screen> 49 50<para> 51The <filename>configure</filename> file is a script generated (via 52<command>autoconf</command>) from the file 53<filename>configure.ac</filename>. 54</para> 55 56 57<para> 58 After the configure process is complete, 59</para> 60 61<screen> 62<computeroutput> 63make all 64</computeroutput> 65</screen> 66 67<para> 68in the build directory starts the build process. The <literal>all</literal> target comes from the <filename>Makefile</filename> file, which is generated via <command>configure</command> from the <filename>Makefile.in</filename> file, which is in turn generated (via 69<command>automake</command>) from the file 70<filename>Makefile.am</filename>. 71</para> 72 73</section> 74 75 76<section xml:id="build_hacking.overview.map"><info><title>What Comes from Where</title></info> 77 78 79 <figure xml:id="fig.build_hacking.deps"> 80 <title>Configure and Build File Dependencies</title> 81 <mediaobject> 82 <imageobject> 83 <imagedata align="center" format="PDF" scale="75" fileref="../images/confdeps.pdf"/> 84 </imageobject> 85 <imageobject> 86 <imagedata align="center" format="PNG" scale="100" fileref="../images/confdeps.png"/> 87 </imageobject> 88 <textobject> 89 <phrase>Dependency Graph for Configure and Build Files</phrase> 90 </textobject> 91 </mediaobject> 92 </figure> 93 94 <para> 95 Regenerate all generated files by using the command 96 <command>autoreconf</command> at the top level of the libstdc++ source 97 directory. 98 </para> 99</section> 100 101</section> <!-- overview --> 102 103 104<section xml:id="build_hacking.configure"> 105<info><title>Configure</title></info> 106 107<section xml:id="build_hacking.configure.scripts"><info><title>Storing Information in non-AC files (like configure.host)</title></info> 108 109 110 <para> 111 Until that glorious day when we can use <literal>AC_TRY_LINK</literal> 112 with a cross-compiler, we have to hardcode the results of what the tests 113 would have shown if they could be run. So we have an inflexible 114 mess like <filename>crossconfig.m4</filename>. 115 </para> 116 117 <para> 118 Wouldn't it be nice if we could store that information in files 119 like configure.host, which can be modified without needing to 120 regenerate anything, and can even be tweaked without really 121 knowing how the configury all works? Perhaps break the pieces of 122 <filename>crossconfig.m4</filename> out and place them in their appropriate 123 <filename class="directory">config/{cpu,os}</filename> directory. 124 </para> 125 126 <para> 127 Alas, writing macros like 128 "<code>AC_DEFINE(HAVE_A_NICE_DAY)</code>" can only be done inside 129 files which are passed through autoconf. Files which are pure 130 shell script can be source'd at configure time. Files which 131 contain autoconf macros must be processed with autoconf. We could 132 still try breaking the pieces out into "config/*/cross.m4" bits, 133 for instance, but then we would need arguments to aclocal/autoconf 134 to properly find them all when generating configure. I would 135 discourage that. 136</para> 137</section> 138 139<section xml:id="build_hacking.configure.conventions"><info><title>Coding and Commenting Conventions</title></info> 140 141 142 <para> 143 Most comments should use {octothorpes, shibboleths, hash marks, 144 pound signs, whatever} rather than "<literal>dnl</literal>". 145 Nearly all comments in <filename>configure.ac</filename> should. 146 Comments inside macros written in ancillary 147 <filename class="extension">.m4</filename> files should. 148 About the only comments which should <emphasis>not</emphasis> 149 use <literal>#</literal>, but use <literal>dnl</literal> instead, 150 are comments <emphasis>outside</emphasis> our own macros in the ancillary 151 files. The difference is that <literal>#</literal> comments show up in 152 <filename>configure</filename> (which is most helpful for debugging), 153 while <literal>dnl</literal>'d lines just vanish. Since the macros 154 in ancillary files generate code which appears in odd places, 155 their "outside" comments tend to not be useful while reading 156 <filename>configure</filename>. 157 </para> 158 159 <para> 160 Do not use any <code>$target*</code> variables, such as 161 <varname>$target_alias</varname>. The single exception is in 162 <filename>configure.ac</filename>, for automake+dejagnu's sake. 163 </para> 164</section> 165 166<section xml:id="build_hacking.configure.acinclude"><info><title>The acinclude.m4 layout</title></info> 167 168 <para> 169 The nice thing about 170 <filename>acinclude.m4</filename>/<filename>aclocal.m4</filename> 171 is that macros aren't 172 actually performed/called/expanded/whatever here, just loaded. So 173 we can arrange the contents however we like. As of this writing, 174 <filename>acinclude.m4</filename> is arranged as follows: 175 </para> 176 <programlisting> 177 GLIBCXX_CHECK_HOST 178 GLIBCXX_TOPREL_CONFIGURE 179 GLIBCXX_CONFIGURE 180 </programlisting> 181 <para> 182 All the major variable "discovery" is done here. 183 <varname>CXX</varname>, multilibs, 184 etc. 185 </para> 186 <programlisting> 187 fragments included from elsewhere 188 </programlisting> 189 <para> 190 Right now, "fragments" == "the math/linkage bits". 191 </para> 192<programlisting> 193 GLIBCXX_CHECK_COMPILER_FEATURES 194 GLIBCXX_CHECK_LINKER_FEATURES 195 GLIBCXX_CHECK_WCHAR_T_SUPPORT 196</programlisting> 197<para> 198 Next come extra compiler/linker feature tests. Wide character 199 support was placed here because I couldn't think of another place 200 for it. It will probably get broken apart like the math tests, 201 because we're still disabling wchars on systems which could actually 202 support them. 203</para> 204<programlisting> 205 GLIBCXX_CHECK_SETRLIMIT_ancilliary 206 GLIBCXX_CHECK_SETRLIMIT 207 GLIBCXX_CHECK_S_ISREG_OR_S_IFREG 208 GLIBCXX_CHECK_POLL 209 GLIBCXX_CHECK_WRITEV 210 211 GLIBCXX_CONFIGURE_TESTSUITE 212</programlisting> 213<para> 214 Feature tests which only get used in one place. Here, things used 215 only in the testsuite, plus a couple bits used in the guts of I/O. 216</para> 217<programlisting> 218 GLIBCXX_EXPORT_INCLUDES 219 GLIBCXX_EXPORT_FLAGS 220 GLIBCXX_EXPORT_INSTALL_INFO 221</programlisting> 222<para> 223 Installation variables, multilibs, working with the rest of the 224 compiler. Many of the critical variables used in the makefiles are 225 set here. 226</para> 227<programlisting> 228 GLIBGCC_ENABLE 229 GLIBCXX_ENABLE_C99 230 GLIBCXX_ENABLE_CHEADERS 231 GLIBCXX_ENABLE_CLOCALE 232 GLIBCXX_ENABLE_CONCEPT_CHECKS 233 GLIBCXX_ENABLE_CSTDIO 234 GLIBCXX_ENABLE_CXX_FLAGS 235 GLIBCXX_ENABLE_C_MBCHAR 236 GLIBCXX_ENABLE_DEBUG 237 GLIBCXX_ENABLE_DEBUG_FLAGS 238 GLIBCXX_ENABLE_LONG_LONG 239 GLIBCXX_ENABLE_PCH 240 GLIBCXX_ENABLE_SYMVERS 241 GLIBCXX_ENABLE_THREADS 242</programlisting> 243<para> 244 All the features which can be controlled with enable/disable 245 configure options. Note how they're alphabetized now? Keep them 246 like that. :-) 247</para> 248<programlisting> 249 AC_LC_MESSAGES 250 libtool bits 251</programlisting> 252<para> 253 Things which we don't seem to use directly, but just has to be 254 present otherwise stuff magically goes wonky. 255</para> 256 257</section> 258 259<section xml:id="build_hacking.configure.enable"><info><title><constant>GLIBCXX_ENABLE</constant>, the <literal>--enable</literal> maker</title></info> 260 261 262 <para> 263 All the <literal>GLIBCXX_ENABLE_FOO</literal> macros use a common 264 helper, <literal>GLIBCXX_ENABLE</literal>. (You don't have to use 265 it, but it's easy.) The helper does two things for us: 266 </para> 267 268<orderedlist> 269 <listitem> 270 <para> 271 Builds the call to the <literal>AC_ARG_ENABLE</literal> macro, with 272 <option>--help</option> text 273 properly quoted and aligned. (Death to changequote!) 274 </para> 275 </listitem> 276 <listitem> 277 <para> 278 Checks the result against a list of allowed possibilities, and 279 signals a fatal error if there's no match. This means that the 280 rest of the <literal>GLIBCXX_ENABLE_FOO</literal> macro doesn't need to test for 281 strange arguments, nor do we need to protect against 282 empty/whitespace strings with the <code>"x$foo" = "xbar"</code> 283 idiom. 284 </para> 285 </listitem> 286</orderedlist> 287 288<para>Doing these things correctly takes some extra autoconf/autom4te code, 289 which made our macros nearly illegible. So all the ugliness is factored 290 out into this one helper macro. 291</para> 292 293<para>Many of the macros take an argument, passed from when they are expanded 294 in configure.ac. The argument controls the default value of the 295 enable/disable switch. Previously, the arguments themselves had defaults. 296 Now they don't, because that's extra complexity with zero gain for us. 297</para> 298 299<para>There are three "overloaded signatures". When reading the descriptions 300 below, keep in mind that the brackets are autoconf's quotation characters, 301 and that they will be stripped. Examples of just about everything occur 302 in acinclude.m4, if you want to look. 303</para> 304 305<programlisting> 306 GLIBCXX_ENABLE (FEATURE, DEFAULT, HELP-ARG, HELP-STRING) 307 GLIBCXX_ENABLE (FEATURE, DEFAULT, HELP-ARG, HELP-STRING, permit a|b|c) 308 GLIBCXX_ENABLE (FEATURE, DEFAULT, HELP-ARG, HELP-STRING, SHELL-CODE-HANDLER) 309</programlisting> 310 311<itemizedlist> 312 <listitem> 313 <para> 314 <literal>FEATURE</literal> is the string that follows 315 <option>--enable</option>. The results of the 316 test (such as it is) will be in the variable 317 <varname>$enable_FEATURE</varname>, 318 where <literal>FEATURE</literal> has been squashed. Example: 319 <code>[extra-foo]</code>, controlled by the 320 <option>--enable-extra-foo</option> 321 option and stored in <varname>$enable_extra_foo</varname>. 322 </para> 323 </listitem> 324 <listitem> 325 <para> 326 <literal>DEFAULT</literal> is the value to store in 327 <varname>$enable_FEATURE</varname> if the user does 328 not pass <option>--enable</option>/<option>--disable</option>. 329 It should be one of the permitted values passed later. 330 Examples: <code>[yes]</code>, or <code>[bar]</code>, or 331 <code>[$1]</code> (which passes the argument given to the 332 <literal>GLIBCXX_ENABLE_FOO</literal> macro as the default). 333 </para> 334 <para> 335 For cases where we need to probe for particular models of things, 336 it is useful to have an undocumented "auto" value here (see 337 <literal>GLIBCXX_ENABLE_CLOCALE</literal> for an example). 338 </para> 339 </listitem> 340 <listitem> 341 <para> 342 <literal>HELP-ARG</literal> is any text to append to the option string 343 itself in the <option>--help</option> output. Examples: 344 <code>[]</code> (i.e., an empty string, which appends nothing), 345 <code>[=BAR]</code>, which produces <code>--enable-extra-foo=BAR</code>, 346 and <code>[@<:@=BAR@:>@]</code>, which produces 347 <code>--enable-extra-foo[=BAR]</code>. See the difference? See 348 what it implies to the user? 349 </para> 350 <para> 351 If you're wondering what that line noise in the last example was, 352 that's how you embed autoconf special characters in output text. 353 They're called <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.gnu.org/software/autoconf/manual/autoconf.html#Quadrigraphs"><emphasis>quadrigraphs</emphasis></link> 354 and you should use them whenever necessary. 355 </para> 356 </listitem> 357 <listitem> 358 <para><literal>HELP-STRING</literal> is what you think it is. Do not include the 359 "default" text like we used to do; it will be done for you by 360 <literal>GLIBCXX_ENABLE</literal>. By convention, these are not full English 361 sentences. Example: <literal>[turn on extra foo]</literal> 362 </para> 363 </listitem> 364</itemizedlist> 365 366<para> 367 With no other arguments, only the standard autoconf patterns are 368 allowed: "<option>--{enable,disable}-foo[={yes,no}]</option>" The 369 <varname>$enable_FEATURE</varname> variable is guaranteed to equal 370 either "<literal>yes</literal>" or "<literal>no</literal>" 371 after the macro. If the user tries to pass something else, an 372 explanatory error message will be given, and configure will halt. 373</para> 374 375<para> 376 The second signature takes a fifth argument, "<code>[permit 377 a | b | c | ...]</code>" 378 This allows <emphasis>a</emphasis> or <emphasis>b</emphasis> or 379 ... after the equals sign in the option, and 380 <varname>$enable_FEATURE</varname> is 381 guaranteed to equal one of them after the macro. Note that if you 382 want to allow plain <option>--enable</option>/<option>--disable</option> 383 with no "<literal>=whatever</literal>", you must 384 include "<literal>yes</literal>" and "<literal>no</literal>" in the 385 list of permitted values. Also note that whatever you passed as 386 <literal>DEFAULT</literal> must be in the list. If the 387 user tries to pass something not on the list, a semi-explanatory 388 error message will be given, and configure will halt. Example: 389 <code>[permit generic|gnu|ieee_1003.1-2001|yes|no|auto]</code> 390</para> 391 392<para> 393 The third signature takes a fifth argument. It is arbitrary shell 394 code to execute if the user actually passes the enable/disable 395 option. (If the user does not, the default is used. Duh.) No 396 argument checking at all is done in this signature. See 397 <literal>GLIBCXX_ENABLE_CXX_FLAGS</literal> for an example of handling, 398 and an error message. 399</para> 400 401</section> 402 403<section xml:id="build_hacking.configure.version"><info><title>Shared Library Versioning</title></info> 404 405<para> 406The <filename class="library">libstdc++.so</filename> shared library must 407be carefully managed to maintain binary compatible with older versions 408of the library. This ensures a new version of the library is still usable by 409programs that were linked against an older version. 410</para> 411 412<para> 413Dependent on the target supporting it, the library uses <link 414xmlns:xlink="http://www.w3.org/1999/xlink" 415xlink:href="https://www.akkadia.org/drepper/symbol-versioning">ELF 416symbol versioning</link> for all exported symbols. The symbol versions 417are defined by a <link xmlns:xlink="http://www.w3.org/1999/xlink" 418xlink:href="https://sourceware.org/binutils/docs/ld/VERSION.html">linker 419script</link> that assigns a version to every symbol. 420The set of symbols in each version is fixed when a GCC 421release is made, and must not change after that. 422</para> 423 424<para> When new symbols are added to the library they must be added 425to a new symbol version, which must be created the first time new symbols 426are added after a release. Adding a new symbol version involves the 427following steps: 428</para> 429 430<itemizedlist> 431<listitem><para> 432Edit <filename>acinclude.m4</filename> to update the "revision" value of 433<varname>libtool_VERSION</varname>, e.g. from <literal>6:22:0</literal> 434to <literal>6:23:0</literal>, which will cause the shared library to be 435built as <filename class="library">libstdc++.so.6.0.23</filename>. 436</para> 437</listitem> 438<listitem><para> 439Regenerate the <filename>configure</filename> script by running the 440<command>autoreconf</command> tool from the correct version of the Autoconf 441package (as dictated by the <link xmlns:xlink="http://www.w3.org/1999/xlink" 442xlink:href="https://gcc.gnu.org/install/prerequisites.html">GCC 443prerequisites</link>). 444</para> 445</listitem> 446<listitem><para> 447Edit the file <filename>config/abi/pre/gnu.ver</filename> to 448add a new version node after the last new node. The node name should be 449<literal>GLIBCXX_3.4.X</literal> where <literal>X</literal> is the new 450revision set in <filename>acinclude.m4</filename>, and the node should 451depend on the previous version e.g. 452<programlisting> 453 GLIBCXX_3.4.23 { 454 455 } GLIBCXX_3.4.22; 456</programlisting> 457For symbols in the ABI runtime, libsupc++, the symbol version naming uses 458<literal>CXXABI_1.3.Y</literal> where <literal>Y</literal> increases 459monotonically with each new version. Again, the new node must depend on the 460previous version node e.g. 461<programlisting> 462 CXXABI_1.3.11 { 463 464 } CXXABI_1.3.10; 465</programlisting> 466</para> 467</listitem> 468<listitem><para> 469In order for the <link linkend="test.run.variations">check-abi</link> test 470target to pass the testsuite must be updated to know about the new symbol 471version(s). Edit the file <filename>testsuite/util/testsuite_abi.cc</filename> 472file to add the new versions to the <varname>known_versions</varname> list, 473and update the checks for the latest versions that set the 474<varname>latestp</varname> variable). 475</para> 476</listitem> 477<listitem><para> 478Add the library (<filename class="library">libstdc++.so.6.0.X</filename>) 479and symbols versions 480(<literal>GLIBCXX_3.4.X</literal> and <literal>CXXABI_1.3.Y</literal>) 481to the <link linkend="abi.versioning.history">History</link> section in 482<filename>doc/xml/manual/abi.xml</filename> at the relevant places. 483</para> 484</listitem> 485</itemizedlist> 486 487<para> 488Once the new symbol version has been added you can add the names of your new 489symbols in the new version node: 490<programlisting> 491 GLIBCXX_3.4.23 { 492 493 # basic_string<C, T, A>::_Alloc_hider::_Alloc_hider(C*, A&&) 494 _ZNSt7__cxx1112basic_stringI[cw]St11char_traitsI[cw]ESaI[cw]EE12_Alloc_hiderC[12]EP[cw]OS3_; 495 496 } GLIBCXX_3.4.22; 497</programlisting> 498You can either use mangled names, or demangled names inside an 499<literal>extern "C++"</literal> block. You might find that the new symbol 500matches an existing pattern in an old symbol version (causing the 501<literal>check-abi</literal> test target to fail). If that happens then the 502existing pattern must be adjusted to be more specific so that it doesn't 503match the new symbol. 504</para> 505 506<para> 507For an example of these steps, including adjusting old patterns to be less 508greedy, see <link xmlns:xlink="http://www.w3.org/1999/xlink" 509xlink:href="https://gcc.gnu.org/ml/gcc-patches/2016-07/msg01926.html">https://gcc.gnu.org/ml/gcc-patches/2016-07/msg01926.html</link> 510and the attached patch. 511</para> 512 513<para> 514If it wasn't done for the last release, you might also need to regenerate 515the <filename>baseline_symbols.txt</filename> file that defines the set 516of expected symbols for old symbol versions. A new baseline file can be 517generated by running <userinput>make new-abi-baseline</userinput> in the 518<filename class="directory"><replaceable>libbuilddir</replaceable>/testsuite</filename> 519directory. Be sure to generate the baseline from a clean build using 520unmodified sources, or you will incorporate your local changes into the 521baseline file. 522</para> 523 524</section> 525</section> <!-- configure --> 526 527<section xml:id="build_hacking.make"><info><title>Make</title></info> 528 529 <para> 530 The build process has to make all of object files needed for 531 static or shared libraries, but first it has to generate some 532 include files. The general order is as follows: 533 </para> 534 535<orderedlist> 536 <listitem> 537 <para> 538 make include files, make pre-compiled headers 539 </para> 540 </listitem> 541 <listitem> 542 <para> 543 make libsupc++ 544 </para> 545 <para> 546 Generates a libtool convenience library, 547 <filename>libsupc++convenience</filename> with language-support 548 routines. Also generates a freestanding static library, 549 <filename>libsupc++.a</filename>. 550 </para> 551 </listitem> 552 <listitem> 553 <para> 554 make src 555 </para> 556 <para> 557 Generates two convenience libraries, one for C++98 and one for 558 C++11, various compatibility files for shared and static 559 libraries, and then collects all the generated bits and creates 560 the final libstdc++ libraries. 561 </para> 562<orderedlist> 563 <listitem> 564 <para> 565 make src/c++98 566 </para> 567 <para> 568 Generates a libtool convenience library, 569 <filename>libc++98convenience</filename> with language-support 570 routines. Uses the <option>-std=gnu++98</option> dialect. 571 </para> 572 </listitem> 573 <listitem> 574 <para> 575 make src/c++11 576 </para> 577 <para> 578 Generates a libtool convenience library, 579 <filename>libc++11convenience</filename> with language-support 580 routines. Uses the <option>-std=gnu++11</option> dialect. 581 </para> 582 </listitem> 583 <listitem> 584 <para> 585 make src 586 </para> 587 <para> 588 Generates needed compatibility objects for shared and static 589 libraries. Shared-only code is seggregated at compile-time via 590 the macro <literal>_GLIBCXX_SHARED</literal>. 591 </para> 592 593 <para> 594 Then, collects all the generated convenience libraries, adds in 595 any required compatibility objects, and creates the final shared 596 and static libraries: <filename>libstdc++.so</filename> and 597 <filename>libstdc++.a</filename>. 598 </para> 599 600 </listitem> 601</orderedlist> 602 </listitem> 603</orderedlist> 604 605</section> <!-- make --> 606 607</section> 608