1<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 2<html> 3<!-- Copyright (C) 1988-2020 Free Software Foundation, Inc. 4 5Permission is granted to copy, distribute and/or modify this document 6under the terms of the GNU Free Documentation License, Version 1.3 or 7any later version published by the Free Software Foundation; with no 8Invariant Sections, the Front-Cover texts being (a) (see below), and 9with the Back-Cover Texts being (b) (see below). A copy of the 10license is included in the section entitled "GNU 11Free Documentation License". 12 13(a) The FSF's Front-Cover Text is: 14 15A GNU Manual 16 17(b) The FSF's Back-Cover Text is: 18 19You have freedom to copy and modify this GNU Manual, like GNU 20 software. Copies published by the Free Software Foundation raise 21 funds for GNU development. --> 22<!-- Created by GNU Texinfo 6.5, http://www.gnu.org/software/texinfo/ --> 23<head> 24<meta http-equiv="Content-Type" content="text/html; charset=utf-8"> 25<title>Installing GCC: Testing</title> 26 27<meta name="description" content="Installing GCC: Testing"> 28<meta name="keywords" content="Installing GCC: Testing"> 29<meta name="resource-type" content="document"> 30<meta name="distribution" content="global"> 31<meta name="Generator" content="makeinfo"> 32<style type="text/css"> 33<!-- 34a.summary-letter {text-decoration: none} 35blockquote.indentedblock {margin-right: 0em} 36blockquote.smallindentedblock {margin-right: 0em; font-size: smaller} 37blockquote.smallquotation {font-size: smaller} 38div.display {margin-left: 3.2em} 39div.example {margin-left: 3.2em} 40div.lisp {margin-left: 3.2em} 41div.smalldisplay {margin-left: 3.2em} 42div.smallexample {margin-left: 3.2em} 43div.smalllisp {margin-left: 3.2em} 44kbd {font-style: oblique} 45pre.display {font-family: inherit} 46pre.format {font-family: inherit} 47pre.menu-comment {font-family: serif} 48pre.menu-preformatted {font-family: serif} 49pre.smalldisplay {font-family: inherit; font-size: smaller} 50pre.smallexample {font-size: smaller} 51pre.smallformat {font-family: inherit; font-size: smaller} 52pre.smalllisp {font-size: smaller} 53span.nolinebreak {white-space: nowrap} 54span.roman {font-family: initial; font-weight: normal} 55span.sansserif {font-family: sans-serif; font-weight: normal} 56ul.no-bullet {list-style: none} 57--> 58</style> 59 60 61</head> 62 63<body lang="en"> 64<h1 class="settitle" align="center">Installing GCC: Testing</h1> 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86<a name="index-Testing"></a> 87<a name="index-Installing-GCC_003a-Testing"></a> 88<a name="index-Testsuite"></a> 89 90<p>Before you install GCC, we encourage you to run the testsuites and to 91compare your results with results from a similar configuration that have 92been submitted to the 93<a href="http://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults mailing list</a>. 94Some of these archived results are linked from the build status lists 95at <a href="http://gcc.gnu.org/buildstat.html">http://gcc.gnu.org/buildstat.html</a>, although not everyone who 96reports a successful build runs the testsuites and submits the results. 97This step is optional and may require you to download additional software, 98but it can give you confidence in your new GCC installation or point out 99problems before you install and start using your new GCC. 100</p> 101<p>First, you must have <a href="download.html">downloaded the testsuites</a>. 102These are part of the full distribution, but if you downloaded the 103“core” compiler plus any front ends, you must download the testsuites 104separately. 105</p> 106<p>Second, you must have the testing tools installed. This includes 107<a href="http://www.gnu.org/software/dejagnu/">DejaGnu</a>, Tcl, and Expect; 108the DejaGnu site has links to these. For running the BRIG frontend 109tests, a tool to assemble the binary BRIGs from HSAIL text, 110<a href="https://github.com/HSAFoundation/HSAIL-Tools/">HSAILasm</a> must 111be installed. 112</p> 113<p>If the directories where <code>runtest</code> and <code>expect</code> were 114installed are not in the <code>PATH</code>, you may need to set the following 115environment variables appropriately, as in the following example (which 116assumes that DejaGnu has been installed under <samp>/usr/local</samp>): 117</p> 118<div class="smallexample"> 119<pre class="smallexample">TCL_LIBRARY = /usr/local/share/tcl8.0 120DEJAGNULIBS = /usr/local/share/dejagnu 121</pre></div> 122 123<p>(On systems such as Cygwin, these paths are required to be actual 124paths, not mounts or links; presumably this is due to some lack of 125portability in the DejaGnu code.) 126</p> 127 128<p>Finally, you can run the testsuite (which may take a long time): 129</p><div class="smallexample"> 130<pre class="smallexample">cd <var>objdir</var>; make -k check 131</pre></div> 132 133<p>This will test various components of GCC, such as compiler 134front ends and runtime libraries. While running the testsuite, DejaGnu 135might emit some harmless messages resembling 136‘<samp>WARNING: Couldn't find the global config file.</samp>’ or 137‘<samp>WARNING: Couldn't find tool init file</samp>’ that can be ignored. 138</p> 139<p>If you are testing a cross-compiler, you may want to run the testsuite 140on a simulator as described at <a href="http://gcc.gnu.org/simtest-howto.html">http://gcc.gnu.org/simtest-howto.html</a>. 141</p> 142<a name="How-can-you-run-the-testsuite-on-selected-tests_003f"></a> 143<h3 class="section">How can you run the testsuite on selected tests?</h3> 144 145<p>In order to run sets of tests selectively, there are targets 146‘<samp>make check-gcc</samp>’ and language specific ‘<samp>make check-c</samp>’, 147‘<samp>make check-c++</samp>’, ‘<samp>make check-d</samp>’ ‘<samp>make check-fortran</samp>’, 148‘<samp>make check-ada</samp>’, ‘<samp>make check-objc</samp>’, ‘<samp>make check-obj-c++</samp>’, 149‘<samp>make check-lto</samp>’ 150in the <samp>gcc</samp> subdirectory of the object directory. You can also 151just run ‘<samp>make check</samp>’ in a subdirectory of the object directory. 152</p> 153 154<p>A more selective way to just run all <code>gcc</code> execute tests in the 155testsuite is to use 156</p> 157<div class="smallexample"> 158<pre class="smallexample">make check-gcc RUNTESTFLAGS="execute.exp <var>other-options</var>" 159</pre></div> 160 161<p>Likewise, in order to run only the <code>g++</code> “old-deja” tests in 162the testsuite with filenames matching ‘<samp>9805*</samp>’, you would use 163</p> 164<div class="smallexample"> 165<pre class="smallexample">make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var>other-options</var>" 166</pre></div> 167 168<p>The file-matching expression following <var>filename</var><code>.exp=</code> is treated 169as a series of whitespace-delimited glob expressions so that multiple patterns 170may be passed, although any whitespace must either be escaped or surrounded by 171single quotes if multiple expressions are desired. For example, 172</p> 173<div class="smallexample"> 174<pre class="smallexample">make check-g++ RUNTESTFLAGS="old-deja.exp=9805*\ virtual2.c <var>other-options</var>" 175make check-g++ RUNTESTFLAGS="'old-deja.exp=9805* virtual2.c' <var>other-options</var>" 176</pre></div> 177 178<p>The <samp>*.exp</samp> files are located in the testsuite directories of the GCC 179source, the most important ones being <samp>compile.exp</samp>, 180<samp>execute.exp</samp>, <samp>dg.exp</samp> and <samp>old-deja.exp</samp>. 181To get a list of the possible <samp>*.exp</samp> files, pipe the 182output of ‘<samp>make check</samp>’ into a file and look at the 183‘<samp>Running … .exp</samp>’ lines. 184</p> 185<a name="Passing-options-and-running-multiple-testsuites"></a> 186<h3 class="section">Passing options and running multiple testsuites</h3> 187 188<p>You can pass multiple options to the testsuite using the 189‘<samp>--target_board</samp>’ option of DejaGNU, either passed as part of 190‘<samp>RUNTESTFLAGS</samp>’, or directly to <code>runtest</code> if you prefer to 191work outside the makefiles. For example, 192</p> 193<div class="smallexample"> 194<pre class="smallexample">make check-g++ RUNTESTFLAGS="--target_board=unix/-O3/-fmerge-constants" 195</pre></div> 196 197<p>will run the standard <code>g++</code> testsuites (“unix” is the target name 198for a standard native testsuite situation), passing 199‘<samp>-O3 -fmerge-constants</samp>’ to the compiler on every test, i.e., 200slashes separate options. 201</p> 202<p>You can run the testsuites multiple times using combinations of options 203with a syntax similar to the brace expansion of popular shells: 204</p> 205<div class="smallexample"> 206<pre class="smallexample">…"--target_board=arm-sim\{-mhard-float,-msoft-float\}\{-O1,-O2,-O3,\}" 207</pre></div> 208 209<p>(Note the empty option caused by the trailing comma in the final group.) 210The following will run each testsuite eight times using the ‘<samp>arm-sim</samp>’ 211target, as if you had specified all possible combinations yourself: 212</p> 213<div class="smallexample"> 214<pre class="smallexample">--target_board='arm-sim/-mhard-float/-O1 \ 215 arm-sim/-mhard-float/-O2 \ 216 arm-sim/-mhard-float/-O3 \ 217 arm-sim/-mhard-float \ 218 arm-sim/-msoft-float/-O1 \ 219 arm-sim/-msoft-float/-O2 \ 220 arm-sim/-msoft-float/-O3 \ 221 arm-sim/-msoft-float' 222</pre></div> 223 224<p>They can be combined as many times as you wish, in arbitrary ways. This 225list: 226</p> 227<div class="smallexample"> 228<pre class="smallexample">…"--target_board=unix/-Wextra\{-O3,-fno-strength\}\{-fomit-frame,\}" 229</pre></div> 230 231<p>will generate four combinations, all involving ‘<samp>-Wextra</samp>’. 232</p> 233<p>The disadvantage to this method is that the testsuites are run in serial, 234which is a waste on multiprocessor systems. For users with GNU Make and 235a shell which performs brace expansion, you can run the testsuites in 236parallel by having the shell perform the combinations and <code>make</code> 237do the parallel runs. Instead of using ‘<samp>--target_board</samp>’, use a 238special makefile target: 239</p> 240<div class="smallexample"> 241<pre class="smallexample">make -j<var>N</var> check-<var>testsuite</var>//<var>test-target</var>/<var>option1</var>/<var>option2</var>/… 242</pre></div> 243 244<p>For example, 245</p> 246<div class="smallexample"> 247<pre class="smallexample">make -j3 check-gcc//sh-hms-sim/{-m1,-m2,-m3,-m3e,-m4}/{,-nofpu} 248</pre></div> 249 250<p>will run three concurrent “make-gcc” testsuites, eventually testing all 251ten combinations as described above. Note that this is currently only 252supported in the <samp>gcc</samp> subdirectory. (To see how this works, try 253typing <code>echo</code> before the example given here.) 254</p> 255 256<a name="How-to-interpret-test-results"></a> 257<h3 class="section">How to interpret test results</h3> 258 259<p>The result of running the testsuite are various <samp>*.sum</samp> and <samp>*.log</samp> 260files in the testsuite subdirectories. The <samp>*.log</samp> files contain a 261detailed log of the compiler invocations and the corresponding 262results, the <samp>*.sum</samp> files summarize the results. These summaries 263contain status codes for all tests: 264</p> 265<ul> 266<li> PASS: the test passed as expected 267</li><li> XPASS: the test unexpectedly passed 268</li><li> FAIL: the test unexpectedly failed 269</li><li> XFAIL: the test failed as expected 270</li><li> UNSUPPORTED: the test is not supported on this platform 271</li><li> ERROR: the testsuite detected an error 272</li><li> WARNING: the testsuite detected a possible problem 273</li></ul> 274 275<p>It is normal for some tests to report unexpected failures. At the 276current time the testing harness does not allow fine grained control 277over whether or not a test is expected to fail. This problem should 278be fixed in future releases. 279</p> 280 281<a name="Submitting-test-results"></a> 282<h3 class="section">Submitting test results</h3> 283 284<p>If you want to report the results to the GCC project, use the 285<samp>contrib/test_summary</samp> shell script. Start it in the <var>objdir</var> with 286</p> 287<div class="smallexample"> 288<pre class="smallexample"><var>srcdir</var>/contrib/test_summary -p your_commentary.txt \ 289 -m gcc-testresults@gcc.gnu.org |sh 290</pre></div> 291 292<p>This script uses the <code>Mail</code> program to send the results, so 293make sure it is in your <code>PATH</code>. The file <samp>your_commentary.txt</samp> is 294prepended to the testsuite summary and should contain any special 295remarks you have on your results or your build environment. Please 296do not edit the testsuite result block or the subject line, as these 297messages may be automatically processed. 298</p> 299<hr /> 300<p> 301<p><a href="./index.html">Return to the GCC Installation page</a> 302</p> 303 304 305 306 307 308 309<hr> 310 311 312 313</body> 314</html> 315