docs: verify/bisect: stable regressions: first stable, then mainline

Rearrange the instructions so that readers facing a regression within a
stable or longterm series first test its latest release before testing
mainline. This is less scary for some people. It also reduces the chance
that something goes sideways for readers that compile their first
kernel, as mainline can cause slightly more trouble.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/efd3cb9c68db450091021326bf9c334553df0ec2.1712647788.git.linux@leemhuis.info

docs: verify/bisect: describe how to use a build host

Describe how to build kernels on another system (with and without
cross-compiling), as building locally can be quite painfully on some
slow systems. This is done in an add-on section, as it would make the
step-by-step guide to complicated if this special case would be
described there.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/288160cb4769e46a3280250ca71da0abc4aa002d.1712647788.git.linux@leemhuis.info

docs: verify/bisect: explain testing reverts, patches and newer code

Rename 'Supplementary tasks' to 'Complementary tasks' while introducing
a section 'Optional tasks: test reverts, patches, or later versions':
the latter is something readers occasionally will have to do after
reporting a bug and thus is best covered here.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/dacf26a4c48e9e8f04ecbc77e0a74c9b2a6a1103.1712647788.git.linux@leemhuis.info

docs: verify/bisect: proper headlines and more spacing

Various small improvements and fixes:

* Separate ref links from their target with a space for better
readability.

* Add a proper heading for the note at the end of the step-by-step
guide.

* Use proper 3rd and 4th level headlines in the reference section and
add short intros for the 2nd level headlines that lacked one.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/f59f0f235a2192ed93899a7338153e4cb71075f0.1712647788.git.linux@leemhuis.info

docs: verify/bisect: add and fetch stable branches ahead of time

Add and fetch all required stable branches ahead of time. This fixes a
bug, as readers that wanted to bisect a regression within a stable or
longterm series otherwise did not have them available at the right time.
This way also matches the flow somewhat better and avoids some "if you
haven't already added it" phrases that otherwise become necessary in
future changes.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/57dcf312959476abe6151bf3d35eb79e3e9a83d1.1712647788.git.linux@leemhuis.info

docs: verify/bisect: use git switch, tag kernel, and various fixes

Various small improvements and fixes:

* Use the more modern 'git switch' instead of 'git checkout', which
makes it more obvious what's happening (among others due to the
--discard-changes parameter that is more clear than --force).

* Provide a hint how a mainline version number and one from a stable
series look like.

* When trying to validate the bisection result with a revert, add a
special tag to facilitate the identification.

* Sync version numbers used in various examples for consistency: stick
to 6.0.13, 6.0.15, and 6.1.5.

* Fix a few typos and oddities.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/85029aa004447b0eeb5043fb014630f2acafacec.1712647788.git.linux@leemhuis.info

docs: verify/bisect: remove a level of indenting

Remove a unnecessary level of indenting in some areas of the reference
section. No text changes.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <01f1a407e92b92d9f8614bd34882956694bab123.1710750972.git.linux@leemhuis.info>

docs: verify/bisect: drop 'v' prefix, EOL aspect, and assorted fixes

A bunch of minor fixes and improvements and two other things:

- Explain the 'v' version prefix when it's first used, but drop it
everywhere in the text for consistency. Also drop single quotes around
a few version numbers.

- Point out that testing a stable/longterm kernel only makes sense if
the series is still supported.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <f13d203d5975419608996300992eaa2e4fcc2dc1.1710750972.git.linux@leemhuis.info>

docs: verify/bisect: check taint flag

Instruct readers to check the taint flag, as the reason why it's set
might directly or indirectly cause the bug or interfere with testing.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <8fcaffa8e85f36d51178d61061355c3c8bc85a0f.1710750972.git.linux@leemhuis.info>

docs: verify/bisect: improve install instructions

These changes among others ensure modules will be installed when
/sbin/installkernel is missing. Furthermore describe better what tasks
the script ideally performs so that users can more easily check if those
have been taken care of. In addition to that point to the distro's
documentation for further details on installing kernels manually.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <e392bd5eb12654bed635f32b24304a712b0c67d1.1710750972.git.linux@leemhuis.info>

docs: verify/bisect: fixes, finetuning, and support for Arch

Assorted changes for the recently added document.

Improvements:

* Add instructions for installing required software on Arch Linux.

Fixes:

* Move a 'git remote add -t master stable [...]' from a totally wrong
to the right place.

* Fix two anchors.

* Add two required packages to the openSUSE install instructions.

Fine tuning:

* Improve the reference section about downloading Linux mainline sources
to make it more obvious that those are alternatives.

* Include the full instructions for git bundles to ensure the remote
gets the right name; that way the text also works stand alone.

* Install ncurses and qt headers for use of menuconfig and xconfig by
default, but tell users that they are free to omit them.

* Mention ahead of time which version number are meant as example in
commands used during the step-by-step guide.

* Mention that 'kernel-install remove' might do a incomplete job.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <6592c9ef4244faa484b4113f088dbc1beca61015.1709716794.git.linux@leemhuis.info>

docs: new text on bisecting which also covers bug validation

Add a second document on bisecting regressions explaining the whole
process from beginning to end -- while also describing how to validate
if a problem is still present in mainline. This "two in one" approach
is possible, as checking whenever a bug is in mainline is one of the
first steps before performing a bisection anyway and thus needs to be
described. Due to this approach the text also works quite nicely in
conjunction with Documentation/admin-guide/reporting-issues.rst, as it
covers all typical cases where users will need to build a kernel in
exactly the same order.

The text targets users that normally run kernels from their Linux
distributor who might never have compiled their own kernel.

This aim is why the first kernel built while following this guide is
generated from the latest mainline codebase. This will rule out that the
regression (a) was fixed already and (b) is caused by config change a
vendor distributor performed; checking mainline will furthermore (c)
determine if the issue is something that needs to be reported to the
regular developers or the stable team (this is needed even when readers
bisect within a stable series).

Only then are readers instructed to build their own variant of the
'good' kernel to validate the trimmed .config file created during early
in the guide, as performing a bisection with a broken one would be a
waste of time. There is a small downside of this order: readers might
have to go back to testing mainline, if it turns out there is a problem
with their .config. But that should be rare -- and if the regression was
already fixed readers might not get to this point anyway. Hence in the
end this order should mean that readers built less kernels overall.

This sequence allows the text to easily cover the "check if a bug is
present in the upstream kernel" case while only making things a tiny bit
more complicated.

The text tries to prevent readers from running into many mistakes users
are known to frequently make. The steps required for this might look
superfluous for people that are already familiar with bisections -- but
anyone with that knowledge should be able to adapt the instructions to
their use-case or will not need this text at all.

Style and structure of the text match the one
Documentation/admin-guide/quickly-build-trimmed-linux.rst uses. Quite a
few paragraphs are even copied from there and not changed at all or only
slightly. This will complicate maintenance, as some future changes to
one of these documents will have to be replicated in the other. But this
is the lesser evil: solutions like "sending readers from one document
over to the other" or "extracting the common parts into a separate
document" might work in other cases, but would be too confusing here
given the topic and the target audience.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
[jc: Undo spurious removal of subsection header line]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <02b084a06de4ad61ac4ecd92b9265d4df4d03d71.1709282441.git.linux@leemhuis.info>