scripts/get_abi.pl: ignore some temp files

When there are filenames of the form ".orig" or ".rej" in
the Documenatation/ABI/ subdirectories, there can be confusing or
erroneous output generated. Example: the file
Documenation/ABI/testing/sysfs-bus-papr-pmem.orig causes this
warning message:

Documentation/ABI/testing/sysfs-bus-papr-pmem:2: WARNING: unknown document: '/powerpc/papr_hcalls'

Prevent this by skipping over filenames that may be created by
patch/diff tools etc.

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20231228233113.5218-1-rdunlap@infradead.org

scripts/get_abi: fix source path leak

The code currently leaks the absolute path of the ABI files into the
rendered documentation.

There exists code to prevent this, but it is not effective when an
absolute path is passed, which it is when $srctree is used.

I consider this to be a minimal, stop-gap fix; a better fix would strip
off the actual prefix instead of hacking it off with a regex.

Link: https://mastodon.social/@vegard/111677490643495163
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: stable@vger.kernel.org
Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20231231235959.3342928-1-vegard.nossum@oracle.com

scripts/get_abi: Fix wrong script file name in the help message

The help message of 'get_abi.pl' is mistakenly saying it's
'abi_book.pl'. This commit fixes the wrong name in the help message.

Fixes: bbc249f2b859 ("scripts: add an script to parse the ABI files")
Signed-off-by: SeongJae Park <sj@kernel.org>
Link: https://lore.kernel.org/r/20220419121636.290407-1-sj@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi: change the file/line number meta info

In order to make it more standard and ReST compatible,
change the meta-tag used with --enable-lineno from:

#define LINENO

to
.. LINENO

In practice, no functional changes.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Link: https://lore.kernel.org/r/125ffd31fbc77ad9eee4d6906e1830b8162fa6ca.1648290305.git.mchehab@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

scripts: get_abi.pl: Fix typo in help message

Fix misspelled word in the script. (s/COMAND/COMMAND/).

Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Link: https://lore.kernel.org/r/1ff3605db3dead41bbde33fbbff6754900eceead.1645789205.git.michal.simek@xilinx.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi.pl: Ignore hidden files

get_abi.pl currently collects every file in Documentation/ABI. This
causes a UnicodeDecodeError in Documentation/sphinx/kernel_abi.py,
when it finds my Vim swap files (.foo.swp) in the directory.

To avoid such issues, ignore hidden files in get_abi.pl.

Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
Link: https://lore.kernel.org/r/20220129005019.2090996-1-j.neuschaefer@gmx.net
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: better generate regex from what fields

Using repeating sequencies of .* seem to slow down the
processing speed on some cases. Also, currently, a "."
character is not properly handled as such.

Change the way regexes are created, in order to produce
better search expressions.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/c69c01c12b1b30466177dcb17e45f833fb47713d.1632994565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: fix fallback rule for undefined symbols

The rule that falls back to the long regex list is wrong:
it is just running again the same loop it did before.

change it to look at the "others" table.

That slows the processing speed, but provides a better
list of undefined symbols.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/a3ba919e9a9208a5f012a13c9674c362a9d73169.1632994565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: make undefined search more deterministic

Sort keys on hashes during undefined search, in order to
make the script more deterministic.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/5dc55fd42e632a24a48f95212aa6c6bc4b2d11fd.1632865873.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: show progress

As parsing the sysfs entries can take a long time, add
progress information.

The progress logic will update the stats on every second,
or on 1% steps of the progress.

When STDERR is a console, it will use a single line, using
a VT-100 command to erase the line before rewriting it.
Otherwise, it will put one message on a separate line.
That would help to identify what parts of sysfs checking
that it is taking more time to process.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/4e581dcbec21ad8a60fff883498018f96f13dd1c.1632823172.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: use STDERR for search-string and show-hints

On undefined checks, use STDOUT only for the not found entries.

All other data (search-string and show-hints) is printed at
STDERR.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/51c6a39c82f73b441030c51bf905a1f382452a67.1632823172.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: update its documentation

The current highlight schema is not working properly. So, use,
instead, Pod::Text.

While here, also update the copyright in order to reflect the latest
changes and the e-mail I'm currently using.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/89fcd301e065ed86dfd8670725144b196266b6a4.1632750315.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: fix parse logic for DT firmware

It doesn't make any sense to parse ABI entries under
/sys/firmware, as those are either specified by ACPI specs
or by Documentation/devicetree.

The current logic to ignore firmware entries is incomplete,
as it ignores just the relative name of the file, and not
its absolute name. This cause errors while parsing the
symlinks.

So, rewrite the logic for it to do a better job.

Tested with both x86 and arm64 (HiKey970) systems.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/1c806eaec96f6706db4b041bbe6a0e2519e9637e.1632750315.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: produce an error if the ref tree is broken

The logic under graph_add_file should create, for every entry, a
__name name array for all entries of the tree. If this fails, the
symlink parsing will break.

Add an error if this ever happens.

While here, improve the output of data dumper to be more
compact and to avoid displaying things like $VAR1=.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/e7dd4d70e206723455d50c851802c8bb6c34941d.1632750315.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: create a valid ReST with duplicated tags

As warned, /sys/bus/iio/devices/iio:deviceX/fault_ovuv is defined 2 times:

Warning: /sys/bus/iio/devices/iio:deviceX/fault_ovuv is defined 2 times: ./Documentation/ABI/testing/sysfs-bus-iio-temperature-max31856:14 ./Documentation/ABI/testing/sysfs-bus-iio-temperature-max31865:0

The logic with joins the two entries is just places the paragraph
for the second entry after the previous one. That could cause more
warnings, as the produced ReST may become invalid, as in the case of
this specific symbol, which ends with a table:

/new_devel/v4l/docs/Documentation/ABI/testing/sysfs-bus-iio-temperature-max31856:2: WARNING: Malformed table.
No bottom table border found or no blank line after table bottom.

=== =======================================================
'1' The input voltage is negative or greater than VDD.
'0' The input voltage is positive and less than VDD (normal
state).
=== =======================================================
/new_devel/v4l/docs/Documentation/ABI/testing/sysfs-bus-iio-temperature-max31856:2: WARNING: Blank line required after table.

Address it by adding two blank lines before joining duplicated
symbols.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/4ad2e3a65f781f0f8d40bb75aa5a07aca80564d6.1632740376.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: ensure that "others" regex will be parsed

The way the search algorithm works is that reduces the number of regex
expressions that will be checked for a given file entry at sysfs. It
does that by looking at the devnode name. For instance, when it checks for
this file:

/sys/bus/pci/drivers/iosf_mbi_pci/bind

The logic will seek only the "What:" expressions that end with "bind".
Currently, there are just a couple of What expressions that matches
it:

What: /sys/bus/fsl\-mc/drivers/.*/bind
What: /sys/bus/pci/drivers/.*/bind

It will then run an O(n²) algorithm to seek, which runs quickly
when there are few regexs to seek. There are, however, some What:
expressions that end with a wildcard. Those are harder to process.
Right now, they're all grouped together at the "others" group.

As those don't depend on the basename of the node, add an extra
loop to ensure that those will be processed at the end, if
not done yet.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/9fe7ab46f67575def5db9e83034e9fab43846d84.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: precompile what match regexes

In order to earn some time during matches, pre-compile regexes.

Before this patch:
$ time ./scripts/get_abi.pl undefined |wc -l
6970

real 0m54,751s
user 0m54,022s
sys 0m0,592s

Afterwards:

$ time ./scripts/get_abi.pl undefined |wc -l
6970

real 0m5,888s
user 0m5,310s
sys 0m0,562s

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/ec45de8fcae791aab0880644974a110424423e68.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: stop check loop earlier when regex is found

Right now, there are two loops used to seek for a regex. Make
sure that both will be skip when a match is found.

While here, drop the unused $defined variable.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/2ba722d2cdbe7c7d0f1d1b58d350052576d1d703.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: ignore some sysfs nodes earlier

When checking for undefined symbols, some nodes aren't easy
or don't make sense to be checked right now. Prevent allocating
memory for those, as they'll be ignored anyway.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/5228789cbef8241d44504ad29fca5cab356cdc53.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: Better handle leaves with wildcards

When the the leaf of a regex ends with a wildcard, the speedup
algorithm to reduce the number of regexes to seek won't work.

So, when those are found, place at the "others" exception.

That slows down the search from 0.14s to 1 minute on my
machine, but the results are a lot more consistent.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/60bb97cf337333783f9f52e114b896439e9cc215.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: improve debug logic

Add a level for debug, in order to allow it to be extended to
debug other parts of the script.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/0203416c6c418abb4fc20577a5f48d0d2a41bae7.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: call get_leave() a little late

The $what conversions need to replace some characters to avoid
breaking regex expressions found on some What:.

only after replacing them back, the script should get the
$leave devnode.

Fixes: ca8e055c2215 ("scripts: get_abi.pl: add a graph to speedup the undefined algorithm")
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/a21631f8a884f50a962beafdd800f27891348d95.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: Fix get_abi.pl search output

Currently, the get_abi.pl will print an invalid symbol
(\xac character). Fix it.

Fixes: ab9c14805b37 ("scripts: get_abi.pl: Better handle multiple What parameters")
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/fb27ac372e38f5ae9d088f9f4e9710c659e0b9e8.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: add a graph to speedup the undefined algorithm

Searching for symlinks is an expensive operation with the current
logic, as it is at the order of O(n^3). In practice, running the
check spends 2-3 minutes to check all symbols.

Fix it by storing the directory tree into a graph, and using
a Breadth First Search (BFS) to find the links for each sysfs node.

With such improvement, it can now report issues with ~11 seconds
on my machine.

It comes with a price, though: there are more symbols reported
as undefined after this change. I suspect it is due to some
sysfs circular loops that are dropped by BFS. Despite such
increase, it seems that the reports are now more coherent.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/f5c1e7b14a27132821c08f0459ba9aea3ed69028.1631957565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: Ignore fs/cgroup sysfs nodes earlier

In order to speedup the parser and store less data, handle
fs/cgroup exceptions a lot earlier.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/caa37831c9e02ae58677d1515ed7cee94f52ea9d.1631957565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: don't skip what that ends with wildcards

The search algorithm used inside check_undefined_symbols
has an optimization: it seeks only whats that have the same
leave name. This helps not only to speedup the search, but
it also allows providing a hint about a partial match.

There's a drawback, however: when "what:" finishes with a
wildcard, the logic will skip the what, reporting it as
"not found".

Fix it by grouping the remaining cases altogether, and
disabing any hints for such cases.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/79ba5139643355230e3bba136b20991cfc92020f.1631957565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: add an option to filter undefined results

The output of this script can be too big. Add an option to
filter out results, in order to help finding issues at the
ABI files.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/7b56c10195bb5e5dfd8b5838a3db8d361231d884.1631957565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: detect softlinks

The way sysfs works is that the same leave may be present under
/sys/devices, /sys/bus and /sys/class, etc, linked via soft
symlinks.

To make it harder to parse, the ABI definition usually refers
only to one of those locations.

So, improve the logic in order to retrieve the symlinks.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/77c49f7d158d88e17f18d40652b75cdde9e179eb.1631957565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: Check for missing symbols at the ABI specs

Check for the symbols that exists under /sys but aren't
defined at Documentation/ABI.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/958e4f3a319148af6d847c0df95e35426f9c4c5f.1631957565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: Better handle multiple What parameters

Using a comma here is problematic, as some What: expressions
may already contain a comma. So, use \xac character instead.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/e83e7ffaf3429f8dfca00d1d01653ecfa36f6119.1631957565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi: ignore code blocks for cross-references

The script should not generate cross-references inside
literal blocks.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/a590f994f8a5742db333bde69e88241a080e4fe0.1616668017.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

scripts: get_abi.pl: parse description line per line

Change the description parsing logic in rst mode in order
to parse it line per line.

The end result is the same, but doing line per line allows
to add some code to escape literal blocks when seeking for
cross-references.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/d33cfa2e59ecf8f28d4ed7de7402468cf2168921.1616668017.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

scripts: get_abi.pl: extend xref match to other types

Currently, there are "What:" symbols for more than just
/sys.

Extend the regex to also cover configfs, /proc /dev and /kvd
symbols.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/a1c7e2b2c37ed6e111dfc8641deb37ed96375a63.1616668017.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

get_abi.pl: fix xref boundaries

There are some issues with the regex that seeks for What:
cross references: basically, it is mis-identifying the start
and the end boundaries of the regex, which causes :ref: to
be inseerted for the wrong symbols at the wrong places.

Fix it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/79a14d2518499b76931b5f29c50979987108152d.1616668017.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

get_abi.pl: seek for all occurrences for Documentation/ABI

Instead of retrieving just one match at most, ensure that the entire
description will be parsed.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/17019b73e106d1b1b353b8880ed189bad3604c13.1616668017.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

scripts: get_abi.pl: better handle escape chars on what:

The parser for the symbols defined on What: doesn't cover all
chars that need to be scaped, like '{' and '}'. Change the logic
to be more generic, and ensure that the same regex will be used
on both What: and when parsing the cross-references.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/29cb56def89b508fe605bcd2ba74a4376cc08e35.1616668017.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

tweewide: Fix most Shebang lines

Change every shebang which does not need an argument to use /usr/bin/env.
This is needed as not every distro has everything under /usr/bin,
sometimes not even bash.

Signed-off-by: Finn Behrens <me@kloenk.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

scripts: get_api.pl: Add sub-titles to ABI output

Instead of adding titles just for the files, add titles
for each part of the ABI output, in order to make easier
to search for a symbol there.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/64752a5de06ab8263c296e3ed01414b25861e1eb.1604312590.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: Don't let ABI files to create subtitles

The ReST output should only contain documentation titles
automatically created by the script.

There are two reasons for that:

1) Consistency.

just a handful ABI docs define titles

2) To avoid critical errors.

Docutils (which is the basis for Sphinx) allows a free
assign of documentation title markups. So, one document
could be doing things like:

Level 1
=======

Level 2
-------

While another one could do the reverse:

Level 1
-------

Level 2
=======

But the same document can't mix.

As the output of get_abi.pl will join contents from multiple
files, if they don't define the levels on a consistent errors,
errors like this can happen:

Sphinx parallel build error:
docutils.utils.SystemMessage: /home/rdunlap/lnx/lnx-510-rc2/Documentation/ABI/testing/sysfs-bus-rapidio:2: (SEVERE/4) Title level inconsistent:

Attributes Common for All RapidIO Devices
-----------------------------------------

Which cause some versions of Sphinx to go into an endless
loop.

It should be noticed that an alternative to that would
be to replace all title occurrences by a single markup,
but that will make the parser more complex, and, due to
(1) it would generate an inconsistent output.

So, better to just remove the titles defined at the ABI
files from the output.

Reported-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/6c62ef5c01d39dee8d891f8390c816d2a889670a.1604312590.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: assume ReST format by default

All ABI files are now compatible with ReST format. So, change
the script default to assume that the ABI source files
are compatible with ReST.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/4b0af257010570e0e917d82498e42992bec5e173.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

docs: ABI: cleanup several ABI documents

There are some ABI documents that, while they don't generate
any warnings, they have issues when parsed by get_abi.pl script
on its output result.

Address them, in order to provide a clean output.

Reviewed-by: Tom Rix <trix@redhat.com> # for fpga-manager
Reviewed-By: Kajol Jain<kjain@linux.ibm.com> # for sysfs-bus-event_source-devices-hv_gpci and sysfs-bus-event_source-devices-hv_24x7
Acked-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> #for IIO
Acked-by: Oded Gabbay <oded.gabbay@gmail.com> # for Habanalabs
Acked-by: Vaibhav Jain <vaibhav@linux.ibm.com> # for sysfs-bus-papr-pmem
Acked-by: Cezary Rojewski <cezary.rojewski@intel.com> # for catpt
Acked-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Acked-by: Ilya Dryomov <idryomov@gmail.com> # for rbd
Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/5bc78e5b68ed1e9e39135173857cb2e753be868f.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

docs: Kconfig/Makefile: add a check for broken ABI files

The files under Documentation/ABI should follow the syntax
as defined at Documentation/ABI/README.

Allow checking if they're following the syntax by running
the ABI parser script on COMPILE_TEST.

With that, when there's a problem with a file under
Documentation/ABI, it would produce a warning like:

Warning: file ./Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats#14:
What '/sys/bus/pci/devices/<dev>/aer_stats/aer_rootport_total_err_cor' doesn't have a description
Warning: file ./Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats#21:
What '/sys/bus/pci/devices/<dev>/aer_stats/aer_rootport_total_err_fatal' doesn't have a description

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/57a38de85cb4b548857207cf1fc1bf1ee08613c9.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: auto-generate cross references

There are several cross-references that can be automatically
generated:

- References to .rst files inside Documentation/
- References to other ABI files;
- References to ABI symbols at /sys/*.

Add a logic to automatically parse them and convert into
cross references.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/abe756d4f94fb6ffcc3dd3902a766c7c3990ea89.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: use bold font for ABI definitions

As we're using tables, let's distinguish the ones used by ABI
by using a bold font.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/9ed645c2519169817379c1df5b8cf03828e92520.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: prevent duplicated file names

The same filename may exist on multiple directories within
ABI. Create separate entries at the internal database for
each of them.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/ef760f68f54e132c4be52f0027189b4ba31554ec.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: output users in ReST format

Right now, the script only outputs Users on search. Print it
also in ReST format.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/83def2a692c4c678f21ee6e76b66a54d2be1e796.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: detect duplicated ABI definitions

The ABI should define only once each What. The current script
logic assumes that.

However, that's not the case, currently: there are several
symbols with a generic definition, and per-driver ones.

Better handle such cases, by preserving the cross-references
with the files that define them, but also track such
cases, producing warnings, as they should be fixed.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/d7a73b8b3aae5b2bff9279996ff9ca4cdfc89196.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: cleanup ABI cross-reference logic

Right now, the cross-references are generated on a single
step, when doing ReST output.

While this is nice optimization, it prevents auto-creating
cross-references for ABI symbols.

So, split it into a separate logic.

While here, turn on Perl warnings, as it helps to debug
problems inside the script.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/dbc97c8c2dfd877921f058134c35b2a8b1f8414b.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: improve its parser to better catch up indentation

The original parser for indentation were relying on having
just one description for each "what". However, that's not
the case: there are a number of ABI symbols that got defined
multiple times.

Improve the parser for it to better handle descriptions
if entries are duplicated.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/eb458bb30be0e5a89192d6057b2e8a7e910dbcb8.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: Allow optionally record from where a line came from

The get_abi.pl reads a lot of files and can join them on a
single output file. Store where each "What:" output came from,
in order to be able to optionally display it.

This is useful for the Sphinx extension, with can now be
able to blame what ABI file has issues, and on what line
the What: description with problems begin.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/befc387011c5e3c6febd285b7f27610e41c90260.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: fix parsing on ReST mode

When the source ABI file is using ReST notation, the script
should handle whitespaces and lines with care, as otherwise
the file won't be properly recognized.

Address the bugs that are on such part of the script.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/5c22c54fbd0cda797b691d52c568be6d0d1079d8.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: get_abi.pl: change script to allow parsing in ReST mode

Right now, several ABI files won't parse as ReST, as they
contain severe violations to the spec, with makes the script
to crash.

So, the code has a sanity logic with escapes bad code and
cleans tags that can cause Sphinx to crash.

Add support for disabling this mode.

Right now, as enabling rst-mode causes crash, it is disabled
by default.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/34b691e3002e8987c24d851fe37640f95e506a92.1604042072.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

doc: ABI scripts: add a SPDX header file

released under GPL v2.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi.pl: add a validate command

Sometimes, we just want the parser to retrieve all symbols from
ABI, in order to check for parsing errors. So, add a new
"validate" command.

While here, update the man/help pages.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi.pl: add a handler for invalid "where" tag

The ABI README file doesn't provide any meaning for a Where:
tag. Yet, a few ABI symbols use it. So, make the parser
handle it, emitting a warning.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi.pl: avoid creating duplicate names

The file the Documentation/ABI/testing/sysfs-class-power has
voltage_min, voltage_max and voltage_now symbols duplicated.

They are defined first for "General Properties" and then for
"USB Properties".

This cause those warnings:

get_abi.pl rest --dir $srctree/Documentation/ABI/testing:26933: WARNING: Duplicate explicit target name: "abi_sys_class_power_supply_supply_name_voltage_max".
get_abi.pl rest --dir $srctree/Documentation/ABI/testing:26968: WARNING: Duplicate explicit target name: "abi_sys_class_power_supply_supply_name_voltage_min".
get_abi.pl rest --dir $srctree/Documentation/ABI/testing:27008: WARNING: Duplicate explicit target name: "abi_sys_class_power_supply_supply_name_voltage_now".

And, as the references are not valid, it will also generate
warnings about links to undefined references.

Fix it by storing labels into a hash table and, when a duplicated
one is found, appending random characters at the end.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi.pl: fix parse issues with some files

A few files are failing to parse:

Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats
Documentation/ABI/testing/sysfs-class-pktcdvd
Documentation/ABI/testing/sysfs-bus-nfit

On all three files, the problem is that there is a ":" character
at the initial file description.

Improve the parse in order to handle those special cases.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi.pl: represent what in tables

Several entries at the ABI have multiple What: with the same
description.

Instead of showing those symbols as sections, let's show them
as tables. That makes easier to read on the final output,
and avoid too much recursion at Sphinx parsing.

We need to put file references at the end, as we don't want
non-file tables to be mangled with other entries.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi.pl: add support for searching for ABI symbols

Change its syntax to allow switching between ReST output mode
and a new search mode, with allows to seek for ABI symbols
using regex.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi.pl: split label naming from xref logic

Instead of using a ReST compilant label while parsing,
move the label to ReST output. That makes the parsing logic
more generic, allowing it to provide other types of output.

As a side effect, now all files used to generate the output
will be output. We can later add command line arguments to
filter.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi.pl: avoid use literal blocks when not needed

The usage of literal blocks make the document very complex,
causing the browser to take a long time to load.

On most ABI descriptions, they're a plain text, and don't
require a literal block.

So, add a logic there with identifies when a literal block
is needed.

As, on literal blocks, we need to respect the original
document space, the most complex part of this patch is
to preserve the original spacing where needed.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts/get_abi.pl: parse files with text at beginning

It sounds usefult o parse files with has some text at the
beginning. Add support for it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

scripts: add an script to parse the ABI files

Add a script to parse the Documentation/ABI files and produce
an output with all entries inside an ABI (sub)directory.

Right now, it outputs its contents on ReST format. It shouldn't
be hard to make it produce other kind of outputs, since the ABI
file parser is implemented in separate than the output generator.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>