kbuild: prefix $(srctree)/ to some included Makefiles

VPATH is used in Kbuild to make pattern rules search for prerequisites
in both $(objtree) and $(srctree). Some of *.c, *.S files are not real
sources, but generated by tools such as flex, bison, perl.

In contrast, I doubt the benefit of --include-dir=$(abs_srctree) because
it is always clear which Makefiles are real sources, and which are not.

So, my hope is to add $(srctree)/ prefix to all check-in Makefiles,
then remove --include-dir=$(abs_srctree) flag in the future.

I am touching only some Kbuild core parts for now. Treewide fixes will
be needed to achieve this goal.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: move headers_check rule to usr/include/Makefile

Currently, some sanity checks for uapi headers are done by
scripts/headers_check.pl, which is wired up to the 'headers_check'
target in the top Makefile.

It is true compiling headers has better test coverage, but there
are still several headers excluded from the compile test. I like
to keep headers_check.pl for a while, but we can delete a lot of
code by moving the build rule to usr/include/Makefile.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: fix 'No such file or directory' warning for headers_install

Since commit d5470d14431e ("kbuild: re-implement Makefile.headersinst
without recursion"), headers_install emits an ugly warning.

$ make headers_install
[ snip ]
UPD include/generated/uapi/linux/version.h
find: ‘./include/uapi/Kbuild’: No such file or directory
HDRINST usr/include/video/uvesafb.h
...

This happens for GNU Make <= 4.2.1

When I wrote that commit, I missed this warning because I was using the
state-of-the-art Make version compiled from the git tree.

$(wildcard $(src)/*/) is intended to match to only existing directories
since it has a trailing slash, but actually matches to regular files too.
(include/uapi/Kbuild in this case)

This is a bug of GNU Make, and was fixed by:

| commit b7acb10e86dc8f5fdf2a2bbd87e1059c315e31d6
| Author: spagoveanu@gmail.com <spagoveanu@gmail.com>
| Date: Wed Jun 20 02:03:48 2018 +0300
|
| * src/dir.c: Preserve glob d_type field

We need to cater to old Make versions. Add '$(filter %/,...) to filter
out the regular files.

Fixes: d5470d14431e ("kbuild: re-implement Makefile.headersinst without recursion")
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: simplify scripts/headers_install.sh

Now that headers_install.sh is invoked per file, remove the for-loop
in the shell script.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: re-implement Makefile.headersinst without recursion

Since commit fcc8487d477a ("uapi: export all headers under uapi
directories"), the headers in uapi directories are all exported by
default although exceptional cases are still allowed by the syntax
'no-export-headers'.

The traditional directory descending has been kept (in a somewhat
hacky way), but it is actually unneeded.

Get rid of it to simplify the code.

Also, handle files one by one instead of the previous per-directory
processing. This will emit much more log, but I like it.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: add 'headers' target to build up uapi headers in usr/include

In Linux build system, build targets and installation targets are
separated.

Examples are:

- 'make vmlinux' -> 'make install'
- 'make modules' -> 'make modules_install'
- 'make dtbs' -> 'make dtbs_install'
- 'make vdso' -> 'make vdso_install'

The intention is to run the build targets under the normal privilege,
then the installation targets under the root privilege since we need
the write permission to the system directories.

We have 'make headers_install' but the corresponding 'make headers'
stage does not exist. The purpose of headers_install is to provide
the kernel interface to C library. So, nobody would try to install
headers to /usr/include directly.

If 'sudo make INSTALL_HDR_PATH=/usr/include headers_install' were run,
some build artifacts in the kernel tree would be owned by root because
some of uapi headers are generated by 'uapi-asm-generic', 'archheaders'
targets.

Anyway, I believe it makes sense to split the header installation into
two stages.

[1] 'make headers'
Process headers in uapi directories by scripts/headers_install.sh
and copy them to usr/include

[2] 'make headers_install'
Copy '*.h' verbatim from usr/include to $(INSTALL_HDR_PATH)/include

For the backward compatibility, 'headers_install' depends on 'headers'.

Some samples expect uapi headers in usr/include. So, the 'headers'
target is useful to build up them in the fixed location usr/include
irrespective of INSTALL_HDR_PATH.

Another benefit is to stop polluting the final destination with the
time-stamp files '.install' and '.check'. Maybe you can see them in
your toolchains.

Lastly, my main motivation is to prepare for compile-testing uapi
headers. To build something, we have to save an object and .*.cmd
somewhere. The usr/include/ will be the work directory for that.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: generate asm-generic wrappers if mandatory headers are missing

Some time ago, Sam pointed out a certain degree of overwrap between
generic-y and mandatory-y. (https://lkml.org/lkml/2017/7/10/121)

I tweaked the meaning of mandatory-y a little bit; now it defines the
minimum set of ASM headers that all architectures must have.

If arch does not have specific implementation of a mandatory header,
Kbuild will let it fallback to the asm-generic one by automatically
generating a wrapper. This will allow to drop lots of redundant
generic-y defines.

Previously, "mandatory" was used in the context of UAPI, but I guess
this can be extended to kernel space ASM headers.

Suggested-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: remove a special handling for *.agh in Makefile.headersinst

scripts/Makefile.headersinst takes care of *.agh just for

arch/cris/include/uapi/arch-v10/arch/sv_addr.agh

because renaming exported headers is difficult (or impossible).

This code is no longer necessary thanks to commit c690eddc2f3b ("CRIS:
Drop support for the CRIS port").

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: specify FORCE in Makefile.headersinst as .PHONY target

Swap the order of ".PHONY: $(PHONY)" and "PHONY += FORCE"
so that FORCE is correctly specified as a .PHONY target.

Use a preferred way for specifying $(subdirs) as .PHONY targets.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: remove redundant $(wildcard ...) for cmd_files calculation

I do not see any reason why $(wildcard ...) needs to be called twice
for computing cmd_files. Remove the first one.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

License cleanup: add SPDX GPL-2.0 license identifier to files with no license

Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.

The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.

The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.

Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).

All documentation files were explicitly excluded.

The following heuristics were used to determine which SPDX license
identifiers to apply.

- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.

For non */uapi/* files that summary was:

SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139

and resulted in the first patch in this series.

If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:

SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930

and resulted in the second patch in this series.

- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:

SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1

and that resulted in the third patch in this series.

- when the two scanners agreed on the detected license(s), that became
the concluded license(s).

- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.

- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).

- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.

- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.

In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.

Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.

Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.

In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.

Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct

This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.

These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.

Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

kbuild: replace $(hdr-arch) with $(SRCARCH)

Since commit 5e53879008b9 ("sparc,sparc64: unify Makefile"), hdr-arch
and SRCARCH always match.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>

kbuild: remove wrapper files handling from Makefile.headersinst

scripts/Makefike.headersinst creates asm-generic wrappers by itself
because scripts/Makefile.asm-generic created some of exported wrappers
outside uapi directories.

Now this distortion has been fixed. scripts/Makefile.headersinst can
simply copy wrappers created by scripts/Makefile.asm-generic.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: do not include old-kbuild-file from Makefile.headersinst

Now asm-generic wrappers to be exported are all listed in
arch/*/include/uapi/asm/Kbuild. "make headers_install" no longer
depends on any Kbuild files outside uapi directories.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: pass dst= to Makefile.headersinst from top Makefile

We can always pass dst= from the top Makefile. This will simplify
the logic in Makefile.headersinst.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: remove useless $(gen) variable in Makefile.headersinst

We have no true case for the $(if $(gen), ...) conditional. Drop it
to simplify the gendir calculation.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: fix header installation under fakechroot environment

Since commit fcc8487d477a ("uapi: export all headers under uapi
directories") fakechroot make bindeb-pkg fails, mismatching files for
directories:
touch: cannot touch 'usr/include/video/uvesafb.h/.install': Not a
directory

This due to a bug in fakechroot:
when using the function $(wildcard $(srcdir)/*/.) in a makefile, under a
fakechroot environment, not only directories but also files are
returned.

To circumvent that, we are using the functions:
$(sort $(dir $(wildcard $(srcdir)/*/))))

Fixes: fcc8487d477a ("uapi: export all headers under uapi directories")
Signed-off-by: Richard Genoud <richard.genoud@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: skip install/check of headers right under uapi directories

Since commit 61562f981e92 ("uapi: export all arch specifics
directories"), "make INSTALL_HDR_PATH=$root/usr headers_install"
deletes standard glibc headers and others in $(root)/usr/include.

The cause of the issue is that headers_install now starts descending
from arch/$(hdr-arch)/include/uapi with $(root)/usr/include for its
destination when installing asm headers. So, headers already there
are assumed to be unwanted.

When headers_install starts descending from include/uapi with
$(root)/usr/include for its destination, it works around the problem
by creating an dummy destination $(root)/usr/include/uapi, but this
is tricky.

To fix the problem in a clean way is to skip headers install/check
in include/uapi and arch/$(hdr-arch)/include/uapi because we know
there are only sub-directories in uapi directories. A good side
effect is the empty destination $(root)/usr/include/uapi will go
away.

I am also removing the trailing slash in the headers_check target to
skip checking in arch/$(hdr-arch)/include/uapi.

Fixes: 61562f981e92 ("uapi: export all arch specifics directories")
Reported-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>

uapi: export all arch specifics directories

This patch removes the need of subdir-y. Now all files/directories under
arch/<arch>/include/uapi/ are exported.

The only change for userland is the layout of the command 'make
headers_install_all': directories asm-<arch> are replaced by arch-<arch>/.
Those new directories contains all files/directories of the specified arch.

Note that only cris and tile have more directories than only asm:
- arch-v[10|32] for cris;
- arch for tile.

Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

uapi: export all headers under uapi directories

Regularly, when a new header is created in include/uapi/, the developer
forgets to add it in the corresponding Kbuild file. This error is usually
detected after the release is out.

In fact, all headers under uapi directories should be exported, thus it's
useless to have an exhaustive list.

After this patch, the following files, which were not exported, are now
exported (with make headers_install_all):
asm-arc/kvm_para.h
asm-arc/ucontext.h
asm-blackfin/shmparam.h
asm-blackfin/ucontext.h
asm-c6x/shmparam.h
asm-c6x/ucontext.h
asm-cris/kvm_para.h
asm-h8300/shmparam.h
asm-h8300/ucontext.h
asm-hexagon/shmparam.h
asm-m32r/kvm_para.h
asm-m68k/kvm_para.h
asm-m68k/shmparam.h
asm-metag/kvm_para.h
asm-metag/shmparam.h
asm-metag/ucontext.h
asm-mips/hwcap.h
asm-mips/reg.h
asm-mips/ucontext.h
asm-nios2/kvm_para.h
asm-nios2/ucontext.h
asm-openrisc/shmparam.h
asm-parisc/kvm_para.h
asm-powerpc/perf_regs.h
asm-sh/kvm_para.h
asm-sh/ucontext.h
asm-tile/shmparam.h
asm-unicore32/shmparam.h
asm-unicore32/ucontext.h
asm-x86/hwcap2.h
asm-xtensa/kvm_para.h
drm/armada_drm.h
drm/etnaviv_drm.h
drm/vgem_drm.h
linux/aspeed-lpc-ctrl.h
linux/auto_dev-ioctl.h
linux/bcache.h
linux/btrfs_tree.h
linux/can/vxcan.h
linux/cifs/cifs_mount.h
linux/coresight-stm.h
linux/cryptouser.h
linux/fsmap.h
linux/genwqe/genwqe_card.h
linux/hash_info.h
linux/kcm.h
linux/kcov.h
linux/kfd_ioctl.h
linux/lightnvm.h
linux/module.h
linux/nbd-netlink.h
linux/nilfs2_api.h
linux/nilfs2_ondisk.h
linux/nsfs.h
linux/pr.h
linux/qrtr.h
linux/rpmsg.h
linux/sched/types.h
linux/sed-opal.h
linux/smc.h
linux/smc_diag.h
linux/stm.h
linux/switchtec_ioctl.h
linux/vfio_ccw.h
linux/wil6210_uapi.h
rdma/bnxt_re-abi.h

Note that I have removed from this list the files which are generated in every
exported directories (like .install or .install.cmd).

Thanks to Julien Floret <julien.floret@6wind.com> for the tip to get all
subdirs with a pure makefile command.

For the record, note that exported files for asm directories are a mix of
files listed by:
- include/uapi/asm-generic/Kbuild.asm;
- arch/<arch>/include/uapi/asm/Kbuild;
- arch/<arch>/include/asm/Kbuild.

Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: Russell King <rmk+kernel@armlinux.org.uk>
Acked-by: Mark Salter <msalter@redhat.com>
Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc)
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Makefile.headersinst: remove destination-y option

This option was added in commit c7bb349e7c25 ("kbuild: introduce destination-y
for exported headers") but never used in-tree.

Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Acked-by: Paul Bolle <pebolle@tiscali.nl>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Makefile.headersinst: cleanup input files

After the last three patches, all exported headers are under uapi/, thus
input-files2 are not needed anymore.
The side effect is that input-files1-name is exactly header-y.

Note also that input-files3-name is genhdr-y.

Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: collect shorthands into scripts/Kbuild.include

The shorthand "clean" is defined in both the top Makefile and
scripts/Makefile.clean. Likewise, the "hdr-inst" is defined in
both the top Makefile and scripts/Makefile.headersinst.

To reduce code duplication, this commit collects them into
scripts/Kbuild.include like the "build" and "modbuiltin" shorthands.
It requires scripts/Makefile.clean to include scripts/Kbuild.include,
but its impact on the performance of "make clean" should be
negligible.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: fix make headers_install when path is too long

If headers_install is executed from a deep/long directory structure, the
shell's maximum argument length can be execeeded, which breaks the operation
with:

| make[2]: execvp: /bin/sh: Argument list too long
| make[2]: ***

Instead of passing each files name with the entire path, I give only the file
name without the source path and give this path as a new argument to
headers_install.pl.

Because there is three possible paths, I have tree input-files list, one per
path.

Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Tested-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

headers_install.pl: convert to headers_install.sh

Remove perl from make headers_install by replacing a perl script (doing a
simple regex search and replace) with a smaller, faster, simpler,
POSIX-2008 shell script implementation. The new shell script is a single
for loop calling sed and piping its output through unifdef to produce the
target file.

Same as last time except for minor tweak to deal with code review from
here: http://lkml.indiana.edu/hypermail/linux/kernel/1302.3/00078.html

(Note that this drops the "arch" argument, which isn't used. Kbuild
already points to the right input files on the command line.)

Signed-off-by: Rob Landley <rob@landley.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Josh Boyer <jwboyer@redhat.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: David Howells <dhowell@redhat.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: fix make headers_check with make 3.80

Commit 10b63956 ("UAPI: Plumb the UAPI Kbuilds into the user header
installation and checking") introduced a dependency of make 3.81
due to use of $(or ...)

We do not want to lift the requirement to gmake 3.81 just yet...
Included are a straightforward conversion to $(if ...)

Bisected-and-tested-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: <stable@vger.kernel.org> [v3.7+]
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: Fix destination-y for installed headers

Commit 10b63956fce7f369cc37fd4d994f09bd5203efe4 which plumbed in UAPI
broke the destination-y functionality of scripts/Makefile.headersinst.

The variable destination-y is used in a := assignment and so is expanded at
declaration time, and the include of the Kbuild fragments that set
destination-y to something is after this time, so it now always expands empty.

There are no in-tree users of destination-y, but it allows any
Kbuild-fragment to redirect where headers are installed.

Just move the assignment of the variable that uses it below the include
of the Kbuild fragment.

Signed-off-by: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: Michal Marek <mmarek@suse.cz>
Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

UAPI: Plumb the UAPI Kbuilds into the user header installation and checking

Plumb the UAPI Kbuilds into the user header installation and checking system.
As the headers are split the entries will be transferred across from the old
Kbuild files to the UAPI Kbuild files.

The changes made in this commit are:

(1) Exported generated files (of which there are currently four) are moved to
uapi/ directories under the appropriate generated/ directory, thus we
get:

include/generated/uapi/linux/version.h
arch/x86/include/generated/uapi/asm/unistd_32.h
arch/x86/include/generated/uapi/asm/unistd_64.h
arch/x86/include/generated/uapi/asm/unistd_x32.h

These paths were added to the build as -I flags in a previous patch.

(2) scripts/Makefile.headersinst is now given the UAPI path to install from
rather than the old path.

It then determines the old path from that and includes that Kbuild also
if it exists, thus permitting the headers to exist in either directory
during the changeover.

I also renamed the "install" variable to "installdir" as it refers to a
directory not the install program.

(3) scripts/headers_install.pl is altered to take a list of source file paths
instead of just their names so that the makefile can tell it exactly
where to find each file.

For the moment, files can be obtained from one of four places for each
output directory:

.../include/uapi/foo/
.../include/generated/uapi/foo/
.../include/foo/
.../include/generated/foo/

The non-UAPI paths will be dropped later.

Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Dave Jones <davej@redhat.com>

UAPI: Remove the objhdr-y export list

Remove the objhdr-y export list as it is no longer used. genhdr-y should be
used instead.

Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Dave Jones <davej@redhat.com>

kbuild: Add support for installing generated asm headers

Generated asm headers are supposed to live in
arch/*/include/generated/asm, but objhdr-y expect them to live in the
same directory they are generated in. Instead of trying to cut that
particular Gordian knot, introduce genhdr-y that takes this into
account; the sole user of objhdr-y, linux/version.h, should be
migrated over at some later date.

Suggested-by: David Woodhouse <dwmw2@infradead.org>
Acked-by: David Woodhouse <dwmw2@infradead.org>
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: Michal Marek <mmarek@suse.cz>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>

kbuild: asm-generic support

There is an increasing amount of header files
shared between individual architectures in asm-generic.
To avoid a lot of dummy wrapper files that just
include the corresponding file in asm-generic provide
some basic support in kbuild for this.

With the following patch an architecture can maintain
a list of files in the file arch/$(ARCH)/include/asm/Kbuild

To use a generic file just add:

generic-y += <name-of-header-file.h>

For each file listed kbuild will generate the necessary
wrapper in arch/$(ARCH)/include/generated/asm.

When installing userspace headers a wrapper is likewise created.

The original inspiration for this came from the unicore32
patchset - although a different method is used.

The patch includes several improvements from Arnd Bergmann.
Michael Marek contributed Makefile.asm-generic.

Remis Baima did an intial implementation along to achive
the same - see https://patchwork.kernel.org/patch/13352/

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Acked-by: Guan Xuetao <guanxuetao@mprc.pku.edu.cn>
Tested-by: Guan Xuetao <guanxuetao@mprc.pku.edu.cn>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Cc: Remis Lima Baima <remis.developer@googlemail.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: drop unifdef-y support

unifdef-y is not used anymore - drop remaining references

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: fix "Argument list too long" error for "make headers_check",

I'm trying to install kernel headers to build a cross-toolchain, but got
the following:

make ARCH=arm
INSTALL_HDR_PATH=/work/psl/eldk-builds/arm-2009-04-21/work/var/tmp/crosstool-0.43-3-root/usr/crosstool/gcc-4.2.2-glibc-20070515T2025-eldk/
+arm-linux-gnueabi/arm-linux-gnueabi/
headers_check
...
CHECK include/linux/raid (2 files)
CHECK include/linux/spi (1 files)
CHECK include/linux/sunrpc (1 files)
CHECK include/linux/tc_act (6 files)
CHECK include/linux/tc_ematch (4 files)
CHECK include/linux/usb (8 files)
make[2]: execvp: /bin/sh: Argument list too long
make[2]: ***
[/work/psl/eldk-builds/arm-2009-04-21/work/var/tmp/crosstool-0.43-3-root/usr/crosstool/gcc-4.2.2-glibc-20070515T2025-eldk/arm-linux-gnueab
+i/arm-linux-gnueabi//include/linux/.check]
Error 127
make[1]: *** [linux] Error 2
make: *** [headers_check] Error 2
->

Introduce use of xargs to fix this.

Signed-off-by: Sergei Poselenov <sposelenov@emcraft.com>
Cc: Wolfgang Denk <wd@denx.de>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: introduce destination-y for exported headers

xtensa and arm have asked for a possibility to export headers
and locate them in a specific directory when exported.
Introduce destiantion-y to support this.

This patch in additiona adds some limited
documentation for the variables used for exported headers.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Oskar Schirmer <os@emlix.com>
Cc: Mikael Starvik <mikael.starvik@axis.com>

kbuild: install all headers when arch is changed

We see some header files that are selected dependent on
the actual architecture so force a reinstallation
of all header files when the arch changes.
This slows down "make headers_check_all" but then
we better reflect reality.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: optimize headers_* targets

Move the core functionality of headers_install
and headers_check to two small perl scripts.
The makefile is adapted to use the perl scrip and
changed to operate on all files in a directory.
So if one file is changed then all files in the
directory is processed.

perl were chosen for the helper scripts because this
is pure text processing which perl is good at and
especially the headers_check.pl script are expected to
see changes / new checks implmented.

The speed is ~300% faster on this box.
And the output generated to the screen is now down to
two lines per directory (one for install, one for check)
so it is easier to scroll back after a kernel build.

The perl scripts has been brought to sanity by patient
feedback from: Vegard Nossum <vegard.nossum@gmail.com>

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: only one call for include/ in make headers_*

Move it to the top-level file to decide if we install/check
the generic headers or the arch specific headers.

This revealed a long standing bug where "make headers_check_all"
relied on the files in asm/ for the current architecture.
So make headers_check_all is now broken by this commit.

In addition:

o add a simpler way to detect if an arch support
exporting header files.

o add 'set -e;' so we error out early if
make headers_check_all fails.

o add sparc64 and cris to arch we do not process
in make headers_*_all because:

sparc64 - use sparc to export headers
cris - is know seriously broken

Includes suggestions from: David Woodhouse
<dwmw2@infradead.org>.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: David Woodhouse <dwmw2@infradead.org>

kbuild: code refactoring in Makefile.headerinst

No functional changes just improved readability

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: drop support of ALTARCH for headers_*

ALTARCH is no longer used by any arch(*) so drop
support for this from Makefile.headerinst

Dropping ALTARCH support simplifies Makefile.headerinst

(*) sparc64 uses it but work is ongoing to drop it
and no furter usage is planned.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: David Miller <davem@davemloft.net>

kbuild: always unifdef files in headers_install*

unifdef utility is fast enough to warrant that we always
run the scripts through unifdef.

This patch runs all headers listed with header-y and unifdef-y
through unifdef.
Next step is to drop unifdef-y in all Kbuild files and
that can now be done in smaller steps.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Adrian Bunk <bunk@kernel.org>

kbuild: use POSIX BRE in headers install target

The sed expression used at the moment in scripts/Makefile.headersinst
relies on the (handy) GNU extension where you can escape ERE's in an
otherwise BRE without using the GNU -r option. The following patch
replaces this "\+" usage with a functionally equivalent POSIX BRE compliant
"\{1,\}". Tested with `make headers_install` against blackfin/x86_64/i386
targets.

Stupid whiny OS X users and their crappy sed ;)

Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

x86_64: fix headers_install

A bug in headers_install for ARCH=x86_64 yields an asm/ directory full of
files all of which are using the same #ifdef guard, "__ASM_STUB_" with no
postfix. So the second and later asm files #included in the same C file
(often through standard headers like ioctl.h) yields no symbols.

Strangeness with the Ubuntu 'tell me if I support something that's not
explcitly mentioned in POSIX, and I'll strip it out' shell, I believe.

We don't need the 'export' but we do need a semicolon at the end of the
FNAME line:

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Rob Landley <rob@landley.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Revert accidental commit to scripts/Makefile.headersinst

Commit 43dfa07fbb6b8bd5b6173a5bab48470f578c8e5b accidentally included a
fix to scripts/Makefile.headersinst which it should not have done. Revert.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>

[JFFS2] Deletion dirents should be REF_NORMAL, not REF_PRISTINE.

Otherwise they'll never actually get garbage-collected.
Noted by Jonathan Larmour.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>

Make 'headerscheck' stop immediately on an error

This should make it stop immediately after printing the _helpful_ error
message, rather than continuing to spit out many pages more of 'CHECK
include/linux/foo.h' before eventually coming to a halt with something
less obvious.

Now I get this...
CHECK include/linux/smb_fs.h
/shiny/git/linux-2.6/usr/include/linux/smb_fs.h requires linux/jiffies.h, which does not exist in exported headers
make[2]: *** [/shiny/git/linux-2.6/usr/include/linux/.check.smb_fs.h] Error 1
make[1]: *** [linux] Error 2
make: *** [headers_check] Error 2

Signed-off-by-if-Sam-says-so: David Woodhouse <dwmw2@infradead.org>
[ Sam had better say so! This made me waste way too much time. - Linus]
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

[PATCH] translate dashes in filenames for headers install

The current filename->define translation does not scrub dashes so when
creating stub defines for like asm-x86_64/ptrace-abi.h, we get: #define
__ASM_STUB_PTRACE-ABI_H

gcc just hates that sort of thing :)

trivial attached patch adds - to the tr list to scrub it to _

Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

[PATCH] fix `make headers_install'

Fix this:

make[3]: *** No rule to make target
`/mnt/md0/devel/linux-git/include/linux/version.h', needed by
`/mnt/md0/devel/linux-git-obj/usr/include/linux/version.h'. Stop.
make[2]: *** [linux] Error 2
make[1]: *** [headers_install] Error 2

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

[PATCH] hdrcheck permission fix

Don't require that scripts/hdrcheck.sh be executable - shit happens...

Cc: Sam Ravnborg <sam@ravnborg.org>
Acked-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

kbuild: use in-kernel unifdef

Let headers_install use in-kernel unifdef

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

Use dependencies for 'make headers_install'.

Re-export header files only if either they or their controlling Kbuild
file has actually changed. Also allow for similar dependencies with
'headers_check', once we properly create the dependencies for those.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>

[PATCH] Fix 'make headers_check' on biarch architectures

We generate an <asm/foo.h> which includes either <asm-$ARCH/foo.h> or
<asm-$ALTARCH/foo.h> as appropriate. But we were doing this dependent on
whether the file in question existed in the _unexported_ tree, not the
exported tree. So if a file was exported to userspace in one asm- directory
but not the other, the generated file in asm/ was incorrect.

This only changed the failure mode if it _was_ included from a nice #error to
a less explicable #include failure -- but it also gave false errors in 'make
headers_check' output. Fix it by looking in the right place instead.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Cc: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

Basic implementation of 'make headers_check'

Based on the 'headers_install' target, this performs a basic sanity check
on the exported headers -- so far only checking that they do not include
any other headers which aren't selected for import, but easily extendable.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>

Basic implementation of 'make headers_install'

This adds a make target which exports a subset of headers which contain
definitions which are useful for system libraries and tools. It uses the
BSD 'unifdef' tool to remove instances of #ifdef __KERNEL__, and uses
sed to remove markers like __user.

Based on an original implementation by Arnd Bergmann <arnd@arndb.de>
Hacked about by David Woodhouse <dwmw2@infradead.org>
Reviewed and cleaned up by Sam Ravnborg <sam@ravnborg.org>

Signed-off-by: David Woodhouse <dwmw2@infradead.org>