kbuild: change DTC_FLAGS_<basetarget>.o to take the path relative to $(obj)

For the same rationale as commit 54b8ae66ae1a ("kbuild: change
*FLAGS_<basetarget>.o to take the path relative to $(obj)").

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

kbuild: change tool coverage variables to take the path relative to $(obj)

Commit 54b8ae66ae1a ("kbuild: change *FLAGS_<basetarget>.o to take the
path relative to $(obj)") changed the syntax of per-file compiler flags.

The situation is the same for the following variables:

OBJECT_FILES_NON_STANDARD_<basetarget>.o
GCOV_PROFILE_<basetarget>.o
KASAN_SANITIZE_<basetarget>.o
KMSAN_SANITIZE_<basetarget>.o
KMSAN_ENABLE_CHECKS_<basetarget>.o
UBSAN_SANITIZE_<basetarget>.o
KCOV_INSTRUMENT_<basetarget>.o
KCSAN_SANITIZE_<basetarget>.o
KCSAN_INSTRUMENT_BARRIERS_<basetarget>.o

The <basetarget> is the filename of the target with its directory and
suffix stripped.

This syntax comes into a trouble when two files with the same basename
appear in one Makefile, for example:

obj-y += dir1/foo.o
obj-y += dir2/foo.o
OBJECT_FILES_NON_STANDARD_foo.o := y

OBJECT_FILES_NON_STANDARD_foo.o is applied to both dir1/foo.o and
dir2/foo.o. This syntax is not flexbile enough to handle cases where
one of them is a standard object, but the other is not.

It is more sensible to use the relative path to the Makefile, like this:

obj-y += dir1/foo.o
OBJECT_FILES_NON_STANDARD_dir1/foo.o := y
obj-y += dir2/foo.o
OBJECT_FILES_NON_STANDARD_dir2/foo.o := y

To maintain the current behavior, I made adjustments to the following two
Makefiles:

- arch/x86/entry/vdso/Makefile, which compiles vclock_gettime.o, vgetcpu.o,
and their vdso32 variants.

- arch/x86/kvm/Makefile, which compiles vmx/vmenter.o and svm/vmenter.o

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Acked-by: Sean Christopherson <seanjc@google.com>

kbuild: create a list of all built DTB files

It is useful to have a list of all *.dtb and *.dtbo files generated
from the current build.

With this commit, 'make dtbs' creates arch/*/boot/dts/dtbs-list, which
lists the dtb(o) files created in the current build. It maintains the
order of the dtb-y additions in Makefiles although the order is not
important for DTBs. It is a (good) side effect through the reuse of the
modules.order rule.

Please note this list only includes the files directly added to dtb-y.

For example, consider this case:

foo-dtbs := foo_base.dtb foo_overlay.dtbo
dtb-y := foo.dtb

In this example, the list will include foo.dtb, but not foo_base.dtb
or foo_overlay.dtbo.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

dtc: Enable dtc interrupt_provider check

Now that all the interrupt warnings have been fixed, enable
'interrupt_provider' check by default. This will also enable
'interrupt_map' check.

Signed-off-by: Rob Herring <robh@kernel.org>
Link: https://lore.kernel.org/r/20240213-arm-dt-cleanups-v1-6-f2dee1292525@kernel.org
Signed-off-by: Arnd Bergmann <arnd@arndb.de>

ubsan: Reintroduce signed overflow sanitizer

In order to mitigate unexpected signed wrap-around[1], bring back the
signed integer overflow sanitizer. It was removed in commit 6aaa31aeb9cf
("ubsan: remove overflow checks") because it was effectively a no-op
when combined with -fno-strict-overflow (which correctly changes signed
overflow from being "undefined" to being explicitly "wrap around").

Compilers are adjusting their sanitizers to trap wrap-around and to
detecting common code patterns that should not be instrumented
(e.g. "var + offset < var"). Prepare for this and explicitly rename
the option from "OVERFLOW" to "WRAP" to more accurately describe the
behavior.

To annotate intentional wrap-around arithmetic, the helpers
wrapping_add/sub/mul_wrap() can be used for individual statements. At
the function level, the __signed_wrap attribute can be used to mark an
entire function as expecting its signed arithmetic to wrap around. For a
single object file the Makefile can use "UBSAN_SIGNED_WRAP_target.o := n"
to mark it as wrapping, and for an entire directory, "UBSAN_SIGNED_WRAP :=
n" can be used.

Additionally keep these disabled under CONFIG_COMPILE_TEST for now.

Link: https://github.com/KSPP/linux/issues/26 [1]
Cc: Miguel Ojeda <ojeda@kernel.org>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Hao Luo <haoluo@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Justin Stitt <justinstitt@google.com>
Signed-off-by: Kees Cook <keescook@chromium.org>

ubsan: Remove CONFIG_UBSAN_SANITIZE_ALL

For simplicity in splitting out UBSan options into separate rules,
remove CONFIG_UBSAN_SANITIZE_ALL, effectively defaulting to "y", which
is how it is generally used anyway. (There are no ":= y" cases beyond
where a specific file is enabled when a top-level ":= n" is in effect.)

Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Marco Elver <elver@google.com>
Cc: linux-doc@vger.kernel.org
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Kees Cook <keescook@chromium.org>

x86/bugs: Rename CONFIG_RETHUNK => CONFIG_MITIGATION_RETHUNK

Step 10/10 of the namespace unification of CPU mitigations related Kconfig options.

[ mingo: Added one more case. ]

Suggested-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Josh Poimboeuf <jpoimboe@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lore.kernel.org/r/20231121160740.1249350-11-leitao@debian.org

x86/bugs: Rename CONFIG_SLS => CONFIG_MITIGATION_SLS

Step 6/10 of the namespace unification of CPU mitigations related Kconfig options.

Suggested-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Josh Poimboeuf <jpoimboe@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lore.kernel.org/r/20231121160740.1249350-7-leitao@debian.org

x86/bugs: Rename CONFIG_RETPOLINE => CONFIG_MITIGATION_RETPOLINE

Step 5/10 of the namespace unification of CPU mitigations related Kconfig options.

[ mingo: Converted a few more uses in comments/messages as well. ]

Suggested-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Ariel Miculas <amiculas@cisco.com>
Acked-by: Josh Poimboeuf <jpoimboe@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lore.kernel.org/r/20231121160740.1249350-6-leitao@debian.org

x86/bugs: Rename CONFIG_CALL_DEPTH_TRACKING => CONFIG_MITIGATION_CALL_DEPTH_TRACKING

Step 3/10 of the namespace unification of CPU mitigations related Kconfig options.

Suggested-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Josh Poimboeuf <jpoimboe@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lore.kernel.org/r/20231121160740.1249350-4-leitao@debian.org

kbuild: determine base DTB by suffix

When using the -dtbs syntax, you need to list the base first, as
follows:

foo-dtbs := foo_base.dtb foo_overlay1.dtbo foo_overlay2.dtbo
dtb-y := foo.dtb

You cannot do this arrangement:

foo-dtbs := foo_overlay1.dtbo foo_overlay2.dtbo foo_base.dtb

This restriction comes from $(firstword ...) in the current
implementation, but it is unneeded to rely on the order in the
-dtbs syntax.

Instead, you can simply determine the base by the suffix because
the base (*.dtb) and overlays (*.dtbo) use different suffixes.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

kbuild: Move the single quotes for image name

Add quotes where UIMAGE_NAME is used, rather than where it is defined.
This allows the UIMAGE_NAME variable to be set by the user.

Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: Disallow DTB overlays to built from .dts named source files

As a follow up to the series allowing DTB overlays to built from .dtso
files. Now that all overlays have been renamed, remove the ability to
build from overlays from .dts files to prevent any files with the old
name from accidental being added.

Signed-off-by: Andrew Davis <afd@ti.com>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Acked-by: Andy Shevchenko <andriy.shevchenko@intel.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: unify cmd_dt_S_dtb and cmd_dt_S_dtbo

cmd_dt_S_dtb and cmd_dt_S_dtbo are almost the same; the only difference
is the prefix of the begin/end symbols. (__dtb vs __dtbo)

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

objtool: Add --mnop as an option to --mcount

Some architectures (powerpc) may not support ftrace locations being nop'ed
out at build time. Introduce CONFIG_HAVE_OBJTOOL_NOP_MCOUNT for objtool, as
a means for architectures to enable nop'ing of ftrace locations. Add --mnop
as an option to objtool --mcount, to indicate support for the same.

Also, make sure that --mnop can be passed as an option to objtool only when
--mcount is passed.

Tested-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Reviewed-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Acked-by: Josh Poimboeuf <jpoimboe@kernel.org>
Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Sathvika Vasireddy <sv@linux.ibm.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20221114175754.1131267-12-sv@linux.ibm.com

x86/ibt: Implement FineIBT

Implement an alternative CFI scheme that merges both the fine-grained
nature of kCFI but also takes full advantage of the coarse grained
hardware CFI as provided by IBT.

To contrast:

kCFI is a pure software CFI scheme and relies on being able to read
text -- specifically the instruction *before* the target symbol, and
does the hash validation *before* doing the call (otherwise control
flow is compromised already).

FineIBT is a software and hardware hybrid scheme; by ensuring every
branch target starts with a hash validation it is possible to place
the hash validation after the branch. This has several advantages:

o the (hash) load is avoided; no memop; no RX requirement.

o IBT WAIT-FOR-ENDBR state is a speculation stop; by placing
the hash validation in the immediate instruction after
the branch target there is a minimal speculation window
and the whole is a viable defence against SpectreBHB.

o Kees feels obliged to mention it is slightly more vulnerable
when the attacker can write code.

Obviously this patch relies on kCFI, but additionally it also relies
on the padding from the call-depth-tracking patches. It uses this
padding to place the hash-validation while the call-sites are
re-written to modify the indirect target to be 16 bytes in front of
the original target, thus hitting this new preamble.

Notably, there is no hardware that needs call-depth-tracking (Skylake)
and supports IBT (Tigerlake and onwards).

Suggested-by: Joao Moreira (Intel) <joao@overdrivepizza.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Link: https://lore.kernel.org/r/20221027092842.634714496@infradead.org

x86: Add prefix symbols for function padding

When code is compiled with:

-fpatchable-function-entry=${PADDING_BYTES},${PADDING_BYTES}

functions will have PADDING_BYTES of NOP in front of them. Unwinders
and other things that symbolize code locations will typically
attribute these bytes to the preceding function.

Given that these bytes nominally belong to the following symbol this
mis-attribution is confusing.

Inspired by the fact that CFI_CLANG emits __cfi_##name symbols to
claim these bytes, use objtool to emit __pfx_##name symbols to do
the same when CFI_CLANG is not used.

This then shows the callthunk for symbol 'name' as:

__pfx_##name+0x6/0x10

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Yujie Liu <yujie.liu@intel.com>
Link: https://lkml.kernel.org/r/20221028194453.592512209@infradead.org

objtool: Add --hacks=skylake

Make the call/func sections selectable via the --hacks option.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20220915111146.120821440@infradead.org

kbuild: Allow DTB overlays to built into .dtbo.S files

DTB files can be built into the kernel by converting them to assembly
files then assembling them into object files. We extend this here
for DTB overlays with the .dtso extensions.

We change the start and end delimiting tag prefix to make it clear that
this data came from overlay files.

[Based on patch by Frank Rowand <frank.rowand@sony.com>]

Signed-off-by: Andrew Davis <afd@ti.com>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>
Reviewed-by: Frank Rowand <frowand.list@gmail.com>
Tested-by: Frank Rowand <frowand.list@gmail.com>
Link: https://lore.kernel.org/r/20221024173434.32518-3-afd@ti.com
Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: Allow DTB overlays to built from .dtso named source files

Currently DTB Overlays (.dtbo) are build from source files with the same
extension (.dts) as the base DTs (.dtb). This may become confusing and
even lead to wrong results. For example, a composite DTB (created from a
base DTB and a set of overlays) might have the same name as one of the
overlays that create it.

Different files should be generated from differently named sources.
.dtb <-> .dts
.dtbo <-> .dtso

We do not remove the ability to compile DTBO files from .dts files here,
only add a new rule allowing the .dtso file name. The current .dts named
overlays can be renamed with time. After all have been renamed we can
remove the other rule.

Signed-off-by: Andrew Davis <afd@ti.com>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>
Reviewed-by: Frank Rowand <frowand.list@gmail.com>
Tested-by: Frank Rowand <frowand.list@gmail.com>
Link: https://lore.kernel.org/r/20221024173434.32518-2-afd@ti.com
Signed-off-by: Rob Herring <robh@kernel.org>

scripts: dtc: only show unique unit address warning for enabled nodes

In some cases an hardware peripheral can be used for two exclusive usages.
For example, on STM32MP15 we have the same peripheral for I2S and SPI. We
have dedicated driver for each usage and so a dedicated device node in
devicetree.
To avoid to get useless warnings running "make W=1 dtbs", this patch adds
the "-Wunique_unit_address_if_enabled" flag for a make with W=1. In this
case we will detect a duplicate address only if both devices are
enabled in the devicetree, which is a real error case.

Signed-off-by: Alexandre Torgue <alexandre.torgue@foss.st.com>
Link: https://lore.kernel.org/r/20221021084447.5550-1-alexandre.torgue@foss.st.com
[robh: Refactor options and keep 'unique_unit_address' for W=2]
Signed-off-by: Rob Herring <robh@kernel.org>

kmsan: add KMSAN runtime core

For each memory location KernelMemorySanitizer maintains two types of
metadata:

1. The so-called shadow of that location - а byte:byte mapping describing
whether or not individual bits of memory are initialized (shadow is 0)
or not (shadow is 1).
2. The origins of that location - а 4-byte:4-byte mapping containing
4-byte IDs of the stack traces where uninitialized values were
created.

Each struct page now contains pointers to two struct pages holding KMSAN
metadata (shadow and origins) for the original struct page. Utility
routines in mm/kmsan/core.c and mm/kmsan/shadow.c handle the metadata
creation, addressing, copying and checking. mm/kmsan/report.c performs
error reporting in the cases an uninitialized value is used in a way that
leads to undefined behavior.

KMSAN compiler instrumentation is responsible for tracking the metadata
along with the kernel memory. mm/kmsan/instrumentation.c provides the
implementation for instrumentation hooks that are called from files
compiled with -fsanitize=kernel-memory.

To aid parameter passing (also done at instrumentation level), each
task_struct now contains a struct kmsan_task_state used to track the
metadata of function parameters and return values for that task.

Finally, this patch provides CONFIG_KMSAN that enables KMSAN, and declares
CFLAGS_KMSAN, which are applied to files compiled with KMSAN. The
KMSAN_SANITIZE:=n Makefile directive can be used to completely disable
KMSAN instrumentation for certain files.

Similarly, KMSAN_ENABLE_CHECKS:=n disables KMSAN checks and makes newly
created stack memory initialized.

Users can also use functions from include/linux/kmsan-checks.h to mark
certain memory regions as uninitialized or initialized (this is called
"poisoning" and "unpoisoning") or check that a particular region is
initialized.

Link: https://lkml.kernel.org/r/20220915150417.722975-12-glider@google.com
Signed-off-by: Alexander Potapenko <glider@google.com>
Acked-by: Marco Elver <elver@google.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Christoph Lameter <cl@linux.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Eric Biggers <ebiggers@google.com>
Cc: Eric Biggers <ebiggers@kernel.org>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Ilya Leoshkevich <iii@linux.ibm.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

kbuild: list sub-directories in ./Kbuild

Use the ordinary obj-y syntax to list subdirectories.

Note1:
Previously, the link order of lib-y depended on CONFIG_MODULES; lib-y
was linked before drivers-y when CONFIG_MODULES=y, otherwise after
drivers-y. This was a bug of commit 7273ad2b08f8 ("kbuild: link lib-y
objects to vmlinux forcibly when CONFIG_MODULES=y"), but it was not a
big deal after all. Now, all objects listed in lib-y are linked last,
irrespective of CONFIG_MODULES.

Note2:
Finally, the single target build in arch/*/lib/ works correctly. There was
a bug report about this. [1]

$ make ARCH=arm arch/arm/lib/findbit.o
CALL scripts/checksyscalls.sh
AS arch/arm/lib/findbit.o

[1]: https://lore.kernel.org/linux-kbuild/YvUQOwL6lD4%2F5%2FU6@shell.armlinux.org.uk/

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

kbuild: use objtool-args-y to clean up objtool arguments

Based on Linus' patch. Refactor scripts/Makefile.vmlinux_o as well.

Link: https://lore.kernel.org/lkml/CAHk-=wgjTMQgiKzBZTmb=uWGDEQxDdyF1+qxBkODYciuNsmwnw@mail.gmail.com/
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

kbuild: take into account DT_SCHEMA_FILES changes while checking dtbs

It is useful to be able to recheck dtbs files against a limited set of
DT schema files. This can be accomplished by using differnt
DT_SCHEMA_FILES argument values while rerunning make dtbs_check. However
for some reason if_changed_rule doesn't pick up the rule_dtc changes
(and doesn't retrigger the build).

Fix this by changing if_changed_rule to if_changed_dep and squashing DTC
and dt-validate into a single new command. Then if_changed_dep triggers
on DT_SCHEMA_FILES changes and reruns the build/check.

Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Link: https://lore.kernel.org/r/20220915114422.79378-1-dmitry.baryshkov@linaro.org
Signed-off-by: Rob Herring <robh@kernel.org>

Kbuild: add Rust support

Having most of the new files in place, we now enable Rust support
in the build system, including `Kconfig` entries related to Rust,
the Rust configuration printer and a few other bits.

Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Co-developed-by: Alex Gaynor <alex.gaynor@gmail.com>
Signed-off-by: Alex Gaynor <alex.gaynor@gmail.com>
Co-developed-by: Finn Behrens <me@kloenk.de>
Signed-off-by: Finn Behrens <me@kloenk.de>
Co-developed-by: Adam Bratschi-Kaye <ark.email@gmail.com>
Signed-off-by: Adam Bratschi-Kaye <ark.email@gmail.com>
Co-developed-by: Wedson Almeida Filho <wedsonaf@google.com>
Signed-off-by: Wedson Almeida Filho <wedsonaf@google.com>
Co-developed-by: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Co-developed-by: Sven Van Asbroeck <thesven73@gmail.com>
Signed-off-by: Sven Van Asbroeck <thesven73@gmail.com>
Co-developed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Gary Guo <gary@garyguo.net>
Co-developed-by: Boris-Chengbiao Zhou <bobo1239@web.de>
Signed-off-by: Boris-Chengbiao Zhou <bobo1239@web.de>
Co-developed-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Co-developed-by: Douglas Su <d0u9.su@outlook.com>
Signed-off-by: Douglas Su <d0u9.su@outlook.com>
Co-developed-by: Dariusz Sosnowski <dsosnowski@dsosnowski.pl>
Signed-off-by: Dariusz Sosnowski <dsosnowski@dsosnowski.pl>
Co-developed-by: Antonio Terceiro <antonio.terceiro@linaro.org>
Signed-off-by: Antonio Terceiro <antonio.terceiro@linaro.org>
Co-developed-by: Daniel Xu <dxu@dxuuu.xyz>
Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
Co-developed-by: Björn Roy Baron <bjorn3_gh@protonmail.com>
Signed-off-by: Björn Roy Baron <bjorn3_gh@protonmail.com>
Co-developed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Signed-off-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>

x86/retbleed: Add fine grained Kconfig knobs

Do fine-grained Kconfig for all the various retbleed parts.

NOTE: if your compiler doesn't support return thunks this will
silently 'upgrade' your mitigation to IBPB, you might not like this.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Borislav Petkov <bp@suse.de>

kbuild: factor out the common objtool arguments

scripts/Makefile.build and scripts/link-vmlinux.sh have similar setups
for the objtool arguments.

It was difficult to factor out them because all the vmlinux build rules
were written in a shell script. It is somewhat tedious to touch the two
files every time a new objtool option is supported.

To reduce the code duplication, move the objtool for vmlinux.o into
scripts/Makefile.vmlinux_o. Then, move the common macros to Makefile.lib
so they are shared between Makefile.build and Makefile.vmlinux_o.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)

kbuild: do not create *.prelink.o for Clang LTO or IBT

When CONFIG_LTO_CLANG=y, additional intermediate *.prelink.o is created
for each module. Also, objtool is postponed until LLVM IR is converted
to ELF.

CONFIG_X86_KERNEL_IBT works in a similar way to postpone objtool until
objects are merged together.

This commit stops generating *.prelink.o, so the build flow will look
similar with/without LTO.

The following figures show how the LTO build currently works, and
how this commit is changing it.

Current build flow
==================

[1] single-object module

$(LD)
$(CC) +objtool $(LD)
foo.c --------------------> foo.o -----> foo.prelink.o -----> foo.ko
(LLVM IR) (ELF) | (ELF)
|
foo.mod.o --/
(LLVM IR)

[2] multi-object module
$(LD)
$(CC) $(AR) +objtool $(LD)
foo1.c -----> foo1.o -----> foo.o -----> foo.prelink.o -----> foo.ko
| (archive) (ELF) | (ELF)
foo2.c -----> foo2.o --/ |
(LLVM IR) foo.mod.o --/
(LLVM IR)

One confusion is that foo.o in multi-object module is an archive
despite of its suffix.

New build flow
==============

[1] single-object module

Since there is only one object, there is no need to keep the LLVM IR.
Use $(CC)+$(LD) to generate an ELF object in one build rule. When LTO
is disabled, $(LD) is unneeded because $(CC) produces an ELF object.

$(CC)+$(LD)+objtool $(LD)
foo.c ----------------------------> foo.o ---------> foo.ko
(ELF) | (ELF)
|
foo.mod.o --/
(LLVM IR)

[2] multi-object module

Previously, $(AR) was used to combine LLVM IR files into an archive,
but there was no technical reason to do so. Use $(LD) to merge them
into a single ELF object.

$(LD)
$(CC) +objtool $(LD)
foo1.c ---------> foo1.o ---------> foo.o ---------> foo.ko
| (ELF) | (ELF)
foo2.c ---------> foo2.o ----/ |
(LLVM IR) foo.mod.o --/
(LLVM IR)

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
Acked-by: Josh Poimboeuf <jpoimboe@kernel.org>

kbuild: make multi_depend work with targets in subdirectory

Precisely speaking, when you get the stem of the path, you should use
$(patsubst $(obj)/%,%,...) instead of $(notdir ...).

I do not see this usecase, but if you create a composite object in a
subdirectory, the Makefile should look like this:

obj-$(CONFIG_FOO) += dir/foo.o
dir/foo-objs := dir/foo1.o dir/foo2.o

The member objects should be assigned to dir/foo-objs instead of
foo-objs.

This syntax is more consistent with commit 54b8ae66ae1a ("kbuild:
change *FLAGS_<basetarget>.o to take the path relative to $(obj)").

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

kbuild: reuse suffix-search to refactor multi_depend

The complicated part of multi_depend is the same as suffix-search.

Reuse it.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

kbuild: replace $(if A,A,B) with $(or A,B)

$(or ...) is available since GNU Make 3.81, and useful to shorten the
code in some places.

Covert as follows:

$(if A,A,B) --> $(or A,B)

This patch also converts:

$(if A, A, B) --> $(or A, B)

Strictly speaking, the latter is not an equivalent conversion because
GNU Make keeps spaces after commas; if A is not empty, $(if A, A, B)
expands to " A", while $(or A, B) expands to "A".

Anyway, preceding spaces are not significant in the code hunks I touched.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

kbuild: unify cmd_copy and cmd_shipped

cmd_copy and cmd_shipped have similar functionality. The difference is
that cmd_copy uses 'cp' while cmd_shipped 'cat'.

Unify them into cmd_copy because this macro name is more intuitive.

Going forward, cmd_copy will use 'cat' to avoid the permission issue.
I also thought of 'cp --no-preserve=mode' but this option is not
mentioned in the POSIX spec [1], so I am keeping the 'cat' command.

[1]: https://pubs.opengroup.org/onlinepubs/009695299/utilities/cp.html
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.com>

kbuild: Fixup the IBT kbuild changes

Masahiro-san deemed my kbuild changes to support whole module objtool
runs too terrible to live and gracefully provided an alternative.

Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/CAK7LNAQ2mYMnOKMQheVi+6byUFE3KEkjm1zcndNUfe0tORGvug@mail.gmail.com

dt-bindings: kbuild: Use DTB files for validation

Switch the DT validation to use DTB files directly instead of a DTS to
YAML conversion.

The original motivation for supporting validation on DTB files was to
enable running validation on a running system (e.g. 'dt-validate
/sys/firmware/fdt') or other cases where the original source DTS is not
available.

The YAML format was not without issues. Using DTBs with the schema type
information solves some of those problems. The YAML format relies on the
DTS source level information including bracketing of properties, size
directives, and phandle tags all of which are lost in a DTB file. While
standardizing the bracketing is a good thing, it does cause a lot of
extra warnings and churn to fix them.

Another issue has been signed types are not validated correctly as sign
information is not propagated to YAML. Using the schema type information
allows for proper handling of signed types. YAML also can't represent
the full range of 64-bit integers as numbers are stored as floats by
most/all parsers.

The DTB validation works by decoding property values using the type
information in the schemas themselves. The main corner case this does
not work for is matrix types where neither dimension is fixed. For
now, checking the dimensions in these cases are skipped.

Signed-off-by: Rob Herring <robh@kernel.org>
Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>
Link: https://lore.kernel.org/r/20220310160513.1708182-3-robh@kernel.org

dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate

In preparation for supporting validation of DTB files, the full
processed schema will always be needed in order to extract type
information from it. Therefore, the processed schema containing only
what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has
gained an option, -l or --limit, to specify which schema(s) to use for
validation.

As the command line option is new, we the minimum dtschema version must be
updated.

Cc: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Rob Herring <robh@kernel.org>
Link: https://lore.kernel.org/r/20220310160513.1708182-2-robh@kernel.org

kbuild: add cmd_file_size

Some architectures support self-extracting kernel, which embeds the
compressed vmlinux.

It has 4 byte data at the end so the decompressor can know the vmlinux
size beforehand.

GZIP natively has it in the trailer, but for the other compression
algorithms, the hand-crafted trailer is added.

It is unneeded to generate such _corrupted_ compressed files because
it is possible to pass the size data as a separate file.

For example, the assembly code:

.incbin "compressed-vmlinux-with-size-data"

can be transformed to:

.incbin "compressed-vmlinux"
.incbin "size-data"

My hope is, after some reworks of the decompressors, the macros
cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}_with_size will go away.

This new macro, cmd_file_size, will be useful to generate a separate
size-data file.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>

kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}

GZIP-compressed files end with 4 byte data that represents the size
of the original input. The decompressors (the self-extracting kernel)
exploit it to know the vmlinux size beforehand. To mimic the GZIP's
trailer, Kbuild provides cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}.
Unfortunately these macros are used everywhere despite the appended
size data is only useful for the decompressors.

There is no guarantee that such hand-crafted trailers are safely ignored.
In fact, the kernel refuses compressed initramdfs with the garbage data.
That is why usr/Makefile overrides size_append to make it no-op.

To limit the use of such broken compressed files, this commit renames
the existing macros as follows:

cmd_bzip2 --> cmd_bzip2_with_size
cmd_lzma --> cmd_lzma_with_size
cmd_lzo --> cmd_lzo_with_size
cmd_lz4 --> cmd_lz4_with_size
cmd_xzkern --> cmd_xzkern_with_size
cmd_zstd22 --> cmd_zstd22_with_size

To keep the decompressors working, I updated the following Makefiles
accordingly:

arch/arm/boot/compressed/Makefile
arch/h8300/boot/compressed/Makefile
arch/mips/boot/compressed/Makefile
arch/parisc/boot/compressed/Makefile
arch/s390/boot/compressed/Makefile
arch/sh/boot/compressed/Makefile
arch/x86/boot/compressed/Makefile

I reused the current macro names for the normal usecases; they produce
the compressed data in the proper format.

I did not touch the following:

arch/arc/boot/Makefile
arch/arm64/boot/Makefile
arch/csky/boot/Makefile
arch/mips/boot/Makefile
arch/riscv/boot/Makefile
arch/sh/boot/Makefile
kernel/Makefile

This means those Makefiles will stop appending the size data.

I dropped the 'override size_append' hack from usr/Makefile.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>

kbuild: drop $(size_append) from cmd_zstd

The appended file size is only used by the decompressors, which some
architectures support.

As the comment "zstd22 is used for kernel compression" says, cmd_zstd22
is used in arch/{mips,s390,x86}/boot/compressed/Makefile.

On the other hand, there is no good reason to append the file size to
cmd_zstd since it is used for other purposes.

Actually cmd_zstd is only used in usr/Makefile, where the appended file
size is rather harmful.

The initramfs with its file size appended is considered as corrupted
data, so commit 65e00e04e5ae ("initramfs: refactor the initramfs build
rules") added 'override size_append := :' to make it no-op.

As a conclusion, this $(size_append) should not exist here.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>

kcsan, kbuild: Add option for barrier instrumentation only

Source files that disable KCSAN via KCSAN_SANITIZE := n, remove all
instrumentation, including explicit barrier instrumentation. With
instrumentation for memory barriers, in few places it is required to
enable just the explicit instrumentation for memory barriers to avoid
false positives.

Providing the Makefile variable KCSAN_INSTRUMENT_BARRIERS_obj.o or
KCSAN_INSTRUMENT_BARRIERS (for all files) set to 'y' only enables the
explicit barrier instrumentation.

Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

kbuild: move objtool_args back to scripts/Makefile.build

Commit b1a1a1a09b46 ("kbuild: lto: postpone objtool") moved objtool_args
to Makefile.lib, so the arguments can be used in Makefile.modfinal as
well as Makefile.build.

With commit 850ded46c642 ("kbuild: Fix TRIM_UNUSED_KSYMS with
LTO_CLANG"), module LTO linking came back to scripts/Makefile.build
again.

So, there is no more reason to keep objtool_args in a separate file.

Get it back to the original place, close to the objtool command.

Remove the stale comment too.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>

kbuild: Enable dtc 'unit_address_format' warning by default

With all the 'unit_address_format' warnings fixed, enable the warning by
default.

Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: linux-kbuild@vger.kernel.org
Acked-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Rob Herring <robh@kernel.org>
Acked-by: Nick Desaulniers <ndesaulniers@google.com>
Link: https://lore.kernel.org/r/20210913192816.1225025-9-robh@kernel.org

kbuild: clean up objtool_args slightly

The code:

$(if $(or $(CONFIG_GCOV_KERNEL),$(CONFIG_LTO_CLANG)), ...)

... can be simpled to:

$(if $(CONFIG_GCOV_KERNEL)$(CONFIG_LTO_CLANG), ...)

Also, remove meaningless commas at the end of $(if ...).

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: Fix TRIM_UNUSED_KSYMS with LTO_CLANG

With CONFIG_LTO_CLANG, we currently link modules into native
code just before modpost, which means with TRIM_UNUSED_KSYMS
enabled, we still look at the LLVM bitcode in the .o files when
generating the list of used symbols. As the bitcode doesn't
yet have calls to compiler intrinsics and llvm-nm doesn't see
function references that only exist in function-level inline
assembly, we currently need a whitelist for TRIM_UNUSED_KSYMS to
work with LTO.

This change moves module LTO linking to happen earlier, and
thus avoids the issue with LLVM bitcode and TRIM_UNUSED_KSYMS
entirely, allowing us to also drop the whitelist from
gen_autoksyms.sh.

Link: https://github.com/ClangBuiltLinux/linux/issues/1369
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Alexander Lobakin <alobakin@pm.me>
Tested-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: Enable dtc 'pci_device_reg' warning by default

There's only a couple of instances of the 'pci_device_reg' warnings left
and they look legit, so let's enable the warning by default.

Cc: Heiko Stuebner <heiko@sntech.de>
Cc: Nicolas Saenz Julienne <nsaenz@kernel.org>
Cc: soc@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Link: https://lore.kernel.org/r/20210820165011.3257112-1-robh@kernel.org/

kbuild: refactor modname-multi by using suffix-search

Improve the readability slightly.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: refactor fdtoverlay rule

Rename overlay-y to multi-dtb-y, which is a consistent name with
multi-obj-y. Also, use multi-search to avoid code duplication.

Introduce real-dtb-y, which is a consistent name with real-obj-y,
to contain primitive blobs compiled from *.dts. This is used to
calculate the list of *.dt.yaml files.

Set -@ to base DTB without using $(eval ).

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: parameterize the .o part of suffix-search

The suffix-search macro hard-codes the suffix, '.o'.

Make it a parameter so that the multi-search and real-search macros
can be reused for foo-dtbs syntax introduced by commit 15d16d6dadf6
("kbuild: Add generic rule to apply fdtoverlay").

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: add comment about why cmd_shipped uses 'cat'

cmd_shipped uses 'cat' instead of 'cp' for copying a file. The reason
is explained in the commit [1], but it was in the pre-git era.

$ touch a
$ chmod -w a
$ cp a b
$ cp a b
cp: cannot create regular file 'b': Permission denied

Add comments so that you can see the reason without looking into
the history.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/commit/?id=a70dba8086160449cc94c5bdaff78419b6b8e3c8

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: move $(strip ) to suffix-search definition

Move $(strip ...) to the callee from the callers of suffix-search. It
shortens the code slightly. Adding a space after a comma will not be
a matter. I also dropped parentheses from single character variables.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: rename multi-used-* to multi-obj-*

I think multi-obj-* is clearer, and more consistent with real-obj-*.

Rename as follows:

multi-used-y -> multi-obj-y
multi-used-m -> multi-obj-m
multi-used -> multi-obj-ym

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: Add generic rule to apply fdtoverlay

Add a generic rule to apply fdtoverlay in Makefile.lib, so every
platform doesn't need to carry the complex rule. This also automatically
adds "DTC_FLAGS_foo_base += -@" for all base files.

The platform's Makefile only needs to have this now:

foo-dtbs := foo_base.dtb foo_overlay1.dtbo foo_overlay2.dtbo
dtb-y := foo.dtb

We don't want to run schema checks on foo.dtb (as foo.dts doesn't exist)
and the Makefile is updated accordingly.

Acked-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Rob Herring <robh@kernel.org>
Co-developed-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Frank Rowand <frank.rowand@sony.com>
Tested-by: Frank Rowand <frank.rowand@sony.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Link: https://lore.kernel.org/r/20920b0df6b067aca4040459a9677d7d1d6d766a.1615354376.git.viresh.kumar@linaro.org

kbuild: Simplify builds with CONFIG_OF_ALL_DTBS

We update 'always-y' based on CONFIG_OF_ALL_DTBS three times. It would be
far more straight forward if we rather update dtb-y to include all .dtb
files if CONFIG_OF_ALL_DTBS is enabled.

Acked-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Frank Rowand <frank.rowand@sony.com>
Tested-by: Frank Rowand <frank.rowand@sony.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Link: https://lore.kernel.org/r/7fe7e5ef6ed75450ddf6c224b8adb53059e504e2.1615354376.git.viresh.kumar@linaro.org

kbuild: Enable DT undocumented compatible checks

dt-validate has an option to warn on any compatible strings which don't
match any schema. The option has recently been improved to fix false
positives, so let's enable the option. This is useful for tracking
compatibles which are undocumented or not yet converted to DT schema.
Previously, the only check of undocumented compatible strings has been
an imperfect checkpatch.pl check.

The option is enabled by default for 'dtbs_check'. This will add more
warnings, but some platforms are down to only a handful of these
warnings (good job!).

There's about 100 cases in the binding examples, so the option is
disabled until these are fixed. In the meantime, they can be checked
with:

make DT_CHECKER_FLAGS=-m dt_binding_check

Cc: Maxime Ripard <mripard@kernel.org>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Link: https://lore.kernel.org/r/20210311233640.1581526-2-robh@kernel.org

kbuild: remove meaningless parameter to $(call if_changed_rule,dtc)

This is a remnant of commit 78046fabe6e7 ("kbuild: determine the output
format of DTC by the target suffix").

The parameter "yaml" is meaningless because cmd_dtc no loner takes $(2).

Reported-by: Rob Herring <robh@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: remove unneeded -O option to dtc

This piece of code converts the target suffix to the dtc -O option:

*.dtb -> -O dtb
*.dt.yaml -> -O yaml

Commit ce88c9c79455 ("kbuild: Add support to build overlays (%.dtbo)")
added the third case:

*.dtbo -> -O dtbo

This works thanks to commit 163f0469bf2e ("dtc: Allow overlays to have
.dtbo extension") in the upstream DTC, which has already been pulled in
the kernel.

However, I think it is a bit odd because "dtbo" is not a format name.
At least, it does not show up in the help message of dtc.

$ scripts/dtc/dtc --help
[ snip ]
-O, --out-format <arg>
Output formats are:
dts - device tree source text
dtb - device tree blob
yaml - device tree encoded as YAML
asm - assembler source

So, I am not a big fan of the second hunk of that change:

} else if (streq(outform, "dtbo")) {
dt_to_blob(outf, dti, outversion);

Anyway, we did not need to do this in Makefile in the first place.

guess_type_by_name() had already understood ".yaml" before commit
4f0e3a57d6eb ("kbuild: Add support for DT binding schema checks"),
and now does ".dtbo" as well.

Makefile does not need to duplicate the same logic. Let's leave it
to dtc.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Rob Herring <robh@kernel.org>

kbuild: remove deprecated 'always' and 'hostprogs-y/m'

These have no more user in the upstream code. The use of them has been
warned for a while for external modules. The migration is finished.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Kbuild: Make composite object searching more generic

Reduce repeated logic around expanding composite objects.

Signed-off-by: Elliot Berman <eberman@codeaurora.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: use always-y instead of extra-y

As commit d0e628cd817f ("kbuild: doc: clarify the difference between
extra-y and always-y") explained, extra-y should be used for listing
the prerequisites of vmlinux.

These targets are not related to vmlinux. always-y is a better fix.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Rob Herring <robh@kernel.org>

kbuild: lto: postpone objtool

With LTO, LLVM bitcode won't be compiled into native code until
modpost_link, or modfinal for modules. This change postpones calls
to objtool until after these steps, and moves objtool_args to
Makefile.lib, so the arguments can be reused in Makefile.modfinal.

As we didn't have objects to process earlier, we use --duplicate
when processing vmlinux.o. This change also disables unreachable
instruction warnings with LTO to avoid warnings about the int3
padding between functions.

Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>

init: lto: ensure initcall ordering

With LTO, the compiler doesn't necessarily obey the link order for
initcalls, and initcall variables need globally unique names to avoid
collisions at link time.

This change exports __KBUILD_MODNAME and adds the initcall_id() macro,
which uses it together with __COUNTER__ and __LINE__ to help ensure
these variables have unique names, and moves each variable to its own
section when LTO is enabled, so the correct order can be specified using
a linker script.

The generate_initcall_ordering.pl script uses nm to find initcalls from
the object files passed to the linker, and generates a linker script
that specifies the same order for initcalls that we would have without
LTO. With LTO enabled, the script is called in link-vmlinux.sh through
jobserver-exec to limit the number of jobs spawned.

Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Kees Cook <keescook@chromium.org>
Link: https://lore.kernel.org/r/20201211184633.3213045-8-samitolvanen@google.com

kbuild: Add support to build overlays (%.dtbo)

Add support for building DT overlays (%.dtbo). The overlay's source file
will have the usual extension, i.e. .dts, though the blob will have
.dtbo extension to distinguish it from normal blobs.

Acked-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rob Herring <robh@kernel.org>
Link: https://lore.kernel.org/r/434ba2467dd0cd011565625aeb3450650afe0aae.1611904394.git.viresh.kumar@linaro.org

kasan, arm64: expand CONFIG_KASAN checks

Some #ifdef CONFIG_KASAN checks are only relevant for software KASAN modes
(either related to shadow memory or compiler instrumentation). Expand
those into CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS.

Link: https://lkml.kernel.org/r/e6971e432dbd72bb897ff14134ebb7e169bdcf0c.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

dt-bindings: Use json for processed-schema*

Change the format of processed-schema* from yaml to json to speed up
validation. With json output, using xargs and appending the output won't
work since json has explicit list begin and end characters. Instead,
we pass the schema files as a list in a temp file.

The parsing time for the processed schema goes down from ~2sec to 70ms.
Also, 'make dtbs_check' becomes 33% faster.

Some error messages are affected by this change. For example, "True was
expected" becomes "... is not of type 'boolean'". The order of messages
is also changed.

Signed-off-by: Andrei Ziureaev <andrei.ziureaev@arm.com>
Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: introduce hostprogs-always-y and userprogs-always-y

To build host programs, you need to add the program names to 'hostprogs'
to use the necessary build rule, but it is not enough to build them
because there is no dependency.

There are two types of host programs: built as the prerequisite of
another (e.g. gen_crc32table in lib/Makefile), or always built when
Kbuild visits the Makefile (e.g. genksyms in scripts/genksyms/Makefile).

The latter is typical in Makefiles under scripts/, which contains host
programs globally used during the kernel build. To build them, you need
to add them to both 'hostprogs' and 'always-y'.

This commit adds hostprogs-always-y as a shorthand.

The same applies to user programs. net/bpfilter/Makefile builds
bpfilter_umh on demand, hence always-y is unneeded. In contrast,
programs under samples/ are added to both 'userprogs' and 'always-y'
so they are always built when Kbuild visits the Makefiles.

userprogs-always-y works as a shorthand.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>

kbuild: introduce ccflags-remove-y and asflags-remove-y

CFLAGS_REMOVE_<file>.o filters out flags when compiling a particular
object, but there is no convenient way to do that for every object in
a directory.

Add ccflags-remove-y and asflags-remove-y to make it easily.

Use ccflags-remove-y to clean up some Makefiles.

The add/remove order works as follows:

[1] KBUILD_CFLAGS specifies compiler flags used globally

[2] ccflags-y adds compiler flags for all objects in the
current Makefile

[3] ccflags-remove-y removes compiler flags for all objects in the
current Makefile (New feature)

[4] CFLAGS_<file> adds compiler flags per file.

[5] CFLAGS_REMOVE_<file> removes compiler flags per file.

Having [3] before [4] allows us to remove flags from most (but not all)
objects in the current Makefile.

For example, kernel/trace/Makefile removes $(CC_FLAGS_FTRACE)
from all objects in the directory, then adds it back to
trace_selftest_dynamic.o and CFLAGS_trace_kprobe_selftest.o

The same applies to lib/livepatch/Makefile.

Please note ccflags-remove-y has no effect to the sub-directories.
In contrast, the previous notation got rid of compiler flags also from
all the sub-directories.

The following are not affected because they have no sub-directories:

arch/arm/boot/compressed/
arch/powerpc/xmon/
arch/sh/
kernel/trace/

However, lib/ has several sub-directories.

To keep the behavior, I added ccflags-remove-y to all Makefiles
in subdirectories of lib/, except the following:

lib/vdso/Makefile - Kbuild does not descend into this Makefile
lib/raid/test/Makefile - This is not used for the kernel build

I think commit 2464a609ded0 ("ftrace: do not trace library functions")
excluded too much. In the next commit, I will remove ccflags-remove-y
from the sub-directories of lib/.

Suggested-by: Sami Tolvanen <samitolvanen@google.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc)
Acked-by: Brendan Higgins <brendanhiggins@google.com> (KUnit)
Tested-by: Anders Roxell <anders.roxell@linaro.org>

init: Add support for zstd compressed kernel

- Add the zstd and zstd22 cmds to scripts/Makefile.lib

- Add the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options

Architecture specific support is still needed for decompression.

Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Link: https://lore.kernel.org/r/20200730190841.2071656-4-nickrterrell@gmail.com

Revert "kbuild: Create directory for target DTB"

This reverts commit 77479b38e2f58890eb221a0418357502a5b41cd6.

Since commit 8a78756eb545 ("kbuild: create object directories simpler
and faster"), all directories for 'targets' are created.

'mkdir -p $(dir ${dtc-tmp})' is no longer needed.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

dt-bindings: copy process-schema-examples.yaml to process-schema.yaml

There are two processed schema files:

- processed-schema-examples.yaml

Used for 'make dt_binding_check'. This is always a full schema.

- processed-schema.yaml

Used for 'make dtbs_check'. This may be a full schema, or a smaller
subset if DT_SCHEMA_FILES is given by a user.

If DT_SCHEMA_FILES is not specified, they are the same. You can copy
the former to the latter instead of running dt-mk-schema twice. This
saves the cpu time a lot when you do 'make dt_binding_check dtbs_check'
because building the full schema takes a couple of seconds.

If DT_SCHEMA_FILES is specified, processed-schema.yaml is generated
based on the specified yaml files.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Link: https://lore.kernel.org/r/20200625170434.635114-4-masahiroy@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>

scripts/dtc: Update to upstream version v1.6.0-11-g9d7888cbf19c

Sync with upstream dtc primarily to pickup the I2C bus check fixes. The
interrupt_provider check is noisy, so turn it off for now.

This adds the following commits from upstream:

9d7888cbf19c dtc: Consider one-character strings as strings
8259d59f59de checks: Improve i2c reg property checking
fdabcf2980a4 checks: Remove warning for I2C_OWN_SLAVE_ADDRESS
2478b1652c8d libfdt: add extern "C" for C++
f68bfc2668b2 libfdt: trivial typo fix
7be250b4d059 libfdt: Correct condition for reordering blocks
81e0919a3e21 checks: Add interrupt provider test
85e5d839847a Makefile: when building libfdt only, do not add unneeded deps
b28464a550c5 Fix some potential unaligned accesses in dtc

Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: fix broken builds because of GZIP,BZIP2,LZOP variables

Redefine GZIP, BZIP2, LZOP variables as KGZIP, KBZIP2, KLZOP resp.
GZIP, BZIP2, LZOP env variables are reserved by the tools. The original
attempt to redefine them internally doesn't work in makefiles/scripts
intercall scenarios, e.g., "make GZIP=gzip bindeb-pkg" and results in
broken builds. There can be other broken build commands because of this,
so the universal solution is to use non-reserved env variables for the
compression tools.

Fixes: 8dfb61dcbace ("kbuild: add variables for compression tools")
Signed-off-by: Denis Efremov <efremov@linux.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: add variables for compression tools

Allow user to use alternative implementations of compression tools,
such as pigz, pbzip2, pxz. For example, multi-threaded tools to
speed up the build:
$ make GZIP=pigz BZIP2=pbzip2

Variables _GZIP, _BZIP2, _LZOP are used internally because original env
vars are reserved by the tools. The use of GZIP in gzip tool is obsolete
since 2015. However, alternative implementations (e.g., pigz) still rely
on it. BZIP2, BZIP, LZOP vars are not obsolescent.

The credit goes to @grsecurity.

As a sidenote, for multi-threaded lzma, xz compression one can use:
$ export XZ_OPT="--threads=0"

Signed-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: update modules.order only when contained modules are updated

Make modules.order depend on $(obj-m), and use if_changed to build it.
This will avoid unneeded update of modules.order, which will be useful
to optimize the modpost stage.

Currently, the second pass of modpost is always invoked. By checking the
timestamp of modules.order, we can avoid the unneeded modpost.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: refactor subdir-ym calculation

Remove the unneeded variables, __subdir-y and __subdir-m.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: make modules.order rule consistent with built-in.a

built-in.a contains the built-in object paths from the current and sub
directories.

module.order collects the module paths from the current and sub
directories.

Make their build rules look more symmetrical.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: move subdir-obj-y to scripts/Makefile.build

Save $(addprefix ...) for subdir-obj-y.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: warn if always, hostprogs-y, or hostprogs-m is used

always, hostprogs-y, and hostprogs-m are deprecated.

There is no user in upstream code, but I will keep them for external
modules. I want to remove them entirely someday. Prompt downstream
users for the migration.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: determine the output format of DTC by the target suffix

cmd_dtc takes the additional parameter $(2) to select the target
format, dtb or yaml. This makes things complicated when it is used
with cmd_and_fixdep and if_changed_rule. I actually stumbled on this.
See commit 3d4b2238684a ("kbuild: fix DT binding schema rule again to
avoid needless rebuilds").

Extract the suffix part of the target instead of passing the parameter.
Fortunately, this works for both $(obj)/%.dtb and $(obj)/%.dt.yaml .

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: use -MMD instead of -MD to exclude system headers from dependency

This omits system headers from the generated header dependency.

System headers are not updated unless you upgrade the compiler. Nor do
they contain CONFIG options, so fixdep does not need to parse them.

Having said that, the effect of this optimization will be quite small
because the kernel code generally does not include system headers
except <stdarg.h>. Host programs include a lot of system headers,
but there are not so many in the kernel tree.

At first, keeping system headers in .*.cmd files might be useful to
detect the compiler update, but there is no guarantee that <stdarg.h>
is included from every file. So, I implemented a more reliable way in
the previous commit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: fix DT binding schema rule again to avoid needless rebuilds

Since commit 7a0496056064 ("kbuild: fix DT binding schema rule to detect
command line changes"), this rule is every time re-run even if you change
nothing.

cmd_dtc takes one additional parameter to pass to the -O option of dtc.

We need to pass 'yaml' to if_changed_rule. Otherwise, cmd-check invoked
from if_changed_rule is false positive.

Fixes: 7a0496056064 ("kbuild: fix DT binding schema rule to detect command line changes")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: Always validate DT binding examples

Most folks only run dt_binding_check on the single schema they care about
by setting DT_SCHEMA_FILES. That means example is only checked against
that one schema which is not always sufficient.

Let's address this by splitting processed-schema.yaml into 2 files: one
that's always all schemas for the examples and one that's just the schema
in DT_SCHEMA_FILES for dtbs.

Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: remove unneeded semicolon at the end of cmd_dtb_check

This trailing semicolon is unneeded.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Rob Herring <robh@kernel.org>

kbuild: fix DT binding schema rule to detect command line changes

This if_change_rule is not working properly; it cannot detect any
command line change.

The reason is because cmd-check in scripts/Kbuild.include compares
$(cmd_$@) and $(cmd_$1), but cmd_dtc_dt_yaml does not exist here.

For if_change_rule to work properly, the stem part of cmd_* and rule_*
must match. Because this cmd_and_fixdep invokes cmd_dtc, this rule must
be named rule_dtc.

Fixes: 4f0e3a57d6eb ("kbuild: Add support for DT binding schema checks")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Rob Herring <robh@kernel.org>

kbuild: rename hostprogs-y/always to hostprogs/always-y

In old days, the "host-progs" syntax was used for specifying host
programs. It was renamed to the current "hostprogs-y" in 2004.

It is typically useful in scripts/Makefile because it allows Kbuild to
selectively compile host programs based on the kernel configuration.

This commit renames like follows:

always -> always-y
hostprogs-y -> hostprogs

So, scripts/Makefile will look like this:

always-$(CONFIG_BUILD_BIN2C) += ...
always-$(CONFIG_KALLSYMS) += ...
...
hostprogs := $(always-y) $(always-m)

I think this makes more sense because a host program is always a host
program, irrespective of the kernel configuration. We want to specify
which ones to compile by CONFIG options, so always-y will be handier.

The "always", "hostprogs-y", "hostprogs-m" will be kept for backward
compatibility for a while.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: pass KBUILD_MODFILE when compiling builtin objects

When compiling, Kbuild passes KBUILD_BASENAME (basename of the object)
and KBUILD_MODNAME (basename of the module).

This commit adds another one, KBUILD_MODFILE, which is the path of
the module. (or, the path of the module it would end up in if it were
compiled as a module.)

The next commit will use this to generate modules.builtin without
tristate.conf.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: add stringify helper to quote a string passed to C files

Make $(squote)$(quote)...$(quote)$(squote) a helper macro.
I will reuse it in the next commit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: do not create orphan built-in.a or obj-y objects

Both 'obj-y += foo/' and 'obj-m += foo/' request Kbuild to visit the
sub-directory foo/, but the difference is that only the former combines
foo/built-in.a into the built-in.a of the current directory because
everything in sub-directories visited by obj-m is supposed to be modular.

So, it makes sense to create built-in.a only if that sub-directory is
reachable by the chain of obj-y. Otherwise, built-in.a will not be
linked into vmlinux anyway. For the same reason, it is pointless to
compile obj-y objects in the directory visited by obj-m.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kcsan: Add Kernel Concurrency Sanitizer infrastructure

Kernel Concurrency Sanitizer (KCSAN) is a dynamic data-race detector for
kernel space. KCSAN is a sampling watchpoint-based data-race detector.
See the included Documentation/dev-tools/kcsan.rst for more details.

This patch adds basic infrastructure, but does not yet enable KCSAN for
any architecture.

Signed-off-by: Marco Elver <elver@google.com>
Acked-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

kbuild: remove header compile test

There are both positive and negative options about this feature.
At first, I thought it was a good idea, but actually Linus stated a
negative opinion (https://lkml.org/lkml/2019/9/29/227). I admit it
is ugly and annoying.

The baseline I'd like to keep is the compile-test of uapi headers.
(Otherwise, kernel developers have no way to ensure the correctness
of the exported headers.)

I will maintain a small build rule in usr/include/Makefile.
Remove the other header test functionality.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: remove ar-option and KBUILD_ARFLAGS

Commit 40df759e2b9e ("kbuild: Fix build with binutils <= 2.19")
introduced ar-option and KBUILD_ARFLAGS to deal with old binutils.

According to Documentation/process/changes.rst, the current minimal
supported version of binutils is 2.21 so you can assume the 'D' option
is always supported. Not only GNU ar but also llvm-ar supports it.

With the 'D' option hard-coded, there is no more user of ar-option
or KBUILD_ARFLAGS.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>

kbuild: rename KBUILD_ENABLE_EXTRA_GCC_CHECKS to KBUILD_EXTRA_WARN

KBUILD_ENABLE_EXTRA_GCC_CHECKS started as a switch to add extra warning
options for GCC, but now it is a historical misnomer since we use it
also for Clang, DTC, and even kernel-doc.

Rename it to more sensible, shorter KBUILD_EXTRA_WARN.

For the backward compatibility, KBUILD_ENABLE_EXTRA_GCC_CHECKS is still
supported (but not advertised in the documentation).

I also fixed up 'make help', and updated the documentation.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nathan Chancellor <natechancellor@gmail.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Sedat Dilek <sedat.dilek@gmail.com>

kbuild: change *FLAGS_<basetarget>.o to take the path relative to $(obj)

Kbuild provides per-file compiler flag addition/removal:

CFLAGS_<basetarget>.o
CFLAGS_REMOVE_<basetarget>.o
AFLAGS_<basetarget>.o
AFLAGS_REMOVE_<basetarget>.o
CPPFLAGS_<basetarget>.lds
HOSTCFLAGS_<basetarget>.o
HOSTCXXFLAGS_<basetarget>.o

The <basetarget> is the filename of the target with its directory and
suffix stripped.

This syntax comes into a trouble when two files with the same basename
appear in one Makefile, for example:

obj-y += foo.o
obj-y += dir/foo.o
CFLAGS_foo.o := <some-flags>

Here, the <some-flags> applies to both foo.o and dir/foo.o

The real world problem is:

scripts/kconfig/util.c
scripts/kconfig/lxdialog/util.c

Both files are compiled into scripts/kconfig/mconf, but only the
latter should be given with the ncurses flags.

It is more sensible to use the relative path to the Makefile, like this:

obj-y += foo.o
CFLAGS_foo.o := <some-flags>
obj-y += dir/foo.o
CFLAGS_dir/foo.o := <other-flags>

At first, I attempted to replace $(basetarget) with $*. The $* variable
is replaced with the stem ('%') part in a pattern rule. This works with
most of cases, but does not for explicit rules.

For example, arch/ia64/lib/Makefile reuses rule_as_o_S in its own
explicit rules, so $* will be empty, resulting in ignoring the per-file
AFLAGS.

I introduced a new variable, target-stem, which can be used also from
explicit rules.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Marc Zyngier <maz@kernel.org>

kbuild: add $(BASH) to run scripts with bash-extension

CONFIG_SHELL falls back to sh when bash is not installed on the system,
but nobody is testing such a case since bash is usually installed.
So, shell scripts invoked by CONFIG_SHELL are only tested with bash.

It makes it difficult to test whether the hashbang #!/bin/sh is real.
For example, #!/bin/sh in arch/powerpc/kernel/prom_init_check.sh is
false. (I fixed it up)

Besides, some shell scripts invoked by CONFIG_SHELL use bash-extension
and #!/bin/bash is specified as the hashbang, while CONFIG_SHELL may
not always be set to bash.

Probably, the right thing to do is to introduce BASH, which is bash by
default, and always set CONFIG_SHELL to sh. Replace $(CONFIG_SHELL)
with $(BASH) for bash scripts.

If somebody tries to add bash-extension to a #!/bin/sh script, it will
be caught in testing because /bin/sh is a symlink to dash on some major
distributions.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: move modkern_{c,a}flags to Makefile.lib from Makefile.build

Makefile.lib is included by Makefile.modfinal as well as Makefile.build.

Move modkern_cflags to Makefile.lib in order to simplify cmd_cc_o_c
in Makefile.modfinal. Move modkern_cflags as well for consistency.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: treat an object as multi-used when $(foo-) is set

Currently, Kbuild treats an object as multi-used when any of
$(foo-objs), $(foo-y), $(foo-m) is set. It makes more sense to
check $(foo-) as well.

In the context of foo-$(CONFIG_FOO_FEATURE1), CONFIG_FOO_FEATURE1
could be unset.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: move flex and bison rules to Makefile.host

Flex and bison are used for kconfig, dtc, genksyms, all of which are
host programs. I never imagine the kernel embeds a parser or a lexer.

Move the flex and bison rules to scripts/Makefile.host. This file is
included only when hostprogs-y etc. is present in the Makefile in the
directory. So, parsing these rules are skipped in most of directories.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: make bison create C file and header in a single pattern rule

We generally expect bison to create not only a C file, but also a
header, which will be included from the lexer.

Currently, Kbuild generates them in separate rules. So, for instance,
when building Kconfig, you will notice bison is invoked twice:

HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.h
HOSTCC scripts/kconfig/lexer.lex.o
YACC scripts/kconfig/parser.tab.c
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTLD scripts/kconfig/conf

Make handles such cases nicely in pattern rules [1]. Merge the two
rules so that one invokcation of bison can generate both of them.

HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTLD scripts/kconfig/conf

[1] Pattern rule

GNU Make manual says:
"Pattern rules may have more than one target. Unlike normal rules,
this does not act as many different rules with the same prerequisites
and recipe. If a pattern rule has multiple targets, make knows that
the rule's recipe is responsible for making all of the targets. The
recipe is executed only once to make all the targets. When searching
for a pattern rule to match a target, the target patterns of a rule
other than the one that matches the target in need of a rule are
incidental: make worries only about giving a recipe and prerequisites
to the file presently in question. However, when this file's recipe is
run, the other targets are marked as having been updated themselves."

https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.html

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: remove unused single-used-m

This is unused since commit 9f69a496f100 ("kbuild: split out *.mod out
of {single,multi}-used-m rules").

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: compile-test headers listed in header-test-m as well

It will be useful to control the header-test by a tristate option.

If CONFIG_FOO is a tristate option, you can write like this:

header-test-$(CONFIG_FOO) += foo.h

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: fix compression errors getting ignored

A missing compression utility or other errors were not picked up by make
and an empty kernel image was produced. By removing the &&, errors will
no longer be ignored.

Signed-off-by: Harald Seiler <hws@denx.de>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: replace KBUILD_SRCTREE with boolean building_out_of_srctree

Commit 25b146c5b8ce ("kbuild: allow Kbuild to start from any directory")
deprecated KBUILD_SRCTREE.

It is only used in tools/testing/selftest/ to distinguish out-of-tree
build. Replace it with a new boolean flag, building_out_of_srctree.

I also replaced the conditional ($(srctree),.) because the next commit
will allow an absolute path to be used for $(srctree) even when building
in the source tree.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: support header-test-pattern-y

In my view, most of headers can be self-contained. So, it would be
tedious to add every header to header-test-y explicitly. We usually
end up with "all headers with some exceptions".

There are two types in exceptions:

[1] headers that are never compiled as standalone units

For examples, include/linux/compiler-gcc.h is not intended for
direct inclusion. We should always exclude such ones.

[2] headers that are conditionally compiled as standalone units

Some headers can be compiled only for particular architectures.
For example, include/linux/arm-cci.h can be compiled only for
arm/arm64 because it requires <asm/arm-cci.h> to exist.
Clang can compile include/soc/nps/mtm.h only for arc because
it contains an arch-specific register in inline assembler.

So, you can write Makefile like this:

header-test- += linux/compiler-gcc.h
header-test-$(CONFIG_ARM) += linux/arm-cci.h
header-test-$(CONFIG_ARM64) += linux/arm-cci.h
header-test-$(CONFIG_ARC) += soc/nps/mtm.h

The new syntax header-test-pattern-y will be useful to specify
"the rest".

The typical usage is like this:

header-test-pattern-y += */*.h

This will add all the headers in sub-directories to the test coverage,
excluding $(header-test-). In this regards, header-test-pattern-y
behaves like a weaker variant of header-test-y.

Caveat:
The patterns in header-test-pattern-y are prefixed with $(srctree)/$(src)/
but not $(objtree)/$(obj)/. Stale generated headers are often left over
when you traverse the git history without cleaning. Wildcard patterns for
$(objtree) may match to stale headers, which could fail to compile.
One pitfall is $(srctree)/$(src)/ and $(objtree)/$(obj)/ point to the
same directory for in-tree building. So, header-test-pattern-y should
be used with care since it can potentially match to stale headers.

Caveat2:
You could use wildcard for header-test-. For example,

header-test- += asm-generic/%

... will exclude headers in asm-generic directory. Unfortunately, the
wildcard character is '%' instead of '*' here because this is evaluated
by $(filter-out ...) whereas header-test-pattern-y is evaluated by
$(wildcard ...). This is a kludge, but seems useful in some places...

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Jani Nikula <jani.nikula@intel.com>

kbuild: do not create wrappers for header-test-y

header-test-y does not work with headers in sub-directories.

For example, you may want to write a Makefile, like this:

include/linux/Kbuild:

header-test-y += mtd/nand.h

This entry will create a wrapper include/linux/mtd/nand.hdrtest.c
with the following content:

#include "mtd/nand.h"

To make this work, we need to add $(srctree)/include/linux to the
header search path. It would be tedious to add ccflags-y.

Instead, we could change the *.hdrtest.c rule to wrap:

#include "nand.h"

This works for in-tree build since #include "..." searches in the
relative path from the header with this directive. For O=... build,
we need to add $(srctree)/include/linux/mtd to the header search path,
which will be even more tedious.

After all, I thought it would be handier to compile headers directly
without creating wrappers.

I added a new build rule to compile %.h into %.h.s

The target is %.h.s instead of %.h.o because it is slightly faster.
Also, as for GCC, an empty assembly is smaller than an empty object.

I wrote the build rule:

$(CC) $(c_flags) -S -o $@ -x c /dev/null -include $<

instead of:

$(CC) $(c_flags) -S -o $@ -x c $<

Both work fine with GCC, but the latter is bad for Clang.

This comes down to the difference in the -Wunused-function policy.
GCC does not warn about unused 'static inline' functions at all.
Clang does not warn about the ones in included headers, but does
about the ones in the source. So, we should handle headers as
headers, not as source files.

In fact, this has been hidden since commit abb2ea7dfd82 ("compiler,
clang: suppress warning for unused static inline functions"), but we
should not rely on that.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Tested-by: Jani Nikula <jani.nikula@intel.com>

kbuild: add support for ensuring headers are self-contained

Sometimes it's useful to be able to explicitly ensure certain headers
remain self-contained, i.e. that they are compilable as standalone
units, by including and/or forward declaring everything they depend on.

Add special target header-test-y where individual Makefiles can add
headers to be tested if CONFIG_HEADER_TEST is enabled. This will
generate a dummy C file per header that gets built as part of extra-y.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

dt-bindings: Pass binding directory to validation tools

In order to have $ref's to schema files within the kernel, we need to
pass the base path of bindings to the schema validation tools.

Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: devicetree@vger.kernel.org
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: remove 'addtree' and 'flags' magic for header search paths

The 'addtree' and 'flags' in scripts/Kbuild.include are so compilecated
and ugly.

As I mentioned in [1], Kbuild should stop automatic prefixing of header
search path options.

I fixed up (almost) all Makefiles in the kernel. Now 'addtree' and
'flags' have been removed.

Kbuild still caters to add $(srctree)/$(src) and $(objtree)/$(obj)
to the header search path for O= building, but never touches extra
compiler options from ccflags-y etc.

[1]: https://patchwork.kernel.org/patch/9632347/

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: use $(srctree) instead of KBUILD_SRC to check out-of-tree build

KBUILD_SRC was conventionally used for some different purposes:
[1] To remember the source tree path
[2] As a flag to check if sub-make is already done
[3] As a flag to check if Kbuild runs out of tree

For [1], we do not need to remember it because the top Makefile
can compute it by $(realpath $(dir $(lastword $(MAKEFILE_LIST))))

[2] has been replaced with self-commenting 'sub_make_done'.

For [3], we can distinguish in-tree/out-of-tree by comparing
$(srctree) and '.'

This commit converts [3] to prepare for the KBUILD_SRC removal.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: move archive command to scripts/Makefile.lib

scripts/Makefile.build and arch/s390/boot/Makefile use the same
command (thin archiving with symbol table creation).

Avoid the code duplication, and move it to scripts/Makefile.lib.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: remove unnecessary in-subshell execution

The commands surrounded by ( ) are executed in a subshell, but in
most cases, we do not need to spawn an extra subshell.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: add real-prereqs shorthand for $(filter-out FORCE,$^)

In Kbuild, if_changed and friends must have FORCE as a prerequisite.

Hence, $(filter-out FORCE,$^) or $(filter-out $(PHONY),$^) is a common
idiom to get the names of all the prerequisites except phony targets.

Add real-prereqs as a shorthand.

Note:
We cannot replace $(filter %.o,$^) in cmd_link_multi-m because $^ may
include auto-generated dependencies from the .*.cmd file when a single
object module is changed into a multi object module. Refer to commit
69ea912fda74 ("kbuild: remove unneeded link_multi_deps"). I added some
comment to avoid accidental breakage.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Rob Herring <robh@kernel.org>

kbuild: simplify rules of data compression with size appending

All the callers of size_append pass $(filter-out FORCE,$^).
Move $(filter-out FORCE,$^) to the definition of size_append.

This makes the callers cleaner because $(call ...) is unneeded
for a macro with no argument.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: skip 'addtree' and 'flags' magic for external module build

When building an external module, $(obj) is the absolute path to it.

The header search paths from ccflags-y etc. should not be tweaked.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: change filechk to surround the given command with { }

filechk_* rules often consist of multiple 'echo' lines. They must be
surrounded with { } or ( ) to work correctly. Otherwise, only the
string from the last 'echo' would be written into the target.

Let's take care of that in the 'filechk' in scripts/Kbuild.include
to clean up filechk_* rules.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: remove redundant target cleaning on failure

Since commit 9c2af1c7377a ("kbuild: add .DELETE_ON_ERROR special
target"), the target file is automatically deleted on failure.

The boilerplate code

... || { rm -f $@; false; }

is unneeded.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: clean up rule_dtc_dt_yaml

Commit 3a2429e1faf4 ("kbuild: change if_changed_rule for multi-line
recipe") and commit 4f0e3a57d6eb ("kbuild: Add support for DT binding
schema checks") came in via different sub-systems.

This is a follow-up cleanup.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: remove UIMAGE_IN and UIMAGE_OUT

The only/last user of UIMAGE_IN/OUT was removed by commit 4722a3e6b716
("microblaze: fix multiple bugs in arch/microblaze/boot/Makefile").

The input and output should always be $< and $@.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: Add support for DT binding schema checks

This adds the build infrastructure for checking DT binding schema
documents and validating dts files using the binding schema.

Check DT binding schema documents:
make dt_binding_check

Build dts files and check using DT binding schema:
make dtbs_check

Optionally, DT_SCHEMA_FILES can be passed in with a schema file(s) to
use for validation. This makes it easier to find and fix errors
generated by a specific schema.

Currently, the validation targets are separate from a normal build to
avoid a hard dependency on the external DT schema project and because
there are lots of warnings generated.

Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Mark Rutland <mark.rutland@arm.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: linux-doc@vger.kernel.org
Cc: devicetree@vger.kernel.org
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: remove redundant 'set -e' from filechk_offsets

The filechk macro in scripts/Kbuild.include already sets 'set -e'.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: Enable dtc graph_port warning by default

All the 'graph_port' warnings have been fixed or have pending fixes, so
we can enable it by default now.

Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: disable dtc simple_bus_reg warnings by default

The updated version of dtc has a bug fix for simple_bus_reg warnings
and lots of warnings are generated now. So disable this warning by
default.

Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: consolidate Devicetree dtb build rules

There is nothing arch specific about building dtb files other than their
location under /arch/*/boot/dts/. Keeping each arch aligned is a pain.
The dependencies and supported targets are all slightly different.
Also, a cross-compiler for each arch is needed, but really the host
compiler preprocessor is perfectly fine for building dtbs. Move the
build rules to a common location and remove the arch specific ones. This
is done in a single step to avoid warnings about overriding rules.

The build dependencies had been a mixture of 'scripts' and/or 'prepare'.
These pull in several dependencies some of which need a target compiler
(specifically devicetable-offsets.h) and aren't needed to build dtbs.
All that is really needed is dtc, so adjust the dependencies to only be
dtc.

This change enables support 'dtbs_install' on some arches which were
missing the target.

Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Paul Burton <paul.burton@mips.com>
Acked-by: Ley Foon Tan <ley.foon.tan@intel.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: James Hogan <jhogan@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Chris Zankel <chris@zankel.net>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: linux-kbuild@vger.kernel.org
Cc: linux-snps-arc@lists.infradead.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: uclinux-h8-devel@lists.sourceforge.jp
Cc: linux-mips@linux-mips.org
Cc: nios2-dev@lists.rocketboards.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-xtensa@linux-xtensa.org
Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: rename LDFLAGS to KBUILD_LDFLAGS

Commit a0f97e06a43c ("kbuild: enable 'make CFLAGS=...' to add
additional options to CC") renamed CFLAGS to KBUILD_CFLAGS.

Commit 222d394d30e7 ("kbuild: enable 'make AFLAGS=...' to add
additional options to AS") renamed AFLAGS to KBUILD_AFLAGS.

Commit 06c5040cdb13 ("kbuild: enable 'make CPPFLAGS=...' to add
additional options to CPP") renamed CPPFLAGS to KBUILD_CPPFLAGS.

For some reason, LDFLAGS was not renamed.

Using a well-known variable like LDFLAGS may result in accidental
override of the variable.

Kbuild generally uses KBUILD_ prefixed variables for the internally
appended options, so here is one more conversion to sanitize the
naming convention.

I did not touch Makefiles under tools/ since the tools build system
is a different world.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Palmer Dabbelt <palmer@sifive.com>

kbuild: do not redirect the first prerequisite for filechk

Currently, filechk unconditionally opens the first prerequisite and
redirects it as the stdin of a filechk_* rule. Hence, every target
using $(call filechk,...) must list something as the first prerequisite
even if it is unneeded.

'< $<' is actually unneeded in most cases. Each rule can explicitly
adds it if necessary.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: change ld_flags to contain LDFLAGS_$(@F)

Put $(LDFLAGS_$(@F)) into ld_flags so that $(LDFLAGS_pcap.o) and
$(LDFLAGS_vde.o) in arch/um/drivers/Makefile are absorbed.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>

kbuild: replace $(LDFLAGS) $(ldflags-y) with $(ld_flags)

$(LDFLAGS) $(ldflags-y) is equivalent to $(ld_flags).

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>

kbuild: disable new dtc graph and unit-address warnings

dtc gained some new warnings for OF graphs and unique unit addresses,
but they are currently much too noisy. So turn off
'graph_child_address', 'graph_port', and 'unique_unit_address' warnings
by default. They can be enabled by building dtbs with W=1.

Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: replace hardcoded bison in cmd_bison_h with $(YACC)

Commit 73a4f6dbe70a ("kbuild: add LEX and YACC variables") missed to
update cmd_bison_h somehow.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: mark $(targets) as .SECONDARY and remove .PRECIOUS markers

GNU Make automatically deletes intermediate files that are updated
in a chain of pattern rules.

Example 1) %.dtb.o <- %.dtb.S <- %.dtb <- %.dts
Example 2) %.o <- %.c <- %.c_shipped

A couple of makefiles mark such targets as .PRECIOUS to prevent Make
from deleting them, but the correct way is to use .SECONDARY.

.SECONDARY
Prerequisites of this special target are treated as intermediate
files but are never automatically deleted.

.PRECIOUS
When make is interrupted during execution, it may delete the target
file it is updating if the file was modified since make started.
If you mark the file as precious, make will never delete the file
if interrupted.

Both can avoid deletion of intermediate files, but the difference is
the behavior when Make is interrupted; .SECONDARY deletes the target,
but .PRECIOUS does not.

The use of .PRECIOUS is relatively rare since we do not want to keep
partially constructed (possibly corrupted) targets.

Another difference is that .PRECIOUS works with pattern rules whereas
.SECONDARY does not.

.PRECIOUS: $(obj)/%.lex.c

works, but

.SECONDARY: $(obj)/%.lex.c

has no effect. However, for the reason above, I do not want to use
.PRECIOUS which could cause obscure build breakage.

The targets specified as .SECONDARY must be explicit. $(targets)
contains all targets that need to include .*.cmd files. So, the
intermediates you want to keep are mostly in there. Therefore, mark
$(targets) as .SECONDARY. It means primary targets are also marked
as .SECONDARY, but I do not see any drawback for this.

I replaced some .SECONDARY / .PRECIOUS markers with 'targets'. This
will make Kbuild search for non-existing .*.cmd files, but this is
not a noticeable performance issue.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Frank Rowand <frowand.list@gmail.com>
Acked-by: Ingo Molnar <mingo@kernel.org>

kbuild: add %.dtb.S and %.dtb to 'targets' automatically

Another common pattern that consists of chained commands is to compile
a DTB as binary data into the kernel image or a module. It is used in
several places in the source tree. Support it in the core Makefile.

$(call if_changed,dt_S_dtb) is more suitable than $(call cmd,dt_S_dtb)
in case cmd_dt_S_dtb is changed in the future.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Frank Rowand <frowand.list@gmail.com>

genksyms: generate lexer and parser during build instead of shipping

Now that the kernel build supports flex and bison, remove the _shipped
files and generate them during the build instead.

There are no more shipped lexer and parser, so I ripped off the rules
in scripts/Malefile.lib that were used for REGENERATE_PARSERS.

The genksyms parser has ambiguous grammar, which would emit warnings:

scripts/genksyms/parse.y: warning: 9 shift/reduce conflicts [-Wconflicts-sr]
scripts/genksyms/parse.y: warning: 5 reduce/reduce conflicts [-Wconflicts-rr]

They are normally suppressed, but displayed when W=1 is given.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: get <linux/compiler_types.h> out of <linux/kconfig.h>

Since commit 28128c61e08e ("kconfig.h: Include compiler types to avoid
missed struct attributes"), <linux/kconfig.h> pulls in kernel-space
headers to unrelated places.

Commit 0f9da844d877 ("MIPS: boot: Define __ASSEMBLY__ for its.S build")
suppress the build error by defining __ASSEMBLY__, but ITS (i.e. DTS)
is not assembly, and should not include <linux/compiler_types.h> in the
first place.

Looking at arch/s390/tools/Makefile, host programs gen_facilities and
gen_opcode_table now pull in <linux/compiler_types.h> as well.

The motivation for that commit was to define necessary attributes
before any struct is defined. Obviously, this happens only in C.

It is enough to include <linux/compiler_types.h> only when compiling
C files, and only when compiling kernel space. Move the include to
c_flags.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: link $(real-obj-y) instead of $(obj-y) into built-in.a

In Kbuild, Makefiles can add the same object to obj-y multiple
times. So,

obj-y += foo.o
obj-y += foo.o

is fine.

However, this is not true when the same object is added multiple
times via composite objects. For example,

obj-y += foo.o bar.o
foo-objs := foo-bar-common.o foo-only.o
bar-objs := foo-bar-common.o bar-only.o

causes build error because two instances of foo-bar-common.o are
linked into the vmlinux.

Makefiles tend to invent ugly work-around, for example
- lib/zstd/Makefile
- drivers/net/ethernet/cavium/liquidio/Makefile

The technique used in Kbuild to avoid the multiple definition error
is to use $(filter $(obj-y), $^). Here, $^ lists the names of all
the prerequisites with duplicated names removed.

By replacing it with $(filter $(real-obj-y), $^) we can do likewise
for composite objects. For built-in objects, we do not need to keep
the composite object structure. We can simply expand them, and link
$(real-obj-y) to built-in.a.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: rename real-objs-y/m to real-obj-y/m

When I was refactoring Makefiles, I stupidly mistook 'real-obj-y' for
'real-objs-y' over and over again. Finally, I decide to rename it to
'real-obj-y'. This is consistent with 'obj-y', 'subdir-obj-y'.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: move modname and modname-multi close to modname_flags

Just a cosmetic change to put related code close together.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>

kbuild: simplify modname calculation

modname can be calculated much more simply. If modname-multi is
empty, it is a single-used object. So, modname = $(basetarget).
Otherwise, modname = $(modname-multi).

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>

kbuild: fix modname for composite modules

Commit cf4f21938e13 ("kbuild: Allow to specify composite modules
with modname-m") added modname-m support, but missed to update the
corresponding multi-objs-m & modname-multi definition.

Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: define KBUILD_MODNAME even if multiple modules share objects

Currently, KBUILD_MODNAME is defined only when $(modname) contains
just one word. If an object is shared among multiple modules,
undefined KBUILD_MODNAME could cause a build error. For example,
if CONFIG_DYNAMIC_DEBUG is enabled, any call of printk() populates
.modname, then fails to build due to undefined KBUILD_MODNAME.

Take the following code as an example:

obj-m += foo.o
obj-m += bar.o
foo-objs := foo-bar-common.o foo-only.o
bar-objs := foo-bar-common.o bar-only.o

In this case, there is room for argument what to define for
KBUILD_MODNAME when foo-bar-common.o is being compiled.
"foo", "bar", or what else?

One idea is to define colon-separated modules that share the object,
in this case, "bar:foo" (modules are sorted alphabetically by
$(sort ...)).

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>

kbuild: remove unnecessary $(subst $(obj)/, , ...) in modname-multi

In the context ...

$(obj)/%.s: $(src)/%.c FORCE
$(call if_changed_dep,cc_s_c)

$(obj)/%.i: $(src)/%.c FORCE
$(call if_changed_dep,cpp_i_c)

$(obj)/%.o: $(src)/%.c $(recordmcount_source) $(objtool_dep) FORCE
$(call cmd,force_checksrc)
$(call if_changed_rule,cc_o_c)

$(obj)/%.lst: $(src)/%.c FORCE
$(call if_changed_dep,cc_lst_c)

'$*' returns the stem of the target (the part of '%'), so $(obj)/ has
already been ripped off.

$(subst $(obj)/,,$*.o) is the same as $*.o

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>

kbuild: Use ls(1) instead of stat(1) to obtain file size

stat(1) is not standardized and different implementations have their own
(conflicting) flags for querying the size of a file.

ls(1) provides the same information (value of st.st_size) in the 5th
column, except when the file is a character or block device. This output
is standardized[0]. The -n option turns on -l, which writes lines
formatted like

"%s %u %s %s %u %s %s\n", <file mode>, <number of links>,
<owner name>, <group name>, <size>, <date and time>,
<pathname>

but instead of writing the <owner name> and <group name>, it writes the
numeric owner and group IDs (this avoids /etc/passwd and /etc/group
lookups as well as potential field splitting issues).

The <size> field is specified as "the value that would be returned for
the file in the st_size field of struct stat".

To avoid duplicating logic in several locations in the tree, create
scripts/file-size.sh and update callers to use that instead of stat(1).

[0] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html#tag_20_73_10

Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: rename built-in.o to built-in.a

Incremental linking is gone, so rename built-in.o to built-in.a, which
is the usual extension for archive files.

This patch does two things, first is a simple search/replace:

git grep -l 'built-in\.o' | xargs sed -i 's/built-in\.o/built-in\.a/g'

The second is to invert nesting of nested text manipulations to avoid
filtering built-in.a out from libs-y2:

-libs-y2 := $(filter-out %.a, $(patsubst %/, %/built-in.a, $(libs-y)))
+libs-y2 := $(patsubst %/, %/built-in.a, $(filter-out %.a, $(libs-y)))

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: Handle builtin dtb file names containing hyphens

cmd_dt_S_dtb constructs the assembly source to incorporate a devicetree
FDT (that is, the .dtb file) as binary data in the kernel image. This
assembly source contains labels before and after the binary data. The
label names incorporate the file name of the corresponding .dtb file.
Hyphens are not legal characters in labels, so .dtb files built into the
kernel with hyphens in the file name result in errors like the
following:

bcm3368-netgear-cvg834g.dtb.S: Assembler messages:
bcm3368-netgear-cvg834g.dtb.S:5: Error: : no such section
bcm3368-netgear-cvg834g.dtb.S:5: Error: junk at end of line, first unrecognized character is `-'
bcm3368-netgear-cvg834g.dtb.S:6: Error: unrecognized opcode `__dtb_bcm3368-netgear-cvg834g_begin:'
bcm3368-netgear-cvg834g.dtb.S:8: Error: unrecognized opcode `__dtb_bcm3368-netgear-cvg834g_end:'
bcm3368-netgear-cvg834g.dtb.S:9: Error: : no such section
bcm3368-netgear-cvg834g.dtb.S:9: Error: junk at end of line, first unrecognized character is `-'

Fix this by updating cmd_dt_S_dtb to transform all hyphens from the file
name to underscores when constructing the labels.

As of v4.16-rc2, 1139 .dts files across ARM64, ARM, MIPS and PowerPC
contain hyphens in their names, but the issue only currently manifests
on Broadcom MIPS platforms, as that is the only place where such files
are built into the kernel. For example when CONFIG_DT_NETGEAR_CVG834G=y,
or on BMIPS kernels when the dtbs target is used (in the latter case it
admittedly shouldn't really build all the dtb.o files, but thats a
separate issue).

Fixes: 695835511f96 ("MIPS: BMIPS: rename bcm96358nb4ser to bcm6358-neufbox4-sercom")
Signed-off-by: James Hogan <jhogan@kernel.org>
Reviewed-by: Frank Rowand <frowand.list@gmail.com>
Cc: Rob Herring <robh+dt@kernel.org>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Florian Fainelli <f.fainelli@gmail.com>
Cc: Kevin Cernekee <cernekee@gmail.com>
Cc: <stable@vger.kernel.org> # 4.9+
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

scripts: turn off some new dtc warnings by default

The latest dtc update adds some new noisy warnings, so turn them off by
default. Disable 'avoid_unnecessary_addr_size' and 'alias_paths'. They
can be re-enabled by building with 'W=1'.

Signed-off-by: Rob Herring <robh@kernel.org>

scripts: re-enable some now fixed dtc warnings

We can re-enable some dtc warnings that have been completely or mostly
fixed. There are a few remaining ones in arm64 dts files which crept in
recently.

Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: add dtc as dependency on .dtb files

If dtc is rebuilt, we should rebuild .dtb files with the new dtc.

Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: remove remaining use of undefined YACC_PREFIX

Commit eea199b445f6 ("kbuild: remove unnecessary LEX_PREFIX and
YACC_PREFIX") removed YACC_PREFIX definition, but left one use of it. There
was not any build error since there is no user of "cmd_bison_h" currently.
Remove the last remaining occurrence of YACC_PREFIX.

Fixes: eea199b445f6 ("kbuild: remove unnecessary LEX_PREFIX and YACC_PREFIX")
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>

kasan: don't emit builtin calls when sanitization is off

With KASAN enabled the kernel has two different memset() functions, one
with KASAN checks (memset) and one without (__memset). KASAN uses some
macro tricks to use the proper version where required. For example
memset() calls in mm/slub.c are without KASAN checks, since they operate
on poisoned slab object metadata.

The issue is that clang emits memset() calls even when there is no
memset() in the source code. They get linked with improper memset()
implementation and the kernel fails to boot due to a huge amount of KASAN
reports during early boot stages.

The solution is to add -fno-builtin flag for files with KASAN_SANITIZE :=
n marker.

Link: http://lkml.kernel.org/r/8ffecfffe04088c52c42b92739c2bd8a0bcb3f5e.1516384594.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Acked-by: Nick Desaulniers <ndesaulniers@google.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

kbuild: fix W= option checks for extra DTC warnings

Kbuild supports 3 levels of extra warnings, and multiple levels can
be combined, like W=12, W=123. It was added by commit a6de553da01c
("kbuild: Allow to combine multiple W= levels").

From the log of commit 8654cb8d0371 ("dtc: update warning settings
for new bus and node/property name checks"), I assume:

- unit_address_vs_reg, simple_bus_reg, etc. belong to level 1
- node_name_chars_strict, property_name_chars_strict belong to level 2

However, the level 1 warnings are displayed by any argument to W=.
On the other hand, the level 2 warnings are displayed by W=2, but
not by W=12, or W=123.

Use $(findstring ...) like scripts/Makefile.extrawarn.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>

kbuild: remove unnecessary LEX_PREFIX and YACC_PREFIX

Kconfig was the only user of these. With Kconfig converted to use
the default 'yy' prefix, we do not need them any more.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>

kbuild: prepare to remove C files pre-generated by flex and bison

In Linux build system convention, pre-generated files are version-
controlled with a "_shipped" suffix. During the kernel building,
they are simply shipped (copied) removing the suffix.

This approach can reduce external tool dependency for the kernel build,
but it is tedious to manually regenerate such artifacts from developers'
point of view. (We need to do "make REGENERATE_PARSERS=1" every time
we touch real source files such as *.l, *.y)

Some months ago, I sent out RFC patches to run flex, bison, and gperf
during the build.

In the review and test, Linus noticed gperf-3.1 had changed the lookup
function prototype. Then, the use of gperf in kernel was entirely
removed by commit bb3290d91695 ("Remove gperf usage from toolchain").

This time, I tested several versions of flex and bison, and I was not
hit by any compatibility issue except a flaw in flex-2.6.3; if you
generate lexer for dtc and genksyms with flex-2.6.3, you will see
"yywrap redefined" warning. This was not intentional, but a bug,
fixed by flex-2.6.4. Otherwise, both flex and bison look fairly
stable for a long time.

This commit prepares some build rules to remove the _shipped files.
Also, document minimal requirement for flex and bison.

Rationale for the minimal version:
The -Wmissing-prototypes option of GCC warns "no previous prototype"
for lexers generated by flex-2.5.34 or older, so I chose 2.5.35 as the
required version for flex. Flex-2.5.35 was released in 2008. Bison
looks more stable. I did not see any problem with bison-2.0, released
in 2004. I did not test bison-1.x, but bison-2.0 should be old enough.

Tested flex versions:
2.5.35
2.5.36
2.5.37
2.5.39
2.6.0
2.6.1
2.6.2
2.6.3 (*)
2.6.4

(*) flex-2.6.3 causes "yywrap redefined" warning

Tested bison versions:
2.0
2.1
2.2
2.3
2.4
2.4.1
2.5.1
2.6
2.6.1
2.6.2
2.6.3
2.6.4
2.6.5
2.7
2.7.1
3.0
3.0.1
3.0.2
3.0.3
3.0.4

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: add LEX and YACC variables

Allow users to use their favorite lexer / parser generators.
This is useful for me to test various flex and bison versions.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: drop $(extra-y) from real-objs-y

$(real-objs-y) in only used in scripts/Makefile.build to form
"targets", but $(extra-y) is added to "targets" in another line.
We do not need to add $(extra-y) twice.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kbuild: create object directories simpler and faster

For the out-of-tree build, scripts/Makefile.build creates output
directories, but this operation is not efficient.

scripts/Makefile.lib calculates obj-dirs as follows:

obj-dirs := $(dir $(multi-objs) $(obj-y))

Please notice $(sort ...) is not used here. Usually the result is
as many "./" as objects here.

For a lot of duplicated paths, the following command is invoked.

_dummy := $(foreach d,$(obj-dirs), $(shell [ -d $(d) ] || mkdir -p $(d)))

Then, the costly shell command is run over and over again.

I see many points for optimization:

[1] Use $(sort ...) to cut down duplicated paths before passing them
to system call
[2] Use single $(shell ...) instead of repeating it with $(foreach ...)
This will reduce forking.
[3] We can calculate obj-dirs more simply. Most of objects are already
accumulated in $(targets). So, $(dir $(targets)) is fine and more
comprehensive.

I also removed ugly code in arch/x86/entry/vdso/Makefile. This is now
really unnecessary.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Douglas Anderson <dianders@chromium.org>

kbuild: handle dtb-y and CONFIG_OF_ALL_DTBS natively in Makefile.lib

If CONFIG_OF_ALL_DTBS is enabled, "make ARCH=arm64 dtbs" compiles each
DTB twice; one from arch/arm64/boot/dts/*/Makefile and the other from
the dtb-$(CONFIG_OF_ALL_DTBS) line in arch/arm64/boot/dts/Makefile.
It could be a race problem when building DTBS in parallel.

Another minor issue is CONFIG_OF_ALL_DTBS covers only *.dts in vendor
sub-directories, so this broke when Broadcom added one more hierarchy
in arch/arm64/boot/dts/broadcom/<soc>/.

One idea to fix the issues in a clean way is to move DTB handling
to Kbuild core scripts. Makefile.dtbinst already recognizes dtb-y
natively, so it should not hurt to do so.

Add $(dtb-y) to extra-y, and $(dtb-) as well if CONFIG_OF_ALL_DTBS is
enabled. All clutter things in Makefiles go away.

As a bonus clean-up, I also removed dts-dirs. Just use subdir-y
directly to traverse sub-directories.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
[robh: corrected BUILTIN_DTB to CONFIG_BUILTIN_DTB]
Signed-off-by: Rob Herring <robh@kernel.org>

License cleanup: add SPDX GPL-2.0 license identifier to files with no license

Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.

The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.

The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.

Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).

All documentation files were explicitly excluded.

The following heuristics were used to determine which SPDX license
identifiers to apply.

- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.

For non */uapi/* files that summary was:

SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139

and resulted in the first patch in this series.

If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:

SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930

and resulted in the second patch in this series.

- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:

SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1

and that resulted in the third patch in this series.

- when the two scanners agreed on the detected license(s), that became
the concluded license(s).

- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.

- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).

- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.

- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.

In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.

Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.

Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.

In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.

Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct

This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.

These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.

Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

kbuild: remove KBUILD_SUBDIR_ASFLAGS and KBUILD_SUBDIR_CCFLAGS

Accumulate subdir-{cc,as}flags-y directly to KBUILD_{A,C}FLAGS.
Remove KBUILD_SUBDIR_{AS,CC}FLAGS.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>

kbuild: comments cleanup in Makefile.lib

It has:
1. Move comments close to what it want to comment.
2. Comments cleanup & improvement.

Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

remove gperf left-overs from build system

I removed all the gperf use, but not the Makefile rules. Sam Ravnborg
says I get bonus points for cleaning this up. I'll hold him to it.

Requested-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

of: search scripts/dtc/include-prefixes path for both CPP and DTC

Since commit d5d332d3f7e8 ("devicetree: Move include prefixes from
arch to separate directory"), cross-arch DT reference works well,
but only for CPP style #include directives.

It makes as much sense to share DT between different architectures
by using DTC's /include/ directives.

So, scripts/dtc/include-prefixes should be passed to both CPP and DTC.
I refactored Makefile.lib a bit to not repeat the same path.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>

of: remove arch/$(SRCARCH)/boot/dts from include search path for CPP

Having arch/$(SRCARCH)/boot/dts as an include search path is not
very useful these days because some architectures such as ARM64,
MIPS have no DT in this directory. Instead, they have DT in vendor
sub-directories.

With some DT files in ARM and PowerPC fixed, we can now drop this
include search path.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>

of: remove drivers/of/testcase-data from include search path for CPP

This search path was added by commit b5190516b282 ("of: Move testcase
FDT data into drivers/of"). At that time, it was needed for platform
DT files to include testcase data.

It became unnecessary when commit ae9304c9d311 ("Adding selftest
testdata dynamically into live tree") introduced dynamic addition of
testcase data, but it missed to delete this search path.

Moreover, the directory drivers/of/testcase-data does not exist since
commit 19fd74879a32 ("of/unittest: Rename selftest.c to unittest.c").

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>

devicetree: Move include prefixes from arch to separate directory

We use a directory under arch/$ARCH/boot/dts as an include path
that has links outside of the subtree to find dt-bindings from under
include/dt-bindings. That's been working well, but new DT architectures
haven't been adding them by default.

Recently there's been a desire to share some of the DT material between
arm and arm64, which originally caused developers to create symlinks or
relative includes between the subtrees. This isn't ideal -- it breaks
if the DT files aren't stored in the exact same hierarchy as the kernel
tree, and generally it's just icky.

As a somewhat cleaner solution we decided to add a $ARCH/ prefix link
once, and allow DTS files to reference dtsi (and dts) files in other
architectures that way.

Original approach was to create these links under each architecture,
but it lead to the problem of recursive symlinks.

As a remedy, move the include link directories out of the architecture
trees into a common location. At the same time, they can now share one
directory and one dt-bindings/ link as well.

Fixes: 4027494ae6e3 ('ARM: dts: add arm/arm64 include symlinks')
Reported-by: Russell King <linux@armlinux.org.uk>
Reported-by: Omar Sandoval <osandov@osandov.com>
Reviewed-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Heiko Stuebner <heiko@sntech.de>
Acked-by: Rob Herring <robh@kernel.org>
Cc: Heiko Stuebner <heiko@sntech.de>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mikael Starvik <starvik@axis.com>
Cc: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Frank Rowand <frowand.list@gmail.com>
Cc: linux-arch <linux-arch@vger.kernel.org>
Signed-off-by: Olof Johansson <olof@lixom.net>

of: per-file dtc compiler flags

The dtc compiler version that adds initial support was available
in 4.11-rc1. Add the ability to set an additional dtc compiler
flag is needed by overlays.

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: fix asm-offset generation to work with clang

KBuild abuses the asm statement to write to a file and
clang chokes about these invalid asm statements. Hack it
even more by fooling this is actual valid asm code.

[masahiro:
Import Jeroen's work for U-Boot:
http://patchwork.ozlabs.org/patch/375026/
Tweak sed script a little to avoid garbage '#' for GCC case, like
#define NR_PAGEFLAGS 23 /* __NR_PAGEFLAGS # */ ]

Signed-off-by: Jeroen Hofstee <jeroen@myspectrum.nl>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Matthias Kaehlcke <mka@chromium.org>
Tested-by: Matthias Kaehlcke <mka@chromium.org>

kbuild: consolidate redundant sed script ASM offset generation

This part ended up in redundant code after touched by multiple
people.

[1] Commit 3234282f33b2 ("x86, asm: Fix CFI macro invocations to
deal with shortcomings in gas") added parentheses for defined
expressions to support old gas for x86.

[2] Commit a22dcdb0032c ("x86, asm: Fix ancient-GAS workaround")
split the pattern into two to avoid parentheses for non-numeric
expressions.

[3] Commit 95a2f6f72d37 ("Partially revert patch that encloses
asm-offset.h numbers in brackets") removed parentheses from numeric
expressions as well because parentheses in MN10300 assembly have a
special meaning (pointer access).

Apparently, there is a conflict between [1] and [3]. After all,
[3] took precedence, and a long time has passed since then.

Now, merge the two patterns again because the first one is covered
by the other.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Matthias Kaehlcke <mka@chromium.org>

kbuild: Consolidate header generation from ASM offset information

Largely redundant code is used in different places to generate C headers
from offset information extracted from assembly language output.
Consolidate the code in Makefile.lib and use this instead.

Signed-off-by: Matthias Kaehlcke <mka@chromium.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

dtc: update warning settings for new bus and node/property name checks

dtc gained new warnings checking PCI and simple buses, unit address
formatting, and stricter node and property name checking. Disable the
new dtc warnings by default as there are 1000s. As before, warnings are
enabled with W=1 or W=2. The strict node and property name checks are a
bit subjective, so they are only enabled for W=2.

Signed-off-by: Rob Herring <robh@kernel.org>

kbuild: external module build warnings when KBUILD_OUTPUT set and W=1

Commit db547ef19064 ("Kbuild: don't add obj tree in additional includes")
causes warnings (-Wmissing-include-dirs) when compiling external modules
with KBUILD_OUTPUT set and W=1. This is because $src can be an absolute
path to the external module source which when prefixed with -I$(srctree)/
generates an incorrect directory path.

Signed-off-by: Bruce Allan <bruce.w.allan@intel.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

kcov: allow more fine-grained coverage instrumentation

For more targeted fuzzing, it's better to disable kernel-wide
instrumentation and instead enable it on a per-subsystem basis. This
follows the pattern of UBSAN and allows you to compile in the kcov
driver without instrumenting the whole kernel.

To instrument a part of the kernel, you can use either

# for a single file in the current directory
KCOV_INSTRUMENT_filename.o := y

or

# for all the files in the current directory (excluding subdirectories)
KCOV_INSTRUMENT := y

or

# (same as above)
ccflags-y += $(CFLAGS_KCOV)

or

# for all the files in the current directory (including subdirectories)
subdir-ccflags-y += $(CFLAGS_KCOV)

Link: http://lkml.kernel.org/r/1464008380-11405-1-git-send-email-vegard.nossum@oracle.com
Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Kbuild: don't add obj tree in additional includes

When building with separate object directories and driver specific
Makefiles that add additional header include paths, Kbuild adjusts
the gcc flags so that we include both the directory in the source
tree and in the object tree.

However, due to another bug I fixed earlier, this did not actually
include the correct directory in the object tree, so we know that
we only really need the source tree here. Also, including the
object tree sometimes causes warnings about nonexisting directories
when the include path only exists in the source.

This changes the logic to only emit the -I argument for the srctree,
not for objects. We still need both $(srctree)/$(src) and $(obj)
though, so I'm adding them manually.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Michal Marek <mmarek@suse.com>

kbuild: Get rid of KBUILD_STR

The compiler can accept -DKBUILD_MODNAME="foo", it's just a matter of
quoting. That way, we reduce the gcc command line a bit.

Signed-off-by: Michal Marek <mmarek@suse.com>

dtc: turn off dtc unit address warnings by default

The newly added dtc warning to check DT unit-address without reg
property and vice-versa generates lots of warnings. Turn off the check
unless building with W=1 or W=2.

Signed-off-by: Rob Herring <robh@kernel.org>
Cc: Michal Marek <mmarek@suse.com>
Cc: linux-kbuild@vger.kernel.org

kernel: add kcov code coverage

kcov provides code coverage collection for coverage-guided fuzzing
(randomized testing). Coverage-guided fuzzing is a testing technique
that uses coverage feedback to determine new interesting inputs to a
system. A notable user-space example is AFL
(http://lcamtuf.coredump.cx/afl/). However, this technique is not
widely used for kernel testing due to missing compiler and kernel
support.

kcov does not aim to collect as much coverage as possible. It aims to
collect more or less stable coverage that is function of syscall inputs.
To achieve this goal it does not collect coverage in soft/hard
interrupts and instrumentation of some inherently non-deterministic or
non-interesting parts of kernel is disbled (e.g. scheduler, locking).

Currently there is a single coverage collection mode (tracing), but the
API anticipates additional collection modes. Initially I also
implemented a second mode which exposes coverage in a fixed-size hash
table of counters (what Quentin used in his original patch). I've
dropped the second mode for simplicity.

This patch adds the necessary support on kernel side. The complimentary
compiler support was added in gcc revision 231296.

We've used this support to build syzkaller system call fuzzer, which has
found 90 kernel bugs in just 2 months:

https://github.com/google/syzkaller/wiki/Found-Bugs

We've also found 30+ bugs in our internal systems with syzkaller.
Another (yet unexplored) direction where kcov coverage would greatly
help is more traditional "blob mutation". For example, mounting a
random blob as a filesystem, or receiving a random blob over wire.

Why not gcov. Typical fuzzing loop looks as follows: (1) reset
coverage, (2) execute a bit of code, (3) collect coverage, repeat. A
typical coverage can be just a dozen of basic blocks (e.g. an invalid
input). In such context gcov becomes prohibitively expensive as
reset/collect coverage steps depend on total number of basic
blocks/edges in program (in case of kernel it is about 2M). Cost of
kcov depends only on number of executed basic blocks/edges. On top of
that, kernel requires per-thread coverage because there are always
background threads and unrelated processes that also produce coverage.
With inlined gcov instrumentation per-thread coverage is not possible.

kcov exposes kernel PCs and control flow to user-space which is
insecure. But debugfs should not be mapped as user accessible.

Based on a patch by Quentin Casasnovas.

[akpm@linux-foundation.org: make task_struct.kcov_mode have type `enum kcov_mode']
[akpm@linux-foundation.org: unbreak allmodconfig]
[akpm@linux-foundation.org: follow x86 Makefile layout standards]
Signed-off-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Cc: syzkaller <syzkaller@googlegroups.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Tavis Ormandy <taviso@google.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Cc: Kostya Serebryany <kcc@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Kees Cook <keescook@google.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Sasha Levin <sasha.levin@oracle.com>
Cc: David Drysdale <drysdale@google.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Kirill A. Shutemov <kirill@shutemov.name>
Cc: Jiri Slaby <jslaby@suse.cz>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

kbuild: Allow using host dtc instead of kernel's copy

Development of dtc happens in its own upstream repository, but testing
dtc changes against the kernel tree is useful. Change dtc to a variable
that users can override.

Signed-off-by: Rob Herring <robh@kernel.org>
Cc: Michal Marek <mmarek@suse.com>
Cc: linux-kbuild@vger.kernel.org

UBSAN: run-time undefined behavior sanity checker

UBSAN uses compile-time instrumentation to catch undefined behavior
(UB). Compiler inserts code that perform certain kinds of checks before
operations that could cause UB. If check fails (i.e. UB detected)
__ubsan_handle_* function called to print error message.

So the most of the work is done by compiler. This patch just implements
ubsan handlers printing errors.

GCC has this capability since 4.9.x [1] (see -fsanitize=undefined
option and its suboptions).
However GCC 5.x has more checkers implemented [2].
Article [3] has a bit more details about UBSAN in the GCC.

[1] - https://gcc.gnu.org/onlinedocs/gcc-4.9.0/gcc/Debugging-Options.html
[2] - https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html
[3] - http://developerblog.redhat.com/2014/10/16/gcc-undefined-behavior-sanitizer-ubsan/

Issues which UBSAN has found thus far are:

Found bugs:

* out-of-bounds access - 97840cb67ff5 ("netfilter: nfnetlink: fix
insufficient validation in nfnetlink_bind")

undefined shifts:

* d48458d4a768 ("jbd2: use a better hash function for the revoke
table")

* 10632008b9e1 ("clockevents: Prevent shift out of bounds")

* 'x << -1' shift in ext4 -
http://lkml.kernel.org/r/<5444EF21.8020501@samsung.com>

* undefined rol32(0) -
http://lkml.kernel.org/r/<1449198241-20654-1-git-send-email-sasha.levin@oracle.com>

* undefined dirty_ratelimit calculation -
http://lkml.kernel.org/r/<566594E2.3050306@odin.com>

* undefined roundown_pow_of_two(0) -
http://lkml.kernel.org/r/<1449156616-11474-1-git-send-email-sasha.levin@oracle.com>

* [WONTFIX] undefined shift in __bpf_prog_run -
http://lkml.kernel.org/r/<CACT4Y+ZxoR3UjLgcNdUm4fECLMx2VdtfrENMtRRCdgHB2n0bJA@mail.gmail.com>

WONTFIX here because it should be fixed in bpf program, not in kernel.

signed overflows:

* 32a8df4e0b33f ("sched: Fix odd values in effective_load()
calculations")

* mul overflow in ntp -
http://lkml.kernel.org/r/<1449175608-1146-1-git-send-email-sasha.levin@oracle.com>

* incorrect conversion into rtc_time in rtc_time64_to_tm() -
http://lkml.kernel.org/r/<1449187944-11730-1-git-send-email-sasha.levin@oracle.com>

* unvalidated timespec in io_getevents() -
http://lkml.kernel.org/r/<CACT4Y+bBxVYLQ6LtOKrKtnLthqLHcw-BMp3aqP3mjdAvr9FULQ@mail.gmail.com>

* [NOTABUG] signed overflow in ktime_add_safe() -
http://lkml.kernel.org/r/<CACT4Y+aJ4muRnWxsUe1CMnA6P8nooO33kwG-c8YZg=0Xc8rJqw@mail.gmail.com>

[akpm@linux-foundation.org: fix unused local warning]
[akpm@linux-foundation.org: fix __int128 build woes]
Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sasha Levin <sasha.levin@oracle.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Michal Marek <mmarek@suse.cz>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Yury Gribov <y.gribov@samsung.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: Kostya Serebryany <kcc@google.com>
Cc: Johannes Berg <johannes@sipsolutions.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

kbuild: add AFLAGS_REMOVE_(basename).o option

It is already possible to remove CFLAGS with the CFLAGS_REMOVE option
that was introduced with commit 656ee82cc855 ("kbuild: create new
CFLAGS_REMOVE_(basename).o option"). However it is not possible to
remove AFLAGS for assembler files.

So this patch just adds the AFLAGS_REMOVE option which works the same
like CFLAGS_REMOVE.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Acked-by: Michal Marek <mmarek@suse.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

kbuild: Allow to specify composite modules with modname-m

This allows to write

drm-$(CONFIG_AGP) += drm_agpsupport.o

without having to handle CONFIG_AGP=y vs. CONFIG_AGP=m. Only support
this syntax for modules, since built-in code depending on something
modular cannot work and init/Makefile actually relies on the current
semantics. There are a few drivers which adapted to the current
semantics out of necessity; these are fixed to also work when the
respective subsystem is modular.

Acked-by: Peter Chen <peter.chen@freescale.com> [chipidea]
Signed-off-by: Michal Marek <mmarek@suse.com>

kbuild: Create directory for target DTB

When building specific DTBs out of the kernel tree the vendor subdirs
(boot/dts/<vendor>) are not created, ensure that they are before
building the DTB.

Signed-off-by: Nathan Rossi <nathan.rossi@xilinx.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Olof Johansson <olof@lixom.net>

kasan: add kernel address sanitizer infrastructure

Kernel Address sanitizer (KASan) is a dynamic memory error detector. It
provides fast and comprehensive solution for finding use-after-free and
out-of-bounds bugs.

KASAN uses compile-time instrumentation for checking every memory access,
therefore GCC > v4.9.2 required. v4.9.2 almost works, but has issues with
putting symbol aliases into the wrong section, which breaks kasan
instrumentation of globals.

This patch only adds infrastructure for kernel address sanitizer. It's
not available for use yet. The idea and some code was borrowed from [1].

Basic idea:

The main idea of KASAN is to use shadow memory to record whether each byte
of memory is safe to access or not, and use compiler's instrumentation to
check the shadow memory on each memory access.

Address sanitizer uses 1/8 of the memory addressable in kernel for shadow
memory and uses direct mapping with a scale and offset to translate a
memory address to its corresponding shadow address.

Here is function to translate address to corresponding shadow address:

unsigned long kasan_mem_to_shadow(unsigned long addr)
{
return (addr >> KASAN_SHADOW_SCALE_SHIFT) + KASAN_SHADOW_OFFSET;
}

where KASAN_SHADOW_SCALE_SHIFT = 3.

So for every 8 bytes there is one corresponding byte of shadow memory.
The following encoding used for each shadow byte: 0 means that all 8 bytes
of the corresponding memory region are valid for access; k (1 <= k <= 7)
means that the first k bytes are valid for access, and other (8 - k) bytes
are not; Any negative value indicates that the entire 8-bytes are
inaccessible. Different negative values used to distinguish between
different kinds of inaccessible memory (redzones, freed memory) (see
mm/kasan/kasan.h).

To be able to detect accesses to bad memory we need a special compiler.
Such compiler inserts a specific function calls (__asan_load*(addr),
__asan_store*(addr)) before each memory access of size 1, 2, 4, 8 or 16.

These functions check whether memory region is valid to access or not by
checking corresponding shadow memory. If access is not valid an error
printed.

Historical background of the address sanitizer from Dmitry Vyukov:

"We've developed the set of tools, AddressSanitizer (Asan),
ThreadSanitizer and MemorySanitizer, for user space. We actively use
them for testing inside of Google (continuous testing, fuzzing,
running prod services). To date the tools have found more than 10'000
scary bugs in Chromium, Google internal codebase and various
open-source projects (Firefox, OpenSSL, gcc, clang, ffmpeg, MySQL and
lots of others): [2] [3] [4].
The tools are part of both gcc and clang compilers.

We have not yet done massive testing under the Kernel AddressSanitizer
(it's kind of chicken and egg problem, you need it to be upstream to
start applying it extensively). To date it has found about 50 bugs.
Bugs that we've found in upstream kernel are listed in [5].
We've also found ~20 bugs in out internal version of the kernel. Also
people from Samsung and Oracle have found some.

[...]

As others noted, the main feature of AddressSanitizer is its
performance due to inline compiler instrumentation and simple linear
shadow memory. User-space Asan has ~2x slowdown on computational
programs and ~2x memory consumption increase. Taking into account that
kernel usually consumes only small fraction of CPU and memory when
running real user-space programs, I would expect that kernel Asan will
have ~10-30% slowdown and similar memory consumption increase (when we
finish all tuning).

I agree that Asan can well replace kmemcheck. We have plans to start
working on Kernel MemorySanitizer that finds uses of unitialized
memory. Asan+Msan will provide feature-parity with kmemcheck. As
others noted, Asan will unlikely replace debug slab and pagealloc that
can be enabled at runtime. Asan uses compiler instrumentation, so even
if it is disabled, it still incurs visible overheads.

Asan technology is easily portable to other architectures. Compiler
instrumentation is fully portable. Runtime has some arch-dependent
parts like shadow mapping and atomic operation interception. They are
relatively easy to port."

Comparison with other debugging features:
========================================

KMEMCHECK:

- KASan can do almost everything that kmemcheck can. KASan uses
compile-time instrumentation, which makes it significantly faster than
kmemcheck. The only advantage of kmemcheck over KASan is detection of
uninitialized memory reads.

Some brief performance testing showed that kasan could be
x500-x600 times faster than kmemcheck:

$ netperf -l 30
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to localhost (127.0.0.1) port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

no debug: 87380 16384 16384 30.00 41624.72

kasan inline: 87380 16384 16384 30.00 12870.54

kasan outline: 87380 16384 16384 30.00 10586.39

kmemcheck: 87380 16384 16384 30.03 20.23

- Also kmemcheck couldn't work on several CPUs. It always sets
number of CPUs to 1. KASan doesn't have such limitation.

DEBUG_PAGEALLOC:
- KASan is slower than DEBUG_PAGEALLOC, but KASan works on sub-page
granularity level, so it able to find more bugs.

SLUB_DEBUG (poisoning, redzones):
- SLUB_DEBUG has lower overhead than KASan.

- SLUB_DEBUG in most cases are not able to detect bad reads,
KASan able to detect both reads and writes.

- In some cases (e.g. redzone overwritten) SLUB_DEBUG detect
bugs only on allocation/freeing of object. KASan catch
bugs right before it will happen, so we always know exact
place of first bad read/write.

[1] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel
[2] https://code.google.com/p/address-sanitizer/wiki/FoundBugs
[3] https://code.google.com/p/thread-sanitizer/wiki/FoundBugs
[4] https://code.google.com/p/memory-sanitizer/wiki/FoundBugs
[5] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel#Trophies

Based on work by Andrey Konovalov.

Signed-off-by: Andrey Ryabinin <a.ryabinin@samsung.com>
Acked-by: Michal Marek <mmarek@suse.cz>
Signed-off-by: Andrey Konovalov <adech.fo@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Konstantin Serebryany <kcc@google.com>
Cc: Dmitry Chernenkov <dmitryc@google.com>
Cc: Yuri Gribov <tetra2005@gmail.com>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: Sasha Levin <sasha.levin@oracle.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

dts, kbuild: Factor out dtbs install rules to Makefile.dtbinst

Move dtbs install rules to Makefile.dtbinst. This change is needed to
implement support for dts vendor subdirs. The change makes Makefiles
easier and smaller as no longer the dtbs_install rule needs to be
defined. Another advantage is that install goals are not encoded in
targets anymore (%.dtb_dtbinst_).

Signed-off-by: Robert Richter <rrichter@cavium.com>

kbuild: handle multi-objs dependency appropriately

The comment in scripts/Makefile.build says as follows:

We would rather have a list of rules like
foo.o: $(foo-objs)
but that's not so easy, so we rather make all composite objects depend
on the set of all their parts

This commit makes it possible!

For example, assume a Makefile like this

obj-m = foo.o bar.o
foo-objs := foo1.o foo2.o
bar-objs := bar1.o bar2.o

Without this patch, foo.o depends on all of
foo1.o foo2.o bar1.o bar2.o.
It looks funny that foo.o is regenerated when bar1.c is updated.

Now we can handle the dependency of foo.o and bar.o separately.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: trivial - remove trailing spaces

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: move "quote" to Kbuild.include to be consistent

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: dtbs_install: new make target

Unlike other build products in the Linux kernel, there is no 'make
*install' mechanism to put devicetree blobs in a standard place.

This commit adds a new 'dtbs_install' make target which copies all of
the dtbs into the INSTALL_DTBS_PATH directory. INSTALL_DTBS_PATH can be
set before calling make to change the default install directory. If not
set then it defaults to:

$INSTALL_PATH/dtbs/$KERNELRELEASE.

This is done to keep dtbs from different kernel versions separate until
things have settled down. Once the dtbs are stable, and not so strongly
linked to the kernel version, the devicetree files will most likely move
to their own repo. Users will need to upgrade install scripts at that
time.

v7: (reworked by Grant Likely)
- Moved rules from arch/arm/Makefile to arch/arm/boot/dts/Makefile so
that each dtb install could have a separate target and be reported as
part of the make output.
- Fixed dependency problem to ensure $KERNELRELEASE is calculated before
attempting to install
- Removed option to call external script. Copying the files should be
sufficient and a build system can post-process the install directory.
Despite the fact an external script is used for installing the kernel,
I don't think that is a pattern that should be encouraged. I would
rather see buildroot type tools post process the install directory to
rename or move dtb files after installing to a staging directory.
- Plus it is easy to add a hook after the fact without blocking the
rest of this feature.
- Move the helper targets into scripts/Makefile.lib with the rest of the
common dtb rules

Signed-off-by: Jason Cooper <jason@lakedaemon.net>
Signed-off-by: Grant Likely <grant.likely@linaro.org>
Cc: Michal Marek <mmarek@suse.cz>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Rob Herring <robh+dt@kernel.org>

of: Move testcase FDT data into drivers/of

The testcase data is usable by any platform. This patch moves it into
the drivers/of directory so it can be included by any architecture.

Using the test cases requires manually adding #include <testcases.dtsi>
to the end of the boards .dtsi file and enabling CONFIG_OF_SELFTEST. Not
pretty though. A useful project would be to make the testcase code
easier to execute.

Signed-off-by: Grant Likely <grant.likely@linaro.org>

lib: add support for LZ4-compressed kernel

Add support for extracting LZ4-compressed kernel images, as well as
LZ4-compressed ramdisk images in the kernel boot process.

Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Florian Fainelli <florian@openwrt.org>
Cc: Yann Collet <yann.collet.73@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

kbuild: create directory for dir/file.o

When add a obj with dir to obj-y, like this

obj-y += dir/file.o

The $(obj)/dir not created, this patch fix this.

When try to add a file(which in a subdir) to my board's obj-y, the build
progress crashed.

For example, I use at91rm9200ek board, and in kernel dir run:

mkdir objtree
make O=objtree at91rm9200_defconfig
mkdir arch/arm/mach-at91/dir
touch arch/arm/mach-at91/dir/file.c

and edit arch/arm/mach-at91/dir/file.c to add some code.
then edit arch/arm/mach-at91/Makefile, change the following line:

obj-$(CONFIG_MACH_AT91RM9200EK) += board-rm9200ek.o

to:

obj-$(CONFIG_MACH_AT91RM9200EK) += board-rm9200ek.o dir/file.o

Now build it:

make O=objtree

Then the error appears:
...
CC arch/arm/mach-at91/board-rm9200dk.o
CC arch/arm/mach-at91/board-rm9200ek.o
CC arch/arm/mach-at91/dir/file.o
linux-2.6/arch/arm/mach-at91/dir/file.c:5:
fatal error: opening dependency file
arch/arm/mach-at91/dir/.file.o.d: No such file or directory

Check the objtree:
LANG=en ls objtree/arch/arm/mach-at91/dir
ls: cannot access objtree/arch/arm/mach-at91/dir: No such file or directory

It's apparently that the target dir not created for file.o

Check kbuild source code. It seems that kbuild create dirs for that in
$(obj-dirs). But if the dir need not to create a built-in.o, It should
never in $(obj-dirs).

So I make this patch to make sure It in $(obj-dirs)

this bug caused by commit
f5fb976520a53f45f8bbf2e851f16b3b5558d485

Signed-off-by: 张忠山 <zzs0213@gmail.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: make sure we clean up DTB temporary files

Various temporary files used when building DTB files were not suffixed with
.tmp and therefore were not cleaned up by "make clean".

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Grant Likely <grant.likely@linaro.org>

Makefile.lib: align DTB quiet_cmd

The unaligned dtb.S filename in make output started to irritate me:
DTC arch/metag/boot/dts/skeleton.dtb
DTB arch/metag/boot/dts/skeleton.dtb.S
AS arch/metag/boot/dts/skeleton.dtb.o
LD arch/metag/boot/dts/built-in.o

Add an extra space to quiet_cmd_dt_S_dtb so the dtb.S filename aligns
with all the others.

Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Dirk Brandewie <dirk.brandewie@gmail.com>
Cc: Grant Likely <grant.likely@secretlab.ca>
Cc: trivial@kernel.org
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: Don't assume dts files live in arch/*/boot/dts

In commit b40b25ff (kbuild: always run gcc -E on *.dts, remove cmd_dtc_cpp),
dts building was changed to always use the C preprocessor. This meant
that the .dts file passed to dtc is not the original, but the
preprocessed one.

When compiling with a separate build directory (i.e., with O=), this
preprocessed file will not live in the same directory as the original.
When the .dts file includes .dtsi files, dtc will look for them in the
build directory, not in the source directory and compilation will fail.

The commit referenced above tried to fix this by passing arch/*/boot/dts
as an include path to dtc. However, for mips, the .dts files are not in
this directory, so dts compilation on mips breaks for some targets.

Instead of hardcoding this particular include path, this commit just
uses the directory of the .dts file that is being compiled, which
effectively restores the previous behaviour wrt includes. For most .dts
files, this path is just the same as the previous hardcoded
arch/*/boot/dts path.

This was tested on a mips (rt3052) and an arm (bcm2835) target.

Signed-off-by: Matthijs Kooijman <matthijs@stdin.nl>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: always run gcc -E on *.dts, remove cmd_dtc_cpp

Replace cmd_dtc with cmd_dtc_cpp, and delete the latter.

Previously, a special file extension (.dtsp) was required to trigger
the C pre-processor to run on device tree files. This was ugly. Now that
previous changes have enhanced cmd_dtc_cpp to collect dependency
information from both gcc -E and dtc, we can transparently run the pre-
processor on all device tree files, irrespective of whether they
use /include/ or #include syntax to include *.dtsi.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Rob Herring <rob.herring@calxeda.com>

kbuild: cmd_dtc_cpp: extract deps from both gcc -E and dtc

Prior to this change, when compiling *.dts to *.dtb, the dependency
output from dtc would be used, and when compiling *.dtsp to *.dtb, the
dependency output from gcc -E alone would be used, despite dtc also
being invoked (on a temporary file that was guaranteed to have no
dependencies).

With this change, when compiling *.dtsp to *.dtb, the dependency files
from both gcc -E and dtc are used. This will allow cmd_dtc_cpp to
replace cmd_dtc in a future change. In turn, that will allow the C pre-
processor to be run transparently on *.dts, without the need to a
separate rule or file extension to trigger it.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Rob Herring <rob.herring@calxeda.com>

kbuild: create an "include chroot" for DT bindings

The recent dtc+cpp support allows header files and C pre-processor
defines/macros to be used when compiling device tree files. These
headers will typically define various constants that are part of the
device tree bindings.

The original patch which set up the dtc+cpp include path only considered
using those headers from device tree files. However, most are also
useful for kernel code which needs to interpret the device tree.

In both the DT files and the kernel, I'd like to include the DT-related
headers in the same way, for example, <dt-bindings/gpio/tegra-gpio.h>.
That will simplify any text which discusses the DT header locations.

Creating a <dt-bindings/> for kernel source to use is as simple as
placing files into include/dt-bindings/.

However, when compiling DT files, the include path should be restricted
so that only the dt-bindings path is available; arbitrary kernel headers
shouldn't be exposed. For this reason, create a specific include
directory for use by dtc+cpp, and symlink dt-bindings from there to the
actual location of include/dt-bindings/. For want of a better location,
place this "include chroot" into the existing dts/ directory.

arch/*/boot/dts/include/dt-bindings -> ../../../../../include/dt-bindings

Some headers used by device tree files may not be useful to the kernel;
they may be used simply to aid in constructing the DT file (e.g. macros
to create a node), but not define any information that the kernel needs
to share. These may be placed directly into arch/*/boot/dts/ along with
the DT files themselves.

Acked-by: Michal Marek <mmarek@suse.cz>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Acked-by: Rob Herring <rob.herring@calxeda.com>
Signed-off-by: Stephen Warren <swarren@nvidia.com>

CONFIG_SYMBOL_PREFIX: cleanup.

We have CONFIG_SYMBOL_PREFIX, which three archs define to the string
"_". But Al Viro broke this in "consolidate cond_syscall and
SYSCALL_ALIAS declarations" (in linux-next), and he's not the first to
do so.

Using CONFIG_SYMBOL_PREFIX is awkward, since we usually just want to
prefix it so something. So various places define helpers which are
defined to nothing if CONFIG_SYMBOL_PREFIX isn't set:

1) include/asm-generic/unistd.h defines __SYMBOL_PREFIX.
2) include/asm-generic/vmlinux.lds.h defines VMLINUX_SYMBOL(sym)
3) include/linux/export.h defines MODULE_SYMBOL_PREFIX.
4) include/linux/kernel.h defines SYMBOL_PREFIX (which differs from #7)
5) kernel/modsign_certificate.S defines ASM_SYMBOL(sym)
6) scripts/modpost.c defines MODULE_SYMBOL_PREFIX
7) scripts/Makefile.lib defines SYMBOL_PREFIX on the commandline if
CONFIG_SYMBOL_PREFIX is set, so that we have a non-string version
for pasting.

(arch/h8300/include/asm/linkage.h defines SYMBOL_NAME(), too).

Let's solve this properly:
1) No more generic prefix, just CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX.
2) Make linux/export.h usable from asm.
3) Define VMLINUX_SYMBOL() and VMLINUX_SYMBOL_STR().
4) Make everyone use them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Reviewed-by: James Hogan <james.hogan@imgtec.com>
Tested-by: James Hogan <james.hogan@imgtec.com> (metag)

kbuild: limit dtc+cpp include path

Device tree source files may now include header files. The intent is
that those header files define/name constants used as part of the DT
bindings. Currently this feature is open to abuse, since any kernel
header file at all can be included, This could allow device tree files
to become dependant on kernel headers files, and thus make them no
longer OS-independent. This would also prevent separating the device
tree source files from the kernel repository.

Solve this by limiting the cpp include path for device tree files to
separate directories.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

kbuild: create a rule to run the pre-processor on *.dts files

Create cmd_dtc_cpp to run the C pre-processor on *.dts file before
passing them to dtc for final compilation. This allows the use of #define
and #include within the .dts file.

Acked-by: Simon Glass <sjg@chromium.org>
Acked-by: Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
Acked-by: Michal Marek <mmarek@suse.cz>
Acked-by: Srinivas Kandagatla <srinivas.kandagatla@st.com>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

kbuild: centralize .dts->.dtb rule

All architectures that use cmd_dtc do so in almost the same way. Create
a central build rule to avoid duplication. The one difference is that
most current uses of dtc build $(obj)/%.dtb from $(src)/dts/%.dts rather
than building the .dtb in the same directory as the .dts file. This
difference will be eliminated arch-by-arch in future patches.

MIPS is the exception here; it already uses the exact same rule as the
new common rule, so the duplicate is removed in this patch to avoid any
conflict. arch/mips changes courtesy of Ralf Baechle.

Update Documentation/kbuild to remove the explicit call to cmd_dtc from
the example, now that the rule exists in a centralized location.

Cc: Arnd Bergmann <arnd@arndb.de>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Olof Johansson <olof@lixom.net>
Cc: Russell King <linux@arm.linux.org.uk>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Jonas Bonn <jonas@southpole.se>
Cc: linux@lists.openrisc.net
Cc: Aurelien Jacquiot <a-jacquiot@ti.com>
Cc: linux-c6x-dev@linux-c6x.org
Cc: Mark Salter <msalter@redhat.com>
Cc: Michal Simek <monstr@monstr.eu>
Cc: microblaze-uclinux@itee.uq.edu.au
Cc: Chris Zankel <chris@zankel.net>
Cc: linux-xtensa@linux-xtensa.org
Cc: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Rob Herring <rob.herring@calxeda.com>

Kbuild: centralize MKIMAGE and cmd_uimage definitions

All ARCHs have the same definition of MKIMAGE. Move it to Makefile.lib
to avoid duplication.

All ARCHs have similar definitions of cmd_uimage. Place a sufficiently
parameterized version in Makefile.lib to avoid duplication.

Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Tested-by: Mike Frysinger <vapier@gentoo.org> [Blackfin]
Tested-by: Michal Simek <monstr@monstr.eu> [Microblaze]
Tested-by: Guan Xuetao <gxt@mprc.pku.edu.cn> [unicore32]
Signed-off-by: Michal Marek <mmarek@suse.cz>

Kbuild: Use dtc's -d (dependency) option

This hooks dtc into Kbuild's dependency system.

Thus, for example, "make dtbs" will rebuild tegra-harmony.dtb if only
tegra20.dtsi has changed yet tegra-harmony.dts has not. The previous
lack of this feature recently caused me to have very confusing "git
bisect" results.

For ARM, it's obvious what to add to $(targets). I'm not familiar enough
with other architectures to know what to add there. Powerpc appears to
already add various .dtb files into $(targets), but the other archs may
need something added to $(targets) to work.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
[mmarek: Dropped arch/c6x part to avoid merging commits from the middle
of the merge window]
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: Fix comment in Makefile.lib

KBUILD_MODNAME is not defined for files that are linked into multiple
modules, and trying to change reality to match documentation would
result in all sorts of trouble. E.g. options for built-in modules would
be called either foo_bar.param, foo.param, or bar.param, depending on
the configuration. So just change the comment.

Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: prevent make from deleting _shipped files

commit 7373f4f (kbuild: add implicit rules for parser generation)
created a implicit rule chain (%.c: %.c_shipped: %.y).
Make considers the _shipped files to be intermediate files which
causes them to be deleted if they didn't exist before make was run.
Mark the _shipped files PRECIOUS to prevent make from deleting them.

Signed-off-by: Peter Foley <pefoley2@verizon.net>
Acked-by: Arnaud Lacombe <lacombar@gmail.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: simplify the %_shipped rule

This is needed to have make(1) correctly link the implicit rules which
generate the _shipped file from the lexer/parser to the final file.

Signed-off-by: Arnaud Lacombe <lacombar@gmail.com>

kbuild: add implicit rules for parser generation

Cc: David Gibson <david@gibson.dropbear.id.au>
Cc: Michal Marek <mmarek@suse.cz>
Signed-off-by: Arnaud Lacombe <lacombar@gmail.com>

kbuild: Call gzip with -n

The timestamps recorded in the .gz files add no value.

Signed-off-by: Michal Marek <mmarek@suse.cz>

decompressors: add XZ decompressor module

In userspace, the .lzma format has become mostly a legacy file format that
got superseded by the .xz format. Similarly, LZMA Utils was superseded by
XZ Utils.

These patches add support for XZ decompression into the kernel. Most of
the code is as is from XZ Embedded <http://tukaani.org/xz/embedded.html>.
It was written for the Linux kernel but is usable in other projects too.

Advantages of XZ over the current LZMA code in the kernel:
- Nice API that can be used by other kernel modules; it's
not limited to kernel, initramfs, and initrd decompression.
- Integrity check support (CRC32)
- BCJ filters improve compression of executable code on
certain architectures. These together with LZMA2 can
produce a few percent smaller kernel or Squashfs images
than plain LZMA without making the decompression slower.

This patch: Add the main decompression code (xz_dec), testing module
(xz_dec_test), wrapper script (xz_wrap.sh) for the xz command line tool,
and documentation. The xz_dec module is enough to have a usable XZ
decompressor e.g. for Squashfs.

Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Alain Knaff <alain@knaff.lu>
Cc: Albin Tonnerre <albin.tonnerre@free-electrons.com>
Cc: Phillip Lougher <phillip@lougher.demon.co.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

of: Add support for linking device tree blobs into vmlinux

This patch adds support for linking device tree blob(s) into
vmlinux. Modifies asm-generic/vmlinux.lds.h to add linking
.dtb sections into vmlinux. To maintain compatiblity with the of/fdt
driver code platforms MUST copy the blob to a non-init memory location
before the kernel frees the .init.* sections in the image.

Modifies scripts/Makefile.lib to add a kbuild command to
compile DTS files to device tree blobs and a rule to create objects to
wrap the blobs for linking.

STRUCT_ALIGNMENT is defined in vmlinux.lds.h for use in the rule to
create wrapper objects for the dtb in Makefile.lib. The
STRUCT_ALIGN() macro in vmlinux.lds.h is modified to use the
STRUCT_ALIGNMENT definition.

The DTB's are placed on 32 byte boundries to allow parsing the blob
with driver/of/fdt.c during early boot without having to copy the blob
to get the structure alignment GCC expects.

A DTB is linked in by adding the DTB object to the list of objects to
be linked into vmlinux in the archtecture specific Makefile using
obj-y += foo.dtb.o

Signed-off-by: Dirk Brandewie <dirk.brandewie@gmail.com>
Acked-by: Michal Marek <mmarek@suse.cz>
[grant.likely@secretlab.ca: cleaned up whitespace inconsistencies]
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>

initramfs: Fix build break on symbol-prefixed archs

Signed-off-by: Mike Frysinger <vapier.adi@gmail.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

jump label: Convert dynamic debug to use jump labels

Convert the 'dynamic debug' infrastructure to use jump labels.

Signed-off-by: Jason Baron <jbaron@redhat.com>
LKML-Reference: <b77627358cea3e27d7be4386f45f66219afb8452.1284733808.git.jbaron@redhat.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>

x86: Add optimized popcnt variants

Add support for the hardware version of the Hamming weight function,
popcnt, present in CPUs which advertize it under CPUID, Function
0x0000_0001_ECX[23]. On CPUs which don't support it, we fallback to the
default lib/hweight.c sw versions.

A synthetic benchmark comparing popcnt with __sw_hweight64 showed almost
a 3x speedup on a F10h machine.

Signed-off-by: Borislav Petkov <borislav.petkov@amd.com>
LKML-Reference: <20100318112015.GC11152@aftab>
Signed-off-by: H. Peter Anvin <hpa@zytor.com>

scripts/Makefile.lib: Align the output of LZO

The output of LZO is not aligned with the other output:
...
CC drivers/usb/mon/usbmon.mod.o
LZO arch/mips/boot/compressed/vmlinux.lzo
...

This patch fixes it.

Signed-off-by: Wu Zhangjin <wuzhangjin@gmail.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: really fix bzImage build with non-bash sh

In an x86 build with CONFIG_KERNEL_LZMA enabled and dash as sh,
arch/x86/boot/compressed/vmlinux.bin.lzma ends with
'\xf0\x7d\x39\x00' (16 bytes) instead of the 4 bytes intended and
the resulting vmlinuz fails to boot. This improves on the
previous behavior, in which the file contained the characters
'-ne ' as well, but not by much.

Previous commits replaced "echo -ne" first with "/bin/echo -ne",
then "printf" in the hope of improving portability, but none of
these commands is guaranteed to support hexadecimal escapes on
POSIX systems. So use the shell to convert from hexadecimal to
octal.

With this change, an LZMA-compressed kernel built with dash as sh
boots correctly again.

Reported-by: Sebastian Dalfuß <sd@sedf.de>
Reported-by: Oliver Hartkopp <oliver@hartkopp.net>
Reported-by: Michael Guntsche <mike@it-loops.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Cc: Michael Tokarev <mjt@tls.msk.ru>
Cc: Alek Du <alek.du@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Michal Marek <mmarek@suse.cz>

lib: add support for LZO-compressed kernels

This patch series adds generic support for creating and extracting
LZO-compressed kernel images, as well as support for using such images on
the x86 and ARM architectures, and support for creating and using
LZO-compressed initrd and initramfs images.

Russell King said:

: Testing on a Cortex A9 model:
: - lzo decompressor is 65% of the time gzip takes to decompress a kernel
: - lzo kernel is 9% larger than a gzip kernel
:
: which I'm happy to say confirms your figures when comparing the two.
:
: However, when comparing your new gzip code to the old gzip code:
: - new is 99% of the size of the old code
: - new takes 42% of the time to decompress than the old code
:
: What this means is that for a proper comparison, the results get even better:
: - lzo is 7.5% larger than the old gzip'd kernel image
: - lzo takes 28% of the time that the old gzip code took
:
: So the expense seems definitely worth the effort. The only reason I
: can think of ever using gzip would be if you needed the additional
: compression (eg, because you have limited flash to store the image.)
:
: I would argue that the default for ARM should therefore be LZO.

This patch:

The lzo compressor is worse than gzip at compression, but faster at
extraction. Here are some figures for an ARM board I'm working on:

Uncompressed size: 3.24Mo
gzip 1.61Mo 0.72s
lzo 1.75Mo 0.48s

So for a compression ratio that is still relatively close to gzip, it's
much faster to extract, at least in that case.

This part contains:
- Makefile routine to support lzo compression
- Fixes to the existing lzo compressor so that it can be used in
compressed kernels
- wrapper around the existing lzo1x_decompress, as it only extracts one
block at a time, while we need to extract a whole file here
- config dialog for kernel compression

[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: cleanup]
Signed-off-by: Albin Tonnerre <albin.tonnerre@free-electrons.com>
Tested-by: Wu Zhangjin <wuzhangjin@gmail.com>
Acked-by: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Russell King <rmk@arm.linux.org.uk>
Acked-by: Russell King <rmk@arm.linux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

module: make MODULE_SYMBOL_PREFIX into a CONFIG option

The next commit will require the use of MODULE_SYMBOL_PREFIX in
.tmp_exports-asm.S. Currently it is mixed in with C structure
definitions in "asm/module.h". Move the definition of this arch option
into Kconfig, so it can be easily accessed by any code.

This also lets modpost.c use the same definition. Previously modpost
relied on a hardcoded list of architectures in mk_elfconfig.c.

A build test for blackfin, one of the two MODULE_SYMBOL_PREFIX archs,
showed the generated code was unchanged. vmlinux was identical save
for build ids, and an apparently randomized suffix on a single "__key"
symbol in the kallsyms data).

Signed-off-by: Alan Jenkins <alan-jenkins@tuffmail.co.uk>
Acked-by: Mike Frysinger <vapier@gentoo.org> (blackfin)
CC: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

kbuild: fix bzImage build for x86

As has been discussed previously (and Sam has been CC'ed), the fix
is still incorrect. It replaces "echo -ne" with "/bin/echo -ne",
but neither of the two are guaranteed to support the necessary
arguments and necessary (hexadecimal) escape sequences. What should
be used here is printf(1). The trivial patch below (on top of these
kbuild changes) fixes this issue.

Signed-Off-By: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Michal Marek <mmarek@suse.cz>

kbuild: Fix size_append issue for bzip2/lzma kernel

The Makefile.lib will call "echo -ne" to append uncompressed kernel size to
bzip2/lzma kernel image.
The "echo" here depends on the shell that /bin/sh pointing to.
On Ubuntu system, the /bin/sh is pointing to dash, which does not support
"echo -e" at all. Use /bin/echo instead of shell echo should always be safe.

Signed-off-by: Alek Du <alek.du@intel.com>
Acked-by: WANG Cong <xiyou.wangcong@gmail.com>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

gcov: add gcov profiling infrastructure

Enable the use of GCC's coverage testing tool gcov [1] with the Linux
kernel. gcov may be useful for:

* debugging (has this code been reached at all?)
* test improvement (how do I change my test to cover these lines?)
* minimizing kernel configurations (do I need this option if the
associated code is never run?)

The profiling patch incorporates the following changes:

* change kbuild to include profiling flags
* provide functions needed by profiling code
* present profiling data as files in debugfs

Note that on some architectures, enabling gcc's profiling option
"-fprofile-arcs" for the entire kernel may trigger compile/link/
run-time problems, some of which are caused by toolchain bugs and
others which require adjustment of architecture code.

For this reason profiling the entire kernel is initially restricted
to those architectures for which it is known to work without changes.
This restriction can be lifted once an architecture has been tested
and found compatible with gcc's profiling. Profiling of single files
or directories is still available on all platforms (see config help
text).

[1] http://gcc.gnu.org/onlinedocs/gcc/Gcov.html

Signed-off-by: Peter Oberparleiter <oberpar@linux.vnet.ibm.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Huang Ying <ying.huang@intel.com>
Cc: Li Wei <W.Li@Sun.COM>
Cc: Michael Ellerman <michaele@au1.ibm.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Heiko Carstens <heicars2@linux.vnet.ibm.com>
Cc: Martin Schwidefsky <mschwid2@linux.vnet.ibm.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: WANG Cong <xiyou.wangcong@gmail.com>
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

kbuild: allow compressors (gzip, bzip2, lzma) to take multiple inputs

Allow the compression commands in Kbuild (i.e. gzip, bzip2, lzma) to
take multiple input files and emit the concatenated compressed
output. This avoids an intermediate step when a kernel image is built
from multiple components, such as the relocatable x86-32 kernel.

Sam Ravnborg integrated the bin_size script into the Makefile.

[ Impact: new build feature, not yet used ]

Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: introduce subdir-ccflags-y

Following patch introduce support for setting options
to gcc that has effect for current directory and all
subdirectories.

The typical use case are an architecture or a subsystem that
decide to cover all files with -Werror.
Today alpha, mips and sparc uses -Werror in almost all their
Makefile- with subdir-ccflag-y it is now simpler to do so
as only the top-level directories needs to be covered.

Likewise if we decide to cover a full subsystem such
as net/ with -Werror this is done by adding a single
line to net/Makefile.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Thomas Gleixner <tglx@linutronix.de>

dynamic debug: combine dprintk and dynamic printk

This patch combines Greg Bank's dprintk() work with the existing dynamic
printk patchset, we are now calling it 'dynamic debug'.

The new feature of this patchset is a richer /debugfs control file interface,
(an example output from my system is at the bottom), which allows fined grained
control over the the debug output. The output can be controlled by function,
file, module, format string, and line number.

for example, enabled all debug messages in module 'nf_conntrack':

echo -n 'module nf_conntrack +p' > /mnt/debugfs/dynamic_debug/control

to disable them:

echo -n 'module nf_conntrack -p' > /mnt/debugfs/dynamic_debug/control

A further explanation can be found in the documentation patch.

Signed-off-by: Greg Banks <gnb@sgi.com>
Signed-off-by: Jason Baron <jbaron@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

bzip2/lzma: handle failures from bzip2 and lzma correctly

Impact: Bug fix

If bzip2 or lzma fails (for example, if they aren't installed on the
system), we need to propagate the failure out to "make". However,
they were masked by being followed by a semicolon.

Signed-off-by: H. Peter Anvin <hpa@zytor.com>

bzip2/lzma: library support for gzip, bzip2 and lzma decompression

Impact: Replaces inflate.c with a wrapper around zlib_inflate; new library code

This is the first part of the bzip2/lzma patch

The bzip patch is based on an idea by Christian Ludwig, includes support for
compressing the kernel with bzip2 or lzma rather than gzip. Both
compressors give smaller sizes than gzip. Lzma's decompresses faster
than bzip2.

It also supports ramdisks and initramfs' compressed using these two
compressors.

The functionality has been successfully used for a couple of years by
the udpcast project

This version applies to "tip" kernel 2.6.28

This part contains:
- changed inflate.c to accomodate rest of patch
- implementation of bzip2 compression (not used at this stage yet)
- implementation of lzma compression (not used at this stage yet)
- Makefile routines to support bzip2 and lzma kernel compression

Signed-off-by: Alain Knaff <alain@knaff.lu>
Signed-off-by: H. Peter Anvin <hpa@zytor.com>

kbuild: expand -I in KBUILD_CPPFLAGS

kbuild failed to expand include flags in KBUILD_CPPFLAGS
resulting in code like this in arch Makefiles:

ifeq ($(KBUILD_SRC),)
KBUILD_CPPFLAGS += -Iinclude/foo
else
KBUILD_CPPFLAGS += -I$(srctree)/include/foo
endif

Move use of LINUXINCLUDE into Makefile.lib to allow
us to expand -I directives of KBUILD_CPPFLAGS so
we can avoid the above code.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

driver core: basic infrastructure for per-module dynamic debug messages

Base infrastructure to enable per-module debug messages.

I've introduced CONFIG_DYNAMIC_PRINTK_DEBUG, which when enabled centralizes
control of debugging statements on a per-module basis in one /proc file,
currently, <debugfs>/dynamic_printk/modules. When, CONFIG_DYNAMIC_PRINTK_DEBUG,
is not set, debugging statements can still be enabled as before, often by
defining 'DEBUG' for the proper compilation unit. Thus, this patch set has no
affect when CONFIG_DYNAMIC_PRINTK_DEBUG is not set.

The infrastructure currently ties into all pr_debug() and dev_dbg() calls. That
is, if CONFIG_DYNAMIC_PRINTK_DEBUG is set, all pr_debug() and dev_dbg() calls
can be dynamically enabled/disabled on a per-module basis.

Future plans include extending this functionality to subsystems, that define
their own debug levels and flags.

Usage:

Dynamic debugging is controlled by the debugfs file,
<debugfs>/dynamic_printk/modules. This file contains a list of the modules that
can be enabled. The format of the file is as follows:

<module_name> <enabled=0/1>
.
.
.

<module_name> : Name of the module in which the debug call resides
<enabled=0/1> : whether the messages are enabled or not

For example:

snd_hda_intel enabled=0
fixup enabled=1
driver enabled=0

Enable a module:

$echo "set enabled=1 <module_name>" > dynamic_printk/modules

Disable a module:

$echo "set enabled=0 <module_name>" > dynamic_printk/modules

Enable all modules:

$echo "set enabled=1 all" > dynamic_printk/modules

Disable all modules:

$echo "set enabled=0 all" > dynamic_printk/modules

Finally, passing "dynamic_printk" at the command line enables
debugging for all modules. This mode can be turned off via the above
disable command.

[gkh: minor cleanups and tweaks to make the build work quietly]

Signed-off-by: Jason Baron <jbaron@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

kbuild: create new CFLAGS_REMOVE_(basename).o option

We currently have a way to add special CFLAGS to code, but we do not have a
way to remove them if needed.

With the case of ftrace, some files should simply not be profiled. Adding
the -pg flag to these files is simply a waste, and adding "notrace" to each
and every function is ugly.

Currently we put in "Makefile turd" [1] to stop the compiler from adding -pg
to certain files. This was clumsy and awkward.

This patch now adds the revese of CFLAGS_(basename).o with
CFLAGS_REMOVE_(basename).o. This allows developers to prevent certain
CFLAGS from being used to compile files. For example, we can now do

CFLAGS_REMOVE_string.o = -pg

to remove the -pg option from the string.o file in the lib directory.

Note: a space delimited list of options may be added to the REMOVE macro.

[1] - what David Miller called the workaronud to remove -pg

Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

kbuild: implement modules.order

When multiple built-in modules (especially drivers) provide the same
capability, they're prioritized by link order specified by the order
listed in Makefile. This implicit ordering is lost for loadable
modules.

When driver modules are loaded by udev, what comes first in
modules.alias file is selected. However, the order in this file is
indeterministic (depends on filesystem listing order of installed
modules). This causes confusion.

The solution is two-parted. This patch updates kbuild such that it
generates and installs modules.order which contains the name of
modules ordered according to Makefile. The second part is update to
depmod such that it generates output files according to this file.

Note that both obj-y and obj-m subdirs can contain modules and
ordering information between those two are lost from beginning.
Currently obj-y subdirs are put before obj-m subdirs.

Sam Ravnborg cleaned up Makefile modifications and suggested using awk
to remove duplicate lines from modules.order instead of using separate
C program.

Signed-off-by: Tejun Heo <htejun@gmail.com>
Acked-by: Greg Kroah-Hartman <gregkh@suse.de>
Cc: Bill Nottingham <notting@redhat.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Kay Sievers <kay.sievers@vrfy.org>
Cc: Jon Masters <jonathan@jonmasters.org>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: introduce ccflags-y, asflags-y and ldflags-y

Introduce ccflags-y, asflags-y and ldflags-y so we soon can
deprecate use of EXTRA_CFLAGS, EXTRA_AFLAGS and EXTRA_LDFLAGS.
This patch does not touch any in-tree users - thats next round.
Lets get this committed first and then fix the users of the
soon to be deprecated variants next.

The rationale behind this change is to introduce support for
makefile fragments like:

ccflags-$(CONFIG_WHATEVER_DEBUG) := -DDEBUG

As a replacement for the uglier:
ifeq ($(CONFIG_WHATEVER_DEBUG),y)
EXTRA_CFLAGS := -DDEBUG
endif

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: enable 'make CPPFLAGS=...' to add additional options to CPP

The variable CPPFLAGS is a wellknown variable and the usage by
kbuild may result in unexpected behaviour.

This patch replace use of CPPFLAGS with KBUILD_CPPFLAGS all over the
tree and enabling one to use:
make CPPFLAGS=...
to specify additional CPP commandline options.

Patch was tested on following architectures:
alpha, arm, i386, x86_64, mips, sparc, sparc64, ia64, m68k, s390

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: enable 'make AFLAGS=...' to add additional options to AS

The variable AFLAGS is a wellknown variable and the usage by
kbuild may result in unexpected behaviour.
On top of that several people over time has asked for a way to
pass in additional flags to gcc.

This patch replace use of AFLAGS with KBUILD_AFLAGS all over
the tree.

Patch was tested on following architectures:
alpha, arm, i386, x86_64, mips, sparc, sparc64, ia64, m68k, s390

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: enable 'make CFLAGS=...' to add additional options to CC

The variable CFLAGS is a wellknown variable and the usage by
kbuild may result in unexpected behaviour.
On top of that several people over time has asked for a way to
pass in additional flags to gcc.

This patch replace use of CFLAGS with KBUILD_CFLAGS all over the
tree and enabling one to use:
make CFLAGS=...
to specify additional gcc commandline options.

One usecase is when trying to find gcc bugs but other
use cases has been requested too.

Patch was tested on following architectures:
alpha, arm, i386, x86_64, mips, sparc, sparc64, ia64, m68k

Test was simple to do a defconfig build, apply the patch and check
that nothing got rebuild.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: kill backward compatibility checks

These checks has been present for several kernel releases (> 5).
So lets just get rid of them.
With this we no longer check for use of:
EXTRA_TARGETS, O_TARGET, L_TARGET, list-multi, export-objs

There were three remaining in-tree users of O_TARGET in some
unmaintained sh64 code - mail sent to the maintainer + list.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: fix directory traversal bug

Previously kbuild choked over the following:
obj-y += ../../../arch/i386/kernel/bootflag.o

This has resulted in some rather ugly workarounds in
current x86_64 tree.
This patch fixes kbuild to allow the above and enable
potential cleanups in x86_64 and maybe in other places.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: fix ia64 breakage after introducing make -rR

kbuild used $¤(*F to get filename of target without extension.
This was used in several places all over kbuild, but introducing
make -rR broke his for all cases where we specified full path to
target/prerequsite. It is assumed that make -rR disables old style
suffix-rules which is why is suddenly failed.

ia64 was impacted by this change because several div* routines in
arch/ia64/lib are build using explicit paths and then kbuild failed.

Thanks to David Mosberger-Tang <David.Mosberger@acm.org> for an explanation
what was the root-cause and for testing on ia64.

This patch also fixes two uses of $(*F) in arch/um

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

Revert "kbuild: fix make -rR breakage"

This reverts commit e5c44fd88c146755da6941d047de4d97651404a9.

Thanks to Daniel Ritz and Michal Piotrowski for noticing the problem.

Daniel says:

"[The] reason is a recent change that made modules always shows as
module.mod. it breaks modprobe and probably many scripts..besides
lsmod looking horrible

stuff like this in modprobe.conf:
install pcmcia_core /sbin/modprobe --ignore-install pcmcia_core; /sbin/modprobe pcmcia
makes modprobe fork/exec endlessly calling itself...until oom
interrupts it"

Signed-off-by: Linus Torvalds <torvalds@osdl.org>

kbuild: fix make -rR breakage

make failed to supply the filename when using make -rR and using $(*F)
to get target filename without extension.
This bug was not reproduceable in small scale but using:
$(basename $(notdir $@)) fixes it with same functionality.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

kbuild: properly pass options to hostcc when doing make O=..

This fix a longstanding bug where proper options was not
passed to hostcc in case of a make O=.. build.
This bug showed up in (not yet merged) klibc, and is not known
to have any counterpart in-kernel.
Fixed by moving the flags macro to Kbuild.include so it can be used
by both Makefile.lib and Makefile.host.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>