patman: Move library functions into a library directory

The patman directory has a number of modules which are used by other tools
in U-Boot. This makes it hard to package the tools using pypi since the
common files must be copied along with the tool that uses them.

To address this, move these files into a new u_boot_pylib library. This
can be packaged separately and listed as a dependency of each tool.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use correct argument name in docstrings

Use correct argument name in docstrings.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Simon Glass <sjg@chromium.org>

binman: Provide general support for updating ELF symbols

The current support for updating variables in a binary is hard-coded to
work with U-Boot:

- It assumes the image starts at __image_copy_start
- It uses the existing U-Boot-specific entry types

It is useful for other projects to use these feature.

Add properties to enable writing symbols for any blob, a way of specifying
the base symbol and a way of providing the ELF filename to allow symbol
lookup to take place.

With this it is possible to update a Zephyr image, such as zephyr.bin
after it has been built.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support optional external blobs

Some blobs are actually not necessary for the board to work correctly. Add
a property to allow this to be indicated. Missing optional blobs do not
cause a build failure.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Handle writing ELF symbols in the Entry class

This feature is used by several etypes and we plan to add more that use
it. Make symbol writing a feature of the base class to reduce the code
duplication.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow mkimage to use a non-zero fake-blob size

Unfortunately mkimage gets upset with zero-sized files. Update the
ObtainContents() method to support specifying the size, if a fake blob is
created.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>

binman: Change how faked blobs are created

At present fake blobs are created but internally an empty blob is used.
Change it to use the contents of the faked file. Also return whether the
blob was faked, in case the caller needs to know that.

Add a TODO to put fake blobs in their own directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>

patman: Convert camel case in tools.py

Convert this file to snake case and update all files which use it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Write fake blobs to the output directory

At present binman writes fake blobs to the current directory. This is not
very helpful, since the files serve no useful purpose once binman has
finished. They clutter up the source directory and affect future runs,
since the files in the current directory are often used in preference to
those in the board directory.

To avoid these problems, write them to the output directory instead.

Move the file-creation code to the Entry base class, so it can be used by
any entry type that needs it. This is required since some entry types,
such as Entry_blob_ext_list, are not subclasses of Entry_blob.

Signed-off-by: Simon Glass <sjg@chromium.org>

Merge branch 'next'

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: add support for creating dummy files for external blobs

While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>

binman: Support lists of external blobs

Sometimes it is useful to have a list of related external blobs in a
single entry. An example is the DDR binaries used by meson. There are
9 files in total. Add support for this, so we don't have to have a
separate entry for each.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Add basic support for debugging performance

One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.

This is for debugging purposes only.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop repetitive heading for each entry

Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up underscores in entry documentation

Several entries currently use an underscore in the entry-type name, but in
fact a hyphen is used. Update the docs to fix this as it might be
confusing.

Also simplify the 'filename' comment and fix the 'operation' typo.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Show an error when a file is missing

The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

binman: Move 'external' support into base class

At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"

This reverts commit 5d3a21df6694ebd66d5c34c9d62a26edc7456fc7, reversing
changes made to 56d37f1c564107e27d873181d838571b7d7860e7.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

patman: Move to absolute imports

At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move to absolute imports

At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow use of help and entry-docs without libfdt

At present if libfdt is not available binman can't do anything much.
Improve the situation a little.

Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.

Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)

binman: Update Entry.ReadEntry() to work through classes

At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow reading an entry from an image

It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow device-tree entries to be compressed

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move compression into the Entry base class

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use tools compression function for blob handling

Avoid duplicate code here by using the new compression function in the
tools module.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use binary mode when compressing data

The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop an unnecessary comma in blob handling

This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support compressed entries

Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename ReadContents() to ReadBlobContents()

This function name is too generic for its purpose and is therefore
confusing. It actually only applies to blobs, so rename it to indicate
this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Expand documentation for entries

At present only the more complex entries are documented. It is useful to
have documentation for all entries in one place.

As a first step, add and expand the documentation to cover all entries.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up setting of entry contents

At present the contents of an entry are set in subclasses simply by
assigning to the data and content_size properties. Add some methods to do
this, so that we have more control. In particular, add a method to set the
contents without changing its size, so we can validate that case.

Add a test case for trying to change the size when this is not allowed.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename Entry property to 'section'

Entries are now passed a Section object rather than an Image. Rename this
property to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>

SPDX: Convert all of our single license tags to Linux Kernel style

When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Add basic entry types for U-Boot

Add entries to support some standard U-Boot binaries, such as u-boot.bin,
u-boot.dtb, etc. Also add some tests for these.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

binman: Provide general support for updating ELF symbols

The current support for updating variables in a binary is hard-coded to
work with U-Boot:

- It assumes the image starts at __image_copy_start
- It uses the existing U-Boot-specific entry types

It is useful for other projects to use these feature.

Add properties to enable writing symbols for any blob, a way of specifying
the base symbol and a way of providing the ELF filename to allow symbol
lookup to take place.

With this it is possible to update a Zephyr image, such as zephyr.bin
after it has been built.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support optional external blobs

Some blobs are actually not necessary for the board to work correctly. Add
a property to allow this to be indicated. Missing optional blobs do not
cause a build failure.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Handle writing ELF symbols in the Entry class

This feature is used by several etypes and we plan to add more that use
it. Make symbol writing a feature of the base class to reduce the code
duplication.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow mkimage to use a non-zero fake-blob size

Unfortunately mkimage gets upset with zero-sized files. Update the
ObtainContents() method to support specifying the size, if a fake blob is
created.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>

binman: Change how faked blobs are created

At present fake blobs are created but internally an empty blob is used.
Change it to use the contents of the faked file. Also return whether the
blob was faked, in case the caller needs to know that.

Add a TODO to put fake blobs in their own directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>

patman: Convert camel case in tools.py

Convert this file to snake case and update all files which use it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Write fake blobs to the output directory

At present binman writes fake blobs to the current directory. This is not
very helpful, since the files serve no useful purpose once binman has
finished. They clutter up the source directory and affect future runs,
since the files in the current directory are often used in preference to
those in the board directory.

To avoid these problems, write them to the output directory instead.

Move the file-creation code to the Entry base class, so it can be used by
any entry type that needs it. This is required since some entry types,
such as Entry_blob_ext_list, are not subclasses of Entry_blob.

Signed-off-by: Simon Glass <sjg@chromium.org>

Merge branch 'next'

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: add support for creating dummy files for external blobs

While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>

binman: Support lists of external blobs

Sometimes it is useful to have a list of related external blobs in a
single entry. An example is the DDR binaries used by meson. There are
9 files in total. Add support for this, so we don't have to have a
separate entry for each.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Add basic support for debugging performance

One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.

This is for debugging purposes only.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop repetitive heading for each entry

Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up underscores in entry documentation

Several entries currently use an underscore in the entry-type name, but in
fact a hyphen is used. Update the docs to fix this as it might be
confusing.

Also simplify the 'filename' comment and fix the 'operation' typo.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Show an error when a file is missing

The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

binman: Move 'external' support into base class

At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"

This reverts commit 5d3a21df6694ebd66d5c34c9d62a26edc7456fc7, reversing
changes made to 56d37f1c564107e27d873181d838571b7d7860e7.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

patman: Move to absolute imports

At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move to absolute imports

At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow use of help and entry-docs without libfdt

At present if libfdt is not available binman can't do anything much.
Improve the situation a little.

Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.

Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)

binman: Update Entry.ReadEntry() to work through classes

At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow reading an entry from an image

It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow device-tree entries to be compressed

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move compression into the Entry base class

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use tools compression function for blob handling

Avoid duplicate code here by using the new compression function in the
tools module.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use binary mode when compressing data

The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop an unnecessary comma in blob handling

This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support compressed entries

Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename ReadContents() to ReadBlobContents()

This function name is too generic for its purpose and is therefore
confusing. It actually only applies to blobs, so rename it to indicate
this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Expand documentation for entries

At present only the more complex entries are documented. It is useful to
have documentation for all entries in one place.

As a first step, add and expand the documentation to cover all entries.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up setting of entry contents

At present the contents of an entry are set in subclasses simply by
assigning to the data and content_size properties. Add some methods to do
this, so that we have more control. In particular, add a method to set the
contents without changing its size, so we can validate that case.

Add a test case for trying to change the size when this is not allowed.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename Entry property to 'section'

Entries are now passed a Section object rather than an Image. Rename this
property to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>

SPDX: Convert all of our single license tags to Linux Kernel style

When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Add basic entry types for U-Boot

Add entries to support some standard U-Boot binaries, such as u-boot.bin,
u-boot.dtb, etc. Also add some tests for these.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

binman: Handle writing ELF symbols in the Entry class

This feature is used by several etypes and we plan to add more that use
it. Make symbol writing a feature of the base class to reduce the code
duplication.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow mkimage to use a non-zero fake-blob size

Unfortunately mkimage gets upset with zero-sized files. Update the
ObtainContents() method to support specifying the size, if a fake blob is
created.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>

binman: Change how faked blobs are created

At present fake blobs are created but internally an empty blob is used.
Change it to use the contents of the faked file. Also return whether the
blob was faked, in case the caller needs to know that.

Add a TODO to put fake blobs in their own directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>

patman: Convert camel case in tools.py

Convert this file to snake case and update all files which use it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Write fake blobs to the output directory

At present binman writes fake blobs to the current directory. This is not
very helpful, since the files serve no useful purpose once binman has
finished. They clutter up the source directory and affect future runs,
since the files in the current directory are often used in preference to
those in the board directory.

To avoid these problems, write them to the output directory instead.

Move the file-creation code to the Entry base class, so it can be used by
any entry type that needs it. This is required since some entry types,
such as Entry_blob_ext_list, are not subclasses of Entry_blob.

Signed-off-by: Simon Glass <sjg@chromium.org>

Merge branch 'next'

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: add support for creating dummy files for external blobs

While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>

binman: Support lists of external blobs

Sometimes it is useful to have a list of related external blobs in a
single entry. An example is the DDR binaries used by meson. There are
9 files in total. Add support for this, so we don't have to have a
separate entry for each.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Add basic support for debugging performance

One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.

This is for debugging purposes only.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop repetitive heading for each entry

Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up underscores in entry documentation

Several entries currently use an underscore in the entry-type name, but in
fact a hyphen is used. Update the docs to fix this as it might be
confusing.

Also simplify the 'filename' comment and fix the 'operation' typo.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Show an error when a file is missing

The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

binman: Move 'external' support into base class

At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"

This reverts commit 5d3a21df6694ebd66d5c34c9d62a26edc7456fc7, reversing
changes made to 56d37f1c564107e27d873181d838571b7d7860e7.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

patman: Move to absolute imports

At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move to absolute imports

At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow use of help and entry-docs without libfdt

At present if libfdt is not available binman can't do anything much.
Improve the situation a little.

Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.

Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)

binman: Update Entry.ReadEntry() to work through classes

At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow reading an entry from an image

It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow device-tree entries to be compressed

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move compression into the Entry base class

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use tools compression function for blob handling

Avoid duplicate code here by using the new compression function in the
tools module.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use binary mode when compressing data

The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop an unnecessary comma in blob handling

This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support compressed entries

Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename ReadContents() to ReadBlobContents()

This function name is too generic for its purpose and is therefore
confusing. It actually only applies to blobs, so rename it to indicate
this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Expand documentation for entries

At present only the more complex entries are documented. It is useful to
have documentation for all entries in one place.

As a first step, add and expand the documentation to cover all entries.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up setting of entry contents

At present the contents of an entry are set in subclasses simply by
assigning to the data and content_size properties. Add some methods to do
this, so that we have more control. In particular, add a method to set the
contents without changing its size, so we can validate that case.

Add a test case for trying to change the size when this is not allowed.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename Entry property to 'section'

Entries are now passed a Section object rather than an Image. Rename this
property to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>

SPDX: Convert all of our single license tags to Linux Kernel style

When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Add basic entry types for U-Boot

Add entries to support some standard U-Boot binaries, such as u-boot.bin,
u-boot.dtb, etc. Also add some tests for these.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

binman: Allow mkimage to use a non-zero fake-blob size

Unfortunately mkimage gets upset with zero-sized files. Update the
ObtainContents() method to support specifying the size, if a fake blob is
created.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>

binman: Change how faked blobs are created

At present fake blobs are created but internally an empty blob is used.
Change it to use the contents of the faked file. Also return whether the
blob was faked, in case the caller needs to know that.

Add a TODO to put fake blobs in their own directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>

patman: Convert camel case in tools.py

Convert this file to snake case and update all files which use it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Write fake blobs to the output directory

At present binman writes fake blobs to the current directory. This is not
very helpful, since the files serve no useful purpose once binman has
finished. They clutter up the source directory and affect future runs,
since the files in the current directory are often used in preference to
those in the board directory.

To avoid these problems, write them to the output directory instead.

Move the file-creation code to the Entry base class, so it can be used by
any entry type that needs it. This is required since some entry types,
such as Entry_blob_ext_list, are not subclasses of Entry_blob.

Signed-off-by: Simon Glass <sjg@chromium.org>

Merge branch 'next'

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: add support for creating dummy files for external blobs

While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>

binman: Support lists of external blobs

Sometimes it is useful to have a list of related external blobs in a
single entry. An example is the DDR binaries used by meson. There are
9 files in total. Add support for this, so we don't have to have a
separate entry for each.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Add basic support for debugging performance

One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.

This is for debugging purposes only.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop repetitive heading for each entry

Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up underscores in entry documentation

Several entries currently use an underscore in the entry-type name, but in
fact a hyphen is used. Update the docs to fix this as it might be
confusing.

Also simplify the 'filename' comment and fix the 'operation' typo.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Show an error when a file is missing

The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

binman: Move 'external' support into base class

At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"

This reverts commit 5d3a21df6694ebd66d5c34c9d62a26edc7456fc7, reversing
changes made to 56d37f1c564107e27d873181d838571b7d7860e7.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

patman: Move to absolute imports

At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move to absolute imports

At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow use of help and entry-docs without libfdt

At present if libfdt is not available binman can't do anything much.
Improve the situation a little.

Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.

Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)

binman: Update Entry.ReadEntry() to work through classes

At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow reading an entry from an image

It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow device-tree entries to be compressed

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move compression into the Entry base class

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use tools compression function for blob handling

Avoid duplicate code here by using the new compression function in the
tools module.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use binary mode when compressing data

The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop an unnecessary comma in blob handling

This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support compressed entries

Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename ReadContents() to ReadBlobContents()

This function name is too generic for its purpose and is therefore
confusing. It actually only applies to blobs, so rename it to indicate
this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Expand documentation for entries

At present only the more complex entries are documented. It is useful to
have documentation for all entries in one place.

As a first step, add and expand the documentation to cover all entries.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up setting of entry contents

At present the contents of an entry are set in subclasses simply by
assigning to the data and content_size properties. Add some methods to do
this, so that we have more control. In particular, add a method to set the
contents without changing its size, so we can validate that case.

Add a test case for trying to change the size when this is not allowed.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename Entry property to 'section'

Entries are now passed a Section object rather than an Image. Rename this
property to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>

SPDX: Convert all of our single license tags to Linux Kernel style

When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Add basic entry types for U-Boot

Add entries to support some standard U-Boot binaries, such as u-boot.bin,
u-boot.dtb, etc. Also add some tests for these.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

patman: Convert camel case in tools.py

Convert this file to snake case and update all files which use it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Write fake blobs to the output directory

At present binman writes fake blobs to the current directory. This is not
very helpful, since the files serve no useful purpose once binman has
finished. They clutter up the source directory and affect future runs,
since the files in the current directory are often used in preference to
those in the board directory.

To avoid these problems, write them to the output directory instead.

Move the file-creation code to the Entry base class, so it can be used by
any entry type that needs it. This is required since some entry types,
such as Entry_blob_ext_list, are not subclasses of Entry_blob.

Signed-off-by: Simon Glass <sjg@chromium.org>

Merge branch 'next'

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: add support for creating dummy files for external blobs

While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>

binman: Support lists of external blobs

Sometimes it is useful to have a list of related external blobs in a
single entry. An example is the DDR binaries used by meson. There are
9 files in total. Add support for this, so we don't have to have a
separate entry for each.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Add basic support for debugging performance

One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.

This is for debugging purposes only.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop repetitive heading for each entry

Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up underscores in entry documentation

Several entries currently use an underscore in the entry-type name, but in
fact a hyphen is used. Update the docs to fix this as it might be
confusing.

Also simplify the 'filename' comment and fix the 'operation' typo.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Show an error when a file is missing

The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

binman: Move 'external' support into base class

At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"

This reverts commit 5d3a21df6694ebd66d5c34c9d62a26edc7456fc7, reversing
changes made to 56d37f1c564107e27d873181d838571b7d7860e7.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

patman: Move to absolute imports

At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move to absolute imports

At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow use of help and entry-docs without libfdt

At present if libfdt is not available binman can't do anything much.
Improve the situation a little.

Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.

Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)

binman: Update Entry.ReadEntry() to work through classes

At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow reading an entry from an image

It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow device-tree entries to be compressed

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move compression into the Entry base class

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use tools compression function for blob handling

Avoid duplicate code here by using the new compression function in the
tools module.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use binary mode when compressing data

The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop an unnecessary comma in blob handling

This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support compressed entries

Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename ReadContents() to ReadBlobContents()

This function name is too generic for its purpose and is therefore
confusing. It actually only applies to blobs, so rename it to indicate
this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Expand documentation for entries

At present only the more complex entries are documented. It is useful to
have documentation for all entries in one place.

As a first step, add and expand the documentation to cover all entries.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up setting of entry contents

At present the contents of an entry are set in subclasses simply by
assigning to the data and content_size properties. Add some methods to do
this, so that we have more control. In particular, add a method to set the
contents without changing its size, so we can validate that case.

Add a test case for trying to change the size when this is not allowed.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename Entry property to 'section'

Entries are now passed a Section object rather than an Image. Rename this
property to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>

SPDX: Convert all of our single license tags to Linux Kernel style

When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Add basic entry types for U-Boot

Add entries to support some standard U-Boot binaries, such as u-boot.bin,
u-boot.dtb, etc. Also add some tests for these.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

binman: Write fake blobs to the output directory

At present binman writes fake blobs to the current directory. This is not
very helpful, since the files serve no useful purpose once binman has
finished. They clutter up the source directory and affect future runs,
since the files in the current directory are often used in preference to
those in the board directory.

To avoid these problems, write them to the output directory instead.

Move the file-creation code to the Entry base class, so it can be used by
any entry type that needs it. This is required since some entry types,
such as Entry_blob_ext_list, are not subclasses of Entry_blob.

Signed-off-by: Simon Glass <sjg@chromium.org>

Merge branch 'next'

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: add support for creating dummy files for external blobs

While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>

binman: Support lists of external blobs

Sometimes it is useful to have a list of related external blobs in a
single entry. An example is the DDR binaries used by meson. There are
9 files in total. Add support for this, so we don't have to have a
separate entry for each.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Add basic support for debugging performance

One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.

This is for debugging purposes only.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop repetitive heading for each entry

Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up underscores in entry documentation

Several entries currently use an underscore in the entry-type name, but in
fact a hyphen is used. Update the docs to fix this as it might be
confusing.

Also simplify the 'filename' comment and fix the 'operation' typo.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Show an error when a file is missing

The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

binman: Move 'external' support into base class

At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"

This reverts commit 5d3a21df6694ebd66d5c34c9d62a26edc7456fc7, reversing
changes made to 56d37f1c564107e27d873181d838571b7d7860e7.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

patman: Move to absolute imports

At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move to absolute imports

At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow use of help and entry-docs without libfdt

At present if libfdt is not available binman can't do anything much.
Improve the situation a little.

Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.

Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)

binman: Update Entry.ReadEntry() to work through classes

At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow reading an entry from an image

It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow device-tree entries to be compressed

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move compression into the Entry base class

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use tools compression function for blob handling

Avoid duplicate code here by using the new compression function in the
tools module.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use binary mode when compressing data

The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop an unnecessary comma in blob handling

This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support compressed entries

Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename ReadContents() to ReadBlobContents()

This function name is too generic for its purpose and is therefore
confusing. It actually only applies to blobs, so rename it to indicate
this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Expand documentation for entries

At present only the more complex entries are documented. It is useful to
have documentation for all entries in one place.

As a first step, add and expand the documentation to cover all entries.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up setting of entry contents

At present the contents of an entry are set in subclasses simply by
assigning to the data and content_size properties. Add some methods to do
this, so that we have more control. In particular, add a method to set the
contents without changing its size, so we can validate that case.

Add a test case for trying to change the size when this is not allowed.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename Entry property to 'section'

Entries are now passed a Section object rather than an Image. Rename this
property to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>

SPDX: Convert all of our single license tags to Linux Kernel style

When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Add basic entry types for U-Boot

Add entries to support some standard U-Boot binaries, such as u-boot.bin,
u-boot.dtb, etc. Also add some tests for these.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

Merge branch 'next'

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: add support for creating dummy files for external blobs

While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>

binman: Support lists of external blobs

Sometimes it is useful to have a list of related external blobs in a
single entry. An example is the DDR binaries used by meson. There are
9 files in total. Add support for this, so we don't have to have a
separate entry for each.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Add basic support for debugging performance

One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.

This is for debugging purposes only.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop repetitive heading for each entry

Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up underscores in entry documentation

Several entries currently use an underscore in the entry-type name, but in
fact a hyphen is used. Update the docs to fix this as it might be
confusing.

Also simplify the 'filename' comment and fix the 'operation' typo.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Show an error when a file is missing

The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

binman: Move 'external' support into base class

At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"

This reverts commit 5d3a21df6694ebd66d5c34c9d62a26edc7456fc7, reversing
changes made to 56d37f1c564107e27d873181d838571b7d7860e7.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

patman: Move to absolute imports

At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move to absolute imports

At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow use of help and entry-docs without libfdt

At present if libfdt is not available binman can't do anything much.
Improve the situation a little.

Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.

Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)

binman: Update Entry.ReadEntry() to work through classes

At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow reading an entry from an image

It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow device-tree entries to be compressed

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move compression into the Entry base class

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use tools compression function for blob handling

Avoid duplicate code here by using the new compression function in the
tools module.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use binary mode when compressing data

The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop an unnecessary comma in blob handling

This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support compressed entries

Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename ReadContents() to ReadBlobContents()

This function name is too generic for its purpose and is therefore
confusing. It actually only applies to blobs, so rename it to indicate
this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Expand documentation for entries

At present only the more complex entries are documented. It is useful to
have documentation for all entries in one place.

As a first step, add and expand the documentation to cover all entries.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up setting of entry contents

At present the contents of an entry are set in subclasses simply by
assigning to the data and content_size properties. Add some methods to do
this, so that we have more control. In particular, add a method to set the
contents without changing its size, so we can validate that case.

Add a test case for trying to change the size when this is not allowed.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename Entry property to 'section'

Entries are now passed a Section object rather than an Image. Rename this
property to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>

SPDX: Convert all of our single license tags to Linux Kernel style

When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Add basic entry types for U-Boot

Add entries to support some standard U-Boot binaries, such as u-boot.bin,
u-boot.dtb, etc. Also add some tests for these.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

binman: add support for creating dummy files for external blobs

While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>

binman: Add basic support for debugging performance

One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.

This is for debugging purposes only.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop repetitive heading for each entry

Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up underscores in entry documentation

Several entries currently use an underscore in the entry-type name, but in
fact a hyphen is used. Update the docs to fix this as it might be
confusing.

Also simplify the 'filename' comment and fix the 'operation' typo.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Show an error when a file is missing

The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

binman: Move 'external' support into base class

At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"

This reverts commit 5d3a21df6694ebd66d5c34c9d62a26edc7456fc7, reversing
changes made to 56d37f1c564107e27d873181d838571b7d7860e7.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

patman: Move to absolute imports

At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move to absolute imports

At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow use of help and entry-docs without libfdt

At present if libfdt is not available binman can't do anything much.
Improve the situation a little.

Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.

Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)

binman: Update Entry.ReadEntry() to work through classes

At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow reading an entry from an image

It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow device-tree entries to be compressed

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move compression into the Entry base class

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use tools compression function for blob handling

Avoid duplicate code here by using the new compression function in the
tools module.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use binary mode when compressing data

The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop an unnecessary comma in blob handling

This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support compressed entries

Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename ReadContents() to ReadBlobContents()

This function name is too generic for its purpose and is therefore
confusing. It actually only applies to blobs, so rename it to indicate
this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Expand documentation for entries

At present only the more complex entries are documented. It is useful to
have documentation for all entries in one place.

As a first step, add and expand the documentation to cover all entries.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up setting of entry contents

At present the contents of an entry are set in subclasses simply by
assigning to the data and content_size properties. Add some methods to do
this, so that we have more control. In particular, add a method to set the
contents without changing its size, so we can validate that case.

Add a test case for trying to change the size when this is not allowed.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename Entry property to 'section'

Entries are now passed a Section object rather than an Image. Rename this
property to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>

SPDX: Convert all of our single license tags to Linux Kernel style

When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Add basic entry types for U-Boot

Add entries to support some standard U-Boot binaries, such as u-boot.bin,
u-boot.dtb, etc. Also add some tests for these.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

binman: Add basic support for debugging performance

One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.

This is for debugging purposes only.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop repetitive heading for each entry

Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up underscores in entry documentation

Several entries currently use an underscore in the entry-type name, but in
fact a hyphen is used. Update the docs to fix this as it might be
confusing.

Also simplify the 'filename' comment and fix the 'operation' typo.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Show an error when a file is missing

The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

binman: Move 'external' support into base class

At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"

This reverts commit 5d3a21df6694ebd66d5c34c9d62a26edc7456fc7, reversing
changes made to 56d37f1c564107e27d873181d838571b7d7860e7.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

patman: Move to absolute imports

At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move to absolute imports

At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow use of help and entry-docs without libfdt

At present if libfdt is not available binman can't do anything much.
Improve the situation a little.

Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.

Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)

binman: Update Entry.ReadEntry() to work through classes

At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow reading an entry from an image

It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow device-tree entries to be compressed

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move compression into the Entry base class

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use tools compression function for blob handling

Avoid duplicate code here by using the new compression function in the
tools module.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use binary mode when compressing data

The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop an unnecessary comma in blob handling

This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support compressed entries

Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename ReadContents() to ReadBlobContents()

This function name is too generic for its purpose and is therefore
confusing. It actually only applies to blobs, so rename it to indicate
this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Expand documentation for entries

At present only the more complex entries are documented. It is useful to
have documentation for all entries in one place.

As a first step, add and expand the documentation to cover all entries.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up setting of entry contents

At present the contents of an entry are set in subclasses simply by
assigning to the data and content_size properties. Add some methods to do
this, so that we have more control. In particular, add a method to set the
contents without changing its size, so we can validate that case.

Add a test case for trying to change the size when this is not allowed.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename Entry property to 'section'

Entries are now passed a Section object rather than an Image. Rename this
property to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>

SPDX: Convert all of our single license tags to Linux Kernel style

When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Add basic entry types for U-Boot

Add entries to support some standard U-Boot binaries, such as u-boot.bin,
u-boot.dtb, etc. Also add some tests for these.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

binman: Drop repetitive heading for each entry

Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up underscores in entry documentation

Several entries currently use an underscore in the entry-type name, but in
fact a hyphen is used. Update the docs to fix this as it might be
confusing.

Also simplify the 'filename' comment and fix the 'operation' typo.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Show an error when a file is missing

The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

binman: Move 'external' support into base class

At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"

This reverts commit 5d3a21df6694ebd66d5c34c9d62a26edc7456fc7, reversing
changes made to 56d37f1c564107e27d873181d838571b7d7860e7.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Use super() instead of specifying parent type

It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

patman: Move to absolute imports

At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move to absolute imports

At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow use of help and entry-docs without libfdt

At present if libfdt is not available binman can't do anything much.
Improve the situation a little.

Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.

Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)

binman: Update Entry.ReadEntry() to work through classes

At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow reading an entry from an image

It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Allow device-tree entries to be compressed

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move compression into the Entry base class

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use tools compression function for blob handling

Avoid duplicate code here by using the new compression function in the
tools module.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Use binary mode when compressing data

The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Drop an unnecessary comma in blob handling

This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Support compressed entries

Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename ReadContents() to ReadBlobContents()

This function name is too generic for its purpose and is therefore
confusing. It actually only applies to blobs, so rename it to indicate
this.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Expand documentation for entries

At present only the more complex entries are documented. It is useful to
have documentation for all entries in one place.

As a first step, add and expand the documentation to cover all entries.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Tidy up setting of entry contents

At present the contents of an entry are set in subclasses simply by
assigning to the data and content_size properties. Add some methods to do
this, so that we have more control. In particular, add a method to set the
contents without changing its size, so we can validate that case.

Add a test case for trying to change the size when this is not allowed.

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Rename Entry property to 'section'

Entries are now passed a Section object rather than an Image. Rename this
property to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>

SPDX: Convert all of our single license tags to Linux Kernel style

When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>

binman: Add basic entry types for U-Boot

Add entries to support some standard U-Boot binaries, such as u-boot.bin,
u-boot.dtb, etc. Also add some tests for these.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Bin Meng <bmeng.cn@gmail.com>

binman: Allow vblock to include devicetree blobs

At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.

Fix this by adjusting how the vblock is created.

Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().

Signed-off-by: Simon Glass <sjg@chromium.org>

binman: Move CompressData() into Entry base class

At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>