mtd: ubi: introduce pre-removal notification for UBI volumes

Introduce a new notification type UBI_VOLUME_SHUTDOWN to inform users
that a volume is just about to be removed.
This is needed because users (such as the NVMEM subsystem) expect that
at the time their removal function is called, the parenting device is
still available (for removal of sysfs nodes, for example, in case of
NVMEM which otherwise WARNs on volume removal).

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Richard Weinberger <richard@nod.at>

mtd: ubi: block: wire-up device parent

ubiblock devices were previously only identifyable by their name, but
not connected to their parent UBI volume device e.g. in sysfs.
Properly parent ubiblock device as descendant of a UBI volume device
to reflect device model hierachy.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Richard Weinberger <richard@nod.at>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156

Based on 1 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license as published by
the free software foundation either version 2 of the license or at
your option any later version this program is distributed in the
hope that it will be useful but without any warranty without even
the implied warranty of merchantability or fitness for a particular
purpose see the gnu general public license for more details you
should have received a copy of the gnu general public license along
with this program if not write to the free software foundation inc
59 temple place suite 330 boston ma 02111 1307 usa

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-or-later

has been chosen to replace the boilerplate/reference in 1334 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Allison Randal <allison@lohutok.net>
Reviewed-by: Richard Fontana <rfontana@redhat.com>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190527070033.113240726@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

UBI: Add initial support for scatter gather

Adds a new set of functions to deal with scatter gather.
ubi_eba_read_leb_sg() will read from a LEB into a scatter gather list.
The new data structure struct ubi_sgl will be used within UBI to
hold the scatter gather list itself and metadata to have a cursor
within the list.

Signed-off-by: Richard Weinberger <richard@nod.at>
Tested-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>

UBI: Implement UBI_METAONLY

UBI_METAONLY is a new open mode for UBI volumes, it indicates
that only meta data is being changed.
Meta data in terms of UBI volumes means data which is stored in the
UBI volume table but not on the volume itself.
While it does not interfere with UBI_READONLY and UBI_READWRITE
it is not allowed to use UBI_METAONLY together with UBI_EXCLUSIVE.

Cc: Ezequiel Garcia <ezequiel.garcia@free-electrons.com>
Cc: Andrew Murray <amurray@embedded-bits.co.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
Tested-by: Guido Martínez <guido@vanguardiasur.com.ar>
Reviewed-by: Guido Martínez <guido@vanguardiasur.com.ar>
Tested-by: Christoph Fritz <chf.fritz@googlemail.com>
Tested-by: Andrew Murray <amurray@embedded-bits.co.uk>

UBI: modify ubi_wl_flush function to clear work queue for a lnum

This patch modifies ubi_wl_flush to force the erasure of
particular volume id / logical eraseblock number pairs. Previous functionality
is preserved when passing UBI_ALL for both values. The locations where ubi_wl_flush
were called are appropriately changed: ubi_leb_erase only flushes for the
erased LEB, and ubi_create_volume forces only flushing for its volume id.
External code can call this new feature via the new function ubi_flush() added
to kapi.c, which simply passes through to ubi_wl_flush().

This was tested by disabling the call to do_work in ubi thread, which results
in the work queue remaining unless explicitly called to remove. UBIFS was
changed to call ubifs_leb_change 50 times for four different LEBs. Then the
new function was called to clear the queue: passing wrong volume ids / lnum,
correct ones, and finally UBI_ALL for both to ensure it was finally all
cleard. The work queue was dumped each time and the selective removal
of the particular LEB numbers was observed. Extra checks were enabled and
ubifs's integck was also run. Finally, the drive was repeatedly filled and
emptied to ensure that the queue was cleared normally.

Artem: amended the patch.

Signed-off-by: Joel Reardon <reardonj@inf.ethz.ch>
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

UBI: introduce UBI_ALL constant

Joel will use it in his 'ubi_flush()' extention to specify all eraseblocks.
Also amend the comment for UBI_UNKNOWN - it is used beyond attaching info
structure now.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

UBI: Kill data type hint

We do not need this feature and to our shame it even was not working
and there was a bug found very recently.
-- Artem Bityutskiy

Without the data type hint UBI2 (fastmap) will be easier to implement.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

UBI: clarify the volume notification types' doc

I realized the new descriptions of ADDED and REMOVED could also be
misleading: they can also be triggered after using a userland util
(ubi{mk,rm}vol).

Artem: amend the commentaries

Signed-off-by David Wagner <david.wagner@free-electrons.com>
Signed-off-by: Artem Bityutskiy <dedekind1@gmail.com>

UBI: fix minor stylistic issues

Fix checkpatch.pl errors and warnings:

* space before tab
* line over 80 characters
* include linux/ioctl.h instead of asm/ioctl.h

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI: provide LEB offset information

Provide the LEB offset information in the UBI device information data
structure. This piece of information is required by UBIFS to find out
what are the LEB offsets which are aligned to the max. write size.

If LEB offset not aligned to max. write size, then UBIFS has to take
this into account to write more optimally.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI: incorporate maximum write size

Incorporate MTD write buffer size into UBI device information
because UBIFS needs this field. UBI does not use it ATM, just
provides to upper layers in 'struct ubi_device_info'.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI: Add ubi_open_volume_path

Add an 'ubi_open_volume_path(path, mode)' function which works like
'open_bdev_exclusive(path, mode, ...)' where path is the special file
representing the UBI volume, typically /dev/ubi0_0.

This is needed to teach UBIFS being able to mount UBI character devices.

[Comments and the patch were amended a bit by Artem]

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI: add notification API

UBI volume notifications are intended to create the API to get clients
notified about volume creation/deletion, renaming and re-sizing. A
client can subscribe to these notifications using 'ubi_volume_register()'
and cancel the subscription using 'ubi_volume_unregister()'. When UBI
volumes change, a blocking notifier is called. Clients also can request
"added" events on all volumes that existed before client subscribed
to the notifications.

If we use notifications instead of calling functions like 'ubi_gluebi_xxx()',
we can make the MTD emulation layer to be more flexible: build it as a
separate module and load/unload it on demand.

[Artem: many cleanups, rework locking, add "updated" event, provide
device/volume info in notifiers]

Signed-off-by: Dmitry Pervushin <dpervushin@embeddedalley.com>
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI: amend commentaries

Hch asked not to use "unit" for sub-systems, let it be so.
Also some other commentaries modifications.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI: add ubi_sync() interface

To flush MTD device caches.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI: introduce atomic LEB change ioctl

We have to be able to change individual LEBs for utilities like
ubifsck, ubifstune. For example, ubifsck has to be able to fix
errors on the media, ubifstune has to be able to change the
the superblock, hence this ioctl.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI: add ubi_leb_map interface

The idea of this interface belongs to Adrian Hunter. The
interface is extremely useful when one has to have a guarantee
that an LEB will contain all 0xFFs even in case of an unclean
reboot. UBI does have an 'ubi_leb_erase()' call which may do
this, but it is stupid and ineffecient, because it flushes whole
queue. I should be re-worked to just be a pair of unmap,
map calls.

The user of the interfaci is UBIFS at the moment.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

UBI: Unsorted Block Images

UBI (Latin: "where?") manages multiple logical volumes on a single
flash device, specifically supporting NAND flash devices. UBI provides
a flexible partitioning concept which still allows for wear-levelling
across the whole flash device.

In a sense, UBI may be compared to the Logical Volume Manager
(LVM). Whereas LVM maps logical sector numbers to physical HDD sector
numbers, UBI maps logical eraseblocks to physical eraseblocks.

More information may be found at
http://www.linux-mtd.infradead.org/doc/ubi.html

Partitioning/Re-partitioning

An UBI volume occupies a certain number of erase blocks. This is
limited by a configured maximum volume size, which could also be
viewed as the partition size. Each individual UBI volume's size can
be changed independently of the other UBI volumes, provided that the
sum of all volume sizes doesn't exceed a certain limit.

UBI supports dynamic volumes and static volumes. Static volumes are
read-only and their contents are protected by CRC check sums.

Bad eraseblocks handling

UBI transparently handles bad eraseblocks. When a physical
eraseblock becomes bad, it is substituted by a good physical
eraseblock, and the user does not even notice this.

Scrubbing

On a NAND flash bit flips can occur on any write operation,
sometimes also on read. If bit flips persist on the device, at first
they can still be corrected by ECC, but once they accumulate,
correction will become impossible. Thus it is best to actively scrub
the affected eraseblock, by first copying it to a free eraseblock
and then erasing the original. The UBI layer performs this type of
scrubbing under the covers, transparently to the UBI volume users.

Erase Counts

UBI maintains an erase count header per eraseblock. This frees
higher-level layers (like file systems) from doing this and allows
for centralized erase count management instead. The erase counts are
used by the wear-levelling algorithm in the UBI layer. The algorithm
itself is exchangeable.

Booting from NAND

For booting directly from NAND flash the hardware must at least be
capable of fetching and executing a small portion of the NAND
flash. Some NAND flash controllers have this kind of support. They
usually limit the window to a few kilobytes in erase block 0. This
"initial program loader" (IPL) must then contain sufficient logic to
load and execute the next boot phase.

Due to bad eraseblocks, which may be randomly scattered over the
flash device, it is problematic to store the "secondary program
loader" (SPL) statically. Also, due to bit-flips it may become
corrupted over time. UBI allows to solve this problem gracefully by
storing the SPL in a small static UBI volume.

UBI volumes vs. static partitions

UBI volumes are still very similar to static MTD partitions:

* both consist of eraseblocks (logical eraseblocks in case of UBI
volumes, and physical eraseblocks in case of static partitions;
* both support three basic operations - read, write, erase.

But UBI volumes have the following advantages over traditional
static MTD partitions:

* there are no eraseblock wear-leveling constraints in case of UBI
volumes, so the user should not care about this;
* there are no bit-flips and bad eraseblocks in case of UBI volumes.

So, UBI volumes may be considered as flash devices with relaxed
restrictions.

Where can it be found?

Documentation, kernel code and applications can be found in the MTD
gits.

What are the applications for?

The applications help to create binary flash images for two purposes: pfi
files (partial flash images) for in-system update of UBI volumes, and plain
binary images, with or without OOB data in case of NAND, for a manufacturing
step. Furthermore some tools are/and will be created that allow flash content
analysis after a system has crashed..

Who did UBI?

The original ideas, where UBI is based on, were developed by Andreas
Arnez, Frank Haverkamp and Thomas Gleixner. Josh W. Boyer and some others
were involved too. The implementation of the kernel layer was done by Artem
B. Bityutskiy. The user-space applications and tools were written by Oliver
Lohmann with contributions from Frank Haverkamp, Andreas Arnez, and Artem.
Joern Engel contributed a patch which modifies JFFS2 so that it can be run on
a UBI volume. Thomas Gleixner did modifications to the NAND layer. Alexander
Schmidt made some testing work as well as core functionality improvements.

Signed-off-by: Artem B. Bityutskiy <dedekind@linutronix.de>
Signed-off-by: Frank Haverkamp <haver@vnet.ibm.com>