#
c375b223 |
|
30-Jan-2024 |
Tejun Heo <tj@kernel.org> |
dm-verity: Convert from tasklet to BH workqueue The only generic interface to execute asynchronously in the BH context is tasklet; however, it's marked deprecated and has some design flaws. To replace tasklets, BH workqueue support was recently added. A BH workqueue behaves similarly to regular workqueues except that the queued work items are executed in the BH context. This commit converts dm-verity from tasklet to BH workqueue. It backfills tasklet code that was removed with commit 0a9bab391e33 ("dm-crypt, dm-verity: disable tasklets") and tweaks to use BH workqueue (and does some renaming). This is a minimal conversion which doesn't rename the related names including the "try_verify_in_tasklet" option. If this patch is applied, a follow-up patch would be necessary. I couldn't decide whether the option name would need to be updated too. Signed-off-by: Tejun Heo <tj@kernel.org> [snitzer: rename 'use_tasklet' to 'use_bh_wq' and 'in_tasklet' to 'in_bh'] Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
9356fcfe |
|
30-Jan-2024 |
Fan Wu <wufan@linux.microsoft.com> |
dm verity: set DM_TARGET_SINGLETON feature flag The device-mapper has a flag to mark targets as singleton, which is a required flag for immutable targets. Without this flag, multiple dm-verity targets can be added to a mapped device, which has no practical use cases and will let dm_table_get_immutable_target return NULL. This patch adds the missing flag, restricting only one dm-verity target per mapped device. Signed-off-by: Fan Wu <wufan@linux.microsoft.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
d95e2c34 |
|
23-Jan-2024 |
Hongyu Jin <hongyu.jin@unisoc.com> |
dm verity: Fix IO priority lost when reading FEC and hash After obtaining the data, verification or error correction process may trigger a new IO that loses the priority of the original IO, that is, the verification of the higher priority IO may be blocked by the lower priority IO. Make the IO used for verification and error correction follow the priority of the original IO. Co-developed-by: Yibin Ding <yibin.ding@unisoc.com> Signed-off-by: Yibin Ding <yibin.ding@unisoc.com> Signed-off-by: Hongyu Jin <hongyu.jin@unisoc.com> Reviewed-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
6e5f0f63 |
|
23-Jan-2024 |
Hongyu Jin <hongyu.jin@unisoc.com> |
dm io: Support IO priority Some IO will dispatch from kworker with different io_context settings than the submitting task, we may need to specify a priority to avoid losing priority. Add IO priority parameter to dm_io() and update all callers. Co-developed-by: Yibin Ding <yibin.ding@unisoc.com> Signed-off-by: Yibin Ding <yibin.ding@unisoc.com> Signed-off-by: Hongyu Jin <hongyu.jin@unisoc.com> Reviewed-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
66ad2fbc |
|
24-Feb-2024 |
Arnd Bergmann <arnd@arndb.de> |
dm-integrity, dm-verity: reduce stack usage for recheck The newly added integrity_recheck() function has another larger stack allocation, just like its caller integrity_metadata(). When it gets inlined, the combination of the two exceeds the warning limit for 32-bit architectures and possibly risks an overflow when this is called from a deep call chain through a file system: drivers/md/dm-integrity.c:1767:13: error: stack frame size (1048) exceeds limit (1024) in 'integrity_metadata' [-Werror,-Wframe-larger-than] 1767 | static void integrity_metadata(struct work_struct *w) Since the caller at this point is done using its checksum buffer, just reuse the same buffer in the new function to avoid the double allocation. [Mikulas: add "noinline" to integrity_recheck and verity_recheck. These functions are only called on error, so they shouldn't bloat the stack frame or code size of the caller.] Fixes: c88f5e553fe3 ("dm-integrity: recheck the integrity tag after a failure") Fixes: 9177f3c0dea6 ("dm-verity: recheck the hash after a failure") Cc: stable@vger.kernel.org Signed-off-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
0e0c50e8 |
|
20-Feb-2024 |
Mike Snitzer <snitzer@kernel.org> |
dm-crypt, dm-integrity, dm-verity: bump target version Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
9177f3c0 |
|
19-Feb-2024 |
Mikulas Patocka <mpatocka@redhat.com> |
dm-verity: recheck the hash after a failure If a userspace process reads (with O_DIRECT) multiple blocks into the same buffer, dm-verity reports an error [1]. This commit fixes dm-verity, so that if hash verification fails, the data is read again into a kernel buffer (where userspace can't modify it) and the hash is rechecked. If the recheck succeeds, the content of the kernel buffer is copied into the user buffer; if the recheck fails, an error is reported. [1] https://people.redhat.com/~mpatocka/testcases/blk-auth-modify/read2.c Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Cc: stable@vger.kernel.org Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
0a9bab39 |
|
31-Jan-2024 |
Mikulas Patocka <mpatocka@redhat.com> |
dm-crypt, dm-verity: disable tasklets Tasklets have an inherent problem with memory corruption. The function tasklet_action_common calls tasklet_trylock, then it calls the tasklet callback and then it calls tasklet_unlock. If the tasklet callback frees the structure that contains the tasklet or if it calls some code that may free it, tasklet_unlock will write into free memory. The commits 8e14f610159d and d9a02e016aaf try to fix it for dm-crypt, but it is not a sufficient fix and the data corruption can still happen [1]. There is no fix for dm-verity and dm-verity will write into free memory with every tasklet-processed bio. There will be atomic workqueues implemented in the kernel 6.9 [2]. They will have better interface and they will not suffer from the memory corruption problem. But we need something that stops the memory corruption now and that can be backported to the stable kernels. So, I'm proposing this commit that disables tasklets in both dm-crypt and dm-verity. This commit doesn't remove the tasklet support, because the tasklet code will be reused when atomic workqueues will be implemented. [1] https://lore.kernel.org/all/d390d7ee-f142-44d3-822a-87949e14608b@suse.de/T/ [2] https://lore.kernel.org/lkml/20240130091300.2968534-1-tj@kernel.org/ Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Cc: stable@vger.kernel.org Fixes: 39d42fa96ba1b ("dm crypt: add flags to optionally bypass kcryptd workqueues") Fixes: 5721d4e5a9cdb ("dm verity: Add optional "try_verify_in_tasklet" feature") Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
0193e396 |
|
21-Nov-2023 |
Wu Bo <bo.wu@vivo.com> |
dm verity: don't perform FEC for failed readahead IO We found an issue under Android OTA scenario that many BIOs have to do FEC where the data under dm-verity is 100% complete and no corruption. Android OTA has many dm-block layers, from upper to lower: dm-verity dm-snapshot dm-origin & dm-cow dm-linear ufs DM tables have to change 2 times during Android OTA merging process. When doing table change, the dm-snapshot will be suspended for a while. During this interval, many readahead IOs are submitted to dm_verity from filesystem. Then the kverity works are busy doing FEC process which cost too much time to finish dm-verity IO. This causes needless delay which feels like system is hung. After adding debugging it was found that each readahead IO needed around 10s to finish when this situation occurred. This is due to IO amplification: dm-snapshot suspend erofs_readahead // 300+ io is submitted dm_submit_bio (dm_verity) dm_submit_bio (dm_snapshot) bio return EIO bio got nothing, it's empty verity_end_io verity_verify_io forloop range(0, io->n_blocks) // each io->nblocks ~= 20 verity_fec_decode fec_decode_rsb fec_read_bufs forloop range(0, v->fec->rsn) // v->fec->rsn = 253 new_read submit_bio (dm_snapshot) end loop end loop dm-snapshot resume Readahead BIOs get nothing while dm-snapshot is suspended, so all of them will cause verity's FEC. Each readahead BIO needs to verify ~20 (io->nblocks) blocks. Each block needs to do FEC, and every block needs to do 253 (v->fec->rsn) reads. So during the suspend interval(~200ms), 300 readahead BIOs trigger ~1518000 (300*20*253) IOs to dm-snapshot. As readahead IO is not required by userspace, and to fix this issue, it is best to pass readahead errors to upper layer to handle it. Cc: stable@vger.kernel.org Fixes: a739ff3f543a ("dm verity: add support for forward error correction") Signed-off-by: Wu Bo <bo.wu@vivo.com> Reviewed-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
7be05bdf |
|
21-Nov-2023 |
Wu Bo <bo.wu@vivo.com> |
dm verity: initialize fec io before freeing it If BIO error, verity_end_io() can call verity_finish_io() before verity_fec_init_io(). Therefore, fec_io->rs is not initialized and may crash when doing memory freeing in verity_fec_finish_io(). Crash call stack: die+0x90/0x2b8 __do_kernel_fault+0x260/0x298 do_bad_area+0x2c/0xdc do_translation_fault+0x3c/0x54 do_mem_abort+0x54/0x118 el1_abort+0x38/0x5c el1h_64_sync_handler+0x50/0x90 el1h_64_sync+0x64/0x6c free_rs+0x18/0xac fec_rs_free+0x10/0x24 mempool_free+0x58/0x148 verity_fec_finish_io+0x4c/0xb0 verity_end_io+0xb8/0x150 Cc: stable@vger.kernel.org # v6.0+ Fixes: 5721d4e5a9cd ("dm verity: Add optional "try_verify_in_tasklet" feature") Signed-off-by: Wu Bo <bo.wu@vivo.com> Reviewed-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
28f07f2a |
|
17-Nov-2023 |
Mikulas Patocka <mpatocka@redhat.com> |
dm-verity: don't use blocking calls from tasklets The commit 5721d4e5a9cd enhanced dm-verity, so that it can verify blocks from tasklets rather than from workqueues. This reportedly improves performance significantly. However, dm-verity was using the flag CRYPTO_TFM_REQ_MAY_SLEEP from tasklets which resulted in warnings about sleeping function being called from non-sleeping context. BUG: sleeping function called from invalid context at crypto/internal.h:206 in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 14, name: ksoftirqd/0 preempt_count: 100, expected: 0 RCU nest depth: 0, expected: 0 CPU: 0 PID: 14 Comm: ksoftirqd/0 Tainted: G W 6.7.0-rc1 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x32/0x50 __might_resched+0x110/0x160 crypto_hash_walk_done+0x54/0xb0 shash_ahash_update+0x51/0x60 verity_hash_update.isra.0+0x4a/0x130 [dm_verity] verity_verify_io+0x165/0x550 [dm_verity] ? free_unref_page+0xdf/0x170 ? psi_group_change+0x113/0x390 verity_tasklet+0xd/0x70 [dm_verity] tasklet_action_common.isra.0+0xb3/0xc0 __do_softirq+0xaf/0x1ec ? smpboot_thread_fn+0x1d/0x200 ? sort_range+0x20/0x20 run_ksoftirqd+0x15/0x30 smpboot_thread_fn+0xed/0x200 kthread+0xdc/0x110 ? kthread_complete_and_exit+0x20/0x20 ret_from_fork+0x28/0x40 ? kthread_complete_and_exit+0x20/0x20 ret_from_fork_asm+0x11/0x20 </TASK> This commit fixes dm-verity so that it doesn't use the flags CRYPTO_TFM_REQ_MAY_SLEEP and CRYPTO_TFM_REQ_MAY_BACKLOG from tasklets. The crypto API would do GFP_ATOMIC allocation instead, it could return -ENOMEM and we catch -ENOMEM in verity_tasklet and requeue the request to the workqueue. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Cc: stable@vger.kernel.org # v6.0+ Fixes: 5721d4e5a9cd ("dm verity: Add optional "try_verify_in_tasklet" feature") Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
05bdb996 |
|
08-Jun-2023 |
Christoph Hellwig <hch@lst.de> |
block: replace fmode_t with a block-specific type for block open flags The only overlap between the block open flags mapped into the fmode_t and other uses of fmode_t are FMODE_READ and FMODE_WRITE. Define a new blk_mode_t instead for use in blkdev_get_by_{dev,path}, ->open and ->ioctl and stop abusing fmode_t. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Jack Wang <jinpu.wang@ionos.com> [rnbd] Reviewed-by: Hannes Reinecke <hare@suse.de> Reviewed-by: Christian Brauner <brauner@kernel.org> Link: https://lore.kernel.org/r/20230608110258.189493-28-hch@lst.de Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
#
3664ff82 |
|
09-Apr-2023 |
Yangtao Li <frank.li@vivo.com> |
dm: add helper macro for simple DM target module init and exit Eliminate duplicate boilerplate code for simple modules that contain a single DM target driver without any additional setup code. Add a new module_dm() macro, which replaces the module_init() and module_exit() with template functions that call dm_register_target() and dm_unregister_target() respectively. Signed-off-by: Yangtao Li <frank.li@vivo.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
b362c733 |
|
18-Mar-2023 |
Yangtao Li <frank.li@vivo.com> |
dm: push error reporting down to dm_register_target() Simplifies each DM target's init method by making dm_register_target() responsible for its error reporting (on behalf of targets). Signed-off-by: Yangtao Li <frank.li@vivo.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
074c4466 |
|
28-Feb-2023 |
Michael Weiß <michael.weiss@aisec.fraunhofer.de> |
dm verity: emit audit events on verification failure and more dm-verity signals integrity violations by returning I/O errors to user space. To identify integrity violations by a controlling instance, the kernel audit subsystem can be used to emit audit events to user space. Analogous to dm-integrity, we also use the dm-audit submodule allowing to emit audit events on verification failures of metadata and data blocks as well as if max corrupted errors are reached. The construction and destruction of verity device mappings are also relevant for auditing a system. Thus, those events are also logged as audit events. Tested by starting a container with the container manager (cmld) of GyroidOS which uses a dm-verity protected rootfs image root.img mapped to /dev/mapper/<uuid>-root. One block was manipulated in the underlying image file and repeated reads of the verity device were performed again until the max corrupted errors is reached, e.g.: dd if=/dev/urandom of=root.img bs=512 count=1 seek=1000 for i in range {1..101}; do \ dd if=/dev/mapper/<uuid>-root of=/dev/null bs=4096 \ count=1 skip=1000 \ done The resulting audit log looks as follows: type=DM_CTRL msg=audit(1677618791.876:962): module=verity op=ctr ppid=4876 pid=29102 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=44 comm="cmld" exe="/usr/sbin/cml/cmld" subj=unconfined dev=254:3 error_msg='success' res=1 type=DM_EVENT msg=audit(1677619463.786:1074): module=verity op=verify-data dev=7:0 sector=1000 res=0 ... type=DM_EVENT msg=audit(1677619596.727:1162): module=verity op=verify-data dev=7:0 sector=1000 res=0 type=DM_EVENT msg=audit(1677619596.731:1163): module=verity op=max-corrupted-errors dev=254:3 sector=? res=0 Signed-off-by: Michael Weiß <michael.weiss@aisec.fraunhofer.de> Acked-by: Paul Moore <paul@paul-moore.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
e8c5d45f |
|
20-Mar-2023 |
Yeongjin Gil <youngjin.gil@samsung.com> |
dm verity: fix error handling for check_at_most_once on FEC In verity_end_io(), if bi_status is not BLK_STS_OK, it can be return directly. But if FEC configured, it is desired to correct the data page through verity_verify_io. And the return value will be converted to blk_status and passed to verity_finish_io(). BTW, when a bit is set in v->validated_blocks, verity_verify_io() skips verification regardless of I/O error for the corresponding bio. In this case, the I/O error could not be returned properly, and as a result, there is a problem that abnormal data could be read for the corresponding block. To fix this problem, when an I/O error occurs, do not skip verification even if the bit related is set in v->validated_blocks. Fixes: 843f38d382b1 ("dm verity: add 'check_at_most_once' option to only validate hashes once") Cc: stable@vger.kernel.org Reviewed-by: Sungjong Seo <sj1557.seo@samsung.com> Signed-off-by: Yeongjin Gil <youngjin.gil@samsung.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
1c3fe2fa |
|
07-Feb-2023 |
Heinz Mauelshagen <heinzm@redhat.com> |
dm: avoid useless 'else' after 'break' or return' Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
6a808034 |
|
06-Feb-2023 |
Heinz Mauelshagen <heinzm@redhat.com> |
dm: avoid using symbolic permissions Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
1c131886 |
|
06-Feb-2023 |
Heinz Mauelshagen <heinzm@redhat.com> |
dm: prefer '"%s...", __func__' Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
0ef0b471 |
|
01-Feb-2023 |
Heinz Mauelshagen <heinzm@redhat.com> |
dm: add missing empty lines Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
03b18887 |
|
30-Jan-2023 |
Heinz Mauelshagen <heinzm@redhat.com> |
dm: fix trailing statements Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
86a3238c |
|
25-Jan-2023 |
Heinz Mauelshagen <heinzm@redhat.com> |
dm: change "unsigned" to "unsigned int" Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
c25da5b7 |
|
01-Feb-2023 |
Nathan Huckleberry <nhuck@google.com> |
dm verity: stop using WQ_UNBOUND for verify_wq Setting WQ_UNBOUND increases scheduler latency on ARM64. This is likely due to the asymmetric architecture of ARM64 processors. I've been unable to reproduce the results that claim WQ_UNBOUND gives a performance boost on x86-64. This flag is causing performance issues for multiple subsystems within Android. Notably, the same slowdown exists for decompression with EROFS. | open-prebuilt-camera | WQ_UNBOUND | ~WQ_UNBOUND | |-----------------------|------------|---------------| | verity wait time (us) | 11746 | 119 (-98%) | | erofs wait time (us) | 357805 | 174205 (-51%) | | sha256 ramdisk random read | WQ_UNBOUND | ~WQ_UNBOUND | |----------------------------|-----------=---|-------------| | arm64 (accelerated) | bw=42.4MiB/s | bw=212MiB/s | | arm64 (generic) | bw=16.5MiB/s | bw=48MiB/s | | x86_64 (generic) | bw=233MiB/s | bw=230MiB/s | Using a alloc_workqueue() @max_active arg of num_online_cpus() only made sense with WQ_UNBOUND. Switch the @max_active arg to 0 (aka default, which is 256 per-cpu). Also, eliminate 'wq_flags' since it really doesn't serve a purpose. Cc: Sami Tolvanen <samitolvanen@google.com> Cc: Eric Biggers <ebiggers@kernel.org> Signed-off-by: Nathan Huckleberry <nhuck@google.com> Reviewed-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
afd41fff |
|
30-Aug-2022 |
Nathan Huckleberry <nhuck@google.com> |
dm verity: enable WQ_HIGHPRI on verify_wq WQ_HIGHPRI increases throughput and decreases disk latency when using dm-verity. This is important in Android for camera startup speed. The following tests were run by doing 60 seconds of random reads using a dm-verity device backed by two ramdisks. Without WQ_HIGHPRI lat (usec): min=13, max=3947, avg=69.53, stdev=50.55 READ: bw=51.1MiB/s (53.6MB/s), 51.1MiB/s-51.1MiB/s (53.6MB/s-53.6MB/s) With WQ_HIGHPRI: lat (usec): min=13, max=7854, avg=31.15, stdev=30.42 READ: bw=116MiB/s (121MB/s), 116MiB/s-116MiB/s (121MB/s-121MB/s) Further testing was done by measuring how long it takes to open a camera on an Android device. Without WQ_HIGHPRI Total verity work queue wait times (ms): 880.960, 789.517, 898.852 With WQ_HIGHPRI: Total verity work queue wait times (ms): 528.824, 439.191, 433.300 The average time to open the camera is reduced by 350ms (or 40-50%). Signed-off-by: Nathan Huckleberry <nhuck@google.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
916ef623 |
|
07-Sep-2022 |
Matthias Kaehlcke <mka@chromium.org> |
dm: verity-loadpin: Only trust verity targets with enforcement Verity targets can be configured to ignore corrupted data blocks. LoadPin must only trust verity targets that are configured to perform some kind of enforcement when data corruption is detected, like returning an error, restarting the system or triggering a panic. Fixes: b6c1c5745ccc ("dm: Add verity helpers for LoadPin") Reported-by: Sarthak Kukreti <sarthakkukreti@chromium.org> Signed-off-by: Matthias Kaehlcke <mka@chromium.org> Reviewed-by: Sarthak Kukreti <sarthakkukreti@chromium.org> Cc: stable@vger.kernel.org Signed-off-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20220907133055.1.Ic8a1dafe960dc0f8302e189642bc88ebb785d274@changeid
|
#
f876df9f |
|
09-Aug-2022 |
Mike Snitzer <snitzer@kernel.org> |
dm verity: fix verity_parse_opt_args parsing Commit df326e7a0699 ("dm verity: allow optional args to alter primary args handling") introduced a bug where verity_parse_opt_args() wouldn't properly shift past an optional argument's additional params (by ignoring them). Fix this by avoiding returning with error if an unknown argument is encountered when @only_modifier_opts=true is passed to verity_parse_opt_args(). In practice this regressed the cryptsetup testsuite's FEC testing because unknown optional arguments were encountered, wherey short-circuiting ever testing FEC mode. With this fix all of the cryptsetup testsuite's verity FEC tests pass. Fixes: df326e7a0699 ("dm verity: allow optional args to alter primary args handling") Reported-by: Milan Broz <gmazyland@gmail.com>> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
8c22816d |
|
09-Aug-2022 |
Mike Snitzer <snitzer@kernel.org> |
dm verity: fix DM_VERITY_OPTS_MAX value yet again Must account for the possibility that "try_verify_in_tasklet" is used. This is the same issue that was fixed with commit 160f99db94322 -- it is far too easy to miss that additional a new argument(s) require bumping DM_VERITY_OPTS_MAX accordingly. Fixes: 5721d4e5a9cd ("dm verity: Add optional "try_verify_in_tasklet" feature") Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
12907efd |
|
04-Aug-2022 |
Mike Snitzer <snitzer@kernel.org> |
dm verity: have verify_wq use WQ_HIGHPRI if "try_verify_in_tasklet" Allow verify_wq to preempt softirq since verification in tasklet will fall-back to using it for error handling (or if the bufio cache doesn't have required hashes). Suggested-by: Nathan Huckleberry <nhuck@google.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
43fa47cb |
|
04-Aug-2022 |
Mike Snitzer <snitzer@kernel.org> |
dm verity: remove WQ_CPU_INTENSIVE flag since using WQ_UNBOUND The documentation [1] says that WQ_CPU_INTENSIVE is "meaningless" for unbound wq. So remove WQ_CPU_INTENSIVE from the verify_wq allocation. 1. https://www.kernel.org/doc/html/latest/core-api/workqueue.html#flags Suggested-by: Maksym Planeta <mplaneta@os.inf.tu-dresden.de> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
e9307e3d |
|
04-Aug-2022 |
Mike Snitzer <snitzer@kernel.org> |
dm verity: only copy bvec_iter in verity_verify_io if in_tasklet Avoid extra bvec_iter copy unless it is needed to allow retrying verification, that failed from a tasklet, from a workqueue. Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
0a36463f |
|
03-Aug-2022 |
Mike Snitzer <snitzer@kernel.org> |
dm verity: optimize verity_verify_io if FEC not configured Only declare and copy bvec_iter if CONFIG_DM_VERITY_FEC is defined and FEC enabled for the verity device. Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
ba2cce82 |
|
25-Jul-2022 |
Mike Snitzer <snitzer@kernel.org> |
dm verity: conditionally enable branching for "try_verify_in_tasklet" Use jump_label to limit the need for branching unless the optional "try_verify_in_tasklet" feature is used. Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
df326e7a |
|
26-Jul-2022 |
Mike Snitzer <snitzer@kernel.org> |
dm verity: allow optional args to alter primary args handling The previous commit ("dm verity: Add optional "try_verify_in_tasklet" feature") imposed that CRYPTO_ALG_ASYNC mask be used even if the optional "try_verify_in_tasklet" feature was not specified. This was because verity_parse_opt_args() was called after handling the primary args (due to it having data dependencies on having first parsed all primary args). Enhance verity_ctr() so that simple optional args, that don't have a data dependency on primary args parsing, can alter how the primary args are handled. In practice this means verity_parse_opt_args() gets called twice. First with the new 'only_modifier_opts' arg set to true, then again with it set to false _after_ parsing all primary args. This allows the v->use_tasklet flag to be properly set and then used when verity_ctr() parses the primary args and then calls crypto_alloc_ahash() with CRYPTO_ALG_ASYNC conditionally set. Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
5721d4e5 |
|
22-Jul-2022 |
Nathan Huckleberry <nhuck@google.com> |
dm verity: Add optional "try_verify_in_tasklet" feature Using tasklets for disk verification can reduce IO latency. When there are accelerated hash instructions it is often better to compute the hash immediately using a tasklet rather than deferring verification to a work-queue. This reduces time spent waiting to schedule work-queue jobs, but requires spending slightly more time in interrupt context. If the dm-bufio cache does not have the required hashes we fallback to the work-queue implementation. FEC is only possible using work-queue because code to support the FEC feature may sleep. The following shows a speed comparison of random reads on a dm-verity device. The dm-verity device uses a 1G ramdisk for data and a 1G ramdisk for hashes. One test was run using tasklets and one test was run using the existing work-queue solution. Both tests were run when the dm-bufio cache was hot. The tasklet implementation performs significantly better since there is no time spent waiting for work-queue jobs to be scheduled. READ: bw=181MiB/s (190MB/s), 181MiB/s-181MiB/s (190MB/s-190MB/s), io=512MiB (537MB), run=2827-2827msec READ: bw=23.6MiB/s (24.8MB/s), 23.6MiB/s-23.6MiB/s (24.8MB/s-24.8MB/s), io=512MiB (537MB), run=21688-21688msec Signed-off-by: Nathan Huckleberry <nhuck@google.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
0fcb100d |
|
22-Jul-2022 |
Nathan Huckleberry <nhuck@google.com> |
dm bufio: Add flags argument to dm_bufio_client_create Add a flags argument to dm_bufio_client_create and update all the callers. This is in preparation to add the DM_BUFIO_NO_SLEEP flag. Signed-off-by: Nathan Huckleberry <nhuck@google.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
b6c1c574 |
|
27-Jun-2022 |
Matthias Kaehlcke <mka@chromium.org> |
dm: Add verity helpers for LoadPin LoadPin limits loading of kernel modules, firmware and certain other files to a 'pinned' file system (typically a read-only rootfs). To provide more flexibility LoadPin is being extended to also allow loading these files from trusted dm-verity devices. For that purpose LoadPin can be provided with a list of verity root digests that it should consider as trusted. Add a bunch of helpers to allow LoadPin to check whether a DM device is a trusted verity device. The new functions broadly fall in two categories: those that need access to verity internals (like the root digest), and the 'glue' between LoadPin and verity. The new file dm-verity-loadpin.c contains the glue functions. Signed-off-by: Matthias Kaehlcke <mka@chromium.org> Acked-by: Mike Snitzer <snitzer@kernel.org> Link: https://lore.kernel.org/lkml/20220627083512.v7.1.I3e928575a23481121e73286874c4c2bdb403355d@changeid Signed-off-by: Kees Cook <keescook@chromium.org>
|
#
20e6fc85 |
|
14-Jun-2022 |
JeongHyeon Lee <jhs2.lee@samsung.com> |
dm verity: fix checkpatch close brace error Resolves: ERROR: else should follow close brace '}' Signed-off-by: JeongHyeon Lee <jhs2.lee@samsung.com> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
4caae584 |
|
31-May-2022 |
Sarthak Kukreti <sarthakkukreti@google.com> |
dm verity: set DM_TARGET_IMMUTABLE feature flag The device-mapper framework provides a mechanism to mark targets as immutable (and hence fail table reloads that try to change the target type). Add the DM_TARGET_IMMUTABLE flag to the dm-verity target's feature flags to prevent switching the verity target with a different target type. Fixes: a4ffc152198e ("dm: add verity target") Cc: stable@vger.kernel.org Signed-off-by: Sarthak Kukreti <sarthakkukreti@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Mike Snitzer <snitzer@kernel.org>
|
#
30495e68 |
|
19-Oct-2021 |
Christoph Hellwig <hch@lst.de> |
dm verity: use bvec_kmap_local in verity_for_bv_block Using local kmaps slightly reduces the chances to stray writes, and the bvec interface cleans up the code a little bit. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
6dcbb52c |
|
17-Oct-2021 |
Christoph Hellwig <hch@lst.de> |
dm: use bdev_nr_sectors and bdev_nr_bytes instead of open coding them Use the proper helpers to read the block device size. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Mike Snitzer <snitzer@redhat.com> Link: https://lore.kernel.org/r/20211018101130.1838532-6-hch@lst.de Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
#
24b83deb |
|
20-Sep-2021 |
Christoph Hellwig <hch@lst.de> |
block: move struct request to blk-mq.h struct request is only used by blk-mq drivers, so move it and all related declarations to blk-mq.h. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Link: https://lore.kernel.org/r/20210920123328.1399408-18-hch@lst.de Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
#
2c0468e0 |
|
13-Sep-2021 |
Akilesh Kailash <akailash@google.com> |
dm verity: skip redundant verity_handle_err() on I/O errors Without FEC, dm-verity won't call verity_handle_err() when I/O fails, but with FEC enabled, it currently does even if an I/O error has occurred. If there is an I/O error and FEC correction fails, return the error instead of calling verity_handle_err() again. Suggested-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Akilesh Kailash <akailash@google.com> Reviewed-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
33ace4ca |
|
13-Aug-2021 |
Tushar Sugandhi <tusharsu@linux.microsoft.com> |
dm ima: update dm target attributes for ima measurements Certain DM targets ('integrity', 'multipath', 'verity') need to update the way their attributes are recorded in the ima log, so that the attestation servers can interpret the data correctly and decide if the devices meet the attestation requirements. For instance, the "mode=%c" attribute in the 'integrity' target is measured twice, the 'verity' target is missing the attribute "root_hash_sig_key_desc=%s", and the 'multipath' target needs to index the attributes properly. Update 'integrity' target to remove the duplicate measurement of the attribute "mode=%c". Add "root_hash_sig_key_desc=%s" attribute for the 'verity' target. Index various attributes in 'multipath' target. Also, add "nr_priority_groups=%u" attribute to 'multipath' target to record the number of priority groups. Signed-off-by: Tushar Sugandhi <tusharsu@linux.microsoft.com> Suggested-by: Thore Sommer <public@thson.de> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
8ec45662 |
|
12-Jul-2021 |
Tushar Sugandhi <tusharsu@linux.microsoft.com> |
dm: update target status functions to support IMA measurement For device mapper targets to take advantage of IMA's measurement capabilities, the status functions for the individual targets need to be updated to handle the status_type_t case for value STATUSTYPE_IMA. Update status functions for the following target types, to log their respective attributes to be measured using IMA. 01. cache 02. crypt 03. integrity 04. linear 05. mirror 06. multipath 07. raid 08. snapshot 09. striped 10. verity For rest of the targets, handle the STATUSTYPE_IMA case by setting the measurement buffer to NULL. For IMA to measure the data on a given system, the IMA policy on the system needs to be updated to have the following line, and the system needs to be restarted for the measurements to take effect. /etc/ima/ima-policy measure func=CRITICAL_DATA label=device-mapper template=ima-buf The measurements will be reflected in the IMA logs, which are located at: /sys/kernel/security/integrity/ima/ascii_runtime_measurements /sys/kernel/security/integrity/ima/binary_runtime_measurements These IMA logs can later be consumed by various attestation clients running on the system, and send them to external services for attesting the system. The DM target data measured by IMA subsystem can alternatively be queried from userspace by setting DM_IMA_MEASUREMENT_FLAG with DM_TABLE_STATUS_CMD. Signed-off-by: Tushar Sugandhi <tusharsu@linux.microsoft.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
219a9b5e |
|
11-Mar-2021 |
JeongHyeon Lee <jhs2.lee@samsung.com> |
dm verity: allow only one error handling mode If more than one one handling mode is requested during DM verity table load, the last requested mode will be used. Change this to impose more strict checking so that the table load will fail if more than one error handling mode is requested. Signed-off-by: JeongHyeon Lee <jhs2.lee@samsung.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
160f99db |
|
11-Mar-2021 |
JeongHyeon Lee <jhs2.lee@samsung.com> |
dm verity: fix DM_VERITY_OPTS_MAX value Three optional parameters must be accepted at once in a DM verity table, e.g.: (verity_error_handling_mode) (ignore_zero_block) (check_at_most_once) Fix this to be possible by incrementing DM_VERITY_OPTS_MAX. Signed-off-by: JeongHyeon Lee <jhs2.lee@samsung.com> Fixes: 843f38d382b1 ("dm verity: add 'check_at_most_once' option to only validate hashes once") Cc: stable@vger.kernel.org Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
252bd125 |
|
02-Dec-2020 |
Hyeongseok Kim <hyeongseok@gmail.com> |
dm verity: skip verity work if I/O error when system is shutting down If emergency system shutdown is called, like by thermal shutdown, a dm device could be alive when the block device couldn't process I/O requests anymore. In this state, the handling of I/O errors by new dm I/O requests or by those already in-flight can lead to a verity corruption state, which is a misjudgment. So, skip verity work in response to I/O error when system is shutting down. Signed-off-by: Hyeongseok Kim <hyeongseok@gmail.com> Reviewed-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
e1fef0b0 |
|
18-Jun-2020 |
JeongHyeon Lee <jhs2.lee@samsung.com> |
dm verity: add "panic_on_corruption" error handling mode Samsung smart phones may need the ability to panic on corruption. Not all devices provide the bootloader support needed to use the existing "restart_on_corruption" mode. Additional details for why Samsung needs this new mode can be found here: https://www.redhat.com/archives/dm-devel/2020-June/msg00235.html Signed-off-by: jhs2.lee <jhs2.lee@samsung.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
ed00aabd |
|
01-Jul-2020 |
Christoph Hellwig <hch@lst.de> |
block: rename generic_make_request to submit_bio_noacct generic_make_request has always been very confusingly misnamed, so rename it to submit_bio_noacct to make it clear that it is submit_bio minus accounting and a few checks. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
#
636be424 |
|
27-Feb-2020 |
Mike Snitzer <snitzer@redhat.com> |
dm: bump version of core and various targets Changes made during the 5.6 cycle warrant bumping the version number for DM core and the targets modified by this commit. It should be noted that dm-thin, dm-crypt and dm-raid already had their target version bumped during the 5.6 merge window. Signed-off-by; Mike Snitzer <snitzer@redhat.com>
|
#
0a531c5a |
|
06-Jan-2020 |
xianrong.zhou <xianrong.zhou@transsion.com> |
dm verity: don't prefetch hash blocks for already-verified data Try to skip prefetching hash blocks that won't be needed due to the "check_at_most_once" option being enabled and the corresponding data blocks already having been verified. Since prefetching operates on a range of data blocks, do this by just trimming the two ends of the range. This doesn't skip every unneeded hash block, since data blocks in the middle of the range could also be unneeded, and hash blocks are still prefetched in large clusters as controlled by dm_verity_prefetch_cluster. But it can still help a lot. In a test on Android Q launching 91 apps every 15s repeated 21 times, prefetching was only done for 447177/4776629 = 9.36% of data blocks. Tested-by: ruxian.feng <ruxian.feng@transsion.com> Co-developed-by: yuanjiong.gao <yuanjiong.gao@transsion.com> Signed-off-by: yuanjiong.gao <yuanjiong.gao@transsion.com> Signed-off-by: xianrong.zhou <xianrong.zhou@transsion.com> [EB: simplified the 'while' loops and improved the commit message] Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
88cd3e6c |
|
17-Jul-2019 |
Jaskaran Khurana <jaskarankhurana@linux.microsoft.com> |
dm verity: add root hash pkcs#7 signature verification The verification is to support cases where the root hash is not secured by Trusted Boot, UEFI Secureboot or similar technologies. One of the use cases for this is for dm-verity volumes mounted after boot, the root hash provided during the creation of the dm-verity volume has to be secure and thus in-kernel validation implemented here will be used before we trust the root hash and allow the block device to be created. The signature being provided for verification must verify the root hash and must be trusted by the builtin keyring for verification to succeed. The hash is added as a key of type "user" and the description is passed to the kernel so it can look it up and use it for verification. Adds CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG which can be turned on if root hash verification is needed. Kernel commandline dm_verity module parameter 'require_signatures' will indicate whether to force root hash signature verification (for all dm verity volumes). Signed-off-by: Jaskaran Khurana <jaskarankhurana@linux.microsoft.com> Tested-and-Reviewed-by: Milan Broz <gmazyland@gmail.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
2eba4e64 |
|
20-Jun-2019 |
Milan Broz <gmazyland@gmail.com> |
dm verity: use message limit for data block corruption message DM verity should also use DMERR_LIMIT to limit repeat data block corruption messages. Signed-off-by: Milan Broz <gmazyland@gmail.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
55716d26 |
|
01-Jun-2019 |
Thomas Gleixner <tglx@linutronix.de> |
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 428 Based on 1 normalized pattern(s): this file is released under the gplv2 extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 68 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Armijn Hemel <armijn@tjaldur.nl> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190531190114.292346262@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
#
bbf6a566 |
|
05-Dec-2018 |
Eric Biggers <ebiggers@google.com> |
dm verity: log the hash algorithm implementation Log the hash algorithm's driver name when a dm-verity target is created. This will help people determine whether the expected implementation is being used. It can make an enormous difference; e.g., SHA-256 on ARM can be 8x faster with the crypto extensions than without. It can also be useful to know if an implementation using an external crypto accelerator is being used instead of a software implementation. Example message: [ 35.281945] device-mapper: verity: sha256 using implementation "sha256-ce" We've already found the similar message in fs/crypto/keyinfo.c to be very useful. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
e4b069e0 |
|
21-Aug-2018 |
Mikulas Patocka <mpatocka@redhat.com> |
dm verity: fix crash on bufio buffer that was allocated with vmalloc Since commit d1ac3ff008fb ("dm verity: switch to using asynchronous hash crypto API") dm-verity uses asynchronous crypto calls for verification, so that it can use hardware with asynchronous processing of crypto operations. These asynchronous calls don't support vmalloc memory, but the buffer data can be allocated with vmalloc if dm-bufio is short of memory and uses a reserved buffer that was preallocated in dm_bufio_client_create(). Fix verity_hash_update() so that it deals with vmalloc'd memory correctly. Reported-by: "Xiao, Jin" <jin.xiao@intel.com> Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Fixes: d1ac3ff008fb ("dm verity: switch to using asynchronous hash crypto API") Cc: stable@vger.kernel.org # 4.12+ Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
778e1cdd |
|
12-Jun-2018 |
Kees Cook <keescook@chromium.org> |
treewide: kvzalloc() -> kvcalloc() The kvzalloc() function has a 2-factor argument form, kvcalloc(). This patch replaces cases of: kvzalloc(a * b, gfp) with: kvcalloc(a * b, gfp) as well as handling cases of: kvzalloc(a * b * c, gfp) with: kvzalloc(array3_size(a, b, c), gfp) as it's slightly less ugly than: kvcalloc(array_size(a, b), c, gfp) This does, however, attempt to ignore constant size factors like: kvzalloc(4 * 1024, gfp) though any constants defined via macros get caught up in the conversion. Any factors with a sizeof() of "unsigned char", "char", and "u8" were dropped, since they're redundant. The Coccinelle script used for this was: // Fix redundant parens around sizeof(). @@ type TYPE; expression THING, E; @@ ( kvzalloc( - (sizeof(TYPE)) * E + sizeof(TYPE) * E , ...) | kvzalloc( - (sizeof(THING)) * E + sizeof(THING) * E , ...) ) // Drop single-byte sizes and redundant parens. @@ expression COUNT; typedef u8; typedef __u8; @@ ( kvzalloc( - sizeof(u8) * (COUNT) + COUNT , ...) | kvzalloc( - sizeof(__u8) * (COUNT) + COUNT , ...) | kvzalloc( - sizeof(char) * (COUNT) + COUNT , ...) | kvzalloc( - sizeof(unsigned char) * (COUNT) + COUNT , ...) | kvzalloc( - sizeof(u8) * COUNT + COUNT , ...) | kvzalloc( - sizeof(__u8) * COUNT + COUNT , ...) | kvzalloc( - sizeof(char) * COUNT + COUNT , ...) | kvzalloc( - sizeof(unsigned char) * COUNT + COUNT , ...) ) // 2-factor product with sizeof(type/expression) and identifier or constant. @@ type TYPE; expression THING; identifier COUNT_ID; constant COUNT_CONST; @@ ( - kvzalloc + kvcalloc ( - sizeof(TYPE) * (COUNT_ID) + COUNT_ID, sizeof(TYPE) , ...) | - kvzalloc + kvcalloc ( - sizeof(TYPE) * COUNT_ID + COUNT_ID, sizeof(TYPE) , ...) | - kvzalloc + kvcalloc ( - sizeof(TYPE) * (COUNT_CONST) + COUNT_CONST, sizeof(TYPE) , ...) | - kvzalloc + kvcalloc ( - sizeof(TYPE) * COUNT_CONST + COUNT_CONST, sizeof(TYPE) , ...) | - kvzalloc + kvcalloc ( - sizeof(THING) * (COUNT_ID) + COUNT_ID, sizeof(THING) , ...) | - kvzalloc + kvcalloc ( - sizeof(THING) * COUNT_ID + COUNT_ID, sizeof(THING) , ...) | - kvzalloc + kvcalloc ( - sizeof(THING) * (COUNT_CONST) + COUNT_CONST, sizeof(THING) , ...) | - kvzalloc + kvcalloc ( - sizeof(THING) * COUNT_CONST + COUNT_CONST, sizeof(THING) , ...) ) // 2-factor product, only identifiers. @@ identifier SIZE, COUNT; @@ - kvzalloc + kvcalloc ( - SIZE * COUNT + COUNT, SIZE , ...) // 3-factor product with 1 sizeof(type) or sizeof(expression), with // redundant parens removed. @@ expression THING; identifier STRIDE, COUNT; type TYPE; @@ ( kvzalloc( - sizeof(TYPE) * (COUNT) * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kvzalloc( - sizeof(TYPE) * (COUNT) * STRIDE + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kvzalloc( - sizeof(TYPE) * COUNT * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kvzalloc( - sizeof(TYPE) * COUNT * STRIDE + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kvzalloc( - sizeof(THING) * (COUNT) * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kvzalloc( - sizeof(THING) * (COUNT) * STRIDE + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kvzalloc( - sizeof(THING) * COUNT * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kvzalloc( - sizeof(THING) * COUNT * STRIDE + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) ) // 3-factor product with 2 sizeof(variable), with redundant parens removed. @@ expression THING1, THING2; identifier COUNT; type TYPE1, TYPE2; @@ ( kvzalloc( - sizeof(TYPE1) * sizeof(TYPE2) * COUNT + array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2)) , ...) | kvzalloc( - sizeof(TYPE1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2)) , ...) | kvzalloc( - sizeof(THING1) * sizeof(THING2) * COUNT + array3_size(COUNT, sizeof(THING1), sizeof(THING2)) , ...) | kvzalloc( - sizeof(THING1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(THING1), sizeof(THING2)) , ...) | kvzalloc( - sizeof(TYPE1) * sizeof(THING2) * COUNT + array3_size(COUNT, sizeof(TYPE1), sizeof(THING2)) , ...) | kvzalloc( - sizeof(TYPE1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(TYPE1), sizeof(THING2)) , ...) ) // 3-factor product, only identifiers, with redundant parens removed. @@ identifier STRIDE, SIZE, COUNT; @@ ( kvzalloc( - (COUNT) * STRIDE * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kvzalloc( - COUNT * (STRIDE) * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kvzalloc( - COUNT * STRIDE * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kvzalloc( - (COUNT) * (STRIDE) * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kvzalloc( - COUNT * (STRIDE) * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kvzalloc( - (COUNT) * STRIDE * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kvzalloc( - (COUNT) * (STRIDE) * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kvzalloc( - COUNT * STRIDE * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) ) // Any remaining multi-factor products, first at least 3-factor products, // when they're not all constants... @@ expression E1, E2, E3; constant C1, C2, C3; @@ ( kvzalloc(C1 * C2 * C3, ...) | kvzalloc( - (E1) * E2 * E3 + array3_size(E1, E2, E3) , ...) | kvzalloc( - (E1) * (E2) * E3 + array3_size(E1, E2, E3) , ...) | kvzalloc( - (E1) * (E2) * (E3) + array3_size(E1, E2, E3) , ...) | kvzalloc( - E1 * E2 * E3 + array3_size(E1, E2, E3) , ...) ) // And then all remaining 2 factors products when they're not all constants, // keeping sizeof() as the second factor argument. @@ expression THING, E1, E2; type TYPE; constant C1, C2, C3; @@ ( kvzalloc(sizeof(THING) * C2, ...) | kvzalloc(sizeof(TYPE) * C2, ...) | kvzalloc(C1 * C2 * C3, ...) | kvzalloc(C1 * C2, ...) | - kvzalloc + kvcalloc ( - sizeof(TYPE) * (E2) + E2, sizeof(TYPE) , ...) | - kvzalloc + kvcalloc ( - sizeof(TYPE) * E2 + E2, sizeof(TYPE) , ...) | - kvzalloc + kvcalloc ( - sizeof(THING) * (E2) + E2, sizeof(THING) , ...) | - kvzalloc + kvcalloc ( - sizeof(THING) * E2 + E2, sizeof(THING) , ...) | - kvzalloc + kvcalloc ( - (E1) * E2 + E1, E2 , ...) | - kvzalloc + kvcalloc ( - (E1) * (E2) + E1, E2 , ...) | - kvzalloc + kvcalloc ( - E1 * E2 + E1, E2 , ...) ) Signed-off-by: Kees Cook <keescook@chromium.org>
|
#
5bd5e8d8 |
|
03-Apr-2018 |
Mike Snitzer <snitzer@redhat.com> |
dm: remove fmode_t argument from .prepare_ioctl hook Use the fmode_t that is passed to dm_blk_ioctl() rather than inconsistently (varies across targets) drop it on the floor by overriding it with the fmode_t stored in 'struct dm_dev'. All the persistent reservation functions weren't using the fmode_t they got back from .prepare_ioctl so remove them. Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
d4b1aaf5 |
|
28-Mar-2018 |
weiyongjun (A) <weiyongjun1@huawei.com> |
dm verity: make verity_for_io_block static Fixes the following sparse warning: drivers/md/dm-verity-target.c:375:6: warning: symbol 'verity_for_io_block' was not declared. Should it be static? Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
843f38d3 |
|
22-Mar-2018 |
Patrik Torstensson <totte@google.com> |
dm verity: add 'check_at_most_once' option to only validate hashes once This allows platforms that are CPU/memory contrained to verify data blocks only the first time they are read from the data device, rather than every time. As such, it provides a reduced level of security because only offline tampering of the data device's content will be detected, not online tampering. Hash blocks are still verified each time they are read from the hash device, since verification of hash blocks is less performance critical than data blocks, and a hash block will not be verified any more after all the data blocks it covers have been verified anyway. This option introduces a bitset that is used to check if a block has been validated before or not. A block can be validated more than once as there is no thread protection for the bitset. These changes were developed and tested on entry-level Android Go devices. Signed-off-by: Patrik Torstensson <totte@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
12f1ffc4 |
|
18-Oct-2017 |
Gilad Ben-Yossef <gilad@benyossef.com> |
dm: move dm-verity to generic async completion dm-verity is starting async. crypto ops and waiting for them to complete. Move it over to generic code doing the same. This also avoids a future potential data coruption bug created by the use of wait_for_completion_interruptible() without dealing correctly with an interrupt aborting the wait prior to the async op finishing, should this code ever move to a context where signals are not masked. Signed-off-by: Gilad Ben-Yossef <gilad@benyossef.com> CC: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
6aa7de05 |
|
23-Oct-2017 |
Mark Rutland <mark.rutland@arm.com> |
locking/atomics: COCCINELLE/treewide: Convert trivial ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE() Please do not apply this to mainline directly, instead please re-run the coccinelle script shown below and apply its output. For several reasons, it is desirable to use {READ,WRITE}_ONCE() in preference to ACCESS_ONCE(), and new code is expected to use one of the former. So far, there's been no reason to change most existing uses of ACCESS_ONCE(), as these aren't harmful, and changing them results in churn. However, for some features, the read/write distinction is critical to correct operation. To distinguish these cases, separate read/write accessors must be used. This patch migrates (most) remaining ACCESS_ONCE() instances to {READ,WRITE}_ONCE(), using the following coccinelle script: ---- // Convert trivial ACCESS_ONCE() uses to equivalent READ_ONCE() and // WRITE_ONCE() // $ make coccicheck COCCI=/home/mark/once.cocci SPFLAGS="--include-headers" MODE=patch virtual patch @ depends on patch @ expression E1, E2; @@ - ACCESS_ONCE(E1) = E2 + WRITE_ONCE(E1, E2) @ depends on patch @ expression E; @@ - ACCESS_ONCE(E) + READ_ONCE(E) ---- Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: davem@davemloft.net Cc: linux-arch@vger.kernel.org Cc: mpe@ellerman.id.au Cc: shuah@kernel.org Cc: snitzer@redhat.com Cc: thor.thayer@linux.intel.com Cc: tj@kernel.org Cc: viro@zeniv.linux.org.uk Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/1508792849-3115-19-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
#
5916a22b |
|
22-Jun-2017 |
Eric Biggers <ebiggers@google.com> |
dm: constify argument arrays The arrays of 'struct dm_arg' are never modified by the device-mapper core, so constify them so that they are placed in .rodata. (Exception: the args array in dm-raid cannot be constified because it is allocated on the stack and modified.) Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
74d46992 |
|
23-Aug-2017 |
Christoph Hellwig <hch@lst.de> |
block: replace bi_bdev with a gendisk pointer and partitions index This way we don't need a block_device structure to submit I/O. The block_device has different life time rules from the gendisk and request_queue and is usually only available when the block device node is open. Other callers need to explicitly create one (e.g. the lightnvm passthrough code, or the new nvme multipathing code). For the actual I/O path all that we need is the gendisk, which exists once per block device. But given that the block layer also does partition remapping we additionally need a partition index, which is used for said remapping in generic_make_request. Note that all the block drivers generally want request_queue or sometimes the gendisk, so this removes a layer of indirection all over the stack. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
#
4e4cbee9 |
|
03-Jun-2017 |
Christoph Hellwig <hch@lst.de> |
block: switch bios to blk_status_t Replace bi_error with a new bi_status to allow for a clear conversion. Note that device mapper overloaded bi_error with a private value, which we'll have to keep arround at least for now and thus propagate to a proper blk_status_t value. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@fb.com>
|
#
846785e6 |
|
03-Jun-2017 |
Christoph Hellwig <hch@lst.de> |
dm: don't return errnos from ->map Instead use the special DM_MAPIO_KILL return value to return -EIO just like we do for the request based path. Note that dm-log-writes returned -ENOMEM in a few places, which now becomes -EIO instead. No consumer treats -ENOMEM special so this shouldn't be an issue (and it should use a mempool to start with to make guaranteed progress). Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mike Snitzer <snitzer@redhat.com> Signed-off-by: Jens Axboe <axboe@fb.com>
|
#
f52236e0 |
|
18-May-2017 |
Gilad Ben-Yossef <gilad@benyossef.com> |
dm verity: fix no salt use case DM-Verity has an (undocumented) mode where no salt is used. This was never handled directly by the DM-Verity code, instead working due to the fact that calling crypto_shash_update() with a zero length data is an implicit noop. This is no longer the case now that we have switched to crypto_ahash_update(). Fix the issue by introducing explicit handling of the no salt use case to DM-Verity. Signed-off-by: Gilad Ben-Yossef <gilad@benyossef.com> Reported-by: Marian Csontos <mcsontos@redhat.com> Fixes: d1ac3ff ("dm verity: switch to using asynchronous hash crypto API") Tested-by: Milan Broz <gmazyland@gmail.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
d1ac3ff0 |
|
19-Feb-2017 |
Gilad Ben-Yossef <gilad@benyossef.com> |
dm verity: switch to using asynchronous hash crypto API Use of the synchronous digest API limits dm-verity to using pure CPU based algorithm providers and rules out the use of off CPU algorithm providers which are normally asynchronous by nature, potentially freeing CPU cycles. This can reduce performance per Watt in situations such as during boot time when a lot of concurrent file accesses are made to the protected volume. Signed-off-by: Gilad Ben-Yossef <gilad@benyossef.com> CC: Eric Biggers <ebiggers3@gmail.com> CC: Ondrej Mosnáček <omosnacek+linux-crypto@gmail.com> Tested-by: Milan Broz <gmazyland@gmail.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
21ffe552 |
|
08-Sep-2016 |
Eric Biggers <ebiggers@google.com> |
dm verity: fix incorrect error message Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
30187e1d |
|
31-Jan-2016 |
Mike Snitzer <snitzer@redhat.com> |
dm: rename target's per_bio_data_size to per_io_data_size Request-based DM will also make use of per_bio_data_size. Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
0cc37c2d |
|
03-Dec-2015 |
Sami Tolvanen <samitolvanen@google.com> |
dm verity: add ignore_zero_blocks feature If ignore_zero_blocks is enabled dm-verity will return zeroes for blocks matching a zero hash without validating the content. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
a739ff3f |
|
03-Dec-2015 |
Sami Tolvanen <samitolvanen@google.com> |
dm verity: add support for forward error correction Add support for correcting corrupted blocks using Reed-Solomon. This code uses RS(255, N) interleaved across data and hash blocks. Each error-correcting block covers N bytes evenly distributed across the combined total data, so that each byte is a maximum distance away from the others. This makes it possible to recover from several consecutive corrupted blocks with relatively small space overhead. In addition, using verity hashes to locate erasures nearly doubles the effectiveness of error correction. Being able to detect corrupted blocks also improves performance, because only corrupted blocks need to corrected. For a 2 GiB partition, RS(255, 253) (two parity bytes for each 253-byte block) can correct up to 16 MiB of consecutive corrupted blocks if erasures can be located, and 8 MiB if they cannot, with 16 MiB space overhead. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
bb4d73ac |
|
03-Dec-2015 |
Sami Tolvanen <samitolvanen@google.com> |
dm verity: factor out verity_for_bv_block() verity_for_bv_block() will be re-used by optional dm-verity object. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
ffa39380 |
|
03-Dec-2015 |
Sami Tolvanen <samitolvanen@google.com> |
dm verity: factor out structures and functions useful to separate object Prepare for an optional verity object to make use of existing dm-verity structures and functions. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|
#
03045cba |
|
03-Dec-2015 |
Sami Tolvanen <samitolvanen@google.com> |
dm verity: move dm-verity.c to dm-verity-target.c Prepare for extending dm-verity with an optional object. Follows the naming convention used by other DM targets (e.g. dm-cache and dm-era). Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com>
|