History log of /linux-master/fs/bcachefs/extents.h
Revision Date Author Comments
# 4409b808 11-Mar-2024 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Repair pass for scanning for btree nodes

If a btree root or interior btree node goes bad, we're going to lose a
lot of data, unless we can recover the nodes that it pointed to by
scanning.

Fortunately btree node headers are fully self describing, and
additionally the magic number is xored with the filesytem UUID, so we
can do so safely.

This implements the scanning - next patch will rework topology repair to
make use of the found nodes.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 47d2080e 25-Mar-2024 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Kill bch2_bkey_ptr_data_type()

Remove some duplication, and inconsistency between check_fix_ptrs and
the main ptr marking paths

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 264b501f 16-Mar-2024 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Avoid extent entry type assertions in .invalid()

After keys have passed bkey_ops.key_invalid we should never see invalid
extent entry types - but .key_invalid itself needs to cope with them.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 88005d5d 08-Mar-2024 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: extent_entry_next_safe()

We need to be able to iterate over extent ptrs that may be corrupted in
order to print them - this fixes a bug where we'd pop an assert in
bch2_bkey_durability_safe().

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# d7e77f53 16-Jan-2024 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: opts->compression can now also be applied in the background

The "apply this compression method in the background" paths now use the
compression option if background_compression is not set; this means that
setting or changing the compression option will cause existing data to
be compressed accordingly in the background.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# f0431c5f 31-Dec-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Combine .trans_trigger, .atomic_trigger

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 4f9ec59f 28-Dec-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: unify extent trigger

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 6cacd0c4 27-Dec-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: unify reservation trigger

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 0beebd92 21-Dec-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: bkey_for_each_ptr() now declares loop iter

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 037a2d9f 23-Dec-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: simplify bch_devs_list

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# b65db750 24-Oct-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Enumerate fsck errors

This patch adds a superblock error counter for every distinct fsck
error; this means that when analyzing filesystems out in the wild we'll
be able to see what sorts of inconsistencies are being found and repair,
and hence what bugs to look for.

Errors validating bkeys are not yet considered distinct fsck errors, but
this patch adds a new helper, bkey_fsck_err(), in order to add distinct
error types for them as well.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# fb3f57bb 20-Oct-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: rebalance_work

This adds a new btree, rebalance_work, to eliminate scanning required
for finding extents that need work done on them in the background - i.e.
for the background_target and background_compression options.

rebalance_work is a bitset btree, where a KEY_TYPE_set corresponds to an
extent in the extents or reflink btree at the same pos.

A new extent field is added, bch_extent_rebalance, which indicates that
this extent has work that needs to be done in the background - and which
options to use. This allows per-inode options to be propagated to
indirect extents - at least in some circumstances. In this patch,
changing IO options on a file will not propagate the new options to
indirect extents pointed to by that file.

Updating (setting/clearing) the rebalance_work btree is done by the
extent trigger, which looks at the bch_extent_rebalance field.

Scanning is still requrired after changing IO path options - either just
for a given inode, or for the whole filesystem. We indicate that
scanning is required by adding a KEY_TYPE_cookie key to the
rebalance_work btree: the cookie counter is so that we can detect that
scanning is still required when an option has been flipped mid-way
through an existing scan.

Future possible work:
- Propagate options to indirect extents when being changed
- Add other IO path options - nr_replicas, ec, to rebalance_work so
they can be applied in the background when they change
- Add a counter, for bcachefs fs usage output, showing the pending
amount of rebalance work: we'll probably want to do this after the
disk space accounting rewrite (moving it to a new btree)

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 9db2f860 22-Oct-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Check for too-large encoded extents

We don't yet repair (split) them, just check.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# e38356d6 20-Oct-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Kill dead code extent_save()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 7413ab70 18-Oct-2023 Kees Cook <keescook@chromium.org>

bcachefs: Refactor memcpy into direct assignment

The memcpy() in bch2_bkey_append_ptr() is operating on an embedded fake
flexible array which looks to the compiler like it has 0 size. This
causes W=1 builds to emit warnings due to -Wstringop-overflow:

In file included from include/linux/string.h:254,
from include/linux/bitmap.h:11,
from include/linux/cpumask.h:12,
from include/linux/smp.h:13,
from include/linux/lockdep.h:14,
from include/linux/radix-tree.h:14,
from include/linux/backing-dev-defs.h:6,
from fs/bcachefs/bcachefs.h:182:
fs/bcachefs/extents.c: In function 'bch2_bkey_append_ptr':
include/linux/fortify-string.h:57:33: warning: writing 8 bytes into a region of size 0 [-Wstringop-overflow=]
57 | #define __underlying_memcpy __builtin_memcpy
| ^
include/linux/fortify-string.h:648:9: note: in expansion of macro '__underlying_memcpy'
648 | __underlying_##op(p, q, __fortify_size); \
| ^~~~~~~~~~~~~
include/linux/fortify-string.h:693:26: note: in expansion of macro '__fortify_memcpy_chk'
693 | #define memcpy(p, q, s) __fortify_memcpy_chk(p, q, s, \
| ^~~~~~~~~~~~~~~~~~~~
fs/bcachefs/extents.c:235:17: note: in expansion of macro 'memcpy'
235 | memcpy((void *) &k->v + bkey_val_bytes(&k->k),
| ^~~~~~
fs/bcachefs/bcachefs_format.h:287:33: note: destination object 'v' of size 0
287 | struct bch_val v;
| ^

Avoid making any structure changes and just replace the u64 copy into a
direct assignment, side-stepping the entire problem.

Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Brian Foster <bfoster@redhat.com>
Cc: linux-bcachefs@vger.kernel.org
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202309192314.VBsjiIm5-lkp@intel.com/
Link: https://lore.kernel.org/r/20231010235609.work.594-kees@kernel.org
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# be47e0ba 29-Sep-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: KEY_TYPE_error now counts towards i_sectors

KEY_TYPE_error is used when all replicas in an extent are marked as
failed; it indicates that data was present, but has been lost.

So that i_sectors doesn't change when replacing extents with
KEY_TYPE_error, we now have to count error keys as allocations - this
fixes fsck errors later.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 88d39fd5 06-Oct-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Switch to unsafe_memcpy() in a few places

The new fortify checking doesn't work for us in all places; this
switches to unsafe_memcpy() where appropriate to silence a few
warnings/errors.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 8726dc93 06-Jul-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Change check for invalid key types

As part of the forward compatibility patch series, we need to allow for
new key types without complaining loudly when running an old version.

This patch changes the flags parameter of bkey_invalid to an enum, and
adds a new flag to indicate we're being called from the transaction
commit path.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 73bd774d 06-Jul-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Assorted sparse fixes

- endianness fixes
- mark some things static
- fix a few __percpu annotations
- fix silent enum conversions

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 2766876d 27-Jun-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: struct bch_extent_rebalance

This adds the extent entry for extents that rebalance needs to do
something with.

We're adding this ahead of the main rebalance_work patchset, because
adding new extent entries can't be done in a forwards-compatible way.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 91ecd41b 13-Jun-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: bch2_extent_ptr_desired_durability()

This adds a new helper for getting a pointer's durability irrespective
of the device state, and uses it in the the data update path.

This fixes a bug where we do a data update but request 0 replicas to be
allocated, because the replica being rewritten is on a device marked as
failed.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 174f930b 29-Apr-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: bkey_ops.min_val_size

This adds a new field to bkey_ops for the minimum size of the value,
which standardizes that check and also enforces the new rule (previously
done somewhat ad-hoc) that we can extend value types by adding new
fields on to the end.

To make that work we do _not_ initialize min_val_size with sizeof,
instead we initialize it to the size of the first version of those
values.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 702ffea2 10-Mar-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Extent helper improvements

- __bch2_bkey_drop_ptr() -> bch2_bkey_drop_ptr_noerror(), now available
outside extents.

- Split bch2_bkey_has_device() and bch2_bkey_has_device_c(), const and
non const versions

- bch2_extent_has_ptr() now returns the pointer it found

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# ac2ccddc 04-Mar-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Drop some anonymous structs, unions

Rust bindgen doesn't cope well with anonymous structs and unions. This
patch drops the fancy anonymous structs & unions in bkey_i that let us
use the same helpers for bkey_i and bkey_packed; since bkey_packed is an
internal type that's never exposed to outside code, it's only a minor
inconvenienc.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 64784ade 22-Feb-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Fix buffer overrun in ec_stripe_update_extent()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# c9163bb0 21-Feb-2023 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Cached pointers should not be erasure coded

There's no reason to erasure code cached pointers: we'll always have
another copy, and it'll be cheaper to read the other copy than do a
reconstruct read. And erasure coded cached pointers would add
complications that we'd rather not have to deal with, so let's make sure
to disallow them.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# facafdcb 20-Dec-2022 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Change bkey_invalid() rw param to flags

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# a8b3a677 02-Nov-2022 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Nocow support

This adds support for nocow mode, where we do writes in-place when
possible. Patch components:

- New boolean filesystem and inode option, nocow: note that when nocow
is enabled, data checksumming and compression are implicitly disabled

- To prevent in-place writes from racing with data moves
(data_update.c) or bucket reuse (i.e. a bucket being reused and
re-allocated while a nocow write is in flight, we have a new locking
mechanism.

Buckets can be locked for either data update or data move, using a
fixed size hash table of two_state_shared locks. We don't have any
chaining, meaning updates and moves to different buckets that hash to
the same lock will wait unnecessarily - we'll want to watch for this
becoming an issue.

- The allocator path also needs to check for in-place writes in flight
to a given bucket before giving it out: thus we add another counter
to bucket_alloc_state so we can track this.

- Fsync now may need to issue cache flushes to block devices instead of
flushing the journal. We add a device bitmask to bch_inode_info,
ei_devs_need_flush, which tracks devices that need to have flushes
issued - note that this will lead to unnecessary flushes when other
codepaths have already issued flushes, we may want to replace this with
a sequence number.

- New nocow write path: look up extents, and if they're writable write
to them - otherwise fall back to the normal COW write path.

XXX: switch to sequence numbers instead of bitmask for devs needing
journal flush

XXX: ei_quota_lock being a mutex means bch2_nocow_write_done() needs to
run in process context - see if we can improve this

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 79203111 13-Nov-2022 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Unwritten extents support

- bch2_extent_merge checks unwritten bit
- read path returns 0s for unwritten extents without actually reading
- reflink path skips over unwritten extents
- bch2_bkey_ptrs_invalid() checks for extents with both written and
unwritten extents, and non-normal extents (stripes, btree ptrs) with
unwritten ptrs
- fiemap checks for unwritten extents and returns
FIEMAP_EXTENT_UNWRITTEN

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 393a1f68 24-Nov-2022 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: Better inlining in core write path

Provide inline versions of some allocation functions
- bch2_alloc_sectors_done_inlined()
- bch2_alloc_sectors_append_ptrs_inlined()

and use them in the core IO path.

Also, inline bch2_extent_update_i_size_sectors() and
bch2_bkey_append_ptr().

In the core write path, function call overhead matters - every function
call is a jump to a new location and a potential cache miss.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# e88a75eb 24-Nov-2022 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: New bpos_cmp(), bkey_cmp() replacements

This patch introduces
- bpos_eq()
- bpos_lt()
- bpos_le()
- bpos_gt()
- bpos_ge()

and equivalent replacements for bkey_cmp().

Looking at the generated assembly these could probably be improved
further, but we already see a significant code size improvement with
this patch.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# a1019576 22-Oct-2022 Kent Overstreet <kent.overstreet@linux.dev>

bcachefs: More style fixes

Fixes for various checkpatch errors.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 7f5c5d20 13-Jun-2022 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Redo data_update interface

This patch significantly cleans up and simplifies the data_update
interface. Instead of only being able to specify a single pointer by
device to rewrite, we're now able to specify any or all of the pointers
in the original extent to be rewrited, as a bitmask.

data_cmd is no more: the various pred functions now just return true if
the extent should be moved/updated. All the data_update path does is
rewrite existing replicas, or add new ones.

This fixes a bug where with background compression on replicated
filesystems, where rebalance -> data_update would incorrectly drop the
wrong old replica, and keep trying to recompress an extent pointer and
each time failing to drop the right replica. Oops.

Now, the data update path doesn't look at the io options to decide which
pointers to keep and which to drop - it only goes off of the
data_update_options passed to it.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>


# 275c8426 03-Apr-2022 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Add rw to .key_invalid()

This adds a new parameter to .key_invalid() methods for whether the key
is being read or written; the idea being that methods can do more
aggressive checks when a key is newly created and being written, when we
wouldn't want to delete the key because of those checks.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>


# f0ac7df2 03-Apr-2022 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Convert .key_invalid methods to printbufs

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>


# 880e2275 12-Mar-2022 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Move trigger fns to bkey_ops

This replaces the switch statements in bch2_mark_key(),
bch2_trans_mark_key() with new bkey methods - prep work for the next
patch, which fixes BTREE_TRIGGER_WANTS_OLD_AND_NEW.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>


# b9a7d8ac 13-Oct-2021 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Fix implementation of KEY_TYPE_error

When force-removing a device, we were silently dropping extents that we
no longer had pointers for - we should have been switching them to
KEY_TYPE_error, so that reads for data that was lost return errors.

This patch adds the logic for switching a key to KEY_TYPE_error to
bch2_bkey_drop_ptr(), and improves the logic somewhat.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>


# 6fed42bb 15-Mar-2021 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Plumb through subvolume id

To implement snapshots, we need every filesystem btree operation (every
btree operation without a subvolume) to start by looking up the
subvolume and getting the current snapshot ID, with
bch2_subvolume_get_snapshot() - then, that snapshot ID is used for doing
btree lookups in BTREE_ITER_FILTER_SNAPSHOTS mode.

This patch adds those bch2_subvolume_get_snapshot() calls, and also
switches to passing around a subvol_inum instead of just an inode
number.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>


# 297d8934 10-Jun-2021 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Extensive triggers cleanups

- We no longer mark subsets of extents, they're marked like regular
keys now - which means we can drop the offset & sectors arguments
to trigger functions
- Drop other arguments that are no longer needed anymore in various
places - fs_usage
- Drop the logic for handling extents in bch2_mark_update() that isn't
needed anymore, to match bch2_trans_mark_update()
- Better logic for hanlding the BTREE_ITER_CACHED_NOFILL case, where we
don't have an old key to mark

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>


# 59ba21d9 28-Apr-2021 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Clean up key merging

This patch simplifies the key merging code by getting rid of partial
merges - it's simpler and saner if we just don't merge extents when
they'd overflow k->size.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>


# e1036ce5 14-May-2021 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Repair code for multiple types of data in same bucket

bch2_check_fix_ptrs() is awkward, we need to find a way to improve it.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 4cf91b02 04-Mar-2021 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Split out bpos_cmp() and bkey_cmp()

With snapshots, we're going to need to differentiate between comparisons
that should and shouldn't include the snapshot field. bpos_cmp is now
the comparison function that does include the snapshot field, used by
core btree code.

Upper level filesystem code generally does _not_ want to compare against
the snapshot field - that code wants keys to compare as equal even when
one of them is in an ancestor snapshot.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# fad7cfed 22-Mar-2021 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Add an .invalid method for bch2_btree_ptr_v2

It was using the method for btree_ptr_v1, but that wasn't checking all
the fields.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 7e6dbac9 19-Mar-2021 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Kill bkey ops->debugcheck method

This code used to be used for running some assertions on alloc info at
runtime, but it long predates fsck and hasn't been good for much in
ages - we can delete it now.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 0507962f 17-Feb-2021 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Drop invalid stripe ptrs in fsck

More repair code, now that we can repair extents during initial gc.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 35a067b4 14-Dec-2020 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Change when we allow overwrites

Originally, we'd check for -ENOSPC when getting a disk reservation
whenever the new extent took up more space on disk than the old extent.

Erasure coding screwed this up, because with erasure coding writes are
initially replicated, and then in the background the extra replicas are
dropped when the stripe is created. This means that with erasure coding
enabled, writes will always take up more space on disk than the data
they're overwriting - but, according to posix, overwrites aren't
supposed to return ENOSPC.

So, in this patch we fudge things: if the new extent has more replicas
than the _effective_ replicas of the old extent, or if the old extent is
compressed and the new one isn't, we check for ENOSPC when getting the
disk reservation - otherwise, we don't.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 801a3de6 24-Oct-2020 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Indirect inline data extents

When inline data extents were added, reflink was forgotten about - we
need indirect inline data extents for reflink + inline data to work
correctly.

This patch adds them, and a new feature bit that's flipped when they're
used.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 59a38a38 31-Mar-2020 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Add print method for bch2_btree_ptr_v2

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 39fb2983 07-Jan-2020 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Kill bkey_type_successor

Previously, BTREE_ID_INODES was special - inodes were indexed by the
inode field, which meant the offset field of struct bpos wasn't used,
which led to special cases in e.g. the btree iterator code.

Now, inodes in the inodes btree are indexed by the offset field.

Also: prevously min_key was special for extents btrees, min_key for
extents would equal max_key for the previous node. Now, min_key =
bkey_successor() of the previous node, same as non extent btrees.

This means we can completely get rid of
btree_type_sucessor/predecessor.

Also make some improvements to the metadata IO validate/compat code.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 548b3d20 07-Feb-2020 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: btree_ptr_v2

Add a new btree ptr type which contains the sequence number (random 64
bit cookie, actually) for that btree node - this lets us verify that
when we read in a btree node it really is the btree node we wanted.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 1f49dafc 06-Feb-2020 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Fix bch2_ptr_swab for indirect extents

bch2_ptr_swab was never updated when the code for generic keys with
pointers was added - it assumed the entire val was only used for
pointers.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# ab05de4c 23-Feb-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Track incompressible data

This fixes the background_compression option: wihout some way of marking
data as incompressible, rebalance will keep rewriting incompressible
data over and over.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 54e86b58 30-Dec-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Make btree_insert_entry more private to update path

This should be private to btree_update_leaf.c, and we might end up
removing it.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 4de77495 16-Nov-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Reorganize extents.c

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 4be1a412 09-Nov-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Inline data extents

This implements extents that have their data inline, in the value,
instead of the bkey value being pointers to the data - and the read and
write paths are updated to read from these new extent types and write
them out, when the write size is small enough.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 08c07fea 15-Nov-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Split out extent_update.c

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 085ab693 09-Nov-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Rework of cut_front & cut_back

This changes bch2_cut_front and bch2_cut_back so that they're able to
shorten the size of the value, and it also changes the extent update
path to update the accounting in the btree node when this happens.

When the size of the value is shortened, they zero out the space that's
no longer used, so it's interpreted as noops (as implemented in the last
patch).

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# aef90ce0 15-Nov-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: kill bch2_extent_has_device()

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 37954a27 08-Oct-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Limit pointers to being in only one stripe

This make the disk accounting code saner, and it's not clear why we'd
ever want the same data to be in multiple stripes simultaneously.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# a9d1f910 14-Sep-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Debug code improvements

.key_debugcheck no longer needs to take a pointer to the btree node

Also, try to make sure wherever we're inserting or modifying keys in the
btree.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# f9c55193 07-Sep-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Drop trans arg to bch2_extent_atomic_end()

Just for consistency

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 8d84260e 07-Sep-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: data move path should not be trying to move reflink_p keys

This was spotted when the move_extent() path tried to allocate a bio for
a reflink_p extent, but adding pages to the bio failed because we
overflowed bi_max_vecs. Oops.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 3fb5ebcd 22-Aug-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Inline some fast paths

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 76426098 16-Aug-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Reflink

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 3c7f3b7a 16-Aug-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Refactor bch2_extent_trim_atomic() for reflink

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 99aaf570 25-Jul-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Refactor various code to not be extent specific

With reflink, various code now has to handle both KEY_TYPE_extent
or KEY_TYPE_reflink_v - so, convert it to be generic across all keys
with pointers.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# b1c9358a 18-Jul-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Refactor __bch2_cut_front()

Minor cleanup - prep work for new key types for reflink

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 436c656d 09-Jun-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: bkey_merge() now takes bkey_s

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 3811aa6d 11-May-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: bch2_bkey_ptrs_invalid()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 0dc17247 13-Mar-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: kill struct btree_insert

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# c93cead0 16-Mar-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Always use bch2_extent_trim_atomic()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 7ef2a73a 21-Jan-2019 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Fix check for if extent update is allocating

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 26609b61 01-Nov-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Make bkey types globally unique

this lets us get rid of a lot of extra switch statements - in a lot of
places we dispatch on the btree node type, and then the key type, so
this is a nice cleanup across a lot of code.

Also improve the on disk format versioning stuff.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 5b8a9227 27-Nov-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Split out bkey_sort.c

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# cd575ddf 01-Nov-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Erasure coding

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 319f9ac3 08-Nov-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: revamp to_text methods

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 5bd95a37 01-Nov-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: new avoid mechanism for io retries

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# abce30b7 30-Sep-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: BCH_EXTENT_ENTRY_TYPES()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 71c9e0ba 27-Sep-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: bch2_extent_ptr_decoded_append()

This new helper for the move path avoids creating a new CRC entry when
we already have one that matches the pointer being added.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# a2753581 30-Sep-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: bch2_extent_drop_ptrs()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 1742237b 27-Sep-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: extent_for_each_ptr_decode()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 642d66d1 02-Oct-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: kill bch_extent_crc_type

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 4cb13156 02-Oct-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: extent_ptr_decoded

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# fc3268c1 08-Aug-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: kill extent_insert_hook

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# e2d9912c 05-Aug-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: bch2_extent_trim_atomic()

Prep work for extents insert hook removal

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# b0004d8d 03-Aug-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Factor out btree_key_can_insert()

working on getting rid of all the reasons bch2_insert_fixup_extent() can
fail/stop partway, which is needed for other refactorings.

One of the reasons we could have to bail out is if we're splitting a
compressed extent we might need to add to our disk reservation - but we
can check that before actually starting the insert.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 277c981c 23-Jul-2018 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: fix bch2_val_to_text()

was returning wrong value

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>


# 1c6fdbd8 17-Mar-2017 Kent Overstreet <kent.overstreet@gmail.com>

bcachefs: Initial commit

Initially forked from drivers/md/bcache, bcachefs is a new copy-on-write
filesystem with every feature you could possibly want.

Website: https://bcachefs.org

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>