Searched +hist:1 +hist:c1d35df (Results 1 - 6 of 6) sorted by relevance

/linux-master/fs/f2fs/
H A Dsysfs.cdiff 9b13a866 Tue Jan 17 06:24:42 MST 2023 Yangtao Li <frank.li@vivo.com> f2fs: fix to check warm_data_age_threshold

hot_data_age_threshold is a non-zero positive number, and
condition 2 includes condition 1, so there is no need to
additionally judge whether t is 0. And let's remove it.

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 45c98f5a Mon Dec 12 06:36:44 MST 2022 Yangtao Li <frank.li@vivo.com> f2fs: convert discard_wake and gc_wake to bool type

discard_wake and gc_wake have only two values, 0 or 1.
So there is no need to use int type to store them.

BTW, move discard_wake to the end of the
discard_cmd_control structure.

Before:

- sizeof(struct discard_cmd_control): 8392

After move:

- sizeof(struct discard_cmd_control): 8384

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 71644dff Thu Dec 01 18:37:15 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: add block_age-based extent cache

This patch introduces a runtime hot/cold data separation method
for f2fs, in order to improve the accuracy for data temperature
classification, reduce the garbage collection overhead after
long-term data updates.

Enhanced hot/cold data separation can record data block update
frequency as "age" of the extent per inode, and take use of the age
info to indicate better temperature type for data block allocation:
- It records total data blocks allocated since mount;
- When file extent has been updated, it calculate the count of data
blocks allocated since last update as the age of the extent;
- Before the data block allocated, it searches for the age info and
chooses the suitable segment for allocation.

Test and result:
- Prepare: create about 30000 files
* 3% for cold files (with cold file extension like .apk, from 3M to 10M)
* 50% for warm files (with random file extension like .FcDxq, from 1K
to 4M)
* 47% for hot files (with hot file extension like .db, from 1K to 256K)
- create(5%)/random update(90%)/delete(5%) the files
* total write amount is about 70G
* fsync will be called for .db files, and buffered write will be used
for other files

The storage of test device is large enough(128G) so that it will not
switch to SSR mode during the test.

Benefit: dirty segment count increment reduce about 14%
- before: Dirty +21110
- after: Dirty +18286

Signed-off-by: qixiaoyu1 <qixiaoyu1@xiaomi.com>
Signed-off-by: xiongping1 <xiongping1@xiaomi.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 71644dff Thu Dec 01 18:37:15 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: add block_age-based extent cache

This patch introduces a runtime hot/cold data separation method
for f2fs, in order to improve the accuracy for data temperature
classification, reduce the garbage collection overhead after
long-term data updates.

Enhanced hot/cold data separation can record data block update
frequency as "age" of the extent per inode, and take use of the age
info to indicate better temperature type for data block allocation:
- It records total data blocks allocated since mount;
- When file extent has been updated, it calculate the count of data
blocks allocated since last update as the age of the extent;
- Before the data block allocated, it searches for the age info and
chooses the suitable segment for allocation.

Test and result:
- Prepare: create about 30000 files
* 3% for cold files (with cold file extension like .apk, from 3M to 10M)
* 50% for warm files (with random file extension like .FcDxq, from 1K
to 4M)
* 47% for hot files (with hot file extension like .db, from 1K to 256K)
- create(5%)/random update(90%)/delete(5%) the files
* total write amount is about 70G
* fsync will be called for .db files, and buffered write will be used
for other files

The storage of test device is large enough(128G) so that it will not
switch to SSR mode during the test.

Benefit: dirty segment count increment reduce about 14%
- before: Dirty +21110
- after: Dirty +18286

Signed-off-by: qixiaoyu1 <qixiaoyu1@xiaomi.com>
Signed-off-by: xiongping1 <xiongping1@xiaomi.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 66aee5aa Mon Nov 14 23:35:37 MST 2022 Yuwei Guan <ssawgyw@gmail.com> f2fs: change type for 'sbi->readdir_ra'

Before this patch, the varibale 'readdir_ra' takes effect if it's equal
to '1' or not, so we can change type for it from 'int' to 'bool'.

Signed-off-by: Yuwei Guan <Yuwei.Guan@zeekrlife.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 225d6795 Tue Oct 25 07:26:38 MDT 2022 Yangtao Li <frank.li@vivo.com> f2fs: add proc entry to show discard_plist info

This patch adds a new proc entry to show discard_plist
information in more detail, which is very helpful to
know the discard pend list count clearly.

Such as:

Discard pend list(Show diacrd_cmd count on each entry, .:not exist):
0 390 156 85 67 46 37 26 14
8 17 12 9 9 6 12 11 10
16 5 9 2 4 8 3 4 1
24 3 2 2 5 2 4 5 4
32 3 3 2 3 . 3 3 1
40 . 4 1 3 2 1 2 1
48 1 . 1 1 . 1 1 .
56 . 1 1 1 . 2 . 1
64 1 2 . . . . . .
72 . 1 . . . . . .
80 3 1 . . 1 1 . .
88 1 . . . 1 . . 1
......

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
H A Dnamei.cdiff 40b2d55e Wed Feb 07 00:05:48 MST 2024 Chao Yu <chao@kernel.org> f2fs: fix to create selinux label during whiteout initialization

generic/700 - output mismatch (see /media/fstests/results//generic/700.out.bad)
--- tests/generic/700.out 2023-03-28 10:40:42.735529223 +0000
+++ /media/fstests/results//generic/700.out.bad 2024-02-06 04:37:56.000000000 +0000
@@ -1,2 +1,4 @@
QA output created by 700
+/mnt/scratch_f2fs/f1: security.selinux: No such attribute
+/mnt/scratch_f2fs/f2: security.selinux: No such attribute
Silence is golden
...
(Run 'diff -u /media/fstests/tests/generic/700.out /media/fstests/results//generic/700.out.bad' to see the entire diff)

HINT: You _MAY_ be missing kernel fix:
70b589a37e1a xfs: add selinux labels to whiteout inodes

Previously, it missed to create selinux labels during whiteout inode
initialization, fix this issue.

Fixes: 7e01e7ad746b ("f2fs: support RENAME_WHITEOUT")
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 40b2d55e Wed Feb 07 00:05:48 MST 2024 Chao Yu <chao@kernel.org> f2fs: fix to create selinux label during whiteout initialization

generic/700 - output mismatch (see /media/fstests/results//generic/700.out.bad)
--- tests/generic/700.out 2023-03-28 10:40:42.735529223 +0000
+++ /media/fstests/results//generic/700.out.bad 2024-02-06 04:37:56.000000000 +0000
@@ -1,2 +1,4 @@
QA output created by 700
+/mnt/scratch_f2fs/f1: security.selinux: No such attribute
+/mnt/scratch_f2fs/f2: security.selinux: No such attribute
Silence is golden
...
(Run 'diff -u /media/fstests/tests/generic/700.out /media/fstests/results//generic/700.out.bad' to see the entire diff)

HINT: You _MAY_ be missing kernel fix:
70b589a37e1a xfs: add selinux labels to whiteout inodes

Previously, it missed to create selinux labels during whiteout inode
initialization, fix this issue.

Fixes: 7e01e7ad746b ("f2fs: support RENAME_WHITEOUT")
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff c10e8558 Fri Jan 12 10:11:13 MST 2024 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: remove unnecessary f2fs_put_page in f2fs_rename

[1] changed the below condition, which made f2fs_put_page() voided.
This patch reapplies the AL's resolution in -next from [2].

- if (S_ISDIR(old_inode->i_mode)) {
+ if (old_is_dir && old_dir != new_dir) {
old_dir_entry = f2fs_parent_dir(old_inode, &old_dir_page);
if (!old_dir_entry) {
if (IS_ERR(old_dir_page))

[1] 7deee77b993a ("f2fs: Avoid reading renamed directory if parent does not change")
[2] https://lore.kernel.org/all/20231220013402.GW1674809@ZenIV/

Suggested-by: Al Viro <viro@zeniv.linux.org.uk>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff c10e8558 Fri Jan 12 10:11:13 MST 2024 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: remove unnecessary f2fs_put_page in f2fs_rename

[1] changed the below condition, which made f2fs_put_page() voided.
This patch reapplies the AL's resolution in -next from [2].

- if (S_ISDIR(old_inode->i_mode)) {
+ if (old_is_dir && old_dir != new_dir) {
old_dir_entry = f2fs_parent_dir(old_inode, &old_dir_page);
if (!old_dir_entry) {
if (IS_ERR(old_dir_page))

[1] 7deee77b993a ("f2fs: Avoid reading renamed directory if parent does not change")
[2] https://lore.kernel.org/all/20231220013402.GW1674809@ZenIV/

Suggested-by: Al Viro <viro@zeniv.linux.org.uk>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 53edb549 Tue Nov 28 02:25:16 MST 2023 Chao Yu <chao@kernel.org> f2fs: fix to avoid dirent corruption

As Al reported in link[1]:

f2fs_rename()
...
if (old_dir != new_dir && !whiteout)
f2fs_set_link(old_inode, old_dir_entry,
old_dir_page, new_dir);
else
f2fs_put_page(old_dir_page, 0);

You want correct inumber in the ".." link. And cross-directory
rename does move the source to new parent, even if you'd been asked
to leave a whiteout in the old place.

[1] https://lore.kernel.org/all/20231017055040.GN800259@ZenIV/

With below testcase, it may cause dirent corruption, due to it missed
to call f2fs_set_link() to update ".." link to new directory.
- mkdir -p dir/foo
- renameat2 -w dir/foo bar

[ASSERT] (__chk_dots_dentries:1421) --> Bad inode number[0x4] for '..', parent parent ino is [0x3]
[FSCK] other corrupted bugs [Fail]

Fixes: 7e01e7ad746b ("f2fs: support RENAME_WHITEOUT")
Cc: Jan Kara <jack@suse.cz>
Reported-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Chao Yu <chao@kernel.org>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 53edb549 Tue Nov 28 02:25:16 MST 2023 Chao Yu <chao@kernel.org> f2fs: fix to avoid dirent corruption

As Al reported in link[1]:

f2fs_rename()
...
if (old_dir != new_dir && !whiteout)
f2fs_set_link(old_inode, old_dir_entry,
old_dir_page, new_dir);
else
f2fs_put_page(old_dir_page, 0);

You want correct inumber in the ".." link. And cross-directory
rename does move the source to new parent, even if you'd been asked
to leave a whiteout in the old place.

[1] https://lore.kernel.org/all/20231017055040.GN800259@ZenIV/

With below testcase, it may cause dirent corruption, due to it missed
to call f2fs_set_link() to update ".." link to new directory.
- mkdir -p dir/foo
- renameat2 -w dir/foo bar

[ASSERT] (__chk_dots_dentries:1421) --> Bad inode number[0x4] for '..', parent parent ino is [0x3]
[FSCK] other corrupted bugs [Fail]

Fixes: 7e01e7ad746b ("f2fs: support RENAME_WHITEOUT")
Cc: Jan Kara <jack@suse.cz>
Reported-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Chao Yu <chao@kernel.org>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff cadfc2f9 Wed May 31 19:37:59 MDT 2023 Wu Bo <bo.wu@vivo.com> f2fs: fix args passed to trace_f2fs_lookup_end

The NULL return of 'd_splice_alias' dosen't mean error. Thus the
successful case will also return NULL, which makes the tracepoint always
print 'err=-ENOENT'.

And the different cases of 'new' & 'err' are list as following:
1) dentry exists: err(0) with new(NULL) --> dentry, err=0
2) dentry exists: err(0) with new(VALID) --> new, err=0
3) dentry exists: err(0) with new(ERR) --> dentry, err=ERR
4) no dentry exists: err(-ENOENT) with new(NULL) --> dentry, err=-ENOENT
5) no dentry exists: err(-ENOENT) with new(VALID) --> new, err=-ENOENT
6) no dentry exists: err(-ENOENT) with new(ERR) --> dentry, err=ERR

Signed-off-by: Wu Bo <bo.wu@vivo.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff c0abbdf2 Sun Mar 19 01:58:22 MDT 2023 Yangtao Li <frank.li@vivo.com> f2fs: convert is_extension_exist() to return bool type

is_extension_exist() only return two values, 0 or 1.
So there is no need to use int type.

Signed-off-by: Yangtao Li <frank.li@vivo.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff cac2f8b8 Thu Sep 22 09:17:00 MDT 2022 Christian Brauner <brauner@kernel.org> fs: rename current get acl method

The current way of setting and getting posix acls through the generic
xattr interface is error prone and type unsafe. The vfs needs to
interpret and fixup posix acls before storing or reporting it to
userspace. Various hacks exist to make this work. The code is hard to
understand and difficult to maintain in it's current form. Instead of
making this work by hacking posix acls through xattr handlers we are
building a dedicated posix acl api around the get and set inode
operations. This removes a lot of hackiness and makes the codepaths
easier to maintain. A lot of background can be found in [1].

The current inode operation for getting posix acls takes an inode
argument but various filesystems (e.g., 9p, cifs, overlayfs) need access
to the dentry. In contrast to the ->set_acl() inode operation we cannot
simply extend ->get_acl() to take a dentry argument. The ->get_acl()
inode operation is called from:

acl_permission_check()
-> check_acl()
-> get_acl()

which is part of generic_permission() which in turn is part of
inode_permission(). Both generic_permission() and inode_permission() are
called in the ->permission() handler of various filesystems (e.g.,
overlayfs). So simply passing a dentry argument to ->get_acl() would
amount to also having to pass a dentry argument to ->permission(). We
should avoid this unnecessary change.

So instead of extending the existing inode operation rename it from
->get_acl() to ->get_inode_acl() and add a ->get_acl() method later that
passes a dentry argument and which filesystems that need access to the
dentry can implement instead of ->get_inode_acl(). Filesystems like cifs
which allow setting and getting posix acls but not using them for
permission checking during lookup can simply not implement
->get_inode_acl().

This is intended to be a non-functional change.

Link: https://lore.kernel.org/all/20220801145520.1532837-1-brauner@kernel.org [1]
Suggested-by/Inspired-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Microsoft) <brauner@kernel.org>
diff cac2f8b8 Thu Sep 22 09:17:00 MDT 2022 Christian Brauner <brauner@kernel.org> fs: rename current get acl method

The current way of setting and getting posix acls through the generic
xattr interface is error prone and type unsafe. The vfs needs to
interpret and fixup posix acls before storing or reporting it to
userspace. Various hacks exist to make this work. The code is hard to
understand and difficult to maintain in it's current form. Instead of
making this work by hacking posix acls through xattr handlers we are
building a dedicated posix acl api around the get and set inode
operations. This removes a lot of hackiness and makes the codepaths
easier to maintain. A lot of background can be found in [1].

The current inode operation for getting posix acls takes an inode
argument but various filesystems (e.g., 9p, cifs, overlayfs) need access
to the dentry. In contrast to the ->set_acl() inode operation we cannot
simply extend ->get_acl() to take a dentry argument. The ->get_acl()
inode operation is called from:

acl_permission_check()
-> check_acl()
-> get_acl()

which is part of generic_permission() which in turn is part of
inode_permission(). Both generic_permission() and inode_permission() are
called in the ->permission() handler of various filesystems (e.g.,
overlayfs). So simply passing a dentry argument to ->get_acl() would
amount to also having to pass a dentry argument to ->permission(). We
should avoid this unnecessary change.

So instead of extending the existing inode operation rename it from
->get_acl() to ->get_inode_acl() and add a ->get_acl() method later that
passes a dentry argument and which filesystems that need access to the
dentry can implement instead of ->get_inode_acl(). Filesystems like cifs
which allow setting and getting posix acls but not using them for
permission checking during lookup can simply not implement
->get_inode_acl().

This is intended to be a non-functional change.

Link: https://lore.kernel.org/all/20220801145520.1532837-1-brauner@kernel.org [1]
Suggested-by/Inspired-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Microsoft) <brauner@kernel.org>
diff cac2f8b8 Thu Sep 22 09:17:00 MDT 2022 Christian Brauner <brauner@kernel.org> fs: rename current get acl method

The current way of setting and getting posix acls through the generic
xattr interface is error prone and type unsafe. The vfs needs to
interpret and fixup posix acls before storing or reporting it to
userspace. Various hacks exist to make this work. The code is hard to
understand and difficult to maintain in it's current form. Instead of
making this work by hacking posix acls through xattr handlers we are
building a dedicated posix acl api around the get and set inode
operations. This removes a lot of hackiness and makes the codepaths
easier to maintain. A lot of background can be found in [1].

The current inode operation for getting posix acls takes an inode
argument but various filesystems (e.g., 9p, cifs, overlayfs) need access
to the dentry. In contrast to the ->set_acl() inode operation we cannot
simply extend ->get_acl() to take a dentry argument. The ->get_acl()
inode operation is called from:

acl_permission_check()
-> check_acl()
-> get_acl()

which is part of generic_permission() which in turn is part of
inode_permission(). Both generic_permission() and inode_permission() are
called in the ->permission() handler of various filesystems (e.g.,
overlayfs). So simply passing a dentry argument to ->get_acl() would
amount to also having to pass a dentry argument to ->permission(). We
should avoid this unnecessary change.

So instead of extending the existing inode operation rename it from
->get_acl() to ->get_inode_acl() and add a ->get_acl() method later that
passes a dentry argument and which filesystems that need access to the
dentry can implement instead of ->get_inode_acl(). Filesystems like cifs
which allow setting and getting posix acls but not using them for
permission checking during lookup can simply not implement
->get_inode_acl().

This is intended to be a non-functional change.

Link: https://lore.kernel.org/all/20220801145520.1532837-1-brauner@kernel.org [1]
Suggested-by/Inspired-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Microsoft) <brauner@kernel.org>
H A Dinode.cdiff 958ccbbf Thu Jul 20 05:29:53 MDT 2023 Chao Yu <chao@kernel.org> Revert "f2fs: fix to do sanity check on extent cache correctly"

syzbot reports a f2fs bug as below:

UBSAN: array-index-out-of-bounds in fs/f2fs/f2fs.h:3275:19
index 1409 is out of range for type '__le32[923]' (aka 'unsigned int[923]')
Call Trace:
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e7/0x2d0 lib/dump_stack.c:106
ubsan_epilogue lib/ubsan.c:217 [inline]
__ubsan_handle_out_of_bounds+0x11c/0x150 lib/ubsan.c:348
inline_data_addr fs/f2fs/f2fs.h:3275 [inline]
__recover_inline_status fs/f2fs/inode.c:113 [inline]
do_read_inode fs/f2fs/inode.c:480 [inline]
f2fs_iget+0x4730/0x48b0 fs/f2fs/inode.c:604
f2fs_fill_super+0x640e/0x80c0 fs/f2fs/super.c:4601
mount_bdev+0x276/0x3b0 fs/super.c:1391
legacy_get_tree+0xef/0x190 fs/fs_context.c:611
vfs_get_tree+0x8c/0x270 fs/super.c:1519
do_new_mount+0x28f/0xae0 fs/namespace.c:3335
do_mount fs/namespace.c:3675 [inline]
__do_sys_mount fs/namespace.c:3884 [inline]
__se_sys_mount+0x2d9/0x3c0 fs/namespace.c:3861
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

The issue was bisected to:

commit d48a7b3a72f121655d95b5157c32c7d555e44c05
Author: Chao Yu <chao@kernel.org>
Date: Mon Jan 9 03:49:20 2023 +0000

f2fs: fix to do sanity check on extent cache correctly

The root cause is we applied both v1 and v2 of the patch, v2 is the right
fix, so it needs to revert v1 in order to fix reported issue.

v1:
commit d48a7b3a72f1 ("f2fs: fix to do sanity check on extent cache correctly")
https://lore.kernel.org/lkml/20230109034920.492914-1-chao@kernel.org/

v2:
commit 269d11948100 ("f2fs: fix to do sanity check on extent cache correctly")
https://lore.kernel.org/lkml/20230207134808.1827869-1-chao@kernel.org/

Reported-by: syzbot+601018296973a481f302@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/linux-f2fs-devel/000000000000fcf0690600e4d04d@google.com/
Fixes: d48a7b3a72f1 ("f2fs: fix to do sanity check on extent cache correctly")
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 958ccbbf Thu Jul 20 05:29:53 MDT 2023 Chao Yu <chao@kernel.org> Revert "f2fs: fix to do sanity check on extent cache correctly"

syzbot reports a f2fs bug as below:

UBSAN: array-index-out-of-bounds in fs/f2fs/f2fs.h:3275:19
index 1409 is out of range for type '__le32[923]' (aka 'unsigned int[923]')
Call Trace:
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e7/0x2d0 lib/dump_stack.c:106
ubsan_epilogue lib/ubsan.c:217 [inline]
__ubsan_handle_out_of_bounds+0x11c/0x150 lib/ubsan.c:348
inline_data_addr fs/f2fs/f2fs.h:3275 [inline]
__recover_inline_status fs/f2fs/inode.c:113 [inline]
do_read_inode fs/f2fs/inode.c:480 [inline]
f2fs_iget+0x4730/0x48b0 fs/f2fs/inode.c:604
f2fs_fill_super+0x640e/0x80c0 fs/f2fs/super.c:4601
mount_bdev+0x276/0x3b0 fs/super.c:1391
legacy_get_tree+0xef/0x190 fs/fs_context.c:611
vfs_get_tree+0x8c/0x270 fs/super.c:1519
do_new_mount+0x28f/0xae0 fs/namespace.c:3335
do_mount fs/namespace.c:3675 [inline]
__do_sys_mount fs/namespace.c:3884 [inline]
__se_sys_mount+0x2d9/0x3c0 fs/namespace.c:3861
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

The issue was bisected to:

commit d48a7b3a72f121655d95b5157c32c7d555e44c05
Author: Chao Yu <chao@kernel.org>
Date: Mon Jan 9 03:49:20 2023 +0000

f2fs: fix to do sanity check on extent cache correctly

The root cause is we applied both v1 and v2 of the patch, v2 is the right
fix, so it needs to revert v1 in order to fix reported issue.

v1:
commit d48a7b3a72f1 ("f2fs: fix to do sanity check on extent cache correctly")
https://lore.kernel.org/lkml/20230109034920.492914-1-chao@kernel.org/

v2:
commit 269d11948100 ("f2fs: fix to do sanity check on extent cache correctly")
https://lore.kernel.org/lkml/20230207134808.1827869-1-chao@kernel.org/

Reported-by: syzbot+601018296973a481f302@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/linux-f2fs-devel/000000000000fcf0690600e4d04d@google.com/
Fixes: d48a7b3a72f1 ("f2fs: fix to do sanity check on extent cache correctly")
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 71644dff Thu Dec 01 18:37:15 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: add block_age-based extent cache

This patch introduces a runtime hot/cold data separation method
for f2fs, in order to improve the accuracy for data temperature
classification, reduce the garbage collection overhead after
long-term data updates.

Enhanced hot/cold data separation can record data block update
frequency as "age" of the extent per inode, and take use of the age
info to indicate better temperature type for data block allocation:
- It records total data blocks allocated since mount;
- When file extent has been updated, it calculate the count of data
blocks allocated since last update as the age of the extent;
- Before the data block allocated, it searches for the age info and
chooses the suitable segment for allocation.

Test and result:
- Prepare: create about 30000 files
* 3% for cold files (with cold file extension like .apk, from 3M to 10M)
* 50% for warm files (with random file extension like .FcDxq, from 1K
to 4M)
* 47% for hot files (with hot file extension like .db, from 1K to 256K)
- create(5%)/random update(90%)/delete(5%) the files
* total write amount is about 70G
* fsync will be called for .db files, and buffered write will be used
for other files

The storage of test device is large enough(128G) so that it will not
switch to SSR mode during the test.

Benefit: dirty segment count increment reduce about 14%
- before: Dirty +21110
- after: Dirty +18286

Signed-off-by: qixiaoyu1 <qixiaoyu1@xiaomi.com>
Signed-off-by: xiongping1 <xiongping1@xiaomi.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 71644dff Thu Dec 01 18:37:15 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: add block_age-based extent cache

This patch introduces a runtime hot/cold data separation method
for f2fs, in order to improve the accuracy for data temperature
classification, reduce the garbage collection overhead after
long-term data updates.

Enhanced hot/cold data separation can record data block update
frequency as "age" of the extent per inode, and take use of the age
info to indicate better temperature type for data block allocation:
- It records total data blocks allocated since mount;
- When file extent has been updated, it calculate the count of data
blocks allocated since last update as the age of the extent;
- Before the data block allocated, it searches for the age info and
chooses the suitable segment for allocation.

Test and result:
- Prepare: create about 30000 files
* 3% for cold files (with cold file extension like .apk, from 3M to 10M)
* 50% for warm files (with random file extension like .FcDxq, from 1K
to 4M)
* 47% for hot files (with hot file extension like .db, from 1K to 256K)
- create(5%)/random update(90%)/delete(5%) the files
* total write amount is about 70G
* fsync will be called for .db files, and buffered write will be used
for other files

The storage of test device is large enough(128G) so that it will not
switch to SSR mode during the test.

Benefit: dirty segment count increment reduce about 14%
- before: Dirty +21110
- after: Dirty +18286

Signed-off-by: qixiaoyu1 <qixiaoyu1@xiaomi.com>
Signed-off-by: xiongping1 <xiongping1@xiaomi.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff fcc2d8cc Tue Sep 13 01:48:12 MDT 2022 Chao Yu <chao@kernel.org> f2fs: fix to detect corrupted meta ino

It is possible that ino of dirent or orphan inode is corrupted in a
fuzzed image, occasionally, if corrupted ino is equal to meta ino:
meta_ino, node_ino or compress_ino, caller of f2fs_iget() from below
call paths will get meta inode directly, it's not allowed, let's
add sanity check to detect such cases.

case #1
- recover_dentry
- __f2fs_find_entry
- f2fs_iget_retry

case #2
- recover_orphan_inode
- f2fs_iget_retry

Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 677a82b4 Wed May 18 06:28:41 MDT 2022 Chao Yu <chao@kernel.org> f2fs: fix to do sanity check for inline inode

Yanming reported a kernel bug in Bugzilla kernel [1], which can be
reproduced. The bug message is:

The kernel message is shown below:

kernel BUG at fs/inode.c:611!
Call Trace:
evict+0x282/0x4e0
__dentry_kill+0x2b2/0x4d0
dput+0x2dd/0x720
do_renameat2+0x596/0x970
__x64_sys_rename+0x78/0x90
do_syscall_64+0x3b/0x90

[1] https://bugzilla.kernel.org/show_bug.cgi?id=215895

The bug is due to fuzzed inode has both inline_data and encrypted flags.
During f2fs_evict_inode(), as the inode was deleted by rename(), it
will cause inline data conversion due to conflicting flags. The page
cache will be polluted and the panic will be triggered in clear_inode().

Try fixing the bug by doing more sanity checks for inline data inode in
sanity_check_inode().

Cc: stable@vger.kernel.org
Reported-by: Ming Yan <yanming@tju.edu.cn>
Signed-off-by: Chao Yu <chao.yu@oppo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 677a82b4 Wed May 18 06:28:41 MDT 2022 Chao Yu <chao@kernel.org> f2fs: fix to do sanity check for inline inode

Yanming reported a kernel bug in Bugzilla kernel [1], which can be
reproduced. The bug message is:

The kernel message is shown below:

kernel BUG at fs/inode.c:611!
Call Trace:
evict+0x282/0x4e0
__dentry_kill+0x2b2/0x4d0
dput+0x2dd/0x720
do_renameat2+0x596/0x970
__x64_sys_rename+0x78/0x90
do_syscall_64+0x3b/0x90

[1] https://bugzilla.kernel.org/show_bug.cgi?id=215895

The bug is due to fuzzed inode has both inline_data and encrypted flags.
During f2fs_evict_inode(), as the inode was deleted by rename(), it
will cause inline data conversion due to conflicting flags. The page
cache will be polluted and the panic will be triggered in clear_inode().

Try fixing the bug by doing more sanity checks for inline data inode in
sanity_check_inode().

Cc: stable@vger.kernel.org
Reported-by: Ming Yan <yanming@tju.edu.cn>
Signed-off-by: Chao Yu <chao.yu@oppo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 2fef99b8 Fri Feb 11 19:56:46 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: fix missing free nid in f2fs_handle_failed_inode

This patch fixes xfstests/generic/475 failure.

[ 293.680694] F2FS-fs (dm-1): May loss orphan inode, run fsck to fix.
[ 293.685358] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691527] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691764] sh (7615): drop_caches: 3
[ 293.691819] sh (7616): drop_caches: 3
[ 293.694017] Buffer I/O error on dev dm-1, logical block 1, async page read
[ 293.695659] sh (7618): drop_caches: 3
[ 293.696979] sh (7617): drop_caches: 3
[ 293.700290] sh (7623): drop_caches: 3
[ 293.708621] sh (7626): drop_caches: 3
[ 293.711386] sh (7628): drop_caches: 3
[ 293.711825] sh (7627): drop_caches: 3
[ 293.716738] sh (7630): drop_caches: 3
[ 293.719613] sh (7632): drop_caches: 3
[ 293.720971] sh (7633): drop_caches: 3
[ 293.727741] sh (7634): drop_caches: 3
[ 293.730783] sh (7636): drop_caches: 3
[ 293.732681] sh (7635): drop_caches: 3
[ 293.732988] sh (7637): drop_caches: 3
[ 293.738836] sh (7639): drop_caches: 3
[ 293.740568] sh (7641): drop_caches: 3
[ 293.743053] sh (7640): drop_caches: 3
[ 293.821889] ------------[ cut here ]------------
[ 293.824654] kernel BUG at fs/f2fs/node.c:3334!
[ 293.826226] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 293.828713] CPU: 0 PID: 7653 Comm: umount Tainted: G OE 5.17.0-rc1-custom #1
[ 293.830946] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[ 293.832526] RIP: 0010:f2fs_destroy_node_manager+0x33f/0x350 [f2fs]
[ 293.833905] Code: e8 d6 3d f9 f9 48 8b 45 d0 65 48 2b 04 25 28 00 00 00 75 1a 48 81 c4 28 03 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 0b
[ 293.837783] RSP: 0018:ffffb04ec31e7a20 EFLAGS: 00010202
[ 293.839062] RAX: 0000000000000001 RBX: ffff9df947db2eb8 RCX: 0000000080aa0072
[ 293.840666] RDX: 0000000000000000 RSI: ffffe86c0432a140 RDI: ffffffffc0b72a21
[ 293.842261] RBP: ffffb04ec31e7d70 R08: ffff9df94ca85780 R09: 0000000080aa0072
[ 293.843909] R10: ffff9df94ca85700 R11: ffff9df94e1ccf58 R12: ffff9df947db2e00
[ 293.845594] R13: ffff9df947db2ed0 R14: ffff9df947db2eb8 R15: ffff9df947db2eb8
[ 293.847855] FS: 00007f5a97379800(0000) GS:ffff9dfa77c00000(0000) knlGS:0000000000000000
[ 293.850647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 293.852940] CR2: 00007f5a97528730 CR3: 000000010bc76005 CR4: 0000000000370ef0
[ 293.854680] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 293.856423] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 293.858380] Call Trace:
[ 293.859302] <TASK>
[ 293.860311] ? ttwu_do_wakeup+0x1c/0x170
[ 293.861800] ? ttwu_do_activate+0x6d/0xb0
[ 293.863057] ? _raw_spin_unlock_irqrestore+0x29/0x40
[ 293.864411] ? try_to_wake_up+0x9d/0x5e0
[ 293.865618] ? debug_smp_processor_id+0x17/0x20
[ 293.866934] ? debug_smp_processor_id+0x17/0x20
[ 293.868223] ? free_unref_page+0xbf/0x120
[ 293.869470] ? __free_slab+0xcb/0x1c0
[ 293.870614] ? preempt_count_add+0x7a/0xc0
[ 293.871811] ? __slab_free+0xa0/0x2d0
[ 293.872918] ? __wake_up_common_lock+0x8a/0xc0
[ 293.874186] ? __slab_free+0xa0/0x2d0
[ 293.875305] ? free_inode_nonrcu+0x20/0x20
[ 293.876466] ? free_inode_nonrcu+0x20/0x20
[ 293.877650] ? debug_smp_processor_id+0x17/0x20
[ 293.878949] ? call_rcu+0x11a/0x240
[ 293.880060] ? f2fs_destroy_stats+0x59/0x60 [f2fs]
[ 293.881437] ? kfree+0x1fe/0x230
[ 293.882674] f2fs_put_super+0x160/0x390 [f2fs]
[ 293.883978] generic_shutdown_super+0x7a/0x120
[ 293.885274] kill_block_super+0x27/0x50
[ 293.886496] kill_f2fs_super+0x7f/0x100 [f2fs]
[ 293.887806] deactivate_locked_super+0x35/0xa0
[ 293.889271] deactivate_super+0x40/0x50
[ 293.890513] cleanup_mnt+0x139/0x190
[ 293.891689] __cleanup_mnt+0x12/0x20
[ 293.892850] task_work_run+0x64/0xa0
[ 293.894035] exit_to_user_mode_prepare+0x1b7/0x1c0
[ 293.895409] syscall_exit_to_user_mode+0x27/0x50
[ 293.896872] do_syscall_64+0x48/0xc0
[ 293.898090] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 293.899517] RIP: 0033:0x7f5a975cd25b

Fixes: 7735730d39d7 ("f2fs: fix to propagate error from __get_meta_page()")
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 2fef99b8 Fri Feb 11 19:56:46 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: fix missing free nid in f2fs_handle_failed_inode

This patch fixes xfstests/generic/475 failure.

[ 293.680694] F2FS-fs (dm-1): May loss orphan inode, run fsck to fix.
[ 293.685358] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691527] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691764] sh (7615): drop_caches: 3
[ 293.691819] sh (7616): drop_caches: 3
[ 293.694017] Buffer I/O error on dev dm-1, logical block 1, async page read
[ 293.695659] sh (7618): drop_caches: 3
[ 293.696979] sh (7617): drop_caches: 3
[ 293.700290] sh (7623): drop_caches: 3
[ 293.708621] sh (7626): drop_caches: 3
[ 293.711386] sh (7628): drop_caches: 3
[ 293.711825] sh (7627): drop_caches: 3
[ 293.716738] sh (7630): drop_caches: 3
[ 293.719613] sh (7632): drop_caches: 3
[ 293.720971] sh (7633): drop_caches: 3
[ 293.727741] sh (7634): drop_caches: 3
[ 293.730783] sh (7636): drop_caches: 3
[ 293.732681] sh (7635): drop_caches: 3
[ 293.732988] sh (7637): drop_caches: 3
[ 293.738836] sh (7639): drop_caches: 3
[ 293.740568] sh (7641): drop_caches: 3
[ 293.743053] sh (7640): drop_caches: 3
[ 293.821889] ------------[ cut here ]------------
[ 293.824654] kernel BUG at fs/f2fs/node.c:3334!
[ 293.826226] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 293.828713] CPU: 0 PID: 7653 Comm: umount Tainted: G OE 5.17.0-rc1-custom #1
[ 293.830946] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[ 293.832526] RIP: 0010:f2fs_destroy_node_manager+0x33f/0x350 [f2fs]
[ 293.833905] Code: e8 d6 3d f9 f9 48 8b 45 d0 65 48 2b 04 25 28 00 00 00 75 1a 48 81 c4 28 03 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 0b
[ 293.837783] RSP: 0018:ffffb04ec31e7a20 EFLAGS: 00010202
[ 293.839062] RAX: 0000000000000001 RBX: ffff9df947db2eb8 RCX: 0000000080aa0072
[ 293.840666] RDX: 0000000000000000 RSI: ffffe86c0432a140 RDI: ffffffffc0b72a21
[ 293.842261] RBP: ffffb04ec31e7d70 R08: ffff9df94ca85780 R09: 0000000080aa0072
[ 293.843909] R10: ffff9df94ca85700 R11: ffff9df94e1ccf58 R12: ffff9df947db2e00
[ 293.845594] R13: ffff9df947db2ed0 R14: ffff9df947db2eb8 R15: ffff9df947db2eb8
[ 293.847855] FS: 00007f5a97379800(0000) GS:ffff9dfa77c00000(0000) knlGS:0000000000000000
[ 293.850647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 293.852940] CR2: 00007f5a97528730 CR3: 000000010bc76005 CR4: 0000000000370ef0
[ 293.854680] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 293.856423] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 293.858380] Call Trace:
[ 293.859302] <TASK>
[ 293.860311] ? ttwu_do_wakeup+0x1c/0x170
[ 293.861800] ? ttwu_do_activate+0x6d/0xb0
[ 293.863057] ? _raw_spin_unlock_irqrestore+0x29/0x40
[ 293.864411] ? try_to_wake_up+0x9d/0x5e0
[ 293.865618] ? debug_smp_processor_id+0x17/0x20
[ 293.866934] ? debug_smp_processor_id+0x17/0x20
[ 293.868223] ? free_unref_page+0xbf/0x120
[ 293.869470] ? __free_slab+0xcb/0x1c0
[ 293.870614] ? preempt_count_add+0x7a/0xc0
[ 293.871811] ? __slab_free+0xa0/0x2d0
[ 293.872918] ? __wake_up_common_lock+0x8a/0xc0
[ 293.874186] ? __slab_free+0xa0/0x2d0
[ 293.875305] ? free_inode_nonrcu+0x20/0x20
[ 293.876466] ? free_inode_nonrcu+0x20/0x20
[ 293.877650] ? debug_smp_processor_id+0x17/0x20
[ 293.878949] ? call_rcu+0x11a/0x240
[ 293.880060] ? f2fs_destroy_stats+0x59/0x60 [f2fs]
[ 293.881437] ? kfree+0x1fe/0x230
[ 293.882674] f2fs_put_super+0x160/0x390 [f2fs]
[ 293.883978] generic_shutdown_super+0x7a/0x120
[ 293.885274] kill_block_super+0x27/0x50
[ 293.886496] kill_f2fs_super+0x7f/0x100 [f2fs]
[ 293.887806] deactivate_locked_super+0x35/0xa0
[ 293.889271] deactivate_super+0x40/0x50
[ 293.890513] cleanup_mnt+0x139/0x190
[ 293.891689] __cleanup_mnt+0x12/0x20
[ 293.892850] task_work_run+0x64/0xa0
[ 293.894035] exit_to_user_mode_prepare+0x1b7/0x1c0
[ 293.895409] syscall_exit_to_user_mode+0x27/0x50
[ 293.896872] do_syscall_64+0x48/0xc0
[ 293.898090] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 293.899517] RIP: 0033:0x7f5a975cd25b

Fixes: 7735730d39d7 ("f2fs: fix to propagate error from __get_meta_page()")
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 2fef99b8 Fri Feb 11 19:56:46 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: fix missing free nid in f2fs_handle_failed_inode

This patch fixes xfstests/generic/475 failure.

[ 293.680694] F2FS-fs (dm-1): May loss orphan inode, run fsck to fix.
[ 293.685358] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691527] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691764] sh (7615): drop_caches: 3
[ 293.691819] sh (7616): drop_caches: 3
[ 293.694017] Buffer I/O error on dev dm-1, logical block 1, async page read
[ 293.695659] sh (7618): drop_caches: 3
[ 293.696979] sh (7617): drop_caches: 3
[ 293.700290] sh (7623): drop_caches: 3
[ 293.708621] sh (7626): drop_caches: 3
[ 293.711386] sh (7628): drop_caches: 3
[ 293.711825] sh (7627): drop_caches: 3
[ 293.716738] sh (7630): drop_caches: 3
[ 293.719613] sh (7632): drop_caches: 3
[ 293.720971] sh (7633): drop_caches: 3
[ 293.727741] sh (7634): drop_caches: 3
[ 293.730783] sh (7636): drop_caches: 3
[ 293.732681] sh (7635): drop_caches: 3
[ 293.732988] sh (7637): drop_caches: 3
[ 293.738836] sh (7639): drop_caches: 3
[ 293.740568] sh (7641): drop_caches: 3
[ 293.743053] sh (7640): drop_caches: 3
[ 293.821889] ------------[ cut here ]------------
[ 293.824654] kernel BUG at fs/f2fs/node.c:3334!
[ 293.826226] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 293.828713] CPU: 0 PID: 7653 Comm: umount Tainted: G OE 5.17.0-rc1-custom #1
[ 293.830946] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[ 293.832526] RIP: 0010:f2fs_destroy_node_manager+0x33f/0x350 [f2fs]
[ 293.833905] Code: e8 d6 3d f9 f9 48 8b 45 d0 65 48 2b 04 25 28 00 00 00 75 1a 48 81 c4 28 03 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 0b
[ 293.837783] RSP: 0018:ffffb04ec31e7a20 EFLAGS: 00010202
[ 293.839062] RAX: 0000000000000001 RBX: ffff9df947db2eb8 RCX: 0000000080aa0072
[ 293.840666] RDX: 0000000000000000 RSI: ffffe86c0432a140 RDI: ffffffffc0b72a21
[ 293.842261] RBP: ffffb04ec31e7d70 R08: ffff9df94ca85780 R09: 0000000080aa0072
[ 293.843909] R10: ffff9df94ca85700 R11: ffff9df94e1ccf58 R12: ffff9df947db2e00
[ 293.845594] R13: ffff9df947db2ed0 R14: ffff9df947db2eb8 R15: ffff9df947db2eb8
[ 293.847855] FS: 00007f5a97379800(0000) GS:ffff9dfa77c00000(0000) knlGS:0000000000000000
[ 293.850647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 293.852940] CR2: 00007f5a97528730 CR3: 000000010bc76005 CR4: 0000000000370ef0
[ 293.854680] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 293.856423] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 293.858380] Call Trace:
[ 293.859302] <TASK>
[ 293.860311] ? ttwu_do_wakeup+0x1c/0x170
[ 293.861800] ? ttwu_do_activate+0x6d/0xb0
[ 293.863057] ? _raw_spin_unlock_irqrestore+0x29/0x40
[ 293.864411] ? try_to_wake_up+0x9d/0x5e0
[ 293.865618] ? debug_smp_processor_id+0x17/0x20
[ 293.866934] ? debug_smp_processor_id+0x17/0x20
[ 293.868223] ? free_unref_page+0xbf/0x120
[ 293.869470] ? __free_slab+0xcb/0x1c0
[ 293.870614] ? preempt_count_add+0x7a/0xc0
[ 293.871811] ? __slab_free+0xa0/0x2d0
[ 293.872918] ? __wake_up_common_lock+0x8a/0xc0
[ 293.874186] ? __slab_free+0xa0/0x2d0
[ 293.875305] ? free_inode_nonrcu+0x20/0x20
[ 293.876466] ? free_inode_nonrcu+0x20/0x20
[ 293.877650] ? debug_smp_processor_id+0x17/0x20
[ 293.878949] ? call_rcu+0x11a/0x240
[ 293.880060] ? f2fs_destroy_stats+0x59/0x60 [f2fs]
[ 293.881437] ? kfree+0x1fe/0x230
[ 293.882674] f2fs_put_super+0x160/0x390 [f2fs]
[ 293.883978] generic_shutdown_super+0x7a/0x120
[ 293.885274] kill_block_super+0x27/0x50
[ 293.886496] kill_f2fs_super+0x7f/0x100 [f2fs]
[ 293.887806] deactivate_locked_super+0x35/0xa0
[ 293.889271] deactivate_super+0x40/0x50
[ 293.890513] cleanup_mnt+0x139/0x190
[ 293.891689] __cleanup_mnt+0x12/0x20
[ 293.892850] task_work_run+0x64/0xa0
[ 293.894035] exit_to_user_mode_prepare+0x1b7/0x1c0
[ 293.895409] syscall_exit_to_user_mode+0x27/0x50
[ 293.896872] do_syscall_64+0x48/0xc0
[ 293.898090] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 293.899517] RIP: 0033:0x7f5a975cd25b

Fixes: 7735730d39d7 ("f2fs: fix to propagate error from __get_meta_page()")
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 2fef99b8 Fri Feb 11 19:56:46 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: fix missing free nid in f2fs_handle_failed_inode

This patch fixes xfstests/generic/475 failure.

[ 293.680694] F2FS-fs (dm-1): May loss orphan inode, run fsck to fix.
[ 293.685358] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691527] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691764] sh (7615): drop_caches: 3
[ 293.691819] sh (7616): drop_caches: 3
[ 293.694017] Buffer I/O error on dev dm-1, logical block 1, async page read
[ 293.695659] sh (7618): drop_caches: 3
[ 293.696979] sh (7617): drop_caches: 3
[ 293.700290] sh (7623): drop_caches: 3
[ 293.708621] sh (7626): drop_caches: 3
[ 293.711386] sh (7628): drop_caches: 3
[ 293.711825] sh (7627): drop_caches: 3
[ 293.716738] sh (7630): drop_caches: 3
[ 293.719613] sh (7632): drop_caches: 3
[ 293.720971] sh (7633): drop_caches: 3
[ 293.727741] sh (7634): drop_caches: 3
[ 293.730783] sh (7636): drop_caches: 3
[ 293.732681] sh (7635): drop_caches: 3
[ 293.732988] sh (7637): drop_caches: 3
[ 293.738836] sh (7639): drop_caches: 3
[ 293.740568] sh (7641): drop_caches: 3
[ 293.743053] sh (7640): drop_caches: 3
[ 293.821889] ------------[ cut here ]------------
[ 293.824654] kernel BUG at fs/f2fs/node.c:3334!
[ 293.826226] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 293.828713] CPU: 0 PID: 7653 Comm: umount Tainted: G OE 5.17.0-rc1-custom #1
[ 293.830946] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[ 293.832526] RIP: 0010:f2fs_destroy_node_manager+0x33f/0x350 [f2fs]
[ 293.833905] Code: e8 d6 3d f9 f9 48 8b 45 d0 65 48 2b 04 25 28 00 00 00 75 1a 48 81 c4 28 03 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 0b
[ 293.837783] RSP: 0018:ffffb04ec31e7a20 EFLAGS: 00010202
[ 293.839062] RAX: 0000000000000001 RBX: ffff9df947db2eb8 RCX: 0000000080aa0072
[ 293.840666] RDX: 0000000000000000 RSI: ffffe86c0432a140 RDI: ffffffffc0b72a21
[ 293.842261] RBP: ffffb04ec31e7d70 R08: ffff9df94ca85780 R09: 0000000080aa0072
[ 293.843909] R10: ffff9df94ca85700 R11: ffff9df94e1ccf58 R12: ffff9df947db2e00
[ 293.845594] R13: ffff9df947db2ed0 R14: ffff9df947db2eb8 R15: ffff9df947db2eb8
[ 293.847855] FS: 00007f5a97379800(0000) GS:ffff9dfa77c00000(0000) knlGS:0000000000000000
[ 293.850647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 293.852940] CR2: 00007f5a97528730 CR3: 000000010bc76005 CR4: 0000000000370ef0
[ 293.854680] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 293.856423] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 293.858380] Call Trace:
[ 293.859302] <TASK>
[ 293.860311] ? ttwu_do_wakeup+0x1c/0x170
[ 293.861800] ? ttwu_do_activate+0x6d/0xb0
[ 293.863057] ? _raw_spin_unlock_irqrestore+0x29/0x40
[ 293.864411] ? try_to_wake_up+0x9d/0x5e0
[ 293.865618] ? debug_smp_processor_id+0x17/0x20
[ 293.866934] ? debug_smp_processor_id+0x17/0x20
[ 293.868223] ? free_unref_page+0xbf/0x120
[ 293.869470] ? __free_slab+0xcb/0x1c0
[ 293.870614] ? preempt_count_add+0x7a/0xc0
[ 293.871811] ? __slab_free+0xa0/0x2d0
[ 293.872918] ? __wake_up_common_lock+0x8a/0xc0
[ 293.874186] ? __slab_free+0xa0/0x2d0
[ 293.875305] ? free_inode_nonrcu+0x20/0x20
[ 293.876466] ? free_inode_nonrcu+0x20/0x20
[ 293.877650] ? debug_smp_processor_id+0x17/0x20
[ 293.878949] ? call_rcu+0x11a/0x240
[ 293.880060] ? f2fs_destroy_stats+0x59/0x60 [f2fs]
[ 293.881437] ? kfree+0x1fe/0x230
[ 293.882674] f2fs_put_super+0x160/0x390 [f2fs]
[ 293.883978] generic_shutdown_super+0x7a/0x120
[ 293.885274] kill_block_super+0x27/0x50
[ 293.886496] kill_f2fs_super+0x7f/0x100 [f2fs]
[ 293.887806] deactivate_locked_super+0x35/0xa0
[ 293.889271] deactivate_super+0x40/0x50
[ 293.890513] cleanup_mnt+0x139/0x190
[ 293.891689] __cleanup_mnt+0x12/0x20
[ 293.892850] task_work_run+0x64/0xa0
[ 293.894035] exit_to_user_mode_prepare+0x1b7/0x1c0
[ 293.895409] syscall_exit_to_user_mode+0x27/0x50
[ 293.896872] do_syscall_64+0x48/0xc0
[ 293.898090] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 293.899517] RIP: 0033:0x7f5a975cd25b

Fixes: 7735730d39d7 ("f2fs: fix to propagate error from __get_meta_page()")
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 2fef99b8 Fri Feb 11 19:56:46 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: fix missing free nid in f2fs_handle_failed_inode

This patch fixes xfstests/generic/475 failure.

[ 293.680694] F2FS-fs (dm-1): May loss orphan inode, run fsck to fix.
[ 293.685358] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691527] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691764] sh (7615): drop_caches: 3
[ 293.691819] sh (7616): drop_caches: 3
[ 293.694017] Buffer I/O error on dev dm-1, logical block 1, async page read
[ 293.695659] sh (7618): drop_caches: 3
[ 293.696979] sh (7617): drop_caches: 3
[ 293.700290] sh (7623): drop_caches: 3
[ 293.708621] sh (7626): drop_caches: 3
[ 293.711386] sh (7628): drop_caches: 3
[ 293.711825] sh (7627): drop_caches: 3
[ 293.716738] sh (7630): drop_caches: 3
[ 293.719613] sh (7632): drop_caches: 3
[ 293.720971] sh (7633): drop_caches: 3
[ 293.727741] sh (7634): drop_caches: 3
[ 293.730783] sh (7636): drop_caches: 3
[ 293.732681] sh (7635): drop_caches: 3
[ 293.732988] sh (7637): drop_caches: 3
[ 293.738836] sh (7639): drop_caches: 3
[ 293.740568] sh (7641): drop_caches: 3
[ 293.743053] sh (7640): drop_caches: 3
[ 293.821889] ------------[ cut here ]------------
[ 293.824654] kernel BUG at fs/f2fs/node.c:3334!
[ 293.826226] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 293.828713] CPU: 0 PID: 7653 Comm: umount Tainted: G OE 5.17.0-rc1-custom #1
[ 293.830946] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[ 293.832526] RIP: 0010:f2fs_destroy_node_manager+0x33f/0x350 [f2fs]
[ 293.833905] Code: e8 d6 3d f9 f9 48 8b 45 d0 65 48 2b 04 25 28 00 00 00 75 1a 48 81 c4 28 03 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 0b
[ 293.837783] RSP: 0018:ffffb04ec31e7a20 EFLAGS: 00010202
[ 293.839062] RAX: 0000000000000001 RBX: ffff9df947db2eb8 RCX: 0000000080aa0072
[ 293.840666] RDX: 0000000000000000 RSI: ffffe86c0432a140 RDI: ffffffffc0b72a21
[ 293.842261] RBP: ffffb04ec31e7d70 R08: ffff9df94ca85780 R09: 0000000080aa0072
[ 293.843909] R10: ffff9df94ca85700 R11: ffff9df94e1ccf58 R12: ffff9df947db2e00
[ 293.845594] R13: ffff9df947db2ed0 R14: ffff9df947db2eb8 R15: ffff9df947db2eb8
[ 293.847855] FS: 00007f5a97379800(0000) GS:ffff9dfa77c00000(0000) knlGS:0000000000000000
[ 293.850647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 293.852940] CR2: 00007f5a97528730 CR3: 000000010bc76005 CR4: 0000000000370ef0
[ 293.854680] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 293.856423] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 293.858380] Call Trace:
[ 293.859302] <TASK>
[ 293.860311] ? ttwu_do_wakeup+0x1c/0x170
[ 293.861800] ? ttwu_do_activate+0x6d/0xb0
[ 293.863057] ? _raw_spin_unlock_irqrestore+0x29/0x40
[ 293.864411] ? try_to_wake_up+0x9d/0x5e0
[ 293.865618] ? debug_smp_processor_id+0x17/0x20
[ 293.866934] ? debug_smp_processor_id+0x17/0x20
[ 293.868223] ? free_unref_page+0xbf/0x120
[ 293.869470] ? __free_slab+0xcb/0x1c0
[ 293.870614] ? preempt_count_add+0x7a/0xc0
[ 293.871811] ? __slab_free+0xa0/0x2d0
[ 293.872918] ? __wake_up_common_lock+0x8a/0xc0
[ 293.874186] ? __slab_free+0xa0/0x2d0
[ 293.875305] ? free_inode_nonrcu+0x20/0x20
[ 293.876466] ? free_inode_nonrcu+0x20/0x20
[ 293.877650] ? debug_smp_processor_id+0x17/0x20
[ 293.878949] ? call_rcu+0x11a/0x240
[ 293.880060] ? f2fs_destroy_stats+0x59/0x60 [f2fs]
[ 293.881437] ? kfree+0x1fe/0x230
[ 293.882674] f2fs_put_super+0x160/0x390 [f2fs]
[ 293.883978] generic_shutdown_super+0x7a/0x120
[ 293.885274] kill_block_super+0x27/0x50
[ 293.886496] kill_f2fs_super+0x7f/0x100 [f2fs]
[ 293.887806] deactivate_locked_super+0x35/0xa0
[ 293.889271] deactivate_super+0x40/0x50
[ 293.890513] cleanup_mnt+0x139/0x190
[ 293.891689] __cleanup_mnt+0x12/0x20
[ 293.892850] task_work_run+0x64/0xa0
[ 293.894035] exit_to_user_mode_prepare+0x1b7/0x1c0
[ 293.895409] syscall_exit_to_user_mode+0x27/0x50
[ 293.896872] do_syscall_64+0x48/0xc0
[ 293.898090] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 293.899517] RIP: 0033:0x7f5a975cd25b

Fixes: 7735730d39d7 ("f2fs: fix to propagate error from __get_meta_page()")
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 2fef99b8 Fri Feb 11 19:56:46 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: fix missing free nid in f2fs_handle_failed_inode

This patch fixes xfstests/generic/475 failure.

[ 293.680694] F2FS-fs (dm-1): May loss orphan inode, run fsck to fix.
[ 293.685358] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691527] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691764] sh (7615): drop_caches: 3
[ 293.691819] sh (7616): drop_caches: 3
[ 293.694017] Buffer I/O error on dev dm-1, logical block 1, async page read
[ 293.695659] sh (7618): drop_caches: 3
[ 293.696979] sh (7617): drop_caches: 3
[ 293.700290] sh (7623): drop_caches: 3
[ 293.708621] sh (7626): drop_caches: 3
[ 293.711386] sh (7628): drop_caches: 3
[ 293.711825] sh (7627): drop_caches: 3
[ 293.716738] sh (7630): drop_caches: 3
[ 293.719613] sh (7632): drop_caches: 3
[ 293.720971] sh (7633): drop_caches: 3
[ 293.727741] sh (7634): drop_caches: 3
[ 293.730783] sh (7636): drop_caches: 3
[ 293.732681] sh (7635): drop_caches: 3
[ 293.732988] sh (7637): drop_caches: 3
[ 293.738836] sh (7639): drop_caches: 3
[ 293.740568] sh (7641): drop_caches: 3
[ 293.743053] sh (7640): drop_caches: 3
[ 293.821889] ------------[ cut here ]------------
[ 293.824654] kernel BUG at fs/f2fs/node.c:3334!
[ 293.826226] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 293.828713] CPU: 0 PID: 7653 Comm: umount Tainted: G OE 5.17.0-rc1-custom #1
[ 293.830946] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[ 293.832526] RIP: 0010:f2fs_destroy_node_manager+0x33f/0x350 [f2fs]
[ 293.833905] Code: e8 d6 3d f9 f9 48 8b 45 d0 65 48 2b 04 25 28 00 00 00 75 1a 48 81 c4 28 03 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 0b
[ 293.837783] RSP: 0018:ffffb04ec31e7a20 EFLAGS: 00010202
[ 293.839062] RAX: 0000000000000001 RBX: ffff9df947db2eb8 RCX: 0000000080aa0072
[ 293.840666] RDX: 0000000000000000 RSI: ffffe86c0432a140 RDI: ffffffffc0b72a21
[ 293.842261] RBP: ffffb04ec31e7d70 R08: ffff9df94ca85780 R09: 0000000080aa0072
[ 293.843909] R10: ffff9df94ca85700 R11: ffff9df94e1ccf58 R12: ffff9df947db2e00
[ 293.845594] R13: ffff9df947db2ed0 R14: ffff9df947db2eb8 R15: ffff9df947db2eb8
[ 293.847855] FS: 00007f5a97379800(0000) GS:ffff9dfa77c00000(0000) knlGS:0000000000000000
[ 293.850647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 293.852940] CR2: 00007f5a97528730 CR3: 000000010bc76005 CR4: 0000000000370ef0
[ 293.854680] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 293.856423] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 293.858380] Call Trace:
[ 293.859302] <TASK>
[ 293.860311] ? ttwu_do_wakeup+0x1c/0x170
[ 293.861800] ? ttwu_do_activate+0x6d/0xb0
[ 293.863057] ? _raw_spin_unlock_irqrestore+0x29/0x40
[ 293.864411] ? try_to_wake_up+0x9d/0x5e0
[ 293.865618] ? debug_smp_processor_id+0x17/0x20
[ 293.866934] ? debug_smp_processor_id+0x17/0x20
[ 293.868223] ? free_unref_page+0xbf/0x120
[ 293.869470] ? __free_slab+0xcb/0x1c0
[ 293.870614] ? preempt_count_add+0x7a/0xc0
[ 293.871811] ? __slab_free+0xa0/0x2d0
[ 293.872918] ? __wake_up_common_lock+0x8a/0xc0
[ 293.874186] ? __slab_free+0xa0/0x2d0
[ 293.875305] ? free_inode_nonrcu+0x20/0x20
[ 293.876466] ? free_inode_nonrcu+0x20/0x20
[ 293.877650] ? debug_smp_processor_id+0x17/0x20
[ 293.878949] ? call_rcu+0x11a/0x240
[ 293.880060] ? f2fs_destroy_stats+0x59/0x60 [f2fs]
[ 293.881437] ? kfree+0x1fe/0x230
[ 293.882674] f2fs_put_super+0x160/0x390 [f2fs]
[ 293.883978] generic_shutdown_super+0x7a/0x120
[ 293.885274] kill_block_super+0x27/0x50
[ 293.886496] kill_f2fs_super+0x7f/0x100 [f2fs]
[ 293.887806] deactivate_locked_super+0x35/0xa0
[ 293.889271] deactivate_super+0x40/0x50
[ 293.890513] cleanup_mnt+0x139/0x190
[ 293.891689] __cleanup_mnt+0x12/0x20
[ 293.892850] task_work_run+0x64/0xa0
[ 293.894035] exit_to_user_mode_prepare+0x1b7/0x1c0
[ 293.895409] syscall_exit_to_user_mode+0x27/0x50
[ 293.896872] do_syscall_64+0x48/0xc0
[ 293.898090] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 293.899517] RIP: 0033:0x7f5a975cd25b

Fixes: 7735730d39d7 ("f2fs: fix to propagate error from __get_meta_page()")
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 2fef99b8 Fri Feb 11 19:56:46 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: fix missing free nid in f2fs_handle_failed_inode

This patch fixes xfstests/generic/475 failure.

[ 293.680694] F2FS-fs (dm-1): May loss orphan inode, run fsck to fix.
[ 293.685358] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691527] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691764] sh (7615): drop_caches: 3
[ 293.691819] sh (7616): drop_caches: 3
[ 293.694017] Buffer I/O error on dev dm-1, logical block 1, async page read
[ 293.695659] sh (7618): drop_caches: 3
[ 293.696979] sh (7617): drop_caches: 3
[ 293.700290] sh (7623): drop_caches: 3
[ 293.708621] sh (7626): drop_caches: 3
[ 293.711386] sh (7628): drop_caches: 3
[ 293.711825] sh (7627): drop_caches: 3
[ 293.716738] sh (7630): drop_caches: 3
[ 293.719613] sh (7632): drop_caches: 3
[ 293.720971] sh (7633): drop_caches: 3
[ 293.727741] sh (7634): drop_caches: 3
[ 293.730783] sh (7636): drop_caches: 3
[ 293.732681] sh (7635): drop_caches: 3
[ 293.732988] sh (7637): drop_caches: 3
[ 293.738836] sh (7639): drop_caches: 3
[ 293.740568] sh (7641): drop_caches: 3
[ 293.743053] sh (7640): drop_caches: 3
[ 293.821889] ------------[ cut here ]------------
[ 293.824654] kernel BUG at fs/f2fs/node.c:3334!
[ 293.826226] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 293.828713] CPU: 0 PID: 7653 Comm: umount Tainted: G OE 5.17.0-rc1-custom #1
[ 293.830946] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[ 293.832526] RIP: 0010:f2fs_destroy_node_manager+0x33f/0x350 [f2fs]
[ 293.833905] Code: e8 d6 3d f9 f9 48 8b 45 d0 65 48 2b 04 25 28 00 00 00 75 1a 48 81 c4 28 03 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 0b
[ 293.837783] RSP: 0018:ffffb04ec31e7a20 EFLAGS: 00010202
[ 293.839062] RAX: 0000000000000001 RBX: ffff9df947db2eb8 RCX: 0000000080aa0072
[ 293.840666] RDX: 0000000000000000 RSI: ffffe86c0432a140 RDI: ffffffffc0b72a21
[ 293.842261] RBP: ffffb04ec31e7d70 R08: ffff9df94ca85780 R09: 0000000080aa0072
[ 293.843909] R10: ffff9df94ca85700 R11: ffff9df94e1ccf58 R12: ffff9df947db2e00
[ 293.845594] R13: ffff9df947db2ed0 R14: ffff9df947db2eb8 R15: ffff9df947db2eb8
[ 293.847855] FS: 00007f5a97379800(0000) GS:ffff9dfa77c00000(0000) knlGS:0000000000000000
[ 293.850647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 293.852940] CR2: 00007f5a97528730 CR3: 000000010bc76005 CR4: 0000000000370ef0
[ 293.854680] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 293.856423] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 293.858380] Call Trace:
[ 293.859302] <TASK>
[ 293.860311] ? ttwu_do_wakeup+0x1c/0x170
[ 293.861800] ? ttwu_do_activate+0x6d/0xb0
[ 293.863057] ? _raw_spin_unlock_irqrestore+0x29/0x40
[ 293.864411] ? try_to_wake_up+0x9d/0x5e0
[ 293.865618] ? debug_smp_processor_id+0x17/0x20
[ 293.866934] ? debug_smp_processor_id+0x17/0x20
[ 293.868223] ? free_unref_page+0xbf/0x120
[ 293.869470] ? __free_slab+0xcb/0x1c0
[ 293.870614] ? preempt_count_add+0x7a/0xc0
[ 293.871811] ? __slab_free+0xa0/0x2d0
[ 293.872918] ? __wake_up_common_lock+0x8a/0xc0
[ 293.874186] ? __slab_free+0xa0/0x2d0
[ 293.875305] ? free_inode_nonrcu+0x20/0x20
[ 293.876466] ? free_inode_nonrcu+0x20/0x20
[ 293.877650] ? debug_smp_processor_id+0x17/0x20
[ 293.878949] ? call_rcu+0x11a/0x240
[ 293.880060] ? f2fs_destroy_stats+0x59/0x60 [f2fs]
[ 293.881437] ? kfree+0x1fe/0x230
[ 293.882674] f2fs_put_super+0x160/0x390 [f2fs]
[ 293.883978] generic_shutdown_super+0x7a/0x120
[ 293.885274] kill_block_super+0x27/0x50
[ 293.886496] kill_f2fs_super+0x7f/0x100 [f2fs]
[ 293.887806] deactivate_locked_super+0x35/0xa0
[ 293.889271] deactivate_super+0x40/0x50
[ 293.890513] cleanup_mnt+0x139/0x190
[ 293.891689] __cleanup_mnt+0x12/0x20
[ 293.892850] task_work_run+0x64/0xa0
[ 293.894035] exit_to_user_mode_prepare+0x1b7/0x1c0
[ 293.895409] syscall_exit_to_user_mode+0x27/0x50
[ 293.896872] do_syscall_64+0x48/0xc0
[ 293.898090] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 293.899517] RIP: 0033:0x7f5a975cd25b

Fixes: 7735730d39d7 ("f2fs: fix to propagate error from __get_meta_page()")
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 2fef99b8 Fri Feb 11 19:56:46 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: fix missing free nid in f2fs_handle_failed_inode

This patch fixes xfstests/generic/475 failure.

[ 293.680694] F2FS-fs (dm-1): May loss orphan inode, run fsck to fix.
[ 293.685358] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691527] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691764] sh (7615): drop_caches: 3
[ 293.691819] sh (7616): drop_caches: 3
[ 293.694017] Buffer I/O error on dev dm-1, logical block 1, async page read
[ 293.695659] sh (7618): drop_caches: 3
[ 293.696979] sh (7617): drop_caches: 3
[ 293.700290] sh (7623): drop_caches: 3
[ 293.708621] sh (7626): drop_caches: 3
[ 293.711386] sh (7628): drop_caches: 3
[ 293.711825] sh (7627): drop_caches: 3
[ 293.716738] sh (7630): drop_caches: 3
[ 293.719613] sh (7632): drop_caches: 3
[ 293.720971] sh (7633): drop_caches: 3
[ 293.727741] sh (7634): drop_caches: 3
[ 293.730783] sh (7636): drop_caches: 3
[ 293.732681] sh (7635): drop_caches: 3
[ 293.732988] sh (7637): drop_caches: 3
[ 293.738836] sh (7639): drop_caches: 3
[ 293.740568] sh (7641): drop_caches: 3
[ 293.743053] sh (7640): drop_caches: 3
[ 293.821889] ------------[ cut here ]------------
[ 293.824654] kernel BUG at fs/f2fs/node.c:3334!
[ 293.826226] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 293.828713] CPU: 0 PID: 7653 Comm: umount Tainted: G OE 5.17.0-rc1-custom #1
[ 293.830946] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[ 293.832526] RIP: 0010:f2fs_destroy_node_manager+0x33f/0x350 [f2fs]
[ 293.833905] Code: e8 d6 3d f9 f9 48 8b 45 d0 65 48 2b 04 25 28 00 00 00 75 1a 48 81 c4 28 03 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 0b
[ 293.837783] RSP: 0018:ffffb04ec31e7a20 EFLAGS: 00010202
[ 293.839062] RAX: 0000000000000001 RBX: ffff9df947db2eb8 RCX: 0000000080aa0072
[ 293.840666] RDX: 0000000000000000 RSI: ffffe86c0432a140 RDI: ffffffffc0b72a21
[ 293.842261] RBP: ffffb04ec31e7d70 R08: ffff9df94ca85780 R09: 0000000080aa0072
[ 293.843909] R10: ffff9df94ca85700 R11: ffff9df94e1ccf58 R12: ffff9df947db2e00
[ 293.845594] R13: ffff9df947db2ed0 R14: ffff9df947db2eb8 R15: ffff9df947db2eb8
[ 293.847855] FS: 00007f5a97379800(0000) GS:ffff9dfa77c00000(0000) knlGS:0000000000000000
[ 293.850647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 293.852940] CR2: 00007f5a97528730 CR3: 000000010bc76005 CR4: 0000000000370ef0
[ 293.854680] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 293.856423] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 293.858380] Call Trace:
[ 293.859302] <TASK>
[ 293.860311] ? ttwu_do_wakeup+0x1c/0x170
[ 293.861800] ? ttwu_do_activate+0x6d/0xb0
[ 293.863057] ? _raw_spin_unlock_irqrestore+0x29/0x40
[ 293.864411] ? try_to_wake_up+0x9d/0x5e0
[ 293.865618] ? debug_smp_processor_id+0x17/0x20
[ 293.866934] ? debug_smp_processor_id+0x17/0x20
[ 293.868223] ? free_unref_page+0xbf/0x120
[ 293.869470] ? __free_slab+0xcb/0x1c0
[ 293.870614] ? preempt_count_add+0x7a/0xc0
[ 293.871811] ? __slab_free+0xa0/0x2d0
[ 293.872918] ? __wake_up_common_lock+0x8a/0xc0
[ 293.874186] ? __slab_free+0xa0/0x2d0
[ 293.875305] ? free_inode_nonrcu+0x20/0x20
[ 293.876466] ? free_inode_nonrcu+0x20/0x20
[ 293.877650] ? debug_smp_processor_id+0x17/0x20
[ 293.878949] ? call_rcu+0x11a/0x240
[ 293.880060] ? f2fs_destroy_stats+0x59/0x60 [f2fs]
[ 293.881437] ? kfree+0x1fe/0x230
[ 293.882674] f2fs_put_super+0x160/0x390 [f2fs]
[ 293.883978] generic_shutdown_super+0x7a/0x120
[ 293.885274] kill_block_super+0x27/0x50
[ 293.886496] kill_f2fs_super+0x7f/0x100 [f2fs]
[ 293.887806] deactivate_locked_super+0x35/0xa0
[ 293.889271] deactivate_super+0x40/0x50
[ 293.890513] cleanup_mnt+0x139/0x190
[ 293.891689] __cleanup_mnt+0x12/0x20
[ 293.892850] task_work_run+0x64/0xa0
[ 293.894035] exit_to_user_mode_prepare+0x1b7/0x1c0
[ 293.895409] syscall_exit_to_user_mode+0x27/0x50
[ 293.896872] do_syscall_64+0x48/0xc0
[ 293.898090] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 293.899517] RIP: 0033:0x7f5a975cd25b

Fixes: 7735730d39d7 ("f2fs: fix to propagate error from __get_meta_page()")
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 2fef99b8 Fri Feb 11 19:56:46 MST 2022 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: fix missing free nid in f2fs_handle_failed_inode

This patch fixes xfstests/generic/475 failure.

[ 293.680694] F2FS-fs (dm-1): May loss orphan inode, run fsck to fix.
[ 293.685358] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691527] Buffer I/O error on dev dm-1, logical block 8388592, async page read
[ 293.691764] sh (7615): drop_caches: 3
[ 293.691819] sh (7616): drop_caches: 3
[ 293.694017] Buffer I/O error on dev dm-1, logical block 1, async page read
[ 293.695659] sh (7618): drop_caches: 3
[ 293.696979] sh (7617): drop_caches: 3
[ 293.700290] sh (7623): drop_caches: 3
[ 293.708621] sh (7626): drop_caches: 3
[ 293.711386] sh (7628): drop_caches: 3
[ 293.711825] sh (7627): drop_caches: 3
[ 293.716738] sh (7630): drop_caches: 3
[ 293.719613] sh (7632): drop_caches: 3
[ 293.720971] sh (7633): drop_caches: 3
[ 293.727741] sh (7634): drop_caches: 3
[ 293.730783] sh (7636): drop_caches: 3
[ 293.732681] sh (7635): drop_caches: 3
[ 293.732988] sh (7637): drop_caches: 3
[ 293.738836] sh (7639): drop_caches: 3
[ 293.740568] sh (7641): drop_caches: 3
[ 293.743053] sh (7640): drop_caches: 3
[ 293.821889] ------------[ cut here ]------------
[ 293.824654] kernel BUG at fs/f2fs/node.c:3334!
[ 293.826226] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 293.828713] CPU: 0 PID: 7653 Comm: umount Tainted: G OE 5.17.0-rc1-custom #1
[ 293.830946] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[ 293.832526] RIP: 0010:f2fs_destroy_node_manager+0x33f/0x350 [f2fs]
[ 293.833905] Code: e8 d6 3d f9 f9 48 8b 45 d0 65 48 2b 04 25 28 00 00 00 75 1a 48 81 c4 28 03 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 0b
[ 293.837783] RSP: 0018:ffffb04ec31e7a20 EFLAGS: 00010202
[ 293.839062] RAX: 0000000000000001 RBX: ffff9df947db2eb8 RCX: 0000000080aa0072
[ 293.840666] RDX: 0000000000000000 RSI: ffffe86c0432a140 RDI: ffffffffc0b72a21
[ 293.842261] RBP: ffffb04ec31e7d70 R08: ffff9df94ca85780 R09: 0000000080aa0072
[ 293.843909] R10: ffff9df94ca85700 R11: ffff9df94e1ccf58 R12: ffff9df947db2e00
[ 293.845594] R13: ffff9df947db2ed0 R14: ffff9df947db2eb8 R15: ffff9df947db2eb8
[ 293.847855] FS: 00007f5a97379800(0000) GS:ffff9dfa77c00000(0000) knlGS:0000000000000000
[ 293.850647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 293.852940] CR2: 00007f5a97528730 CR3: 000000010bc76005 CR4: 0000000000370ef0
[ 293.854680] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 293.856423] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 293.858380] Call Trace:
[ 293.859302] <TASK>
[ 293.860311] ? ttwu_do_wakeup+0x1c/0x170
[ 293.861800] ? ttwu_do_activate+0x6d/0xb0
[ 293.863057] ? _raw_spin_unlock_irqrestore+0x29/0x40
[ 293.864411] ? try_to_wake_up+0x9d/0x5e0
[ 293.865618] ? debug_smp_processor_id+0x17/0x20
[ 293.866934] ? debug_smp_processor_id+0x17/0x20
[ 293.868223] ? free_unref_page+0xbf/0x120
[ 293.869470] ? __free_slab+0xcb/0x1c0
[ 293.870614] ? preempt_count_add+0x7a/0xc0
[ 293.871811] ? __slab_free+0xa0/0x2d0
[ 293.872918] ? __wake_up_common_lock+0x8a/0xc0
[ 293.874186] ? __slab_free+0xa0/0x2d0
[ 293.875305] ? free_inode_nonrcu+0x20/0x20
[ 293.876466] ? free_inode_nonrcu+0x20/0x20
[ 293.877650] ? debug_smp_processor_id+0x17/0x20
[ 293.878949] ? call_rcu+0x11a/0x240
[ 293.880060] ? f2fs_destroy_stats+0x59/0x60 [f2fs]
[ 293.881437] ? kfree+0x1fe/0x230
[ 293.882674] f2fs_put_super+0x160/0x390 [f2fs]
[ 293.883978] generic_shutdown_super+0x7a/0x120
[ 293.885274] kill_block_super+0x27/0x50
[ 293.886496] kill_f2fs_super+0x7f/0x100 [f2fs]
[ 293.887806] deactivate_locked_super+0x35/0xa0
[ 293.889271] deactivate_super+0x40/0x50
[ 293.890513] cleanup_mnt+0x139/0x190
[ 293.891689] __cleanup_mnt+0x12/0x20
[ 293.892850] task_work_run+0x64/0xa0
[ 293.894035] exit_to_user_mode_prepare+0x1b7/0x1c0
[ 293.895409] syscall_exit_to_user_mode+0x27/0x50
[ 293.896872] do_syscall_64+0x48/0xc0
[ 293.898090] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 293.899517] RIP: 0033:0x7f5a975cd25b

Fixes: 7735730d39d7 ("f2fs: fix to propagate error from __get_meta_page()")
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
H A Dfile.cdiff 2f6d721e Tue Mar 05 20:47:46 MST 2024 Xiuhong Wang <xiuhong.wang@unisoc.com> f2fs: compress: fix reserve_cblocks counting error when out of space

When a file only needs one direct_node, performing the following
operations will cause the file to be unrepairable:

unisoc # ./f2fs_io compress test.apk
unisoc #df -h | grep dm-48
/dev/block/dm-48 112G 112G 1.2M 100% /data

unisoc # ./f2fs_io release_cblocks test.apk
924
unisoc # df -h | grep dm-48
/dev/block/dm-48 112G 112G 4.8M 100% /data

unisoc # dd if=/dev/random of=file4 bs=1M count=3
3145728 bytes (3.0 M) copied, 0.025 s, 120 M/s
unisoc # df -h | grep dm-48
/dev/block/dm-48 112G 112G 1.8M 100% /data

unisoc # ./f2fs_io reserve_cblocks test.apk
F2FS_IOC_RESERVE_COMPRESS_BLOCKS failed: No space left on device

adb reboot
unisoc # df -h | grep dm-48
/dev/block/dm-48 112G 112G 11M 100% /data
unisoc # ./f2fs_io reserve_cblocks test.apk
0

This is because the file has only one direct_node. After returning
to -ENOSPC, reserved_blocks += ret will not be executed. As a result,
the reserved_blocks at this time is still 0, which is not the real
number of reserved blocks. Therefore, fsck cannot be set to repair
the file.

After this patch, the fsck flag will be set to fix this problem.

unisoc # df -h | grep dm-48
/dev/block/dm-48 112G 112G 1.8M 100% /data
unisoc # ./f2fs_io reserve_cblocks test.apk
F2FS_IOC_RESERVE_COMPRESS_BLOCKS failed: No space left on device

adb reboot then fsck will be executed
unisoc # df -h | grep dm-48
/dev/block/dm-48 112G 112G 11M 100% /data
unisoc # ./f2fs_io reserve_cblocks test.apk
924

Fixes: c75488fb4d82 ("f2fs: introduce F2FS_IOC_RESERVE_COMPRESS_BLOCKS")
Signed-off-by: Xiuhong Wang <xiuhong.wang@unisoc.com>
Signed-off-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff c7115e09 Fri Jan 12 12:41:32 MST 2024 Chao Yu <chao@kernel.org> f2fs: introduce FAULT_BLKADDR_CONSISTENCE

We will encounter below inconsistent status when FAULT_BLKADDR type
fault injection is on.

Info: checkpoint state = d6 : nat_bits crc fsck compacted_summary orphan_inodes sudden-power-off
[ASSERT] (fsck_chk_inode_blk:1254) --> ino: 0x1c100 has i_blocks: 000000c0, but has 191 blocks
[FIX] (fsck_chk_inode_blk:1260) --> [0x1c100] i_blocks=0x000000c0 -> 0xbf
[FIX] (fsck_chk_inode_blk:1269) --> [0x1c100] i_compr_blocks=0x00000026 -> 0x27
[ASSERT] (fsck_chk_inode_blk:1254) --> ino: 0x1cadb has i_blocks: 0000002f, but has 46 blocks
[FIX] (fsck_chk_inode_blk:1260) --> [0x1cadb] i_blocks=0x0000002f -> 0x2e
[FIX] (fsck_chk_inode_blk:1269) --> [0x1cadb] i_compr_blocks=0x00000011 -> 0x12
[ASSERT] (fsck_chk_inode_blk:1254) --> ino: 0x1c62c has i_blocks: 00000002, but has 1 blocks
[FIX] (fsck_chk_inode_blk:1260) --> [0x1c62c] i_blocks=0x00000002 -> 0x1

After we inject fault into f2fs_is_valid_blkaddr() during truncation,
a) it missed to increase @nr_free or @valid_blocks
b) it can cause in blkaddr leak in truncated dnode
Which may cause inconsistent status.

This patch separates FAULT_BLKADDR_CONSISTENCE from FAULT_BLKADDR,
and rename FAULT_BLKADDR to FAULT_BLKADDR_VALIDITY
so that we can:
a) use FAULT_BLKADDR_CONSISTENCE in f2fs_truncate_data_blocks_range()
to simulate inconsistent issue independently, then it can verify fsck
repair flow.
b) FAULT_BLKADDR_VALIDITY fault will not cause any inconsistent status,
we can just use it to check error path handling in kernel side.

Reviewed-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff a6ec8378 Thu Jun 29 05:11:44 MDT 2023 Chao Yu <chao@kernel.org> f2fs: fix to do sanity check on direct node in truncate_dnode()

syzbot reports below bug:

BUG: KASAN: slab-use-after-free in f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
Read of size 4 at addr ffff88802a25c000 by task syz-executor148/5000

CPU: 1 PID: 5000 Comm: syz-executor148 Not tainted 6.4.0-rc7-syzkaller-00041-ge660abd551f1 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/27/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x150 lib/dump_stack.c:106
print_address_description.constprop.0+0x2c/0x3c0 mm/kasan/report.c:351
print_report mm/kasan/report.c:462 [inline]
kasan_report+0x11c/0x130 mm/kasan/report.c:572
f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
truncate_dnode+0x229/0x2e0 fs/f2fs/node.c:944
f2fs_truncate_inode_blocks+0x64b/0xde0 fs/f2fs/node.c:1154
f2fs_do_truncate_blocks+0x4ac/0xf30 fs/f2fs/file.c:721
f2fs_truncate_blocks+0x7b/0x300 fs/f2fs/file.c:749
f2fs_truncate.part.0+0x4a5/0x630 fs/f2fs/file.c:799
f2fs_truncate include/linux/fs.h:825 [inline]
f2fs_setattr+0x1738/0x2090 fs/f2fs/file.c:1006
notify_change+0xb2c/0x1180 fs/attr.c:483
do_truncate+0x143/0x200 fs/open.c:66
handle_truncate fs/namei.c:3295 [inline]
do_open fs/namei.c:3640 [inline]
path_openat+0x2083/0x2750 fs/namei.c:3791
do_filp_open+0x1ba/0x410 fs/namei.c:3818
do_sys_openat2+0x16d/0x4c0 fs/open.c:1356
do_sys_open fs/open.c:1372 [inline]
__do_sys_creat fs/open.c:1448 [inline]
__se_sys_creat fs/open.c:1442 [inline]
__x64_sys_creat+0xcd/0x120 fs/open.c:1442
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

The root cause is, inodeA references inodeB via inodeB's ino, once inodeA
is truncated, it calls truncate_dnode() to truncate data blocks in inodeB's
node page, it traverse mapping data from node->i.i_addr[0] to
node->i.i_addr[ADDRS_PER_BLOCK() - 1], result in out-of-boundary access.

This patch fixes to add sanity check on dnode page in truncate_dnode(),
so that, it can help to avoid triggering such issue, and once it encounters
such issue, it will record newly introduced ERROR_INVALID_NODE_REFERENCE
error into superblock, later fsck can detect such issue and try repairing.

Also, it removes f2fs_truncate_data_blocks() for cleanup due to the
function has only one caller, and uses f2fs_truncate_data_blocks_range()
instead.

Reported-and-tested-by: syzbot+12cb4425b22169b52036@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/linux-f2fs-devel/000000000000f3038a05fef867f8@google.com
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff a6ec8378 Thu Jun 29 05:11:44 MDT 2023 Chao Yu <chao@kernel.org> f2fs: fix to do sanity check on direct node in truncate_dnode()

syzbot reports below bug:

BUG: KASAN: slab-use-after-free in f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
Read of size 4 at addr ffff88802a25c000 by task syz-executor148/5000

CPU: 1 PID: 5000 Comm: syz-executor148 Not tainted 6.4.0-rc7-syzkaller-00041-ge660abd551f1 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/27/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x150 lib/dump_stack.c:106
print_address_description.constprop.0+0x2c/0x3c0 mm/kasan/report.c:351
print_report mm/kasan/report.c:462 [inline]
kasan_report+0x11c/0x130 mm/kasan/report.c:572
f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
truncate_dnode+0x229/0x2e0 fs/f2fs/node.c:944
f2fs_truncate_inode_blocks+0x64b/0xde0 fs/f2fs/node.c:1154
f2fs_do_truncate_blocks+0x4ac/0xf30 fs/f2fs/file.c:721
f2fs_truncate_blocks+0x7b/0x300 fs/f2fs/file.c:749
f2fs_truncate.part.0+0x4a5/0x630 fs/f2fs/file.c:799
f2fs_truncate include/linux/fs.h:825 [inline]
f2fs_setattr+0x1738/0x2090 fs/f2fs/file.c:1006
notify_change+0xb2c/0x1180 fs/attr.c:483
do_truncate+0x143/0x200 fs/open.c:66
handle_truncate fs/namei.c:3295 [inline]
do_open fs/namei.c:3640 [inline]
path_openat+0x2083/0x2750 fs/namei.c:3791
do_filp_open+0x1ba/0x410 fs/namei.c:3818
do_sys_openat2+0x16d/0x4c0 fs/open.c:1356
do_sys_open fs/open.c:1372 [inline]
__do_sys_creat fs/open.c:1448 [inline]
__se_sys_creat fs/open.c:1442 [inline]
__x64_sys_creat+0xcd/0x120 fs/open.c:1442
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

The root cause is, inodeA references inodeB via inodeB's ino, once inodeA
is truncated, it calls truncate_dnode() to truncate data blocks in inodeB's
node page, it traverse mapping data from node->i.i_addr[0] to
node->i.i_addr[ADDRS_PER_BLOCK() - 1], result in out-of-boundary access.

This patch fixes to add sanity check on dnode page in truncate_dnode(),
so that, it can help to avoid triggering such issue, and once it encounters
such issue, it will record newly introduced ERROR_INVALID_NODE_REFERENCE
error into superblock, later fsck can detect such issue and try repairing.

Also, it removes f2fs_truncate_data_blocks() for cleanup due to the
function has only one caller, and uses f2fs_truncate_data_blocks_range()
instead.

Reported-and-tested-by: syzbot+12cb4425b22169b52036@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/linux-f2fs-devel/000000000000f3038a05fef867f8@google.com
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 38b57833 Wed May 31 06:59:18 MDT 2023 Yangtao Li <frank.li@vivo.com> f2fs: flag as supporting buffered async reads

The f2fs uses generic_file_buffered_read(), which supports buffered async
reads since commit 1a0a7853b901 ("mm: support async buffered reads in
generic_file_buffered_read()").

Let's enable it to match other file-systems. The read performance has been
greatly improved under io_uring:

167M/s -> 234M/s, Increase ratio by 40%

Test w/:
./fio --name=onessd --filename=/data/test/local/io_uring_test
--size=256M --rw=randread --bs=4k --direct=0 --overwrite=0
--numjobs=1 --iodepth=1 --time_based=0 --runtime=10
--ioengine=io_uring --registerfiles --fixedbufs
--gtod_reduce=1 --group_reporting --sqthread_poll=1

Signed-off-by: Lu Hongfei <luhongfei@vivo.com>
Signed-off-by: Yangtao Li <frank.li@vivo.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 38b57833 Wed May 31 06:59:18 MDT 2023 Yangtao Li <frank.li@vivo.com> f2fs: flag as supporting buffered async reads

The f2fs uses generic_file_buffered_read(), which supports buffered async
reads since commit 1a0a7853b901 ("mm: support async buffered reads in
generic_file_buffered_read()").

Let's enable it to match other file-systems. The read performance has been
greatly improved under io_uring:

167M/s -> 234M/s, Increase ratio by 40%

Test w/:
./fio --name=onessd --filename=/data/test/local/io_uring_test
--size=256M --rw=randread --bs=4k --direct=0 --overwrite=0
--numjobs=1 --iodepth=1 --time_based=0 --runtime=10
--ioengine=io_uring --registerfiles --fixedbufs
--gtod_reduce=1 --group_reporting --sqthread_poll=1

Signed-off-by: Lu Hongfei <luhongfei@vivo.com>
Signed-off-by: Yangtao Li <frank.li@vivo.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 38b57833 Wed May 31 06:59:18 MDT 2023 Yangtao Li <frank.li@vivo.com> f2fs: flag as supporting buffered async reads

The f2fs uses generic_file_buffered_read(), which supports buffered async
reads since commit 1a0a7853b901 ("mm: support async buffered reads in
generic_file_buffered_read()").

Let's enable it to match other file-systems. The read performance has been
greatly improved under io_uring:

167M/s -> 234M/s, Increase ratio by 40%

Test w/:
./fio --name=onessd --filename=/data/test/local/io_uring_test
--size=256M --rw=randread --bs=4k --direct=0 --overwrite=0
--numjobs=1 --iodepth=1 --time_based=0 --runtime=10
--ioengine=io_uring --registerfiles --fixedbufs
--gtod_reduce=1 --group_reporting --sqthread_poll=1

Signed-off-by: Lu Hongfei <luhongfei@vivo.com>
Signed-off-by: Yangtao Li <frank.li@vivo.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 38b57833 Wed May 31 06:59:18 MDT 2023 Yangtao Li <frank.li@vivo.com> f2fs: flag as supporting buffered async reads

The f2fs uses generic_file_buffered_read(), which supports buffered async
reads since commit 1a0a7853b901 ("mm: support async buffered reads in
generic_file_buffered_read()").

Let's enable it to match other file-systems. The read performance has been
greatly improved under io_uring:

167M/s -> 234M/s, Increase ratio by 40%

Test w/:
./fio --name=onessd --filename=/data/test/local/io_uring_test
--size=256M --rw=randread --bs=4k --direct=0 --overwrite=0
--numjobs=1 --iodepth=1 --time_based=0 --runtime=10
--ioengine=io_uring --registerfiles --fixedbufs
--gtod_reduce=1 --group_reporting --sqthread_poll=1

Signed-off-by: Lu Hongfei <luhongfei@vivo.com>
Signed-off-by: Yangtao Li <frank.li@vivo.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 38b57833 Wed May 31 06:59:18 MDT 2023 Yangtao Li <frank.li@vivo.com> f2fs: flag as supporting buffered async reads

The f2fs uses generic_file_buffered_read(), which supports buffered async
reads since commit 1a0a7853b901 ("mm: support async buffered reads in
generic_file_buffered_read()").

Let's enable it to match other file-systems. The read performance has been
greatly improved under io_uring:

167M/s -> 234M/s, Increase ratio by 40%

Test w/:
./fio --name=onessd --filename=/data/test/local/io_uring_test
--size=256M --rw=randread --bs=4k --direct=0 --overwrite=0
--numjobs=1 --iodepth=1 --time_based=0 --runtime=10
--ioengine=io_uring --registerfiles --fixedbufs
--gtod_reduce=1 --group_reporting --sqthread_poll=1

Signed-off-by: Lu Hongfei <luhongfei@vivo.com>
Signed-off-by: Yangtao Li <frank.li@vivo.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff d8189834 Tue May 23 00:17:25 MDT 2023 Chao Yu <chao@kernel.org> f2fs: fix to avoid NULL pointer dereference f2fs_write_end_io()

butt3rflyh4ck reports a bug as below:

When a thread always calls F2FS_IOC_RESIZE_FS to resize fs, if resize fs is
failed, f2fs kernel thread would invoke callback function to update f2fs io
info, it would call f2fs_write_end_io and may trigger null-ptr-deref in
NODE_MAPPING.

general protection fault, probably for non-canonical address
KASAN: null-ptr-deref in range [0x0000000000000030-0x0000000000000037]
RIP: 0010:NODE_MAPPING fs/f2fs/f2fs.h:1972 [inline]
RIP: 0010:f2fs_write_end_io+0x727/0x1050 fs/f2fs/data.c:370
<TASK>
bio_endio+0x5af/0x6c0 block/bio.c:1608
req_bio_endio block/blk-mq.c:761 [inline]
blk_update_request+0x5cc/0x1690 block/blk-mq.c:906
blk_mq_end_request+0x59/0x4c0 block/blk-mq.c:1023
lo_complete_rq+0x1c6/0x280 drivers/block/loop.c:370
blk_complete_reqs+0xad/0xe0 block/blk-mq.c:1101
__do_softirq+0x1d4/0x8ef kernel/softirq.c:571
run_ksoftirqd kernel/softirq.c:939 [inline]
run_ksoftirqd+0x31/0x60 kernel/softirq.c:931
smpboot_thread_fn+0x659/0x9e0 kernel/smpboot.c:164
kthread+0x33e/0x440 kernel/kthread.c:379
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308

The root cause is below race case can cause leaving dirty metadata
in f2fs after filesystem is remount as ro:

Thread A Thread B
- f2fs_ioc_resize_fs
- f2fs_readonly --- return false
- f2fs_resize_fs
- f2fs_remount
- write_checkpoint
- set f2fs as ro
- free_segment_range
- update meta_inode's data

Then, if f2fs_put_super() fails to write_checkpoint due to readonly
status, and meta_inode's dirty data will be writebacked after node_inode
is put, finally, f2fs_write_end_io will access NULL pointer on
sbi->node_inode.

Thread A IRQ context
- f2fs_put_super
- write_checkpoint fails
- iput(node_inode)
- node_inode = NULL
- iput(meta_inode)
- write_inode_now
- f2fs_write_meta_page
- f2fs_write_end_io
- NODE_MAPPING(sbi)
: access NULL pointer on node_inode

Fixes: b4b10061ef98 ("f2fs: refactor resize_fs to avoid meta updates in progress")
Reported-by: butt3rflyh4ck <butterflyhuangxx@gmail.com>
Closes: https://lore.kernel.org/r/1684480657-2375-1-git-send-email-yangtiezhu@loongson.cn
Tested-by: butt3rflyh4ck <butterflyhuangxx@gmail.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
H A Df2fs.hdiff 4b99ecd3 Mon Feb 26 00:35:38 MST 2024 Chao Yu <chao@kernel.org> f2fs: ro: compress: fix to avoid caching unaligned extent

Mapping info from dump.f2fs:
i_addr[0x2d] cluster flag [0xfffffffe : 4294967294]
i_addr[0x2e] [0x 10428 : 66600]
i_addr[0x2f] [0x 10429 : 66601]
i_addr[0x30] [0x 1042a : 66602]

f2fs_io fiemap 37 1 /mnt/f2fs/disk-58390c8c.raw

Previsouly, it missed to align fofs and ofs_in_node to cluster_size,
result in adding incorrect read extent cache, fix it.

Before:
f2fs_update_read_extent_tree_range: dev = (253,48), ino = 5, pgofs = 37, len = 4, blkaddr = 66600, c_len = 3

After:
f2fs_update_read_extent_tree_range: dev = (253,48), ino = 5, pgofs = 36, len = 4, blkaddr = 66600, c_len = 3

Fixes: 94afd6d6e525 ("f2fs: extent cache: support unaligned extent")
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 40b2d55e Wed Feb 07 00:05:48 MST 2024 Chao Yu <chao@kernel.org> f2fs: fix to create selinux label during whiteout initialization

generic/700 - output mismatch (see /media/fstests/results//generic/700.out.bad)
--- tests/generic/700.out 2023-03-28 10:40:42.735529223 +0000
+++ /media/fstests/results//generic/700.out.bad 2024-02-06 04:37:56.000000000 +0000
@@ -1,2 +1,4 @@
QA output created by 700
+/mnt/scratch_f2fs/f1: security.selinux: No such attribute
+/mnt/scratch_f2fs/f2: security.selinux: No such attribute
Silence is golden
...
(Run 'diff -u /media/fstests/tests/generic/700.out /media/fstests/results//generic/700.out.bad' to see the entire diff)

HINT: You _MAY_ be missing kernel fix:
70b589a37e1a xfs: add selinux labels to whiteout inodes

Previously, it missed to create selinux labels during whiteout inode
initialization, fix this issue.

Fixes: 7e01e7ad746b ("f2fs: support RENAME_WHITEOUT")
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 40b2d55e Wed Feb 07 00:05:48 MST 2024 Chao Yu <chao@kernel.org> f2fs: fix to create selinux label during whiteout initialization

generic/700 - output mismatch (see /media/fstests/results//generic/700.out.bad)
--- tests/generic/700.out 2023-03-28 10:40:42.735529223 +0000
+++ /media/fstests/results//generic/700.out.bad 2024-02-06 04:37:56.000000000 +0000
@@ -1,2 +1,4 @@
QA output created by 700
+/mnt/scratch_f2fs/f1: security.selinux: No such attribute
+/mnt/scratch_f2fs/f2: security.selinux: No such attribute
Silence is golden
...
(Run 'diff -u /media/fstests/tests/generic/700.out /media/fstests/results//generic/700.out.bad' to see the entire diff)

HINT: You _MAY_ be missing kernel fix:
70b589a37e1a xfs: add selinux labels to whiteout inodes

Previously, it missed to create selinux labels during whiteout inode
initialization, fix this issue.

Fixes: 7e01e7ad746b ("f2fs: support RENAME_WHITEOUT")
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 21ec6823 Wed Jan 24 07:49:15 MST 2024 Chao Yu <chao@kernel.org> f2fs: fix to avoid potential panic during recovery

During recovery, if FAULT_BLOCK is on, it is possible that
f2fs_reserve_new_block() will return -ENOSPC during recovery,
then it may trigger panic.

Also, if fault injection rate is 1 and only FAULT_BLOCK fault
type is on, it may encounter deadloop in loop of block reservation.

Let's change as below to fix these issues:
- remove bug_on() to avoid panic.
- limit the loop count of block reservation to avoid potential
deadloop.

Fixes: 956fa1ddc132 ("f2fs: fix to check return value of f2fs_reserve_new_block()")
Reported-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff c7115e09 Fri Jan 12 12:41:32 MST 2024 Chao Yu <chao@kernel.org> f2fs: introduce FAULT_BLKADDR_CONSISTENCE

We will encounter below inconsistent status when FAULT_BLKADDR type
fault injection is on.

Info: checkpoint state = d6 : nat_bits crc fsck compacted_summary orphan_inodes sudden-power-off
[ASSERT] (fsck_chk_inode_blk:1254) --> ino: 0x1c100 has i_blocks: 000000c0, but has 191 blocks
[FIX] (fsck_chk_inode_blk:1260) --> [0x1c100] i_blocks=0x000000c0 -> 0xbf
[FIX] (fsck_chk_inode_blk:1269) --> [0x1c100] i_compr_blocks=0x00000026 -> 0x27
[ASSERT] (fsck_chk_inode_blk:1254) --> ino: 0x1cadb has i_blocks: 0000002f, but has 46 blocks
[FIX] (fsck_chk_inode_blk:1260) --> [0x1cadb] i_blocks=0x0000002f -> 0x2e
[FIX] (fsck_chk_inode_blk:1269) --> [0x1cadb] i_compr_blocks=0x00000011 -> 0x12
[ASSERT] (fsck_chk_inode_blk:1254) --> ino: 0x1c62c has i_blocks: 00000002, but has 1 blocks
[FIX] (fsck_chk_inode_blk:1260) --> [0x1c62c] i_blocks=0x00000002 -> 0x1

After we inject fault into f2fs_is_valid_blkaddr() during truncation,
a) it missed to increase @nr_free or @valid_blocks
b) it can cause in blkaddr leak in truncated dnode
Which may cause inconsistent status.

This patch separates FAULT_BLKADDR_CONSISTENCE from FAULT_BLKADDR,
and rename FAULT_BLKADDR to FAULT_BLKADDR_VALIDITY
so that we can:
a) use FAULT_BLKADDR_CONSISTENCE in f2fs_truncate_data_blocks_range()
to simulate inconsistent issue independently, then it can verify fsck
repair flow.
b) FAULT_BLKADDR_VALIDITY fault will not cause any inconsistent status,
we can just use it to check error path handling in kernel side.

Reviewed-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff bb6e1c8f Sun Dec 10 02:20:36 MST 2023 Chao Yu <chao@kernel.org> f2fs: delete obsolete FI_DROP_CACHE

FI_DROP_CACHE was introduced in commit 1e84371ffeef ("f2fs: change
atomic and volatile write policies") for volatile write feature,
after commit 7bc155fec5b3 ("f2fs: kill volatile write support"),
we won't support volatile write, let's delete related codes.

Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 5c13e238 Fri Aug 18 12:34:32 MDT 2023 Jaegeuk Kim <jaegeuk@kernel.org> f2fs: avoid false alarm of circular locking

======================================================
WARNING: possible circular locking dependency detected
6.5.0-rc5-syzkaller-00353-gae545c3283dc #0 Not tainted
------------------------------------------------------
syz-executor273/5027 is trying to acquire lock:
ffff888077fe1fb0 (&fi->i_sem){+.+.}-{3:3}, at: f2fs_down_write fs/f2fs/f2fs.h:2133 [inline]
ffff888077fe1fb0 (&fi->i_sem){+.+.}-{3:3}, at: f2fs_add_inline_entry+0x300/0x6f0 fs/f2fs/inline.c:644

but task is already holding lock:
ffff888077fe07c8 (&fi->i_xattr_sem){.+.+}-{3:3}, at: f2fs_down_read fs/f2fs/f2fs.h:2108 [inline]
ffff888077fe07c8 (&fi->i_xattr_sem){.+.+}-{3:3}, at: f2fs_add_dentry+0x92/0x230 fs/f2fs/dir.c:783

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:

-> #1 (&fi->i_xattr_sem){.+.+}-{3:3}:
down_read+0x9c/0x470 kernel/locking/rwsem.c:1520
f2fs_down_read fs/f2fs/f2fs.h:2108 [inline]
f2fs_getxattr+0xb1e/0x12c0 fs/f2fs/xattr.c:532
__f2fs_get_acl+0x5a/0x900 fs/f2fs/acl.c:179
f2fs_acl_create fs/f2fs/acl.c:377 [inline]
f2fs_init_acl+0x15c/0xb30 fs/f2fs/acl.c:420
f2fs_init_inode_metadata+0x159/0x1290 fs/f2fs/dir.c:558
f2fs_add_regular_entry+0x79e/0xb90 fs/f2fs/dir.c:740
f2fs_add_dentry+0x1de/0x230 fs/f2fs/dir.c:788
f2fs_do_add_link+0x190/0x280 fs/f2fs/dir.c:827
f2fs_add_link fs/f2fs/f2fs.h:3554 [inline]
f2fs_mkdir+0x377/0x620 fs/f2fs/namei.c:781
vfs_mkdir+0x532/0x7e0 fs/namei.c:4117
do_mkdirat+0x2a9/0x330 fs/namei.c:4140
__do_sys_mkdir fs/namei.c:4160 [inline]
__se_sys_mkdir fs/namei.c:4158 [inline]
__x64_sys_mkdir+0xf2/0x140 fs/namei.c:4158
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x38/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #0 (&fi->i_sem){+.+.}-{3:3}:
check_prev_add kernel/locking/lockdep.c:3142 [inline]
check_prevs_add kernel/locking/lockdep.c:3261 [inline]
validate_chain kernel/locking/lockdep.c:3876 [inline]
__lock_acquire+0x2e3d/0x5de0 kernel/locking/lockdep.c:5144
lock_acquire kernel/locking/lockdep.c:5761 [inline]
lock_acquire+0x1ae/0x510 kernel/locking/lockdep.c:5726
down_write+0x93/0x200 kernel/locking/rwsem.c:1573
f2fs_down_write fs/f2fs/f2fs.h:2133 [inline]
f2fs_add_inline_entry+0x300/0x6f0 fs/f2fs/inline.c:644
f2fs_add_dentry+0xa6/0x230 fs/f2fs/dir.c:784
f2fs_do_add_link+0x190/0x280 fs/f2fs/dir.c:827
f2fs_add_link fs/f2fs/f2fs.h:3554 [inline]
f2fs_mkdir+0x377/0x620 fs/f2fs/namei.c:781
vfs_mkdir+0x532/0x7e0 fs/namei.c:4117
ovl_do_mkdir fs/overlayfs/overlayfs.h:196 [inline]
ovl_mkdir_real+0xb5/0x370 fs/overlayfs/dir.c:146
ovl_workdir_create+0x3de/0x820 fs/overlayfs/super.c:309
ovl_make_workdir fs/overlayfs/super.c:711 [inline]
ovl_get_workdir fs/overlayfs/super.c:864 [inline]
ovl_fill_super+0xdab/0x6180 fs/overlayfs/super.c:1400
vfs_get_super+0xf9/0x290 fs/super.c:1152
vfs_get_tree+0x88/0x350 fs/super.c:1519
do_new_mount fs/namespace.c:3335 [inline]
path_mount+0x1492/0x1ed0 fs/namespace.c:3662
do_mount fs/namespace.c:3675 [inline]
__do_sys_mount fs/namespace.c:3884 [inline]
__se_sys_mount fs/namespace.c:3861 [inline]
__x64_sys_mount+0x293/0x310 fs/namespace.c:3861
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x38/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
rlock(&fi->i_xattr_sem);
lock(&fi->i_sem);
lock(&fi->i_xattr_sem);
lock(&fi->i_sem);

Cc: <stable@vger.kernel.org>
Reported-and-tested-by: syzbot+e5600587fa9cbf8e3826@syzkaller.appspotmail.com
Fixes: 5eda1ad1aaff "f2fs: fix deadlock in i_xattr_sem and inode page lock"
Tested-by: Guenter Roeck <linux@roeck-us.net>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff a6ec8378 Thu Jun 29 05:11:44 MDT 2023 Chao Yu <chao@kernel.org> f2fs: fix to do sanity check on direct node in truncate_dnode()

syzbot reports below bug:

BUG: KASAN: slab-use-after-free in f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
Read of size 4 at addr ffff88802a25c000 by task syz-executor148/5000

CPU: 1 PID: 5000 Comm: syz-executor148 Not tainted 6.4.0-rc7-syzkaller-00041-ge660abd551f1 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/27/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x150 lib/dump_stack.c:106
print_address_description.constprop.0+0x2c/0x3c0 mm/kasan/report.c:351
print_report mm/kasan/report.c:462 [inline]
kasan_report+0x11c/0x130 mm/kasan/report.c:572
f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
truncate_dnode+0x229/0x2e0 fs/f2fs/node.c:944
f2fs_truncate_inode_blocks+0x64b/0xde0 fs/f2fs/node.c:1154
f2fs_do_truncate_blocks+0x4ac/0xf30 fs/f2fs/file.c:721
f2fs_truncate_blocks+0x7b/0x300 fs/f2fs/file.c:749
f2fs_truncate.part.0+0x4a5/0x630 fs/f2fs/file.c:799
f2fs_truncate include/linux/fs.h:825 [inline]
f2fs_setattr+0x1738/0x2090 fs/f2fs/file.c:1006
notify_change+0xb2c/0x1180 fs/attr.c:483
do_truncate+0x143/0x200 fs/open.c:66
handle_truncate fs/namei.c:3295 [inline]
do_open fs/namei.c:3640 [inline]
path_openat+0x2083/0x2750 fs/namei.c:3791
do_filp_open+0x1ba/0x410 fs/namei.c:3818
do_sys_openat2+0x16d/0x4c0 fs/open.c:1356
do_sys_open fs/open.c:1372 [inline]
__do_sys_creat fs/open.c:1448 [inline]
__se_sys_creat fs/open.c:1442 [inline]
__x64_sys_creat+0xcd/0x120 fs/open.c:1442
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

The root cause is, inodeA references inodeB via inodeB's ino, once inodeA
is truncated, it calls truncate_dnode() to truncate data blocks in inodeB's
node page, it traverse mapping data from node->i.i_addr[0] to
node->i.i_addr[ADDRS_PER_BLOCK() - 1], result in out-of-boundary access.

This patch fixes to add sanity check on dnode page in truncate_dnode(),
so that, it can help to avoid triggering such issue, and once it encounters
such issue, it will record newly introduced ERROR_INVALID_NODE_REFERENCE
error into superblock, later fsck can detect such issue and try repairing.

Also, it removes f2fs_truncate_data_blocks() for cleanup due to the
function has only one caller, and uses f2fs_truncate_data_blocks_range()
instead.

Reported-and-tested-by: syzbot+12cb4425b22169b52036@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/linux-f2fs-devel/000000000000f3038a05fef867f8@google.com
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff a6ec8378 Thu Jun 29 05:11:44 MDT 2023 Chao Yu <chao@kernel.org> f2fs: fix to do sanity check on direct node in truncate_dnode()

syzbot reports below bug:

BUG: KASAN: slab-use-after-free in f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
Read of size 4 at addr ffff88802a25c000 by task syz-executor148/5000

CPU: 1 PID: 5000 Comm: syz-executor148 Not tainted 6.4.0-rc7-syzkaller-00041-ge660abd551f1 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/27/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x150 lib/dump_stack.c:106
print_address_description.constprop.0+0x2c/0x3c0 mm/kasan/report.c:351
print_report mm/kasan/report.c:462 [inline]
kasan_report+0x11c/0x130 mm/kasan/report.c:572
f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
truncate_dnode+0x229/0x2e0 fs/f2fs/node.c:944
f2fs_truncate_inode_blocks+0x64b/0xde0 fs/f2fs/node.c:1154
f2fs_do_truncate_blocks+0x4ac/0xf30 fs/f2fs/file.c:721
f2fs_truncate_blocks+0x7b/0x300 fs/f2fs/file.c:749
f2fs_truncate.part.0+0x4a5/0x630 fs/f2fs/file.c:799
f2fs_truncate include/linux/fs.h:825 [inline]
f2fs_setattr+0x1738/0x2090 fs/f2fs/file.c:1006
notify_change+0xb2c/0x1180 fs/attr.c:483
do_truncate+0x143/0x200 fs/open.c:66
handle_truncate fs/namei.c:3295 [inline]
do_open fs/namei.c:3640 [inline]
path_openat+0x2083/0x2750 fs/namei.c:3791
do_filp_open+0x1ba/0x410 fs/namei.c:3818
do_sys_openat2+0x16d/0x4c0 fs/open.c:1356
do_sys_open fs/open.c:1372 [inline]
__do_sys_creat fs/open.c:1448 [inline]
__se_sys_creat fs/open.c:1442 [inline]
__x64_sys_creat+0xcd/0x120 fs/open.c:1442
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

The root cause is, inodeA references inodeB via inodeB's ino, once inodeA
is truncated, it calls truncate_dnode() to truncate data blocks in inodeB's
node page, it traverse mapping data from node->i.i_addr[0] to
node->i.i_addr[ADDRS_PER_BLOCK() - 1], result in out-of-boundary access.

This patch fixes to add sanity check on dnode page in truncate_dnode(),
so that, it can help to avoid triggering such issue, and once it encounters
such issue, it will record newly introduced ERROR_INVALID_NODE_REFERENCE
error into superblock, later fsck can detect such issue and try repairing.

Also, it removes f2fs_truncate_data_blocks() for cleanup due to the
function has only one caller, and uses f2fs_truncate_data_blocks_range()
instead.

Reported-and-tested-by: syzbot+12cb4425b22169b52036@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/linux-f2fs-devel/000000000000f3038a05fef867f8@google.com
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 901c12d1 Wed May 17 20:14:12 MDT 2023 Chao Yu <chao@kernel.org> f2fs: flush error flags in workqueue

In IRQ context, it wakes up workqueue to record errors into on-disk
superblock fields rather than in-memory fields.

Fixes: 1aa161e43106 ("f2fs: fix scheduling while atomic in decompression path")
Fixes: 95fa90c9e5a7 ("f2fs: support recording errors into superblock")
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
/linux-master/include/linux/
H A Df2fs_fs.hdiff a6ec8378 Thu Jun 29 05:11:44 MDT 2023 Chao Yu <chao@kernel.org> f2fs: fix to do sanity check on direct node in truncate_dnode()

syzbot reports below bug:

BUG: KASAN: slab-use-after-free in f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
Read of size 4 at addr ffff88802a25c000 by task syz-executor148/5000

CPU: 1 PID: 5000 Comm: syz-executor148 Not tainted 6.4.0-rc7-syzkaller-00041-ge660abd551f1 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/27/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x150 lib/dump_stack.c:106
print_address_description.constprop.0+0x2c/0x3c0 mm/kasan/report.c:351
print_report mm/kasan/report.c:462 [inline]
kasan_report+0x11c/0x130 mm/kasan/report.c:572
f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
truncate_dnode+0x229/0x2e0 fs/f2fs/node.c:944
f2fs_truncate_inode_blocks+0x64b/0xde0 fs/f2fs/node.c:1154
f2fs_do_truncate_blocks+0x4ac/0xf30 fs/f2fs/file.c:721
f2fs_truncate_blocks+0x7b/0x300 fs/f2fs/file.c:749
f2fs_truncate.part.0+0x4a5/0x630 fs/f2fs/file.c:799
f2fs_truncate include/linux/fs.h:825 [inline]
f2fs_setattr+0x1738/0x2090 fs/f2fs/file.c:1006
notify_change+0xb2c/0x1180 fs/attr.c:483
do_truncate+0x143/0x200 fs/open.c:66
handle_truncate fs/namei.c:3295 [inline]
do_open fs/namei.c:3640 [inline]
path_openat+0x2083/0x2750 fs/namei.c:3791
do_filp_open+0x1ba/0x410 fs/namei.c:3818
do_sys_openat2+0x16d/0x4c0 fs/open.c:1356
do_sys_open fs/open.c:1372 [inline]
__do_sys_creat fs/open.c:1448 [inline]
__se_sys_creat fs/open.c:1442 [inline]
__x64_sys_creat+0xcd/0x120 fs/open.c:1442
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

The root cause is, inodeA references inodeB via inodeB's ino, once inodeA
is truncated, it calls truncate_dnode() to truncate data blocks in inodeB's
node page, it traverse mapping data from node->i.i_addr[0] to
node->i.i_addr[ADDRS_PER_BLOCK() - 1], result in out-of-boundary access.

This patch fixes to add sanity check on dnode page in truncate_dnode(),
so that, it can help to avoid triggering such issue, and once it encounters
such issue, it will record newly introduced ERROR_INVALID_NODE_REFERENCE
error into superblock, later fsck can detect such issue and try repairing.

Also, it removes f2fs_truncate_data_blocks() for cleanup due to the
function has only one caller, and uses f2fs_truncate_data_blocks_range()
instead.

Reported-and-tested-by: syzbot+12cb4425b22169b52036@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/linux-f2fs-devel/000000000000f3038a05fef867f8@google.com
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff a6ec8378 Thu Jun 29 05:11:44 MDT 2023 Chao Yu <chao@kernel.org> f2fs: fix to do sanity check on direct node in truncate_dnode()

syzbot reports below bug:

BUG: KASAN: slab-use-after-free in f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
Read of size 4 at addr ffff88802a25c000 by task syz-executor148/5000

CPU: 1 PID: 5000 Comm: syz-executor148 Not tainted 6.4.0-rc7-syzkaller-00041-ge660abd551f1 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/27/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x150 lib/dump_stack.c:106
print_address_description.constprop.0+0x2c/0x3c0 mm/kasan/report.c:351
print_report mm/kasan/report.c:462 [inline]
kasan_report+0x11c/0x130 mm/kasan/report.c:572
f2fs_truncate_data_blocks_range+0x122a/0x14c0 fs/f2fs/file.c:574
truncate_dnode+0x229/0x2e0 fs/f2fs/node.c:944
f2fs_truncate_inode_blocks+0x64b/0xde0 fs/f2fs/node.c:1154
f2fs_do_truncate_blocks+0x4ac/0xf30 fs/f2fs/file.c:721
f2fs_truncate_blocks+0x7b/0x300 fs/f2fs/file.c:749
f2fs_truncate.part.0+0x4a5/0x630 fs/f2fs/file.c:799
f2fs_truncate include/linux/fs.h:825 [inline]
f2fs_setattr+0x1738/0x2090 fs/f2fs/file.c:1006
notify_change+0xb2c/0x1180 fs/attr.c:483
do_truncate+0x143/0x200 fs/open.c:66
handle_truncate fs/namei.c:3295 [inline]
do_open fs/namei.c:3640 [inline]
path_openat+0x2083/0x2750 fs/namei.c:3791
do_filp_open+0x1ba/0x410 fs/namei.c:3818
do_sys_openat2+0x16d/0x4c0 fs/open.c:1356
do_sys_open fs/open.c:1372 [inline]
__do_sys_creat fs/open.c:1448 [inline]
__se_sys_creat fs/open.c:1442 [inline]
__x64_sys_creat+0xcd/0x120 fs/open.c:1442
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

The root cause is, inodeA references inodeB via inodeB's ino, once inodeA
is truncated, it calls truncate_dnode() to truncate data blocks in inodeB's
node page, it traverse mapping data from node->i.i_addr[0] to
node->i.i_addr[ADDRS_PER_BLOCK() - 1], result in out-of-boundary access.

This patch fixes to add sanity check on dnode page in truncate_dnode(),
so that, it can help to avoid triggering such issue, and once it encounters
such issue, it will record newly introduced ERROR_INVALID_NODE_REFERENCE
error into superblock, later fsck can detect such issue and try repairing.

Also, it removes f2fs_truncate_data_blocks() for cleanup due to the
function has only one caller, and uses f2fs_truncate_data_blocks_range()
instead.

Reported-and-tested-by: syzbot+12cb4425b22169b52036@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/linux-f2fs-devel/000000000000f3038a05fef867f8@google.com
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 4260c406 Wed Feb 24 12:03:13 MST 2021 Gustavo A. R. Silva <gustavoars@kernel.org> f2fs: Replace one-element array with flexible-array member

There is a regular need in the kernel to provide a way to declare having
a dynamically sized set of trailing elements in a structure. Kernel code
should always use “flexible array members”[1] for these cases. The older
style of one-element or zero-length arrays should no longer be used[2].

Refactor the code according to the use of a flexible-array member in
struct f2fs_checkpoint, instead of a one-element arrays.

Notice that a temporary pointer to void '*tmp_ptr' was used in order to
fix the following errors when using a flexible array instead of a one
element array in struct f2fs_checkpoint:

CC [M] fs/f2fs/dir.o
In file included from fs/f2fs/dir.c:13:
fs/f2fs/f2fs.h: In function ‘__bitmap_ptr’:
fs/f2fs/f2fs.h:2227:40: error: invalid use of flexible array member
2227 | return &ckpt->sit_nat_version_bitmap + offset + sizeof(__le32);
| ^
fs/f2fs/f2fs.h:2227:49: error: invalid use of flexible array member
2227 | return &ckpt->sit_nat_version_bitmap + offset + sizeof(__le32);
| ^
fs/f2fs/f2fs.h:2238:40: error: invalid use of flexible array member
2238 | return &ckpt->sit_nat_version_bitmap + offset;
| ^
make[2]: *** [scripts/Makefile.build:287: fs/f2fs/dir.o] Error 1
make[1]: *** [scripts/Makefile.build:530: fs/f2fs] Error 2
make: *** [Makefile:1819: fs] Error 2

[1] https://en.wikipedia.org/wiki/Flexible_array_member
[2] https://www.kernel.org/doc/html/v5.9/process/deprecated.html#zero-length-and-one-element-arrays

Link: https://github.com/KSPP/linux/issues/79
Build-tested-by: kernel test robot <lkp@intel.com>
Link: https://lore.kernel.org/lkml/603647e4.DeEFbl4eqljuwAUe%25lkp@intel.com/
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 4260c406 Wed Feb 24 12:03:13 MST 2021 Gustavo A. R. Silva <gustavoars@kernel.org> f2fs: Replace one-element array with flexible-array member

There is a regular need in the kernel to provide a way to declare having
a dynamically sized set of trailing elements in a structure. Kernel code
should always use “flexible array members”[1] for these cases. The older
style of one-element or zero-length arrays should no longer be used[2].

Refactor the code according to the use of a flexible-array member in
struct f2fs_checkpoint, instead of a one-element arrays.

Notice that a temporary pointer to void '*tmp_ptr' was used in order to
fix the following errors when using a flexible array instead of a one
element array in struct f2fs_checkpoint:

CC [M] fs/f2fs/dir.o
In file included from fs/f2fs/dir.c:13:
fs/f2fs/f2fs.h: In function ‘__bitmap_ptr’:
fs/f2fs/f2fs.h:2227:40: error: invalid use of flexible array member
2227 | return &ckpt->sit_nat_version_bitmap + offset + sizeof(__le32);
| ^
fs/f2fs/f2fs.h:2227:49: error: invalid use of flexible array member
2227 | return &ckpt->sit_nat_version_bitmap + offset + sizeof(__le32);
| ^
fs/f2fs/f2fs.h:2238:40: error: invalid use of flexible array member
2238 | return &ckpt->sit_nat_version_bitmap + offset;
| ^
make[2]: *** [scripts/Makefile.build:287: fs/f2fs/dir.o] Error 1
make[1]: *** [scripts/Makefile.build:530: fs/f2fs] Error 2
make: *** [Makefile:1819: fs] Error 2

[1] https://en.wikipedia.org/wiki/Flexible_array_member
[2] https://www.kernel.org/doc/html/v5.9/process/deprecated.html#zero-length-and-one-element-arrays

Link: https://github.com/KSPP/linux/issues/79
Build-tested-by: kernel test robot <lkp@intel.com>
Link: https://lore.kernel.org/lkml/603647e4.DeEFbl4eqljuwAUe%25lkp@intel.com/
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 4260c406 Wed Feb 24 12:03:13 MST 2021 Gustavo A. R. Silva <gustavoars@kernel.org> f2fs: Replace one-element array with flexible-array member

There is a regular need in the kernel to provide a way to declare having
a dynamically sized set of trailing elements in a structure. Kernel code
should always use “flexible array members”[1] for these cases. The older
style of one-element or zero-length arrays should no longer be used[2].

Refactor the code according to the use of a flexible-array member in
struct f2fs_checkpoint, instead of a one-element arrays.

Notice that a temporary pointer to void '*tmp_ptr' was used in order to
fix the following errors when using a flexible array instead of a one
element array in struct f2fs_checkpoint:

CC [M] fs/f2fs/dir.o
In file included from fs/f2fs/dir.c:13:
fs/f2fs/f2fs.h: In function ‘__bitmap_ptr’:
fs/f2fs/f2fs.h:2227:40: error: invalid use of flexible array member
2227 | return &ckpt->sit_nat_version_bitmap + offset + sizeof(__le32);
| ^
fs/f2fs/f2fs.h:2227:49: error: invalid use of flexible array member
2227 | return &ckpt->sit_nat_version_bitmap + offset + sizeof(__le32);
| ^
fs/f2fs/f2fs.h:2238:40: error: invalid use of flexible array member
2238 | return &ckpt->sit_nat_version_bitmap + offset;
| ^
make[2]: *** [scripts/Makefile.build:287: fs/f2fs/dir.o] Error 1
make[1]: *** [scripts/Makefile.build:530: fs/f2fs] Error 2
make: *** [Makefile:1819: fs] Error 2

[1] https://en.wikipedia.org/wiki/Flexible_array_member
[2] https://www.kernel.org/doc/html/v5.9/process/deprecated.html#zero-length-and-one-element-arrays

Link: https://github.com/KSPP/linux/issues/79
Build-tested-by: kernel test robot <lkp@intel.com>
Link: https://lore.kernel.org/lkml/603647e4.DeEFbl4eqljuwAUe%25lkp@intel.com/
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 4260c406 Wed Feb 24 12:03:13 MST 2021 Gustavo A. R. Silva <gustavoars@kernel.org> f2fs: Replace one-element array with flexible-array member

There is a regular need in the kernel to provide a way to declare having
a dynamically sized set of trailing elements in a structure. Kernel code
should always use “flexible array members”[1] for these cases. The older
style of one-element or zero-length arrays should no longer be used[2].

Refactor the code according to the use of a flexible-array member in
struct f2fs_checkpoint, instead of a one-element arrays.

Notice that a temporary pointer to void '*tmp_ptr' was used in order to
fix the following errors when using a flexible array instead of a one
element array in struct f2fs_checkpoint:

CC [M] fs/f2fs/dir.o
In file included from fs/f2fs/dir.c:13:
fs/f2fs/f2fs.h: In function ‘__bitmap_ptr’:
fs/f2fs/f2fs.h:2227:40: error: invalid use of flexible array member
2227 | return &ckpt->sit_nat_version_bitmap + offset + sizeof(__le32);
| ^
fs/f2fs/f2fs.h:2227:49: error: invalid use of flexible array member
2227 | return &ckpt->sit_nat_version_bitmap + offset + sizeof(__le32);
| ^
fs/f2fs/f2fs.h:2238:40: error: invalid use of flexible array member
2238 | return &ckpt->sit_nat_version_bitmap + offset;
| ^
make[2]: *** [scripts/Makefile.build:287: fs/f2fs/dir.o] Error 1
make[1]: *** [scripts/Makefile.build:530: fs/f2fs] Error 2
make: *** [Makefile:1819: fs] Error 2

[1] https://en.wikipedia.org/wiki/Flexible_array_member
[2] https://www.kernel.org/doc/html/v5.9/process/deprecated.html#zero-length-and-one-element-arrays

Link: https://github.com/KSPP/linux/issues/79
Build-tested-by: kernel test robot <lkp@intel.com>
Link: https://lore.kernel.org/lkml/603647e4.DeEFbl4eqljuwAUe%25lkp@intel.com/
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 4c8ff709 Fri Nov 01 04:07:14 MDT 2019 Chao Yu <chao@kernel.org> f2fs: support data compression

This patch tries to support compression in f2fs.

- New term named cluster is defined as basic unit of compression, file can
be divided into multiple clusters logically. One cluster includes 4 << n
(n >= 0) logical pages, compression size is also cluster size, each of
cluster can be compressed or not.

- In cluster metadata layout, one special flag is used to indicate cluster
is compressed one or normal one, for compressed cluster, following metadata
maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
data including compress header and compressed data.

- In order to eliminate write amplification during overwrite, F2FS only
support compression on write-once file, data can be compressed only when
all logical blocks in file are valid and cluster compress ratio is lower
than specified threshold.

- To enable compression on regular inode, there are three ways:
* chattr +c file
* chattr +c dir; touch dir/file
* mount w/ -o compress_extension=ext; touch file.ext

Compress metadata layout:
[Dnode Structure]
+-----------------------------------------------+
| cluster 1 | cluster 2 | ......... | cluster N |
+-----------------------------------------------+
. . . .
. . . .
. Compressed Cluster . . Normal Cluster .
+----------+---------+---------+---------+ +---------+---------+---------+---------+
|compr flag| block 1 | block 2 | block 3 | | block 1 | block 2 | block 3 | block 4 |
+----------+---------+---------+---------+ +---------+---------+---------+---------+
. .
. .
. .
+-------------+-------------+----------+----------------------------+
| data length | data chksum | reserved | compressed data |
+-------------+-------------+----------+----------------------------+

Changelog:

20190326:
- fix error handling of read_end_io().
- remove unneeded comments in f2fs_encrypt_one_page().

20190327:
- fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
- don't jump into loop directly to avoid uninitialized variables.
- add TODO tag in error path of f2fs_write_cache_pages().

20190328:
- fix wrong merge condition in f2fs_read_multi_pages().
- check compressed file in f2fs_post_read_required().

20190401
- allow overwrite on non-compressed cluster.
- check cluster meta before writing compressed data.

20190402
- don't preallocate blocks for compressed file.

- add lz4 compress algorithm
- process multiple post read works in one workqueue
Now f2fs supports processing post read work in multiple workqueue,
it shows low performance due to schedule overhead of multiple
workqueue executing orderly.

20190921
- compress: support buffered overwrite
C: compress cluster flag
V: valid block address
N: NEW_ADDR

One cluster contain 4 blocks

before overwrite after overwrite

- VVVV -> CVNN
- CVNN -> VVVV

- CVNN -> CVNN
- CVNN -> CVVV

- CVVV -> CVNN
- CVVV -> CVVV

20191029
- add kconfig F2FS_FS_COMPRESSION to isolate compression related
codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
note that: will remove lzo backend if Jaegeuk agreed that too.
- update codes according to Eric's comments.

20191101
- apply fixes from Jaegeuk

20191113
- apply fixes from Jaegeuk
- split workqueue for fsverity

20191216
- apply fixes from Jaegeuk

20200117
- fix to avoid NULL pointer dereference

[Jaegeuk Kim]
- add tracepoint for f2fs_{,de}compress_pages()
- fix many bugs and add some compression stats
- fix overwrite/mmap bugs
- address 32bit build error, reported by Geert.
- bug fixes when handling errors and i_compressed_blocks

Reported-by: <noreply@ellerman.id.au>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 4c8ff709 Fri Nov 01 04:07:14 MDT 2019 Chao Yu <chao@kernel.org> f2fs: support data compression

This patch tries to support compression in f2fs.

- New term named cluster is defined as basic unit of compression, file can
be divided into multiple clusters logically. One cluster includes 4 << n
(n >= 0) logical pages, compression size is also cluster size, each of
cluster can be compressed or not.

- In cluster metadata layout, one special flag is used to indicate cluster
is compressed one or normal one, for compressed cluster, following metadata
maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
data including compress header and compressed data.

- In order to eliminate write amplification during overwrite, F2FS only
support compression on write-once file, data can be compressed only when
all logical blocks in file are valid and cluster compress ratio is lower
than specified threshold.

- To enable compression on regular inode, there are three ways:
* chattr +c file
* chattr +c dir; touch dir/file
* mount w/ -o compress_extension=ext; touch file.ext

Compress metadata layout:
[Dnode Structure]
+-----------------------------------------------+
| cluster 1 | cluster 2 | ......... | cluster N |
+-----------------------------------------------+
. . . .
. . . .
. Compressed Cluster . . Normal Cluster .
+----------+---------+---------+---------+ +---------+---------+---------+---------+
|compr flag| block 1 | block 2 | block 3 | | block 1 | block 2 | block 3 | block 4 |
+----------+---------+---------+---------+ +---------+---------+---------+---------+
. .
. .
. .
+-------------+-------------+----------+----------------------------+
| data length | data chksum | reserved | compressed data |
+-------------+-------------+----------+----------------------------+

Changelog:

20190326:
- fix error handling of read_end_io().
- remove unneeded comments in f2fs_encrypt_one_page().

20190327:
- fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
- don't jump into loop directly to avoid uninitialized variables.
- add TODO tag in error path of f2fs_write_cache_pages().

20190328:
- fix wrong merge condition in f2fs_read_multi_pages().
- check compressed file in f2fs_post_read_required().

20190401
- allow overwrite on non-compressed cluster.
- check cluster meta before writing compressed data.

20190402
- don't preallocate blocks for compressed file.

- add lz4 compress algorithm
- process multiple post read works in one workqueue
Now f2fs supports processing post read work in multiple workqueue,
it shows low performance due to schedule overhead of multiple
workqueue executing orderly.

20190921
- compress: support buffered overwrite
C: compress cluster flag
V: valid block address
N: NEW_ADDR

One cluster contain 4 blocks

before overwrite after overwrite

- VVVV -> CVNN
- CVNN -> VVVV

- CVNN -> CVNN
- CVNN -> CVVV

- CVVV -> CVNN
- CVVV -> CVVV

20191029
- add kconfig F2FS_FS_COMPRESSION to isolate compression related
codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
note that: will remove lzo backend if Jaegeuk agreed that too.
- update codes according to Eric's comments.

20191101
- apply fixes from Jaegeuk

20191113
- apply fixes from Jaegeuk
- split workqueue for fsverity

20191216
- apply fixes from Jaegeuk

20200117
- fix to avoid NULL pointer dereference

[Jaegeuk Kim]
- add tracepoint for f2fs_{,de}compress_pages()
- fix many bugs and add some compression stats
- fix overwrite/mmap bugs
- address 32bit build error, reported by Geert.
- bug fixes when handling errors and i_compressed_blocks

Reported-by: <noreply@ellerman.id.au>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 4c8ff709 Fri Nov 01 04:07:14 MDT 2019 Chao Yu <chao@kernel.org> f2fs: support data compression

This patch tries to support compression in f2fs.

- New term named cluster is defined as basic unit of compression, file can
be divided into multiple clusters logically. One cluster includes 4 << n
(n >= 0) logical pages, compression size is also cluster size, each of
cluster can be compressed or not.

- In cluster metadata layout, one special flag is used to indicate cluster
is compressed one or normal one, for compressed cluster, following metadata
maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
data including compress header and compressed data.

- In order to eliminate write amplification during overwrite, F2FS only
support compression on write-once file, data can be compressed only when
all logical blocks in file are valid and cluster compress ratio is lower
than specified threshold.

- To enable compression on regular inode, there are three ways:
* chattr +c file
* chattr +c dir; touch dir/file
* mount w/ -o compress_extension=ext; touch file.ext

Compress metadata layout:
[Dnode Structure]
+-----------------------------------------------+
| cluster 1 | cluster 2 | ......... | cluster N |
+-----------------------------------------------+
. . . .
. . . .
. Compressed Cluster . . Normal Cluster .
+----------+---------+---------+---------+ +---------+---------+---------+---------+
|compr flag| block 1 | block 2 | block 3 | | block 1 | block 2 | block 3 | block 4 |
+----------+---------+---------+---------+ +---------+---------+---------+---------+
. .
. .
. .
+-------------+-------------+----------+----------------------------+
| data length | data chksum | reserved | compressed data |
+-------------+-------------+----------+----------------------------+

Changelog:

20190326:
- fix error handling of read_end_io().
- remove unneeded comments in f2fs_encrypt_one_page().

20190327:
- fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
- don't jump into loop directly to avoid uninitialized variables.
- add TODO tag in error path of f2fs_write_cache_pages().

20190328:
- fix wrong merge condition in f2fs_read_multi_pages().
- check compressed file in f2fs_post_read_required().

20190401
- allow overwrite on non-compressed cluster.
- check cluster meta before writing compressed data.

20190402
- don't preallocate blocks for compressed file.

- add lz4 compress algorithm
- process multiple post read works in one workqueue
Now f2fs supports processing post read work in multiple workqueue,
it shows low performance due to schedule overhead of multiple
workqueue executing orderly.

20190921
- compress: support buffered overwrite
C: compress cluster flag
V: valid block address
N: NEW_ADDR

One cluster contain 4 blocks

before overwrite after overwrite

- VVVV -> CVNN
- CVNN -> VVVV

- CVNN -> CVNN
- CVNN -> CVVV

- CVVV -> CVNN
- CVVV -> CVVV

20191029
- add kconfig F2FS_FS_COMPRESSION to isolate compression related
codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
note that: will remove lzo backend if Jaegeuk agreed that too.
- update codes according to Eric's comments.

20191101
- apply fixes from Jaegeuk

20191113
- apply fixes from Jaegeuk
- split workqueue for fsverity

20191216
- apply fixes from Jaegeuk

20200117
- fix to avoid NULL pointer dereference

[Jaegeuk Kim]
- add tracepoint for f2fs_{,de}compress_pages()
- fix many bugs and add some compression stats
- fix overwrite/mmap bugs
- address 32bit build error, reported by Geert.
- bug fixes when handling errors and i_compressed_blocks

Reported-by: <noreply@ellerman.id.au>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 4c8ff709 Fri Nov 01 04:07:14 MDT 2019 Chao Yu <chao@kernel.org> f2fs: support data compression

This patch tries to support compression in f2fs.

- New term named cluster is defined as basic unit of compression, file can
be divided into multiple clusters logically. One cluster includes 4 << n
(n >= 0) logical pages, compression size is also cluster size, each of
cluster can be compressed or not.

- In cluster metadata layout, one special flag is used to indicate cluster
is compressed one or normal one, for compressed cluster, following metadata
maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
data including compress header and compressed data.

- In order to eliminate write amplification during overwrite, F2FS only
support compression on write-once file, data can be compressed only when
all logical blocks in file are valid and cluster compress ratio is lower
than specified threshold.

- To enable compression on regular inode, there are three ways:
* chattr +c file
* chattr +c dir; touch dir/file
* mount w/ -o compress_extension=ext; touch file.ext

Compress metadata layout:
[Dnode Structure]
+-----------------------------------------------+
| cluster 1 | cluster 2 | ......... | cluster N |
+-----------------------------------------------+
. . . .
. . . .
. Compressed Cluster . . Normal Cluster .
+----------+---------+---------+---------+ +---------+---------+---------+---------+
|compr flag| block 1 | block 2 | block 3 | | block 1 | block 2 | block 3 | block 4 |
+----------+---------+---------+---------+ +---------+---------+---------+---------+
. .
. .
. .
+-------------+-------------+----------+----------------------------+
| data length | data chksum | reserved | compressed data |
+-------------+-------------+----------+----------------------------+

Changelog:

20190326:
- fix error handling of read_end_io().
- remove unneeded comments in f2fs_encrypt_one_page().

20190327:
- fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
- don't jump into loop directly to avoid uninitialized variables.
- add TODO tag in error path of f2fs_write_cache_pages().

20190328:
- fix wrong merge condition in f2fs_read_multi_pages().
- check compressed file in f2fs_post_read_required().

20190401
- allow overwrite on non-compressed cluster.
- check cluster meta before writing compressed data.

20190402
- don't preallocate blocks for compressed file.

- add lz4 compress algorithm
- process multiple post read works in one workqueue
Now f2fs supports processing post read work in multiple workqueue,
it shows low performance due to schedule overhead of multiple
workqueue executing orderly.

20190921
- compress: support buffered overwrite
C: compress cluster flag
V: valid block address
N: NEW_ADDR

One cluster contain 4 blocks

before overwrite after overwrite

- VVVV -> CVNN
- CVNN -> VVVV

- CVNN -> CVNN
- CVNN -> CVVV

- CVVV -> CVNN
- CVVV -> CVVV

20191029
- add kconfig F2FS_FS_COMPRESSION to isolate compression related
codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
note that: will remove lzo backend if Jaegeuk agreed that too.
- update codes according to Eric's comments.

20191101
- apply fixes from Jaegeuk

20191113
- apply fixes from Jaegeuk
- split workqueue for fsverity

20191216
- apply fixes from Jaegeuk

20200117
- fix to avoid NULL pointer dereference

[Jaegeuk Kim]
- add tracepoint for f2fs_{,de}compress_pages()
- fix many bugs and add some compression stats
- fix overwrite/mmap bugs
- address 32bit build error, reported by Geert.
- bug fixes when handling errors and i_compressed_blocks

Reported-by: <noreply@ellerman.id.au>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
diff 4c8ff709 Fri Nov 01 04:07:14 MDT 2019 Chao Yu <chao@kernel.org> f2fs: support data compression

This patch tries to support compression in f2fs.

- New term named cluster is defined as basic unit of compression, file can
be divided into multiple clusters logically. One cluster includes 4 << n
(n >= 0) logical pages, compression size is also cluster size, each of
cluster can be compressed or not.

- In cluster metadata layout, one special flag is used to indicate cluster
is compressed one or normal one, for compressed cluster, following metadata
maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
data including compress header and compressed data.

- In order to eliminate write amplification during overwrite, F2FS only
support compression on write-once file, data can be compressed only when
all logical blocks in file are valid and cluster compress ratio is lower
than specified threshold.

- To enable compression on regular inode, there are three ways:
* chattr +c file
* chattr +c dir; touch dir/file
* mount w/ -o compress_extension=ext; touch file.ext

Compress metadata layout:
[Dnode Structure]
+-----------------------------------------------+
| cluster 1 | cluster 2 | ......... | cluster N |
+-----------------------------------------------+
. . . .
. . . .
. Compressed Cluster . . Normal Cluster .
+----------+---------+---------+---------+ +---------+---------+---------+---------+
|compr flag| block 1 | block 2 | block 3 | | block 1 | block 2 | block 3 | block 4 |
+----------+---------+---------+---------+ +---------+---------+---------+---------+
. .
. .
. .
+-------------+-------------+----------+----------------------------+
| data length | data chksum | reserved | compressed data |
+-------------+-------------+----------+----------------------------+

Changelog:

20190326:
- fix error handling of read_end_io().
- remove unneeded comments in f2fs_encrypt_one_page().

20190327:
- fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
- don't jump into loop directly to avoid uninitialized variables.
- add TODO tag in error path of f2fs_write_cache_pages().

20190328:
- fix wrong merge condition in f2fs_read_multi_pages().
- check compressed file in f2fs_post_read_required().

20190401
- allow overwrite on non-compressed cluster.
- check cluster meta before writing compressed data.

20190402
- don't preallocate blocks for compressed file.

- add lz4 compress algorithm
- process multiple post read works in one workqueue
Now f2fs supports processing post read work in multiple workqueue,
it shows low performance due to schedule overhead of multiple
workqueue executing orderly.

20190921
- compress: support buffered overwrite
C: compress cluster flag
V: valid block address
N: NEW_ADDR

One cluster contain 4 blocks

before overwrite after overwrite

- VVVV -> CVNN
- CVNN -> VVVV

- CVNN -> CVNN
- CVNN -> CVVV

- CVVV -> CVNN
- CVVV -> CVVV

20191029
- add kconfig F2FS_FS_COMPRESSION to isolate compression related
codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
note that: will remove lzo backend if Jaegeuk agreed that too.
- update codes according to Eric's comments.

20191101
- apply fixes from Jaegeuk

20191113
- apply fixes from Jaegeuk
- split workqueue for fsverity

20191216
- apply fixes from Jaegeuk

20200117
- fix to avoid NULL pointer dereference

[Jaegeuk Kim]
- add tracepoint for f2fs_{,de}compress_pages()
- fix many bugs and add some compression stats
- fix overwrite/mmap bugs
- address 32bit build error, reported by Geert.
- bug fixes when handling errors and i_compressed_blocks

Reported-by: <noreply@ellerman.id.au>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>

Completed in 694 milliseconds