storage revision 1.28
1$NetBSD: storage,v 1.28 2021/10/21 13:21:53 andvar Exp $ 2 3NetBSD Storage Roadmap 4====================== 5 6This is a small roadmap document, and deals with the storage and file 7systems side of the operating system. It discusses elements, projects, 8and goals that are under development or under discussion; and it is 9divided into three categories based on perceived priority. 10 11The following elements, projects, and goals are considered strategic 12priorities for the project: 13 14 1. Improving iscsi 15 2. nfsv4 support 16 3. A better journaling file system solution 17 4. Getting zfs working for real 18 5. Seamless full-disk encryption 19 6. Finish tls-maxphys 20 21The following elements, projects, and goals are not strategic 22priorities but are still important undertakings worth doing: 23 24 7. nvme support 25 8. lfs64 26 9. Per-process namespaces 27 10. lvm tidyup 28 11. Flash translation layer 29 12. Shingled disk support 30 13. ext3/ext4 support 31 14. Port hammer from Dragonfly 32 15. afs maintenance 33 16. execute-in-place 34 17. extended attributes for acl and capability storage 35 36The following elements, projects, and goals are perhaps less pressing; 37this doesn't mean one shouldn't work on them but the expected payoff 38is perhaps less than for other things: 39 40 18. coda maintenance 41 42 43Explanations 44============ 45 461. Improving iscsi 47------------------ 48 49Both the existing iscsi target and initiator are fairly bad code, and 50neither works terribly well. Fixing this is fairly important as iscsi 51is where it's at for remote block devices. Note that there appears to 52be no compelling reason to move the target to the kernel or otherwise 53make major architectural changes. 54 55 - As of January 2017 nobody is known to be working on this. 56 - There is currently no clear timeframe or release target. 57 - Contact agc for further information. 58 59 602. nfsv4 support 61---------------- 62 63nfsv4 is at this point the de facto standard for FS-level (as opposed 64to block-level) network volumes in production settings. The legacy nfs 65code currently in NetBSD only supports nfsv2 and nfsv3. 66 67The intended plan is to port FreeBSD's nfsv4 code, which also includes 68nfsv2 and nfsv3 support, and eventually transition to it completely, 69dropping our current nfs code. (Which is kind of a mess.) So far the 70only step that has been taken is to import the code from FreeBSD. The 71next step is to update that import (since it was done a while ago now) 72and then work on getting it to configure and compile. 73 74 - As of January 2017 pgoyette has done a bit of prodding of the code 75 recently, but otherwise nobody is working on this, and a volunteer to 76 take charge and move it forward rapidly is urgently needed. 77 - There is no clear timeframe or release target, although having an 78 experimental version ready for -8 would be great. 79 - Contact dholland for further information. 80 81 823. A better journaling file system solution 83------------------------------------------- 84 85WAPBL, the journaling FFS that NetBSD rolled out some time back, has a 86critical problem: it does not address the historic ffs behavior of 87allowing stale on-disk data to leak into user files in crashes. And 88because it runs faster, this happens more often and with more data. 89This situation is both a correctness and a security liability. Fixing 90it has turned out to be difficult. It is not really clear what the 91best option at this point is: 92 93+ Fixing WAPBL (e.g. to flush newly allocated/newly written blocks to 94disk early) has been examined by several people who know the code base 95and judged difficult. Also, some other problems have come to light 96more recently; e.g. PR 50725, and 45676. Still, it might be the best 97way forward. Some performance and stability issues were resolved 98in netbsd-8, and more work is planned. 99 100+ There is another journaling FFS; the Harvard one done by Margo 101Seltzer's group some years back. We have a copy of this, but as it was 102written in BSD/OS circa 1999 it needs a lot of merging, and then will 103undoubtedly also need a certain amount of polishing to be ready for 104production use. It does record-based rather than block-based 105journaling and does not share the stale data problem. 106 107+ We could bring back softupdates (in the softupdates-with-journaling 108form found today in FreeBSD) -- this code is even more complicated 109than the softupdates code we removed back in 2009, and it's not clear 110that it's any more robust either. However, it would solve the stale 111data problem if someone wanted to port it over. It isn't clear that 112this would be any less work than getting the Harvard journaling FFS 113running... or than writing a whole new file system either. 114 115+ We could write a whole new journaling file system. (That is, not 116FFS. Doing a new journaling FFS implementation is probably not 117sensible relative to merging the Harvard journaling FFS.) This is a 118big project. 119 120Right now it is not clear which of these avenues is the best way 121forward. Given the general manpower shortage, it may be that the best 122way is whatever looks best to someone who wants to work on the 123problem. 124 125 - There is no clear timeframe or release target; but given that WAPBL 126 has been disabled by default for new installs in -7 this problem 127 can reasonably be said to have become critical. 128 - jdolecek fixed some WAPBL stability issues, that work is included 129 in netbsd-8, could be possibly enough for making it default for new 130 installs again; there is kern/47030 which seems to be triggered by WAPBL 131 however 132 - There has been some interest in the Harvard journaling FFS but no 133 significant progress. Nobody is known to be working on or particularly 134 interested in porting softupdates-with-journaling. And, while 135 dholland has been mumbling for some time about a plan for a 136 specific new file system to solve this problem, there isn't any 137 realistic prospect of significant progress on that in the 138 foreseeable future, and nobody else is known to have or be working 139 on even that much. 140 - Contact joerg, martin, or jdolecek regarding WAPBL; contact dholland 141 regarding the Harvard journaling FFS. 142 143 1444. Getting zfs working for real 145------------------------------- 146 147ZFS has been almost working for years now. It is high time we got it 148really working. One of the things this entails is updating the ZFS 149code, as what we have is rather old. The Illumos version is probably 150what we want for this. 151 152 - There has been intermittent work on zfs, but as of January 2017 153 nobody is known to be actively working on it 154 - There is no clear timeframe or release target. 155 - Contact riastradh or ?? for further information. 156 157 1585. Seamless full-disk encryption 159-------------------------------- 160 161(This is only sort of a storage issue.) We have cgd, and it is 162believed to still be cryptographically suitable, at least for the time 163being. However, we don't have any of the following things: 164 165+ An easy way to install a machine with full-disk encryption. It 166should really just be a checkbox item in sysinst, or not much more 167than that. 168 169+ Ideally, also an easy way to turn on full-disk encryption for a 170machine that's already been installed, though this is harder. 171 172+ A good story for booting off a disk that is otherwise encrypted; 173obviously one cannot encrypt the bootblocks, but it isn't clear where 174in boot the encrypted volume should take over, or how to make a best 175effort at protecting the unencrypted elements needed to boot. (At 176least, in the absence of something like UEFI secure boot combined with 177an cryptographic oracle to sign your bootloader image so UEFI will 178accept it.) There's also the question of how one runs cgdconfig(8) and 179where the cgdconfig binary comes from. 180 181+ A reasonable way to handle volume passphrases. MacOS apparently uses 182login passwords for this (or as passphrases for secondary keys, or 183something) and this seems to work well enough apart from the somewhat 184surreal experience of sometimes having to log in twice. However, it 185will complicate the bootup story. 186 187Given the increasing regulatory-level importance of full-disk 188encryption, this is at least a de facto requirement for using NetBSD 189on laptops in many circumstances. 190 191 - As of January 2017 nobody is known to be working on this. 192 - There is no clear timeframe or release target. 193 - Contact dholland for further information. 194 195 1966. Finish tls-maxphys 197--------------------- 198 199The tls-maxphys branch changes MAXPHYS (the maximum size of a single 200I/O request) from a global fixed constant to a value that's probed 201separately for each particular I/O channel based on its 202capabilities. Large values are highly desirable for e.g. feeding large 203disk arrays and SSDs, but do not work with all hardware. 204 205The code is nearly done and just needs more testing and support in 206more drivers. 207 208 - On October 2017 jdolecek re-synced the branch, intention is to wrap 209 this up for future netbsd-9 210 - Contact jdolecek or tls for further information. 211 212 2137. nvme support 214---------------- 215 216nvme ("NVM Express") is a hardware interface standard for PCI-attached 217SSDs. NetBSD now has a driver for these. 218 219Driver is now MPSAFE and uses bufq fcfs (i.e. no disksort()) already, 220so the most obvious software bottlenecks were treated. It still needs 221more testing on real hardware, and it may be good to investigate some further 222optimizations, such as DragonFly pbuf(9) or something similar. 223 224Semi-relatedly, it is also time for scsipi to become MPSAFE. 225 226 - As of May 2016 a port of OpenBSD's driver has been committed. This 227 will be in -8. 228 - The nvme driver is a backend to ld(4) and is fully is MPSAFE, but we 229 still need to attend to I/O path bottlenecks like kern/53124. 230 Better instrumentation is needed. 231 - Contact msaitoh, agc, or jdolecek for further information. 232 233 2348. lfs64 235-------- 236 237LFS currently only supports volumes up to 2 TB. As LFS is of interest 238for use on shingled disks (which are larger than 2 TB) and also for 239use on disk arrays (ditto) this is something of a problem. A 64-bit 240version of LFS for large volumes is in the works. 241 242 - dholland was working on this in fall 2015 but time to finish it 243 dried up. 244 - The goal now is to get a few remaining things done in time for 8.0 245 so it will at least be ready for experimental use there. 246 - Responsible: dholland 247 248 2499. Per-process namespaces 250------------------------- 251 252Support for per-process variation of the file system namespace enables 253a number of things; more flexible chroots, for example, and also 254potentially more efficient pkgsrc builds. dholland thought up a 255somewhat hackish but low-footprint way to implement this, and has a 256preliminary implementation, but concluded the scheme was too fragile 257for production. A different approach is probably needed, although the 258existing code could be tidied up and committed if that seems desirable. 259 260 - As of January 2017 nobody is working on this. 261 - Contact: dholland 262 263 26410. lvm tidyup 265-------------- 266 267[agc says someone should look at our lvm stuff; XXX fill this in] 268 269 - As of January 2017 nobody is known to be working on this. 270 - There is no clear timeframe or release target. 271 - Contact agc for further information. 272 273 27411. Flash translation layer 275--------------------------- 276 277SSDs ship with firmware called a "flash translation layer" that 278arbitrates between the block device software expects to see and the 279raw flash chips. FTLs handle wear leveling, lifetime management, and 280also internal caching, striping, and other performance concerns. While 281NetBSD has a file system for raw flash (chfs), it seems that given 282things NetBSD is often used for it ought to come with a flash 283translation layer as well. 284 285Note that this is an area where writing your own is probably a bad 286plan; it is a complicated area with a lot of prior art that's also 287reportedly full of patent mines. There are a couple of open FTL 288implementations that we might be able to import. 289 290 - As of January 2017 nobody is known to be working on this. 291 - There is no clear timeframe or release target. 292 - Contact dholland for further information. 293 294 29512. Shingled disk support 296------------------------- 297 298Shingled disks (or more technically, disks with "shingled magnetic 299recording" or SMR) can only write whole tracks at once. Thus, to 300operate effectively they require translation support similar to the 301flash translation layers found in SSDs. The nature and structure of 302shingle translation layers is still being researched; however, at some 303point we will want to support these things in NetBSD. 304 305 - As of 2016 one of dholland's coworkers was looking at this. 306 - There is no clear timeframe or release target. 307 - Contact dholland for further information. 308 309 31013. ext3/ext4 support 311--------------------- 312 313We would like to be able to read and write Linux ext3fs and ext4fs 314volumes. (We can already read clean ext3fs volumes as they're the same 315as ext2fs, modulo volume features our ext2fs code does not support; 316but we can't write them.) 317 318Ideally someone would write ext3 and/or ext4 code, whether integrated 319with or separate from the ext2 code we already have. It might also 320make sense to port or wrap the Linux ext3 or ext4 code so it can be 321loaded as a GPL'd kernel module; it isn't clear if that would be more 322or less work than doing an implementation. 323 324Note however that implementing ext3 has already defeated several 325people; this is a harder project than it looks. 326 327 - GSoc 2016 brought support for extents, and also ro support for dir 328 hashes; jdolecek also implemented several frequently used ext4 features 329 so most contemporary ext filesystems should be possible to mount 330 read-write 331 - still need rw dir_nhash and xattr (semi-easy), and eventually journaling 332 (hard) 333 - There is no clear timeframe or release target. 334 - jdolecek is working on improving ext3/ext4 support (particularly 335 journaling) 336 337 33814. Port hammer from Dragonfly 339------------------------------ 340 341While the motivation for and role of hammer isn't perhaps super 342persuasive, it would still be good to have it. Porting it from 343Dragonfly is probably not that painful (compared to, say, zfs) but as 344the Dragonfly and NetBSD VFS layers have diverged in different 345directions from the original 4.4BSD, may not be entirely trivial 346either. 347 348 - As of January 2017 nobody is known to be working on this. 349 - There is no clear timeframe or release target. 350 - There probably isn't any particular person to contact; for VFS 351 concerns contact dholland or hannken. 352 353 35415. afs maintenance 355------------------- 356 357AFS needs periodic care and feeding to continue working as NetBSD 358changes, because the kernel-level bits aren't kept in the NetBSD tree 359and don't get updated with other things. This is an ongoing issue that 360always seems to need more manpower than it gets. It might make sense 361to import some of the kernel AFS code, or maybe even just some of the 362glue layer that it uses, in order to keep it more current. 363 364 - jakllsch sometimes works on this. 365 - We would like every release to have working AFS by the time it's 366 released. 367 - Contact jakllsch or gendalia about AFS; for VFS concerns contact 368 dholland or hannken. 369 370 37116. execute-in-place 372-------------------- 373 374It is likely that the future includes non-volatile storage (so-called 375"nvram") that looks like RAM from the perspective of software. Most 376importantly: the storage is memory-mapped rather than looking like a 377disk controller. There are a number of things NetBSD ought to have to 378be ready for this, of which probably the most important is 379"execute-in-place": when an executable is run from such storage, and 380mapped into user memory with mmap, the storage hardware pages should 381be able to appear directly in user memory. Right now they get 382gratuitously copied into RAM, which is slow and wasteful. There are 383also other reasons (e.g. embedded device ROMs) to want execute-in- 384place support. 385 386Note that at the implementation level this is a UVM issue rather than 387strictly a storage issue. 388 389Also note that one does not need access to nvram hardware to work on 390this issue; given the performance profiles touted for nvram 391technologies, a plain RAM disk like md(4) is sufficient both 392structurally and for performance analysis. 393 394 - As of January 2017 nobody is known to be working on this. Some 395 time back, uebayasi wrote some preliminary patches, but they were 396 rejected by the UVM maintainers. 397 - There is no clear timeframe or release target. 398 - Contact dholland for further information. 399 400 40117. use extended attributes for ACL and capability storage 402---------------------------------------------------------- 403 404Currently there is some support for extended attributes in ffs, 405but nothing really uses it. I would be nice if we came up with 406a standard format to store ACL's and capabilities like Linux has. 407The various tools must be modified to understand this and be able 408to copy them if requested. Also tools to manipulate the data will 409need to be written. 410 411 41218. coda maintenance 413-------------------- 414 415Coda only sort of works. [And I think it's behind relative to 416upstream, or something of the sort; XXX fill this in.] Also the code 417appears to have an ugly incestuous relationship with FFS. This should 418really be cleaned up. That or maybe it's time to remove Coda. 419 420 - As of January 2017 nobody is known to be working on this. 421 - There is no clear timeframe or release target. 422 - There isn't anyone in particular to contact. 423 - Circa 2012 christos made it work read-write and split it 424 into modules. Since then christos has not tested it. 425 426 427Alistair Crooks, David Holland 428Fri Nov 20 02:17:53 EST 2015 429Sun May 1 16:50:42 EDT 2016 (some updates) 430Fri Jan 13 00:40:50 EST 2017 (some more updates) 431 432