storage revision 1.23
1$NetBSD: storage,v 1.23 2017/10/31 19:03:32 jdolecek Exp $ 2 3NetBSD Storage Roadmap 4====================== 5 6This is a small roadmap document, and deals with the storage and file 7systems side of the operating system. It discusses elements, projects, 8and goals that are under development or under discussion; and it is 9divided into three categories based on perceived priority. 10 11The following elements, projects, and goals are considered strategic 12priorities for the project: 13 14 1. Improving iscsi 15 2. nfsv4 support 16 3. A better journaling file system solution 17 4. Getting zfs working for real 18 5. Seamless full-disk encryption 19 6. Finish tls-maxphys 20 21The following elements, projects, and goals are not strategic 22priorities but are still important undertakings worth doing: 23 24 7. nvme support 25 8. lfs64 26 9. Per-process namespaces 27 10. lvm tidyup 28 11. Flash translation layer 29 12. Shingled disk support 30 13. ext3/ext4 support 31 14. Port hammer from Dragonfly 32 15. afs maintenance 33 16. execute-in-place 34 17. extended attributes for acl and capability storage 35 36The following elements, projects, and goals are perhaps less pressing; 37this doesn't mean one shouldn't work on them but the expected payoff 38is perhaps less than for other things: 39 40 18. coda maintenance 41 42 43Explanations 44============ 45 461. Improving iscsi 47------------------ 48 49Both the existing iscsi target and initiator are fairly bad code, and 50neither works terribly well. Fixing this is fairly important as iscsi 51is where it's at for remote block devices. Note that there appears to 52be no compelling reason to move the target to the kernel or otherwise 53make major architectural changes. 54 55 - As of January 2017 nobody is known to be working on this. 56 - There is currently no clear timeframe or release target. 57 - Contact agc for further information. 58 59 602. nfsv4 support 61---------------- 62 63nfsv4 is at this point the de facto standard for FS-level (as opposed 64to block-level) network volumes in production settings. The legacy nfs 65code currently in NetBSD only supports nfsv2 and nfsv3. 66 67The intended plan is to port FreeBSD's nfsv4 code, which also includes 68nfsv2 and nfsv3 support, and eventually transition to it completely, 69dropping our current nfs code. (Which is kind of a mess.) So far the 70only step that has been taken is to import the code from FreeBSD. The 71next step is to update that import (since it was done a while ago now) 72and then work on getting it to configure and compile. 73 74 - As of January 2017 pgoyette has done a bit of prodding of the code 75 recently, but otherwise nobody is working on this, and a volunteer to 76 take charge and move it forward rapidly is urgently needed. 77 - There is no clear timeframe or release target, although having an 78 experimental version ready for -8 would be great. 79 - Contact dholland for further information. 80 81 823. A better journaling file system solution 83------------------------------------------- 84 85WAPBL, the journaling FFS that NetBSD rolled out some time back, has a 86critical problem: it does not address the historic ffs behavior of 87allowing stale on-disk data to leak into user files in crashes. And 88because it runs faster, this happens more often and with more data. 89This situation is both a correctness and a security liability. Fixing 90it has turned out to be difficult. It is not really clear what the 91best option at this point is: 92 93+ Fixing WAPBL (e.g. to flush newly allocated/newly written blocks to 94disk early) has been examined by several people who know the code base 95and judged difficult. Also, some other problems have come to light 96more recently; e.g. PR 50725, and 45676. Still, it might be the best 97way forward. Some performance and stability issues were resolved 98in netbsd-8, and more work is planned. 99 100+ There is another journaling FFS; the Harvard one done by Margo 101Seltzer's group some years back. We have a copy of this, but as it was 102written in BSD/OS circa 1999 it needs a lot of merging, and then will 103undoubtedly also need a certain amount of polishing to be ready for 104production use. It does record-based rather than block-based 105journaling and does not share the stale data problem. 106 107+ We could bring back softupdates (in the softupdates-with-journaling 108form found today in FreeBSD) -- this code is even more complicated 109than the softupdates code we removed back in 2009, and it's not clear 110that it's any more robust either. However, it would solve the stale 111data problem if someone wanted to port it over. It isn't clear that 112this would be any less work than getting the Harvard journaling FFS 113running... or than writing a whole new file system either. 114 115+ We could write a whole new journaling file system. (That is, not 116FFS. Doing a new journaling FFS implementation is probably not 117sensible relative to merging the Harvard journaling FFS.) This is a 118big project. 119 120Right now it is not clear which of these avenues is the best way 121forward. Given the general manpower shortage, it may be that the best 122way is whatever looks best to someone who wants to work on the 123problem. 124 125 - There is no clear timeframe or release target; but given that WAPBL 126 has been disabled by default for new installs in -7 this problem 127 can reasonably be said to have become critical. 128 - jdolecek fixed some WAPBL stability issues, that work is included 129 in netbsd-8, could be possibly enough for making it default for new 130 installs again; there is kern/47030 which seems to be triggered by WAPBL 131 however 132 - There has been some interest in the Harvard journaling FFS but no 133 significant progress. Nobody is known to be working on or particularly 134 interested in porting softupdates-with-journaling. And, while 135 dholland has been mumbling for some time about a plan for a 136 specific new file system to solve this problem, there isn't any 137 realistic prospect of significant progress on that in the 138 foreseeable future, and nobody else is known to have or be working 139 on even that much. 140 - Contact joerg, martin, or jdolecek regarding WAPBL; contact dholland 141 regarding the Harvard journaling FFS. 142 143 1444. Getting zfs working for real 145------------------------------- 146 147ZFS has been almost working for years now. It is high time we got it 148really working. One of the things this entails is updating the ZFS 149code, as what we have is rather old. The Illumos version is probably 150what we want for this. 151 152 - There has been intermittent work on zfs, but as of January 2017 153 nobody is known to be actively working on it 154 - There is no clear timeframe or release target. 155 - Contact riastradh or ?? for further information. 156 157 1585. Seamless full-disk encryption 159-------------------------------- 160 161(This is only sort of a storage issue.) We have cgd, and it is 162believed to still be cryptographically suitable, at least for the time 163being. However, we don't have any of the following things: 164 165+ An easy way to install a machine with full-disk encryption. It 166should really just be a checkbox item in sysinst, or not much more 167than that. 168 169+ Ideally, also an easy way to turn on full-disk encryption for a 170machine that's already been installed, though this is harder. 171 172+ A good story for booting off a disk that is otherwise encrypted; 173obviously one cannot encrypt the bootblocks, but it isn't clear where 174in boot the encrypted volume should take over, or how to make a best 175effort at protecting the unencrypted elements needed to boot. (At 176least, in the absence of something like UEFI secure boot combined with 177an cryptographic oracle to sign your bootloader image so UEFI will 178accept it.) There's also the question of how one runs cgdconfig(8) and 179where the cgdconfig binary comes from. 180 181+ A reasonable way to handle volume passphrases. MacOS apparently uses 182login passwords for this (or as passphrases for secondary keys, or 183something) and this seems to work well enough apart from the somewhat 184surreal experience of sometimes having to log in twice. However, it 185will complicate the bootup story. 186 187Given the increasing regulatory-level importance of full-disk 188encryption, this is at least a de facto requirement for using NetBSD 189on laptops in many circumstances. 190 191 - As of January 2017 nobody is known to be working on this. 192 - There is no clear timeframe or release target. 193 - Contact dholland for further information. 194 195 1966. Finish tls-maxphys 197--------------------- 198 199The tls-maxphys branch changes MAXPHYS (the maximum size of a single 200I/O request) from a global fixed constant to a value that's probed 201separately for each particular I/O channel based on its 202capabilities. Large values are highly desirable for e.g. feeding large 203disk arrays and SSDs, but do not work with all hardware. 204 205The code is nearly done and just needs more testing and support in 206more drivers. 207 208 - As of January 2017 nobody is known to be working on this. 209 - There is no clear timeframe or release target. 210 - Contact tls for further information. 211 212 2137. nvme suppport 214---------------- 215 216nvme ("NVM Express") is a hardware interface standard for PCI-attached 217SSDs. NetBSD now has a driver for these. 218 219Driver is now MPSAFE and uses bufq fcfs (i.e. no disksort()) already, 220so the most obvious software bottlenecks were treated. It still needs 221more testing on real hardware, and it may be good to investigate some further 222optimizations, such as DragonFly pbuf(9) or something similar. 223 224Semi-relatedly, it is also time for scsipi to become MPSAFE. 225 226 - As of May 2016 a port of OpenBSD's driver has been commited. This 227 will be in -8. 228 - The nvme driver is a backend to ld(4) which is MPSAFE, but we still 229 need to attend to I/O path bottlenecks. Better instrumentation 230 is needed. 231 - Flush cache commands via DIOCCACHESYNC currently doesn't wait for completion; 232 it must not poll since that corrupts command queue, but it should use 233 a condition variable to wait for the flush to actually finish 234 - There is no clear timeframe or release target for these points. 235 - Contact msaitoh or agc for further information. 236 237 2388. lfs64 239-------- 240 241LFS currently only supports volumes up to 2 TB. As LFS is of interest 242for use on shingled disks (which are larger than 2 TB) and also for 243use on disk arrays (ditto) this is something of a problem. A 64-bit 244version of LFS for large volumes is in the works. 245 246 - dholland was working on this in fall 2015 but time to finish it 247 dried up. 248 - The goal now is to get a few remaining things done in time for 8.0 249 so it will at least be ready for experimental use there. 250 - Responsible: dholland 251 252 2539. Per-process namespaces 254------------------------- 255 256Support for per-process variation of the file system namespace enables 257a number of things; more flexible chroots, for example, and also 258potentially more efficient pkgsrc builds. dholland thought up a 259somewhat hackish but low-footprint way to implement this, and has a 260preliminary implementation, but concluded the scheme was too fragile 261for production. A different approach is probably needed, although the 262existing code could be tidied up and committed if that seems desirable. 263 264 - As of January 2017 nobody is working on this. 265 - Contact: dholland 266 267 26810. lvm tidyup 269-------------- 270 271[agc says someone should look at our lvm stuff; XXX fill this in] 272 273 - As of January 2017 nobody is known to be working on this. 274 - There is no clear timeframe or release target. 275 - Contact agc for further information. 276 277 27811. Flash translation layer 279--------------------------- 280 281SSDs ship with firmware called a "flash translation layer" that 282arbitrates between the block device software expects to see and the 283raw flash chips. FTLs handle wear leveling, lifetime management, and 284also internal caching, striping, and other performance concerns. While 285NetBSD has a file system for raw flash (chfs), it seems that given 286things NetBSD is often used for it ought to come with a flash 287translation layer as well. 288 289Note that this is an area where writing your own is probably a bad 290plan; it is a complicated area with a lot of prior art that's also 291reportedly full of patent mines. There are a couple of open FTL 292implementations that we might be able to import. 293 294 - As of January 2017 nobody is known to be working on this. 295 - There is no clear timeframe or release target. 296 - Contact dholland for further information. 297 298 29912. Shingled disk support 300------------------------- 301 302Shingled disks (or more technically, disks with "shingled magnetic 303recording" or SMR) can only write whole tracks at once. Thus, to 304operate effectively they require translation support similar to the 305flash translation layers found in SSDs. The nature and structure of 306shingle translation layers is still being researched; however, at some 307point we will want to support these things in NetBSD. 308 309 - As of 2016 one of dholland's coworkers was looking at this. 310 - There is no clear timeframe or release target. 311 - Contact dholland for further information. 312 313 31413. ext3/ext4 support 315--------------------- 316 317We would like to be able to read and write Linux ext3fs and ext4fs 318volumes. (We can already read clean ext3fs volumes as they're the same 319as ext2fs, modulo volume features our ext2fs code does not support; 320but we can't write them.) 321 322Ideally someone would write ext3 and/or ext4 code, whether integrated 323with or separate from the ext2 code we already have. It might also 324make sense to port or wrap the Linux ext3 or ext4 code so it can be 325loaded as a GPL'd kernel module; it isn't clear if that would be more 326or less work than doing an implementation. 327 328Note however that implementing ext3 has already defeated several 329people; this is a harder project than it looks. 330 331 - GSoc 2016 brought support for extents, and also ro support for dir 332 hashes; jdolecek also implemented several frequently used ext4 features 333 so most contemporary ext filesystems should be possible to mount 334 read-write 335 - still need rw dir_nhash and xattr (semi-easy), and eventually journalling 336 (hard) 337 - There is no clear timeframe or release target. 338 - jdolecek is working on improving ext3/ext4 support (particularily 339 journalling) 340 341 34214. Port hammer from Dragonfly 343------------------------------ 344 345While the motivation for and role of hammer isn't perhaps super 346persuasive, it would still be good to have it. Porting it from 347Dragonfly is probably not that painful (compared to, say, zfs) but as 348the Dragonfly and NetBSD VFS layers have diverged in different 349directions from the original 4.4BSD, may not be entirely trivial 350either. 351 352 - As of January 2017 nobody is known to be working on this. 353 - There is no clear timeframe or release target. 354 - There probably isn't any particular person to contact; for VFS 355 concerns contact dholland or hannken. 356 357 35815. afs maintenance 359------------------- 360 361AFS needs periodic care and feeding to continue working as NetBSD 362changes, because the kernel-level bits aren't kept in the NetBSD tree 363and don't get updated with other things. This is an ongoing issue that 364always seems to need more manpower than it gets. It might make sense 365to import some of the kernel AFS code, or maybe even just some of the 366glue layer that it uses, in order to keep it more current. 367 368 - jakllsch sometimes works on this. 369 - We would like every release to have working AFS by the time it's 370 released. 371 - Contact jakllsch or gendalia about AFS; for VFS concerns contact 372 dholland or hannken. 373 374 37516. execute-in-place 376-------------------- 377 378It is likely that the future includes non-volatile storage (so-called 379"nvram") that looks like RAM from the perspective of software. Most 380importantly: the storage is memory-mapped rather than looking like a 381disk controller. There are a number of things NetBSD ought to have to 382be ready for this, of which probably the most important is 383"execute-in-place": when an executable is run from such storage, and 384mapped into user memory with mmap, the storage hardware pages should 385be able to appear directly in user memory. Right now they get 386gratuitously copied into RAM, which is slow and wasteful. There are 387also other reasons (e.g. embedded device ROMs) to want execute-in- 388place support. 389 390Note that at the implementation level this is a UVM issue rather than 391strictly a storage issue. 392 393Also note that one does not need access to nvram hardware to work on 394this issue; given the performance profiles touted for nvram 395technologies, a plain RAM disk like md(4) is sufficient both 396structurally and for performance analysis. 397 398 - As of January 2017 nobody is known to be working on this. Some 399 time back, uebayasi wrote some preliminary patches, but they were 400 rejected by the UVM maintainers. 401 - There is no clear timeframe or release target. 402 - Contact dholland for further information. 403 404 40517. use extended attributes for ACL and capability storage 406---------------------------------------------------------- 407 408Currently there is some support for extended attributes in ffs, 409but nothing really uses it. I would be nice if we came up with 410a standard format to store ACL's and capabilities like Linux has. 411The various tools must be modified to understand this and be able 412to copy them if requested. Also tools to manipulate the data will 413need to be written. 414 415 41618. coda maintenance 417-------------------- 418 419Coda only sort of works. [And I think it's behind relative to 420upstream, or something of the sort; XXX fill this in.] Also the code 421appears to have an ugly incestuous relationship with FFS. This should 422really be cleaned up. That or maybe it's time to remove Coda. 423 424 - As of January 2017 nobody is known to be working on this. 425 - There is no clear timeframe or release target. 426 - There isn't anyone in particular to contact. 427 - Circa 2012 christos made it work read-write and split it 428 into modules. Since then christos has not tested it. 429 430 431Alistair Crooks, David Holland 432Fri Nov 20 02:17:53 EST 2015 433Sun May 1 16:50:42 EDT 2016 (some updates) 434Fri Jan 13 00:40:50 EST 2017 (some more updates) 435 436