storage revision 1.20
1$NetBSD: storage,v 1.20 2016/11/10 21:28:15 jdolecek Exp $ 2 3NetBSD Storage Roadmap 4====================== 5 6This is a small roadmap document, and deals with the storage and file 7systems side of the operating system. It discusses elements, projects, 8and goals that are under development or under discussion; and it is 9divided into three categories based on perceived priority. 10 11The following elements, projects, and goals are considered strategic 12priorities for the project: 13 14 1. Improving iscsi 15 2. nfsv4 support 16 3. A better journaling file system solution 17 4. Getting zfs working for real 18 5. Seamless full-disk encryption 19 6. Finish tls-maxphys 20 21The following elements, projects, and goals are not strategic 22priorities but are still important undertakings worth doing: 23 24 7. nvme support 25 8. lfs64 26 9. Per-process namespaces 27 10. lvm tidyup 28 11. Flash translation layer 29 12. Shingled disk support 30 13. ext3/ext4 support 31 14. Port hammer from Dragonfly 32 15. afs maintenance 33 16. execute-in-place 34 17. extended attributes for acl and capability storage 35 36The following elements, projects, and goals are perhaps less pressing; 37this doesn't mean one shouldn't work on them but the expected payoff 38is perhaps less than for other things: 39 40 18. coda maintenance 41 42 43Explanations 44============ 45 461. Improving iscsi 47------------------ 48 49Both the existing iscsi target and initiator are fairly bad code, and 50neither works terribly well. Fixing this is fairly important as iscsi 51is where it's at for remote block devices. Note that there appears to 52be no compelling reason to move the target to the kernel or otherwise 53make major architectural changes. 54 55 - As of November 2015 nobody is known to be working on this. 56 - There is currently no clear timeframe or release target. 57 - Contact agc for further information. 58 59 602. nfsv4 support 61---------------- 62 63nfsv4 is at this point the de facto standard for FS-level (as opposed 64to block-level) network volumes in production settings. The legacy nfs 65code currently in NetBSD only supports nfsv2 and nfsv3. 66 67The intended plan is to port FreeBSD's nfsv4 code, which also includes 68nfsv2 and nfsv3 support, and eventually transition to it completely, 69dropping our current nfs code. (Which is kind of a mess.) So far the 70only step that has been taken is to import the code from FreeBSD. The 71next step is to update that import (since it was done a while ago now) 72and then work on getting it to configure and compile. 73 74 - As of November 2015 nobody is working on this, and a volunteer to 75 take charge is urgently needed. 76 - There is no clear timeframe or release target, although having an 77 experimental version ready for -8 would be great. 78 - Contact dholland for further information. 79 80 813. A better journaling file system solution 82------------------------------------------- 83 84WAPBL, the journaling FFS that NetBSD rolled out some time back, has a 85critical problem: it does not address the historic ffs behavior of 86allowing stale on-disk data to leak into user files in crashes. And 87because it runs faster, this happens more often and with more data. 88This situation is both a correctness and a security liability. Fixing 89it has turned out to be difficult. It is not really clear what the 90best option at this point is: 91 92+ Fixing WAPBL (e.g. to flush newly allocated/newly written blocks to 93disk early) has been examined by several people who know the code base 94and judged difficult. Also, some other problems have come to light 95more recently; e.g. PR 50725, and 45676. Still, it might be the best 96way forward. 97 98+ There is another journaling FFS; the Harvard one done by Margo 99Seltzer's group some years back. We have a copy of this, but as it was 100written in BSD/OS circa 1999 it needs a lot of merging, and then will 101undoubtedly also need a certain amount of polishing to be ready for 102production use. It does record-based rather than block-based 103journaling and does not share the stale data problem. 104 105+ We could bring back softupdates (in the softupdates-with-journaling 106form found today in FreeBSD) -- this code is even more complicated 107than the softupdates code we removed back in 2009, and it's not clear 108that it's any more robust either. However, it would solve the stale 109data problem if someone wanted to port it over. It isn't clear that 110this would be any less work than getting the Harvard journaling FFS 111running... or than writing a whole new file system either. 112 113+ We could write a whole new journaling file system. (That is, not 114FFS. Doing a new journaling FFS implementation is probably not 115sensible relative to merging the Harvard journaling FFS.) This is a 116big project. 117 118Right now it is not clear which of these avenues is the best way 119forward. Given the general manpower shortage, it may be that the best 120way is whatever looks best to someone who wants to work on the 121problem. 122 123 - There has been some interest in the Harvard journaling FFS but no 124 significant progress. Nobody is known to be working on or particularly 125 interested in porting softupdates-with-journaling. And, while 126 dholland has been mumbling for some time about a plan for a 127 specific new file system to solve this problem, there isn't any 128 realistic prospect of significant progress on that in the 129 foreseeable future, and nobody else is known to have or be working 130 on even that much. 131 - There is no clear timeframe or release target; but given that WAPBL 132 has been disabled by default for new installs in -7 this problem 133 can reasonably be said to have become critical. 134 - jdolecek is working on fixing WAPBL, goal is to get WAPBL fixed 135 enough to be safe to re-enable as default for -8 136 - Contact joerg or martin regarding WAPBL; contact dholland regarding 137 the Harvard journaling FFS. 138 139 1404. Getting zfs working for real 141------------------------------- 142 143ZFS has been almost working for years now. It is high time we got it 144really working. One of the things this entails is updating the ZFS 145code, as what we have is rather old. The Illumos version is probably 146what we want for this. 147 148 - There has been intermittent work on zfs, but as of November 2015 149 nobody is known to be actively working on it 150 - There is no clear timeframe or release target. 151 - Contact riastradh or ?? for further information. 152 153 1545. Seamless full-disk encryption 155-------------------------------- 156 157(This is only sort of a storage issue.) We have cgd, and it is 158believed to still be cryptographically suitable, at least for the time 159being. However, we don't have any of the following things: 160 161+ An easy way to install a machine with full-disk encryption. It 162should really just be a checkbox item in sysinst, or not much more 163than that. 164 165+ Ideally, also an easy way to turn on full-disk encryption for a 166machine that's already been installed, though this is harder. 167 168+ A good story for booting off a disk that is otherwise encrypted; 169obviously one cannot encrypt the bootblocks, but it isn't clear where 170in boot the encrypted volume should take over, or how to make a best 171effort at protecting the unencrypted elements needed to boot. (At 172least, in the absence of something like UEFI secure boot combined with 173an cryptographic oracle to sign your bootloader image so UEFI will 174accept it.) There's also the question of how one runs cgdconfig(8) and 175where the cgdconfig binary comes from. 176 177+ A reasonable way to handle volume passphrases. MacOS apparently uses 178login passwords for this (or as passphrases for secondary keys, or 179something) and this seems to work well enough apart from the somewhat 180surreal experience of sometimes having to log in twice. However, it 181will complicate the bootup story. 182 183Given the increasing regulatory-level importance of full-disk 184encryption, this is at least a de facto requirement for using NetBSD 185on laptops in many circumstances. 186 187 - As of November 2015 nobody is known to be working on this. 188 - There is no clear timeframe or release target. 189 - Contact dholland for further information. 190 191 1926. Finish tls-maxphys 193--------------------- 194 195The tls-maxphys branch changes MAXPHYS (the maximum size of a single 196I/O request) from a global fixed constant to a value that's probed 197separately for each particular I/O channel based on its 198capabilities. Large values are highly desirable for e.g. feeding large 199disk arrays but do not work with all hardware. 200 201The code is nearly done and just needs more testing and support in 202more drivers. 203 204 - As of November 2015 nobody is known to be working on this. 205 - There is no clear timeframe or release target. 206 - Contact tls for further information. 207 208 2097. nvme suppport 210---------------- 211 212nvme ("NVM Express") is a hardware interface standard for PCI-attached 213SSDs. NetBSD now has a driver for these. 214 215Driver is now MPSAFE and uses bufq fcfs (i.e. no disksort()) already, 216so the most obvious software bottlenecks were treated. It still needs 217more testing on real hardware, and it may be good to investigate some further 218optimizations, such as DragonFly pbuf(9) or something similar. 219 220Semi-relatedly, it is also time for scsipi to become MPSAFE. 221 222 - As of May 2016 a port of OpenBSD's driver has been commited. This 223 will be in -8. 224 - The nvme driver is a backend to ld(4) which is MPSAFE, but we still 225 need to attend to I/O path bottlenecks. Better instrumentation 226 is needed. 227 - Flush cache commands via DIOCCACHESYNC currently doesn't wait for completion; 228 it must not poll since that corrupts command queue, but it should use 229 a condition variable to wait for the flush to actually finish 230 - NVMe controllers supports write cache administration via GET/SET FEATURE, but 231 driver doesn't currently implement the cache ioctls, leading to somewhat 232 ugly dkctl(1) output; it would be fairly simple to add this, but would 233 require ld(4) attachment code changed to support passing arbitrary ioctls 234 to attachments 235 - There is no clear timeframe or release target for these points. 236 - Contact msaitoh or agc for further information. 237 238 2398. lfs64 240-------- 241 242LFS currently only supports volumes up to 2 TB. As LFS is of interest 243for use on shingled disks (which are larger than 2 TB) and also for 244use on disk arrays (ditto) this is something of a problem. A 64-bit 245version of LFS for large volumes is in the works. 246 247 - As of November 2015 dholland is working on this. 248 - It is close to being ready for at least experimental use and is 249 expected to be in 8.0. 250 - Responsible: dholland 251 252 2539. Per-process namespaces 254------------------------- 255 256Support for per-process variation of the file system namespace enables 257a number of things; more flexible chroots, for example, and also 258potentially more efficient pkgsrc builds. dholland thought up a 259somewhat hackish but low-footprint way to implement this. 260 261 - As of November 2015 dholland is working on this. 262 - It is scheduled to be in 8.0. 263 - Responsible: dholland 264 265 26610. lvm tidyup 267-------------- 268 269[agc says someone should look at our lvm stuff; XXX fill this in] 270 271 - As of November 2015 nobody is known to be working on this. 272 - There is no clear timeframe or release target. 273 - Contact agc for further information. 274 275 27611. Flash translation layer 277--------------------------- 278 279SSDs ship with firmware called a "flash translation layer" that 280arbitrates between the block device software expects to see and the 281raw flash chips. FTLs handle wear leveling, lifetime management, and 282also internal caching, striping, and other performance concerns. While 283NetBSD has a file system for raw flash (chfs), it seems that given 284things NetBSD is often used for it ought to come with a flash 285translation layer as well. 286 287Note that this is an area where writing your own is probably a bad 288plan; it is a complicated area with a lot of prior art that's also 289reportedly full of patent mines. There are a couple of open FTL 290implementations that we might be able to import. 291 292 - As of November 2015 nobody is known to be working on this. 293 - There is no clear timeframe or release target. 294 - Contact dholland for further information. 295 296 29712. Shingled disk support 298------------------------- 299 300Shingled disks (or more technically, disks with "shingled magnetic 301recording" or SMR) can only write whole tracks at once. Thus, to 302operate effectively they require translation support similar to the 303flash translation layers found in SSDs. The nature and structure of 304shingle translation layers is still being researched; however, at some 305point we will want to support these things in NetBSD. 306 307 - As of November 2015 one of dholland's coworkers is looking at this. 308 - There is no clear timeframe or release target. 309 - Contact dholland for further information. 310 311 31213. ext3/ext4 support 313--------------------- 314 315We would like to be able to read and write Linux ext3fs and ext4fs 316volumes. (We can already read clean ext3fs volumes as they're the same 317as ext2fs, modulo volume features our ext2fs code does not support; 318but we can't write them.) 319 320Ideally someone would write ext3 and/or ext4 code, whether integrated 321with or separate from the ext2 code we already have. It might also 322make sense to port or wrap the Linux ext3 or ext4 code so it can be 323loaded as a GPL'd kernel module; it isn't clear if that would be more 324or less work than doing an implementation. 325 326Note however that implementing ext3 has already defeated several 327people; this is a harder project than it looks. 328 329 - GSoc 2016 brought support for extents, and also ro support for dir 330 hashes; jdolecek also implemented several frequently used ext4 features 331 so most contemporary ext filesystems should be possible to mount 332 read-write 333 - still need rw dir_nhash and xattr (semi-easy), and eventually journalling 334 (hard) 335 - There is no clear timeframe or release target. 336 - jdolecek is working on improving ext3/ext4 support (particularily 337 journalling) 338 339 34014. Port hammer from Dragonfly 341------------------------------ 342 343While the motivation for and role of hammer isn't perhaps super 344persuasive, it would still be good to have it. Porting it from 345Dragonfly is probably not that painful (compared to, say, zfs) but as 346the Dragonfly and NetBSD VFS layers have diverged in different 347directions from the original 4.4BSD, may not be entirely trivial 348either. 349 350 - As of November 2015 nobody is known to be working on this. 351 - There is no clear timeframe or release target. 352 - There probably isn't any particular person to contact; for VFS 353 concerns contact dholland or hannken. 354 355 35615. afs maintenance 357------------------- 358 359AFS needs periodic care and feeding to continue working as NetBSD 360changes, because the kernel-level bits aren't kept in the NetBSD tree 361and don't get updated with other things. This is an ongoing issue that 362always seems to need more manpower than it gets. It might make sense 363to import some of the kernel AFS code, or maybe even just some of the 364glue layer that it uses, in order to keep it more current. 365 366 - jakllsch sometimes works on this. 367 - We would like every release to have working AFS by the time it's 368 released. 369 - Contact jakllsch or gendalia about AFS; for VFS concerns contact 370 dholland or hannken. 371 372 37316. execute-in-place 374-------------------- 375 376It is likely that the future includes non-volatile storage (so-called 377"nvram") that looks like RAM from the perspective of software. Most 378importantly: the storage is memory-mapped rather than looking like a 379disk controller. There are a number of things NetBSD ought to have to 380be ready for this, of which probably the most important is 381"execute-in-place": when an executable is run from such storage, and 382mapped into user memory with mmap, the storage hardware pages should 383be able to appear directly in user memory. Right now they get 384gratuitously copied into RAM, which is slow and wasteful. There are 385also other reasons (e.g. embedded device ROMs) to want execute-in- 386place support. 387 388Note that at the implementation level this is a UVM issue rather than 389strictly a storage issue. 390 391Also note that one does not need access to nvram hardware to work on 392this issue; given the performance profiles touted for nvram 393technologies, a plain RAM disk like md(4) is sufficient both 394structurally and for performance analysis. 395 396 - As of November 2015 nobody is known to be working on this. Some 397 time back, uebayasi wrote some preliminary patches, but they were 398 rejected by the UVM maintainers. 399 - There is no clear timeframe or release target. 400 - Contact dholland for further information. 401 402 40317. use extended attributes for ACL and capability storage 404---------------------------------------------------------- 405 406Currently there is some support for extended attributes in ffs, 407but nothing really uses it. I would be nice if we came up with 408a standard format to store ACL's and capabilities like Linux has. 409The various tools must be modified to understand this and be able 410to copy them if requested. Also tools to manipulate the data will 411need to be written. 412 41318. coda maintenance 414-------------------- 415 416Coda only sort of works. [And I think it's behind relative to 417upstream, or something of the sort; XXX fill this in.] Also the code 418appears to have an ugly incestuous relationship with FFS. This should 419really be cleaned up. That or maybe it's time to remove Coda. 420 421 - As of November 2015 nobody is known to be working on this. 422 - There is no clear timeframe or release target. 423 - There isn't anyone in particular to contact. 424 - Circa 2012 christos made it work read-write and split it 425 into modules. Since then christos has not tested it. 426 427Alistair Crooks, David Holland 428Fri Nov 20 02:17:53 EST 2015 429Sun May 1 16:50:42 EDT 2016 (some updates) 430 431