system revision 1.4
1$NetBSD: system,v 1.4 2008/03/21 08:53:06 spz Exp $ 2 3NetBSD System Roadmap 4===================== 5 6This is a small roadmap document, and deals with the main system 7aspects of the operating system. 8 9NetBSD 5.0 will ship with the following main changes to the system: 10 111. Modularized scheduler 122. Real-time scheduling classes and priorities 133. Processor sets, processor affinity and processor control 144. Multiprocessor optimized scheduler 155. High-performance 1:1 threading implementation 166. Pushback of the global kernel lock 177. New kernel concurrency model 188. Multiprocessor optimized memory allocators 199. POSIX asynchronous I/O and message queues 2010. In-kernel linker 2111. SysV IPC tuneables 2212. Improved observability: minidumps, lockstat and tprof 2313. Power management framework 24 25We are working on the following elements, and expect them to be in 5.0: 26 2714. Full kernel preemption for real-time threads 2815. 64-bit time values supported 2916. POSIX shared memory 3017. namei() tactical changes 31 32The following projects are expected to be included in NetBSD 6.0 33 3418. Better resource controls 3519. Improved observability: online crashdumps, remote debugging 3620. Processor and cache topology aware scheduler 37 38We expect to branch for the 5.0 release in April or May 2008, which would 39put a NetBSD 5.0 Release date in Q4 2008. An annual release cycle would 40thus mean that 6.0 would be branched in April or May 2009. 41 42Pictorially, this looks like: 43 44 2008 2009 45 Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug 46|===|===|===|===|===|===|===|===|===|===|===|===|===|===|===|===|===|===|===| 47 48 | ------- 5.0 --------- | | ------- 6.0 49 branch release branch 50 51 1-8 11,15-17,18 9,10,12-14 19 20 52 53We'll continue to update this roadmap as features and dates get firmed up. 54 55 56Some explanations 57================= 58 591. Modularized scheduler 60------------------------ 61 62Traditionally the only method of control on process scheduling was the 63'nice' value assigned to each process. The scheduler interface has been 64redesiged to allow for pluggable schedulers, selected at compile time. 65At the current time, there are no plans to switch schedulers at run-time, 66since there is little appreciable gain to be had from that, and the extra 67performance hit to provide this functionality is thought not to be worth 68it. 69 70The in-kernel scheduler interface has been enhanced to provide a framework 71for adding new schedulers, called the common scheduler framework - more 72information can be found in the csf(9) manual page. 73 74Responsible: ad, dsieger, rmind, yamt 75 762. Real-time scheduling classes and priorities 77---------------------------------------------- 78 79The scheduler has been extended to allow provide multiple new priority 80bands, including real-time. POSIX standard interfaces for controlling 81thread priority and scheduling class have been implemented, along with 82a command line tool to allow control by the system administrator. 83 843. Processor sets, processor affinity and processor control 85----------------------------------------------------------- 86 87A Solaris and HP-UX compatible interface for defining and controlling 88processor sets has been added. Processor sets allow applications and 89the administrator complete flexibility in partitioning CPU resources 90among applications, down to thread-level granularity. 91 92Linux compatibile interface controlling processor affinity, similar 93in spirit to processor sets, is provided. 94 95A new utility to control CPU status (cpuctl) is provided. cpuctl 96allows the administrator to enable and disable individual CPUs at 97the software level, while the system is running. It is expected that 98this will in time be extended to support full dynamic reconfiguration, 99in concert with a hypervisor such as Xen. 100 1014. Multiprocessor optimized scheduler 102------------------------------------- 103 104An intelligent, pluggable scheduler named M2 that is optimized for 105multiprocessor systems, supports POSIX real-time extensions, 106time-sharing class, and implements thread affinity. 107 1085. High-performance 1:1 threading implementation 109------------------------------------------------ 110 111A new lightweight 1:1 threading implementation, replacing the M:N based 112implementation found in NetBSD 4.0 and earlier. The new implementation is 113more correct according to POSIX thread standards, and provides a massive 114performance boost to threaded workloads in both uni- and multi-processor 115configurations. 116 1176. Pushback of the global kernel lock 118------------------------------------- 119 120Previously, most access to the kernel was single threaded on multiprocessor 121systems by the global kernel_lock. The kernel_lock has been pushed back to 122to the device driver and wire-protocol layers, providing a significant 123performance boost on heavily loaded multiprocessor systems. 124 1257. New kernel concurrency model 126------------------------------- 127 128The non-preemptive spinlock and "interrupt priority level" synchronization 129model has been replaced wholesale with a hybrid thread/interrupt model. A 130full range of new, lightweight synchronization primitives are available to 131the kernel programmer, including: adaptive mutexes, reader/writer locks, 132memory barriers, atomic operations, threaded soft interrupts, generic cross 133calls, workqueues, priority inheritance, and per-CPU storage. 134 1358. Multiprocessor optimized memory allocators 136--------------------------------------------- 137 138The memory allocators in both the kernel and user space are now fully 139optimized for multiprocessor systems and eliminate the performance 140degradation typically associated with memory allocators in an MP setting. 141 1429. POSIX asynchronous I/O and message queues 143--------------------------------------------- 144 145A full implementation of the POSIX asynchronous I/O and message 146queue facilities is now available. 147 14810. In-kernel linker 149-------------------- 150 151A in-kernel ELF object linker has been added, and a revamped kernel module 152infrastructure developed to accompany it. It is expected that the kernel 153will become completely modular over time, while continuing to retain the 154ability to link to a single binary image for embedded and hobby systems. 155 15611. SysV IPC tuneables 157---------------------- 158 159Parameters for the SVR3-compatible IPC mechanisms can now be tuned 160completely at runtime. 161 16212. Improved observability: minidumps, lockstat and tprof 163--------------------------------------------------------- 164 165The x86 architecture now supports mini crash-dumps as a support aid for 166kernel debugging. Only memory contents actively in use by the kernel at 167the time of crash are dumped to and recovered from disk, an improvement 168over the traditional scheme where the complete contents of memory is 169dumped to disk. 170 171The lockstat and tprof commands have been addded to the system. lockstat 172provides a high-resolution description of lock activity in a running system. 173 174tprof uses sample based profiling in conjuction with the available 175performance counters in order to better profile system activity. 176 17713. Power management framework 178------------------------------ 179 180A new power management framework has been introduced that improves 181handling of device power state transitions. As power management support 182is now integrated with the auto-configuration subsystem, the kernel can 183ensure that a parent device is powered on before attempting to access 184the device. 185 186With these changes comes an updated release of the Intel ACPI 187Component Architecture and an x86 emulator which assists in restoring 188uninitialized display adapters. 189 190Leveraging this work, the i386 and amd64 kernels now support suspend 191to RAM in uni- and multi-processor configurations on ACPI-capable 192machines. This support has been successfully tested on a wide variety of 193laptops, including (but not limited to) recent systems from Dell, IBM/Lenovo, 194Fujitsu, Toshiba, and Sony. 195 196Responsible: jmcneill, joerg 197 19814. Full kernel preemption for real-time threads 199------------------------------------------------ 200 201With the revamp of the kernel concurrency model, much of the kernel is fully 202multi-threaded and can therefore be preempted at any time. In support of 203lower context switch and dispatch times for real-time threads, full kernel 204preemption is being implemented. 205 20615. 64-bit time_t support 207------------------------- 208 209The Unix 32-bit time_t value will overflow in 2037 - any mortgage calculations 210which use a time_t value are in danger of overflowing at the present time - 211and to address this, 64-bit time_t values will be used to contain the number 212of seconds since 1970. 213 214Responsible: christos 215 21616. POSIX shared memory 217----------------------- 218 219Implement POSIX shared memory facilities, which can be used to create the 220shared memory objects and add the memory locations to the address space of 221a process. 222 223Responsible: rmind 224 22517. Incremental namei improvements, Phase 1 226------------------------------------------- 227 228Implement the rest of the changes to namei outlined in Message-ID: 229<20080319053709.GB3951@netbsd.org>. Simplify the locking and behavior 230of namei() calls within the kernel to resolve path names within file 231systems. This phase simplifies the majority of calls to namei(). 232 233Responsible: dholland 234 23518. Better resource controls 236---------------------------- 237 238A resource provisioning and control framework that extends beyond the 239traditional Unix process limits. 240 24119. Improved observability: online crashdumps, remote debugging 242--------------------------------------------------------------- 243 244XXX crashdumps while the system is running 245XXX firewire support in libkvm 246 24720. Processor and cache topology aware scheduler 248------------------------------------------------ 249 250Implement the detection of the topology of the processors and caches. 251Improve the scheduler to make decisions about thread migration 252according to the topology, to get better thread affinity and less 253cache thrashing, and thus improve overall performance in modern SMP 254systems. 255 256Responsible: rmind 257 25829. Incremental namei improvements, Phase 2 259------------------------------------------- 260 261Implement the rest of the changes to namei outlined in Message-ID: 262<20080319053709.GB3951@netbsd.org>. Simplify the locking and behavior 263of namei() calls within the kernel to resolve path names within file 264systems. 265 266Responsible: dholland 267 268 269 270Andrew Doran 271Wed Mar 19 18:26:49 GMT 2008 272