| buf.9 (89124) | buf.9 (107788) |
|---|---|
| 1.\" Copyright (c) 1998 2.\" The Regents of the University of California. All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. --- 15 unchanged lines hidden (view full) --- 24.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 26.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 27.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 28.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 29.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 30.\" SUCH DAMAGE. 31.\" | 1.\" Copyright (c) 1998 2.\" The Regents of the University of California. All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. --- 15 unchanged lines hidden (view full) --- 24.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 26.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 27.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 28.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 29.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 30.\" SUCH DAMAGE. 31.\" |
| 32.\" $FreeBSD: head/share/man/man9/buf.9 89124 2002-01-09 11:43:48Z mpp $ | 32.\" $FreeBSD: head/share/man/man9/buf.9 107788 2002-12-12 17:26:04Z ru $ |
| 33.\" 34.Dd December 22, 1998 35.Dt BUF 9 36.Os 37.Sh NAME 38.Nm buf 39.Nd "kernel buffer I/O scheme used in FreeBSD VM system" 40.Sh DESCRIPTION 41The kernel implements a KVM abstraction of the buffer cache which allows it 42to map potentially disparate vm_page's into contiguous KVM for use by | 33.\" 34.Dd December 22, 1998 35.Dt BUF 9 36.Os 37.Sh NAME 38.Nm buf 39.Nd "kernel buffer I/O scheme used in FreeBSD VM system" 40.Sh DESCRIPTION 41The kernel implements a KVM abstraction of the buffer cache which allows it 42to map potentially disparate vm_page's into contiguous KVM for use by |
| 43(mainly filesystem) devices and device I/O. This abstraction supports | 43(mainly file system) devices and device I/O. This abstraction supports |
| 44block sizes from DEV_BSIZE (usually 512) to upwards of several pages or more. 45It also supports a relatively primitive byte-granular valid range and dirty 46range currently hardcoded for use by NFS. The code implementing the 47VM Buffer abstraction is mostly concentrated in 48.Pa /usr/src/sys/kern/vfs_bio.c . 49.Pp 50One of the most important things to remember when dealing with buffer pointers 51(struct buf) is that the underlying pages are mapped directly from the buffer | 44block sizes from DEV_BSIZE (usually 512) to upwards of several pages or more. 45It also supports a relatively primitive byte-granular valid range and dirty 46range currently hardcoded for use by NFS. The code implementing the 47VM Buffer abstraction is mostly concentrated in 48.Pa /usr/src/sys/kern/vfs_bio.c . 49.Pp 50One of the most important things to remember when dealing with buffer pointers 51(struct buf) is that the underlying pages are mapped directly from the buffer |
| 52cache. No data copying occurs in the scheme proper, though some filesystems | 52cache. No data copying occurs in the scheme proper, though some file systems |
| 53such as UFS do have to copy a little when dealing with file fragments. The 54second most important thing to remember is that due to the underlying page 55mapping, the b_data base pointer in a buf is always *page* aligned, not 56*block* aligned. When you have a VM buffer representing some b_offset and 57b_size, the actual start of the buffer is (b_data + (b_offset & PAGE_MASK)) 58and not just b_data. Finally, the VM system's core buffer cache supports 59valid and dirty bits (m->valid, m->dirty) for pages in DEV_BSIZE chunks. Thus 60a platform with a hardware page size of 4096 bytes has 8 valid and 8 dirty --- 17 unchanged lines hidden (view full) --- 78the (vnode,b_offset,b_size). The kernel typically unmaps VM buffers the moment 79they are no longer needed but often keeps the 'struct buf' structure 80instantiated and even bp->b_pages array instantiated despite having unmapped 81them from KVM. If a page making up a VM buffer is about to undergo I/O, the 82system typically unmaps it from KVM and replaces the page in the b_pages[] 83array with a place-marker called bogus_page. The place-marker forces any kernel 84subsystems referencing the associated struct buf to re-lookup the associated 85page. I believe the place-marker hack is used to allow sophisticated devices | 53such as UFS do have to copy a little when dealing with file fragments. The 54second most important thing to remember is that due to the underlying page 55mapping, the b_data base pointer in a buf is always *page* aligned, not 56*block* aligned. When you have a VM buffer representing some b_offset and 57b_size, the actual start of the buffer is (b_data + (b_offset & PAGE_MASK)) 58and not just b_data. Finally, the VM system's core buffer cache supports 59valid and dirty bits (m->valid, m->dirty) for pages in DEV_BSIZE chunks. Thus 60a platform with a hardware page size of 4096 bytes has 8 valid and 8 dirty --- 17 unchanged lines hidden (view full) --- 78the (vnode,b_offset,b_size). The kernel typically unmaps VM buffers the moment 79they are no longer needed but often keeps the 'struct buf' structure 80instantiated and even bp->b_pages array instantiated despite having unmapped 81them from KVM. If a page making up a VM buffer is about to undergo I/O, the 82system typically unmaps it from KVM and replaces the page in the b_pages[] 83array with a place-marker called bogus_page. The place-marker forces any kernel 84subsystems referencing the associated struct buf to re-lookup the associated 85page. I believe the place-marker hack is used to allow sophisticated devices |
| 86such as filesystem devices to remap underlying pages in order to deal with, | 86such as file system devices to remap underlying pages in order to deal with, |
| 87for example, re-mapping a file fragment into a file block. 88.Pp 89VM buffers are used to track I/O operations within the kernel. Unfortunately, 90the I/O implementation is also somewhat of a hack because the kernel wants 91to clear the dirty bit on the underlying pages the moment it queues the I/O 92to the VFS device, not when the physical I/O is actually initiated. This | 87for example, re-mapping a file fragment into a file block. 88.Pp 89VM buffers are used to track I/O operations within the kernel. Unfortunately, 90the I/O implementation is also somewhat of a hack because the kernel wants 91to clear the dirty bit on the underlying pages the moment it queues the I/O 92to the VFS device, not when the physical I/O is actually initiated. This |
| 93can create confusion within filesystem devices that use delayed-writes because | 93can create confusion within file system devices that use delayed-writes because |
| 94you wind up with pages marked clean that are actually still dirty. If not 95treated carefully, these pages could be thrown away! Indeed, a number of 96serious bugs related to this hack were not fixed until the 2.2.8/3.0 release. 97The kernel uses an instantiated VM buffer (i.e. struct buf) to place-mark pages 98in this special state. The buffer is typically flagged B_DELWRI. When a 99device no longer needs a buffer it typically flags it as B_RELBUF. Due to 100the underlying pages being marked clean, the B_DELWRI|B_RELBUF combination must 101be interpreted to mean that the buffer is still actually dirty and must be --- 23 unchanged lines hidden --- | 94you wind up with pages marked clean that are actually still dirty. If not 95treated carefully, these pages could be thrown away! Indeed, a number of 96serious bugs related to this hack were not fixed until the 2.2.8/3.0 release. 97The kernel uses an instantiated VM buffer (i.e. struct buf) to place-mark pages 98in this special state. The buffer is typically flagged B_DELWRI. When a 99device no longer needs a buffer it typically flags it as B_RELBUF. Due to 100the underlying pages being marked clean, the B_DELWRI|B_RELBUF combination must 101be interpreted to mean that the buffer is still actually dirty and must be --- 23 unchanged lines hidden --- |