MFC r303738: report sector size and number of sectors in lsdev output
for bios disks

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Add the flags parameter to the disk_open() function and DISK_F_NOCACHE
flag, that disables the caching of partition tables metadata.
Use this flag for floppies in the libi386/biosdisk driver.

Almost each time when loader opens a file, this leads to calling
disk_open(). Very often this is called several times for one file.
This leads to reading partition table metadata for each call. To
reduce the number of disk I/O we have a simple block cache, but it
is very dumb and more than half of I/O operations related to reading
metadata, misses this cache.

Introduce new cache layer to resolve this problem. It is independent
and doesn't need initialization like bcache, and will work by default
for all loaders which use the new DISK API. A successful disk_open()
call to each new disk or partition produces new entry in the cache.
Even more, when disk was already open, now opening of any nested
partitions does not require reading top level partition table.
So, if without this cache, partition table metadata was read around
20-50 times during boot, now it reads only once. This affects the booting
from GPT and MBR from the UFS.

After r239066, reinitialize v86.ctl and v86.addr for int 13 EDD probing
in sys/boot/i386/libi386/biosdisk.c. Otherwise, when DISK_DEBUG is
enabled, the DEBUG() macros will clobber those fields, and cause the
probing to always fail mysteriously when debugging is enabled.

Add another debug message.

Add offset field to the i386_devdesc structure to be compatible with
disk_devdesc structure. Update biosdisk driver to the new disk API.

i386 boot: consolidate MAXBDDEV definition

MFC after: 1 month

- Add a new header for the x86 boot code that defines various structures
and constants related to the BIOS Enhanced Disk Drive Specification.
- Use this header instead of magic numbers and various duplicate structure
definitions for doing I/O.
- Use an actual structure for the request to fetch drive parameters in
drvsize() rather than a gross hack of a char array with some magic
size. While here, change drvsize() to only pass the 1.1 version of
the structure and not request device path information. If we want
device path information you have to set the length of the device
path information as an input (along with probably checking the actual
EDD version to see which size one should use as the device path
information is variable-length). This fixes data smashing problems
from passing an EDD 3 structure to BIOSes supporting EDD 4.

Reviewed by: avg
Tested by: Dennis Koegel dk neveragain.de
MFC after: 1 week

Consolidate duplicate definitions of V86_CY() and V86_ZR() which check for
the carry and zero flags being set, respectively, in <btxv86.h> and use
them throughout the x86 boot code.

Before VirtualBox is fixed, mark with #ifdef what has to be done to make
it possible to boot from ZFS RAIDZ for example from within VirtualBox.
The problem with VirtualBox is that its BIOS reports only one disk present.
If we choose to ignore this report, we can find all the disks available.
We can't have this work-around to be turned on by default, because some broken
BIOSes report true when it comes to number of disks, but present the same disk
multiple times.

Fix debug messages of bd_io().

MFC after: 1 week

On special systems where the MBR and the GPT are in sync (up to the 4th
slicei, Apple EFI hardware), the bootloader will fail to recognize the GPT
if it finds anything else but the EFI partition. Change the check to continue
detecting the GPT by looking at the EFI partition on the MBR but
stopping successfuly after finding it.

PR: kern/134590
Submitted by: Christoph Langguth <christoph at rosenkeller.org>
Reviewed by: jhb
MFC after: 2 weeks
Approved by: re (kib)

Don't attempt to free the GPT partition list for a disk with an empty GPT.

Submitted by: Yuri Pankov yuri.pankov of gmail
MFC after: 3 days

Add a missing parameter when displaying GPT partitions with an unknown
UUID.

Submitted by: Pawel Worach pawel.worach | gmail
MFC after: 1 week

Use a disk address instead of an int to hold the starting offset of an
open partition. This fixes access to partitions whose starting offset
is >= 2 TB.

Submitted by: "James R. Van Artsdalen" james jrv.org
MFC after: 3 days

The recent change to use memory > 1MB for the heap by default broke CD
booting because the CD driver did not use bounce buffers to ensure
request buffers sent to the BIOS were always in the first 1MB. Copy over
the bounce buffer logic from the BIOS disk driver (minus the 64k boundary
code for floppies) to fix this.

Reported by: kensmith

- Make it possible to disable GPT support by setting LOADER_NO_GPT_SUPPORT
in make.conf or src.conf.
- When GPT is enabled (which it is by default), use memory above 1 MB and
leave the memory from the end of the bss to the end of the 640k window
purely for the stack. The loader has grown and now it is much more
common for the heap and stack to grow into each other when both are
located in the 640k window.

PR: kern/129526
MFC after: 1 week

If we free the GPT partition list in bd_open_gpt() because of an error, don't
try to free it again in bd_closedisk(). While I'm here, fix a DEBUG print.

Tweak the verbose disk printing a bit:
- Consolidate the code to humanize the size of a disk partition into a
single function based on the code for GPT partitions and use it for
GPT partitions, BSD slices, and BSD partitions.
- Teach the humanize code to use KB for small partitions (e.g. GPT boot
partitions now show up as 64KB rather than 0MB).
- Pad a few partition type names out so that things line up in the
common case.

MFC after: 1 week

Though we are currently not interested in the EDD3 flag,
Enhanced Disk Drive Specification Ver 3.0 defines that the version
of extension in AH would be 30h.
Correct the check for that to be >=30h instead of >3h.
MFC after: 2 months

Break out of the I/O retry loop as soon as an I/O operation succeeds rather
than always retrying operations three times.

Submitted by: nyan

First cut at support for booting a GPT labeled disk via the BIOS bootstrap
on i386 and amd64 machines. The overall process is that /boot/pmbr lives
in the PMBR (similar to /boot/mbr for MBR disks) and is responsible for
locating and loading /boot/gptboot. /boot/gptboot is similar to /boot/boot
except that it groks GPT rather than MBR + bsdlabel. Unlike /boot/boot,
/boot/gptboot lives in its own dedicated GPT partition with a new
"FreeBSD boot" type. This partition does not have a fixed size in that
/boot/pmbr will load the entire partition into the lower 640k. However,
it is limited in that it can only be 545k. That's still a lot better than
the current 7.5k limit for boot2 on MBR. gptboot mostly acts just like
boot2 in that it reads /boot.config and loads up /boot/loader. Some more
details:
- Include uuid_equal() and uuid_is_nil() in libstand.
- Add a new 'boot' command to gpt(8) which makes a GPT disk bootable using
/boot/pmbr and /boot/gptboot. Note that the disk must have some free
space for the boot partition.
- This required exposing the backend of the 'add' function as a
gpt_add_part() function to the rest of gpt(8). 'boot' uses this to
create a boot partition if needed.
- Don't cripple cgbase() in the UFS boot code for /boot/gptboot so that
it can handle a filesystem > 1.5 TB.
- /boot/gptboot has a simple loader (gptldr) that doesn't do any I/O
unlike boot1 since /boot/pmbr loads all of gptboot up front. The
C portion of gptboot (gptboot.c) has been repocopied from boot2.c.
The primary changes are to parse the GPT to find a root filesystem
and to use 64-bit disk addresses. Currently gptboot assumes that the
first UFS partition on the disk is the / filesystem, but this algorithm
will likely be improved in the future.
- Teach the biosdisk driver in /boot/loader to understand GPT tables.
GPT partitions are identified as 'disk0pX:' (e.g. disk0p2:) which is
similar to the /dev names the kernel uses (e.g. /dev/ad0p2).
- Add a new "freebsd-boot" alias to g_part() for the new boot UUID.

MFC after: 1 month
Discussed with: marcel (some things might still change, but am committing
what I have so far)

Rework the read/write support in the bios disk driver some to cut down
on duplicated code and support 64-bit LBAs for GPT.
- The code to manage an EDD or C/H/S I/O request are now in their own
routines. The EDD routine now handles a full 64-bit LBA instead of
truncating LBAs to the lower 32-bits. (MBRs and BSD labels only
have 32-bit LBAs anyway, so the only LBAs ever passed down were 32-bit).
- All of the bounce buffer and retry logic duplicated in bd_read() and
bd_write() are merged into a single bd_io() routine that takes an
extra direction argument. bd_read() and bd_write() are now simple
wrappers around bd_io().
- If a disk supports EDD then always use it rather than only using it if
the cylinder is > 1023. Other parts of the boot code already do
something similar to this. Also, GPT just uses LBAs, so for a GPT disk
it's probably best to ignore C/H/S completely. Always using EDD when
it is supported by a disk is an easy way to accomplish this.

MFC after: 1 week

Reindent the read/write code of bd_realstrategy() so it is more readable.

MFC after: 1 week

Slightly cleanup the 'bootdev' concept on x86 by changing the various
macros to treat the 'slice' field as a real part of the bootdev instead
of as hack that spans two other fields (adaptor (sic) and controller)
that are not used in any modern FreeBSD boot code.

MFC after: 1 week

Extend struct devdesc with a unit field, called d_unit. Promote the
device (kind) specific unit field to the common field. This change
allows a future version of libefi to work without requiring anything
more than what is defined in struct devdesc and as such makes it
possible to compile said version of libefi for different platforms
without requiring that those platforms have identical derivatives
of struct devdesc.

Increment the disk block offset after writing, not before. This
fixes filesystem corruption when nextboot.conf is located after
cylinder 1023. The bug appears to have been introduced at the time
bd_read was copied to create bd_write.

PR: bin/98005
Reported by: yar
MFC after: 1 week

Long-long time ago, when the trees were large and memory expensive amount of
memory directly available to loader(8) and friends was limited to 640K on i386.
Those times have passed long time ago and now loader(8) can directly access
up to 4GB of RAM at least theoretically. At the same time, there are several
places where it's assumed that malloc() will only allocate memory within
first megabyte.

Remove that assumption by allocating appropriate bounce buffers for BIOS
calls on stack where necessary.

This allows using memory above first megabyte for heap if necessary.

Trap invalid sector size 0 in disk probe, refusing to add such a
device to the list. This prevents crashes on /0 errors in 'lsdev'
et al.

Reviewed-by: jhb@
MT5 after: RE approval

Unspam sys/boot, the dev_t commit should not have touched these.

Spotted by: peter

Do the dreaded s/dev_t/struct cdev */
Bump __FreeBSD_version accordingly.

Premptively fix GCC3.4 issue: add break; after empty default: clauses.

PR: 65493
Submitted by: Pawel Worach <sajd@telia.com>

If a transfer to or from a floppy disk crosses a 64k boundary, we have to
use a bounce buffer for the actual transfer to avoid crossing a 64k
boundary. To do this, we malloc a buffer twice as big as we need and then
find an aligned block within that buffer to do the transfer. The check
to see which part of the block we use used the wrong variable for part of
the condition meaning that in certain edge cases we would ask the BIOS to
cross a 64k boundary. The BIOS request would then fail resulting in file
transfers that just magically fail in the middle without any apparent
reason. Specifically, my tests for the splitfs boot floppies managed to
trigger this edge case.

MFC after: 1 week
X-MFC-info: along with fixes to libstand filesystems

Use __FBSDID().
Also some minor copyright style cleanups.

Libdisk does not need to include <sys/diskslice.h> any more.

Move the remaining bits of <sys/diskslice.h> to <i386/include/bootinfo.h>

Move i386/pc98 specific bits from <sys/reboot.h> to
<i386/include/bootinfo.h> as well.

Adjust includes in sys/boot accordingly.

Use NEXTDOSPART instead of MAX_SLICES.

Correctly recognize both bogus and genuine BSD disklabels.

Don't expect me to participate in a discussion which is which.

Sponsored by: DARPA & NAI Labs.

Split MBR and PC98 on-disk sliceformats out from disklabel.h, step 1:

Peter had repocopied sys/disklabel.h to sys/diskpc98.h and sys/diskmbr.h.

These two new copies are still intact copies of disklabel.h and
therefore protected by #ifndef _SYS_DISKLABEL_H_ so #including them
in programs which already include <sys.disklabel.h> is currently a
no-op.

This commit adds a number of such #includes.

Once I have verified that I have fixed all the places which need fixing,
I will commit the updated versions of the three #include files.

Sponsored by: DARPA & NAI Labs.

Add support for writing to BIOS disks.

PR: kern/32389
Submitted by: Jonathan Mini <mini@haikugeek.com>
Sponsored by: ClickArray, Inc.

Update to C99, s/__FUNCTION__/__func__/,
also don't use ANSI string concatenation.

The Number of Fixed Disks at memory location 0x475 is only 1 byte, not a
2 byte word. This fixes machines that probe 30-odd hard drives during boot
in the loader.

Submitted by: Helpful folks at Tyan via ps

Fix several BDECFLAGS warnings. Also, actually use the 'verbose' parameter
for lsdev when printing out slice information. Now, you only get the
sizes and LBA's of slices if you use 'lsdev -v'.

- Add some more details to the 'lsdev' output for FAT file systems.
- Add in support for the EDD (Enhanced Disk Drive) BIOS extensions to
use LBA mode for accessing drives past cylinder 1024. This should allow
us to load a kernel from anywhere on a newer drive up to 2 TB. Part
of this came from the PR below.

PR: i386/13847
Submitted by: Tor Egge <Tor.Egge@fast.no>

Teach the loader about the ext2fs filesystem, extended partitions, and
the new readdir function.

Mega i386 loader commit.
- Don't hard code 0x10000 as the entry point for the loader. Instead add
src/sys/boot/i386/Makefile.inc which defines a make variable with the
entry point for the loader. Move the loader's entry point up to
0x20000, which makes PXE happy.
- Don't try to use cpp to parse btxldr for the optional BTXLDR_VERBOSE,
instead use m4 to achieve this. Also, add a BTXLDR_VERBOSE knob in the
btxldr Makefile to turn this option on.
- Redo parts of cdldr's Makefile so that it now builds and installs cdboot
instead of having i386/loader/Makefile do that. Also, add in some more
variables to make the pxeldr Makefile almost identical and thus to ease
maintainability.
- Teach cdldr about the a.out format. Cdldr now parsers the a.out header
of the loader binary and relocates it based on that. The entry point of
the loader no longer has to be hardcoded into cdldr. Also, the boot
info table from mkisofs is no longer required to get a useful cdboot.
- Update the lsdev function for BIOS disks to parse other file systems
(such as DOS FAT) that we currently support. This is still buggy as
it assumes that a floppy with a DOS boot sector actually has a MBR and
parses it as such. I'll be fixing this in the future.
- The biggie: Add in support for booting off of PXE-enabled network
adapters. Currently, we use the TFTP API provided by the PXE BIOS.
Eventually we will switch to using the low-level NIC driver thus
allowing both TFTP and NFS to be used, but for now it's just TFTP.

Submitted by: ps, alfred
Testing by: Benno Rice <benno@netizen.com.au>

Remove some extra spammage that made it into this commit. This will be
revisited later with a better fix, or at least one that compiles. :)

Approved by: dcs

Pass an unit number to bcache_strategy, so it can flush the cache when
necessary. Pass an absolute block number too, instead of receiving a
relative one in realstrategy(), as bcache_strategy() requires this.

The fix is sligthly different from the one in the PR.

PR: 17098
Submitted by: John Hood <jhood@sitaranetworks.com>

Remove BAD144 support, it has already been disabled for some time.

$Id$ -> $FreeBSD$

Fill in bi_bios_geom[] in the bootinfo structure passed to the kernel.
This should resolve the problem raised in PR 12315, and incidentally
makes it easier to determine what geometry the BIOS is actually using
(by way of boot -v and dmesg).

Fix a couple of further bugs: missing argument to sprintf() and
"==" used for "=".

Bleh. Missing 'unit =' doesn't help legacy num_ide_disks code.

PR: bin/9651
Submitted by: "Daniel C. Sobral" <dcs@newsguy.com>

biosdisk.c
Implement a new variable 'root_disk_unit' which supersedes
'num_ide_disks' and makes it possible to explicitly set the
root device unit number regardless of type considerations.

bootinfo.c
If we can't calculate a dev_t for the root disk, complain and
don't proceed to boot with an invalid boot device.

Remove redundant line of code.

PR: 9364
Submitted by: Daniel C. Sobral <dcs@newsguy.com>

Extend bd_print() so that it displays slices and partitions on disks.

Add a new variable $num_ide_disks which is used to offset the unit number
for SCSI disks when converting from BIOS unit numbers to da unit numbers.

Prompted by Kevin Street <street@iname.com>

Implement a simple LRU block cache. By default this is initialised to 16k,
and will bypass transfers for more than 8k. Blocks are invalidated after
2 seconds, so removable media should not confuse the cache.

The 8k threshold is a compromise; all UFS transfers performed by
libstand are 8k or less, so large file reads thrash the cache.
However many filesystem metadata operations are also performed using
8k blocks, so using a lower threshold gives poor performance.

Those of you with an eye for cache algorithms are welcome to tell me
how badly this one sucks; you can start with the 'bcachestats' command
which will print the contents of the cache and access statistics.

- Add a new command 'lsdev' to list devices which might be likely to host
filesystems.
- New 'help' command and data in the help.* files (not yet installed),
provides topic and subtopic help, indexes, etc.
- Don't crash if the user tries to set an invalid console. Be helpful
instead.
- Expand tabs (badly) on the i386 video console.
- Some minor cosmetic changes.

Fix an uninitialized variable in the `dangerously dedicated disk' case.

Fix a warning that's been bugging me for ages.

Warn that one of the DEBUG statements has a v86 recursion bug.

Remove some debugging code.
Do a much better job of DWIM with partial device specifications.
Fix the module metadata build process, which was completely broken.
Use a larger read buffer when copying large objects in; this
improves performance marginally and will avoid flushning any small caches
we might choose to implement.

bootinfo.c
Strip any device name information from the kernel name
before passing it in.

biosdisk.c
Be more strict about matching device names to slice entries.
Only allow unsliced syntax on unsliced disks.

biosdisk.c
Allocate space for, and copy, NDOSPART slice entries from the
MBR, not just one. Add some extra debugging while we're at it.

elf_freebsd.c
Initialise the symbol table start/end pointers in case we don't
have them.

aout_freebsd.c
Use bd_getdev() to work out a dev_t for the root device.
Allow $rootdev to override $currdev as the root device.

biosdisk.c
Save the slice table and disklabel when opening a disk.
Add bd_getdev(), which attempts to return a dev_t corresponding
to a given device. Cases which it still doesn't get right:
- The inevitable da-when-wd-also-exists
- Disks with no slice table (the slice number is not set correctly)
The first is difficult to get right, the second will be
fixed in an upcoming commit.

comconsole.c
vidconsole.c
getchar() should return an 8-bit value; some BIOSsen pack extra
information in %eax.

libi386.h
Remove some stale prototypes, add new ones.

MBR magic is 0x55aa not 0xffaa.

Precedence bug (?) causing probe problems.

The BIOS can't perform a floppy operation where the destination crosses
a physical 64k boundary. Allocate a bounce buffer for such a transaction.

- Improve debugging code.
- Make the "what do we do with a drunken disklabel" if-then-else-regardless
tangle easier to read.
- Don't count on the v86 structure being preserved between loop iterations,
as it may be trampled eg. by the DEBUG call.

Synch with development version. Compiles and opens but doesn't work yet.

We lost all the files in crt/, so define the BIOS sector size here instead.

Initial integration of the i386 bootloader and BTX.

- Discard large amounts of BIOS-related code in favour of the more compact
BTX vm86 interface.
- Build the loader module as ELF, although the resulting object is a.out,
make gensetdefs 32/64-bit sensitive and use a single copy of it.
- Throw away installboot, as it's no longer required.
- Use direct bcopy operations in the i386_copy module, as BTX
maps the first 16M of memory. Check operations against the
detected size of actual memory.

This commit was generated by cvs2svn to compensate for changes in r38465,
which included commits to RCS files with non-trunk default branches.

This is the new unified bootstrap, sometimes known previously as the
'three-stage' bootstrap.
There are a number of caveats with the code in its current state:
- The i386 bootstrap only supports booting from a floppy.
- The kernel and kld do not yet know how to deal with the extended
information and module summary passed in.
- PnP-based autodetection and demand loading of modules is not implemented.
- i386 ELF kernel loading is not ready yet.
- The i386 bootstrap is loaded via an ugly blockmap.

On the alpha, both net- and disk-booting (SRM console machines only) is
supported. No blockmaps are used by this code.

Obtained from: Parts from the NetBSD/i386 standalone bootstrap.