Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r243074:
Add a missing check for fread

Approved by: cperciva (implicit)

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Merge binutils 2.17.50 to head. This brings a number of improvements to
x86 CPU support, better support for powerpc64, some new directives, and
many other things. Bump __FreeBSD_version, and add a note to UPDATING.

Thanks to the many people that have helped to test this.

Obtained from: projects/binutils-2.17

ANSIfy some more tools in usr.sbin/.

Most of these tools build with WARNS=6, except for their use of K&R
function declarations.

Use a ndis_ prefix on the C variable instead of directly using the .sys
filename, this would fail if the filename started with a number.

PR: bin/84911
Submitted by: Fredrik Lindberg

Silence GCC4 signed/unsigned pointer mismatch warning.

Clean up and apply the fix for PR 83477. The calculation for locating
the start of the section headers has to take into account the fact
that the image_nt_header is really variable sized. It happens that
the existing calculation is correct for _most_ production binaries
produced by the Windows DDK, but if we get a binary with oddball
offsets, the PE loader could crash.

Changes from the supplied patch are:

- We don't really need to use the IMAGE_SIZEOF_NT_HEADER() macro when
computing how much of the header to return to callers of
pe_get_optional_header(). While it's important to take the variable
size of the header into account in other calculations, we never
actually look at anything outside the non-variable portion of the
header. This saves callers from having to allocate a variable sized
buffer off the heap (I purposely tried to avoid using malloc()
in subr_pe.c to make it easier to compile in both the -D_KERNEL and
!-D_KERNEL case), and since we're copying into a buffer on the
stack, we always have to copy the same amount of data or else
we'll trash the stack something fierce.

- We need <stddef.h> to get offsetof() in the !-D_KERNEL case.

- ndiscvt.c needs the IMAGE_FIRST_SECTION() macro too, since it does
a little bit of section pre-processing.

PR: kern/83477

Fix a small bug in firmcvt: outfile must be strdup()ed.
Also, add conditional code to allow different invokations for objcopy
depending on whether we're compiled on an i386 arch or amd64 arch, so
that we can produce x86-64 object files on amd64.

Next step on the road to IRPs: create and use an imitation of the
Windows DRIVER_OBJECT and DEVICE_OBJECT mechanism so that we can
simulate driver stacking.

In Windows, each loaded driver image is attached to a DRIVER_OBJECT
structure. Windows uses the registry to match up a given vendor/device
ID combination with a corresponding DRIVER_OBJECT. When a driver image
is first loaded, its DriverEntry() routine is invoked, which sets up
the AddDevice() function pointer in the DRIVER_OBJECT and creates
a dispatch table (based on IRP major codes). When a Windows bus driver
detects a new device, it creates a Physical Device Object (PDO) for
it. This is a DEVICE_OBJECT structure, with semantics analagous to
that of a device_t in FreeBSD. The Windows PNP manager will invoke
the driver's AddDevice() function and pass it pointers to the DRIVER_OBJECT
and the PDO.

The AddDevice() function then creates a new DRIVER_OBJECT structure of
its own. This is known as the Functional Device Object (FDO) and
corresponds roughly to a private softc instance. The driver uses
IoAttachDeviceToDeviceStack() to add this device object to the
driver stack for this PDO. Subsequent drivers (called filter drivers
in Windows-speak) can be loaded which add themselves to the stack.
When someone issues an IRP to a device, it travel along the stack
passing through several possible filter drivers until it reaches
the functional driver (which actually knows how to talk to the hardware)
at which point it will be completed. This is how Windows achieves
driver layering.

Project Evil now simulates most of this. if_ndis now has a modevent
handler which will use MOD_LOAD and MOD_UNLOAD events to drive the
creation and destruction of DRIVER_OBJECTs. (The load event also
does the relocation/dynalinking of the image.) We don't have a registry,
so the DRIVER_OBJECTS are stored in a linked list for now. Eventually,
the list entry will contain the vendor/device ID list extracted from
the .INF file. When ndis_probe() is called and detectes a supported
device, it will create a PDO for the device instance and attach it
to the DRIVER_OBJECT just as in Windows. ndis_attach() will then call
our NdisAddDevice() handler to create the FDO. The NDIS miniport block
is now a device extension hung off the FDO, just as it is in Windows.
The miniport characteristics table is now an extension hung off the
DRIVER_OBJECT as well (the characteristics are the same for all devices
handled by a given driver, so they don't need to be per-instance.)
We also do an IoAttachDeviceToDeviceStack() to put the FDO on the
stack for the PDO. There are a couple of fake bus drivers created
for the PCI and pccard buses. Eventually, there will be one for USB,
which will actually accept USB IRP.s

Things should still work just as before, only now we do things in
the proper order and maintain the correct framework to support passing
IRPs between drivers.

Various changes:

- corrected the comments about IRQL handling in subr_hal.c to more
accurately reflect reality
- update ndiscvt to make the drv_data symbol in ndis_driver_data.h a
global so that if_ndis_pci.o and/or if_ndis_pccard.o can see it.
- Obtain the softc pointer from the miniport block by referencing
the PDO rather than a private pointer of our own (nmb_ifp is no
longer used)
- implement IoAttachDeviceToDeviceStack(), IoDetachDevice(),
IoGetAttachedDevice(), IoAllocateDriverObjectExtension(),
IoGetDriverObjectExtension(), IoCreateDevice(), IoDeleteDevice(),
IoAllocateIrp(), IoReuseIrp(), IoMakeAssociatedIrp(), IoFreeIrp(),
IoInitializeIrp()
- fix a few mistakes in the driver_object and device_object definitions
- add a new module, kern_windrv.c, to handle the driver registration
and relocation/dynalinkign duties (which don't really belong in
kern_ndis.c).
- made ndis_block and ndis_chars in the ndis_softc stucture pointers
and modified all references to it
- fixed NdisMRegisterMiniport() and NdisInitializeWrapper() so they
work correctly with the new driver_object mechanism
- changed ndis_attach() to call NdisAddDevice() instead of ndis_load_driver()
(which is now deprecated)
- used ExAllocatePoolWithTag()/ExFreePool() in lookaside list routines
instead of kludged up alloc/free routines
- added kern_windrv.c to sys/modules/ndis/Makefile and files.i386.

Today, RealTek sent me a driver to test which had been compiled with
some debug support turned on. It turns out the sections in this driver
binary had relative virtual addresses (RVAs) that were different
from the raw addresses, and it had a .data section where the virtual size
was much larger than the raw size. (Most production binaries produced
with the Microsoft DDK have RVA == PA.)

There's code in the ndiscvt(8) utility that's supposed to handle
the vsize != rsize case, but it turns out it was slightly broken,
and it failed to handle the RVA != RA case at all. Hopefully, this
commit will fix all that.

Big mess 'o changes:

- Give ndiscvt(8) the ability to process a .SYS file directly into
a .o file so that we don't have to emit big messy char arrays into
the ndis_driver_data.h file. This behavior is currently optional, but
may become the default some day.

- Give ndiscvt(8) the ability to turn arbitrary files into .ko files
so that they can be pre-loaded or kldloaded. (Both this and the
previous change involve using objcopy(1)).

- Give NdisOpenFile() the ability to 'read' files out of kernel memory
that have been kldloaded or pre-loaded, and disallow the use of
the normal vn_open() file opening method during bootstrap (when no
filesystems have been mounted yet). Some people have reported that
kldloading if_ndis.ko works fine when the system is running multiuser
but causes a panic when the modile is pre-loaded by /boot/loader. This
happens with drivers that need to use NdisOpenFile() to access
external files (i.e. firmware images). NdisOpenFile() won't work
during kernel bootstrapping because no filesystems have been mounted.
To get around this, you can now do the following:

o Say you have a firmware file called firmware.img
o Do: ndiscvt -f firmware.img -- this creates firmware.img.ko
o Put the firmware.img.ko in /boot/kernel
o add firmware.img_load="YES" in /boot/loader.conf
o add if_ndis_load="YES" and ndis_load="YES" as well

Now the loader will suck the additional file into memory as a .ko. The
phony .ko has two symbols in it: filename_start and filename_end, which
are generated by objcopy(1). ndis_open_file() will traverse each module
in the module list looking for these symbols and, if it finds them, it'll
use them to generate the file mapping address and length values that
the caller of NdisOpenFile() wants.

As a bonus, this will even work if the file has been statically linked
into the kernel itself, since the "kernel" module is searched too.
(ndiscvt(8) will generate both filename.o and filename.ko for you).

- Modify the mechanism used to provide make-pretend FASTCALL support.
Rather than using inline assembly to yank the first two arguments
out of %ecx and %edx, we now use the __regparm__(3) attribute (and
the __stdcall__ attribute) and use some macro magic to re-order
the arguments and provide dummy arguments as needed so that the
arguments passed in registers end up in the right place. Change
taken from DragonflyBSD version of the NDISulator.

Add preliminary support for PCMCIA devices in addition to PCI/cardbus.
if_ndis.c has been split into if_ndis_pci.c and if_ndis_pccard.c.
The ndiscvt(8) utility should be able to parse device info for PCMCIA
devices now. The ndis_alloc_amem() has moved from kern_ndis.c to
if_ndis_pccard.c so that kern_ndis.c no longer depends on pccard.

NOTE: this stuff is not guaranteed to work 100% correctly yet. So
far I have been able to load/init my PCMCIA Cisco Aironet 340 card,
but it crashes in the interrupt handler. The existing support for
PCI/cardbus devices should still work as before.

Fix usage() (-d is really -n).

Handle WinNT .inf files with a $windows nt$ signature but no .NT decorated
AddReg sections.

Also insert extra newline after emitting device name overrides.

Clean up ndiscvt a bit (leaving out the -i flag didn't work) and add
copyrights to the inf parser files.

Add a -n flag to ndiscvt to allow the user to override the default
device name of NDIS devices. Instead of "ndis0, ndis1, etc..."
you can have "foo0, foo1, etc..." This allows you to have more than
one kind of NDIS device in the kernel at the same time.

Convert from printf() to device_printf() in if_ndis.c, kern_ndis.c
and subr_ndis.c.

Create UMA zones for ndis_packet and ndis_buffer structs allocated
on transmit. The zones are created and destroyed in the modevent
handler in kern_ndis.c.

printf() and UMA changes submitted by green@freebsd.org

Make ndiscvt(8) emit the binary image array as inline assembly code rather
than a char array. Emitting the data as a big char array works fine in
the typical case, where a .sys file may be ~50K in size. Unfortunately,
some .sys files can be several hundred Kbytes in size, or even several
megabytes in size. One extreme case is the Intel centrino wireless
driver, which is 2.4MB. This causes us to emit an ndis_driver_data.h
file that's on the order of 15MB in size, and gcc consumes enormous
amounts of virtual memory while trying to compile it. On my laptop,
with 128MB of RAM and 256MB of swap space, gcc consumed all available
VM and crashed without being able to compile if_ndis.o.

By emitting the array as assembler, we bypass the C compiler and consume
much less memory. I was able to easily test compile if_ndis.ko with the
centrino driver on my laptop after this change.

This is merely a convenience, and should not have any operational effect
on the NDISulator itself.

Fix getopt() string so -o works.

Commit the ndiscvt(8) utility too. (Missed it in the last import.)