boot/loader: relocate kernel to virtual address space on 32-bit architectures

Change-Id: Ic86ca6874f358df8755a0c691cccb155d145e06c
Reviewed-on: https://review.haiku-os.org/c/haiku/+/4675
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: Fredrik Holmqvist <fredrik.holmqvist@gmail.com>

system: Build using public elf.h header

Reduce duplication of code by

* Removing from elf_common.h definitions available in os/kernel/elf.h
* Deleting elf32.h and elf64.h
* Renaming elf_common.h to elf_private.h
* Updating source to build using public and private ELF header files
together

Signed-off-by: Jessica Hamilton <jessica.l.hamilton@gmail.com>

Support ELF64 in the kernel.

This has been done by adding typedefs in elf_common.h to the correct ELF
structures for the architecture, and changing all Elf32_* uses to those
types. I don't know whether image loading works as I cannot test it yet,
there may be some 64-bit safety issues around. However, symbol lookup for
the kernel is working correctly.

Cleaned up ELF64 address handling.

* platform_allocate_elf_region() is removed, it is implemented in platform-
independent code now (ELF*Class::AllocateRegion). For ELF64 it is now
assumed that 64-bit addresses are mapped in the loader's 32-bit address space
as (address - KERNEL_BASE_64BIT + KERNEL_BASE).
* mapped_delta field from preloaded_*_image removed, now handled compile-time
using the ELF*Class::Map method.
* Also link the kernel with -z max-page-size=0x1000, removes the need for
2MB alignment on the data segment (not going to map the kernel with large
pages for the time being).

Handle 64-bit load addresses for ELF64 images in the bootloader.

The ELF loader now uses a new platform function, platform_allocate_elf_region,
which returns 2 addresses: the real load address and an address where the
region is mapped in the loader's address space. All of the ELF loading code
has been changed to access the load region through the mapped address rather
than the addresses contained in the ELF image. The ELF64 version of
platform_allocate_elf_region on x86 uses the existing MMU code, which maps
everything at 0x80000000, but returns the correct 64-bit address. The long
mode switch code will just set up the 64-bit address space with everything
remapped at the correct address.

FixedWidthPointer: Fix ==/!= operators, remove OtherType casts

* FixedWidthPointer:
- operators ==/!=: Change second operand type from void* to const
Type*. Also add non-const version to resolve ambiguity warning when
comparing with non-const pointer.
- Add Pointer() getter.
- Remove templatized cast operators. They are nice for casting the
pointer directly to another pointer type, but result in ambiguity.
* Make preloaded_image::debug_string_table non-const. Avoids clashes of
the const and non-coast FixedWidthPointer comparison operators. A
cleaner (but more verbose) solution would be to spezialize
FixedWidthPointer for const types.

Changed the boot ELF code to use templates and added ELF64 support.

The actual implementation of the ELF loading methods have been put into
an ELFLoader template class that takes a single template parameter, which
is a structure containing all the necessary ELF typedefs. It's a bit
verbose, but I thought it was a neater solution than using a bunch of
standalone functions with a huge number of template parameters. There is
no change to code outside of elf.cpp, the ELF32/ELF64 differences are
handled internally.

Added an ELF64 version of preloaded_image.

* There is now 2 structures, preloaded_elf32_image and preloaded_elf64_image,
which both inherit from preloaded_image.
* For now I've just hardcoded in use of preloaded_elf32_image, but the
bootloader ELF code will shortly be converted to use templates which use
the appropriate structure. The kernel will be changed later when I add
ELF64 support to it.
* All kernel_args data is now compatible between 32-bit and 64-bit kernels.

Changes to kernel_args to make it identical for x86 and x86_64.

* Added a FixedWidthPointer template class which uses 64-bit storage to hold
a pointer. This is used in place of raw pointers in kernel_args.
* Added __attribute__((packed)) to kernel_args and all structures contained
within it. This is necessary due to different alignment behaviour for
32-bit and 64-bit compilation with GCC.
* With these changes, kernel_args will now come out the same size for both
the x86_64 kernel and the loader, excluding the preloaded_image structure
which has not yet been changed.
* Tested both an x86 GCC2 and GCC4 build, no problems caused by these changes.

[Sorry, couldn't split this one up any further.]
* Images preloaded by the boot loader had to be modules to be of any use
to the kernel. Extended the mechanism so that any images not accepted
by the module code would later be tried to be added as drivers by the
devfs. This is a little hacky ATM, since the devfs manages the drivers
using a hash map keyed by the drivers inode ID, which those drivers
obviously don't have.
* The devfs emulates read_pages() using read(), if the device driver
doesn't implement the former (all old-style drivers), thus making it
possible to BFS, which uses the file cache which in turn requires
read_pages(), on the device. write_pages() emulation is still missing.
* Replaced the kernel_args::boot_disk structure by a KMessage, which can
more flexibly be extended and deals more gracefully with
arbitrarily-size data. The disk_identifier structure still exists,
though. It is added as message field in cases where needed (non net
boot). Moved the boot_drive_number field of the bios_ia32 platform
specific args into the message.
* Made the stage 1 PXE boot loader superfluous. Moved the relevant
initialization code into the stage 2 loader, which can now be loaded
directly via PXE.
* The PXE boot loader does now download a boot tgz archive via TFTP. It
does no longer use the RemoteDisk protocol (it could actually be
removed from the boot loader). It also parses the DHCP options in the
DHCPACK packet provided by PXE and extracts the root path to be
mounted by the kernel.
* Reorganized the boot volume search in the kernel (vfs_boot.cpp) and
added support for network boot. In this case the net stack is
initialized and the network interface the boot loader used is brought
up and configured. Since NBD and RemoteDisk are our only options for
net boot (and those aren't really configurable dynamically) ATM, the
the boot device is found automatically by the disk device manager.

Booting via PXE does work to some degree now. The most grievous problem
is that loading certain drivers or kernel modules (or related activity)
causes a reboot (likely a triple fault, though one wonders where our
double fault handler is on vacation). Namely the keyboard and mouse input
server add-ons need to be deactivated as well as the media server.
A smaller problem is the net server, which apparently tries to
(re-)configure the network interface we're using to boot, which
obviously doesn't work out that well. So, if all this stuff is disabled
Haiku does fully boot, when using the RemoteDisk protocol (not being
able to use keyboard or mouse doesn't make this a particular fascinating
experience, though ;-)). I had no luck with NBD -- it seemed to have
protocol problems with the servers I tried.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21611 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced new build system variables
{HAIKU,HOST,TARGET}_KERNEL_PIC_{CC,LINK}FLAGS which define the
compiler/linker flags specifying the kind of position independence
the kernel shall have. For x86 we had and still have -fno-pic, but the
PPC kernel has -fPIE (position independent executable) now, as we
need to relocate it.
* The boot loader relocates the kernel now. Mostly copied the relocation
code from the kernel ELF loader. Almost completely rewrote the PPC
specific relocation code, though. It's more correct and more complete now
(some things are still missing though).
* Added boot platform awareness to the kernel. Moved the generic
Open Firmware code (openfirmware.c/h) from the boot loader to the kernel.
* The kernel PPC serial debug output is sent to the console for the time
being.
* The PPC boot loader counts the CPUs now and allocates the kernel stacks
(made OF device iteration a bit more flexible on the way -- the search
can be restricted to subtree). Furthermore we really enter the kernel...
(Yay! :-) ... and crash in the first dprintf() (in the atomic_set()
called by acquire_spinlock()). kprintf() works, though.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15756 a95241bf-73f2-0310-859d-f6bbb57e9c96

Now stores an inode number along with the other preloaded image data.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7985 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved the addr_range struct out of kernel_args.h to a separate file.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7703 a95241bf-73f2-0310-859d-f6bbb57e9c96

For debugging purposes, the boot loader is now able to load an additional
symbol table into memory and hand it over to the kernel.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7695 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added an ID field that connects the real kernel image with the preloaded image structure.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7530 a95241bf-73f2-0310-859d-f6bbb57e9c96

Actually, elf_region already contains an area_id field...


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7517 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added two fields that ease the kernel initialization of the preloaded images.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7503 a95241bf-73f2-0310-859d-f6bbb57e9c96

Is now also usable from plain C.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7329 a95241bf-73f2-0310-859d-f6bbb57e9c96

We also need the size of the dynamic section.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7275 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced addr with addr_t.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7262 a95241bf-73f2-0310-859d-f6bbb57e9c96

Changes for the ELF loader in the boot loader.
Removed the arch_mmu_alloc() function, added (empty) arch_mmu_free() function.
Added platform prototypes for allocating and freeing a region.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@5044 a95241bf-73f2-0310-859d-f6bbb57e9c96

Support ELF64 in the kernel.

This has been done by adding typedefs in elf_common.h to the correct ELF
structures for the architecture, and changing all Elf32_* uses to those
types. I don't know whether image loading works as I cannot test it yet,
there may be some 64-bit safety issues around. However, symbol lookup for
the kernel is working correctly.

Cleaned up ELF64 address handling.

* platform_allocate_elf_region() is removed, it is implemented in platform-
independent code now (ELF*Class::AllocateRegion). For ELF64 it is now
assumed that 64-bit addresses are mapped in the loader's 32-bit address space
as (address - KERNEL_BASE_64BIT + KERNEL_BASE).
* mapped_delta field from preloaded_*_image removed, now handled compile-time
using the ELF*Class::Map method.
* Also link the kernel with -z max-page-size=0x1000, removes the need for
2MB alignment on the data segment (not going to map the kernel with large
pages for the time being).

Handle 64-bit load addresses for ELF64 images in the bootloader.

The ELF loader now uses a new platform function, platform_allocate_elf_region,
which returns 2 addresses: the real load address and an address where the
region is mapped in the loader's address space. All of the ELF loading code
has been changed to access the load region through the mapped address rather
than the addresses contained in the ELF image. The ELF64 version of
platform_allocate_elf_region on x86 uses the existing MMU code, which maps
everything at 0x80000000, but returns the correct 64-bit address. The long
mode switch code will just set up the 64-bit address space with everything
remapped at the correct address.

FixedWidthPointer: Fix ==/!= operators, remove OtherType casts

* FixedWidthPointer:
- operators ==/!=: Change second operand type from void* to const
Type*. Also add non-const version to resolve ambiguity warning when
comparing with non-const pointer.
- Add Pointer() getter.
- Remove templatized cast operators. They are nice for casting the
pointer directly to another pointer type, but result in ambiguity.
* Make preloaded_image::debug_string_table non-const. Avoids clashes of
the const and non-coast FixedWidthPointer comparison operators. A
cleaner (but more verbose) solution would be to spezialize
FixedWidthPointer for const types.

Changed the boot ELF code to use templates and added ELF64 support.

The actual implementation of the ELF loading methods have been put into
an ELFLoader template class that takes a single template parameter, which
is a structure containing all the necessary ELF typedefs. It's a bit
verbose, but I thought it was a neater solution than using a bunch of
standalone functions with a huge number of template parameters. There is
no change to code outside of elf.cpp, the ELF32/ELF64 differences are
handled internally.

Added an ELF64 version of preloaded_image.

* There is now 2 structures, preloaded_elf32_image and preloaded_elf64_image,
which both inherit from preloaded_image.
* For now I've just hardcoded in use of preloaded_elf32_image, but the
bootloader ELF code will shortly be converted to use templates which use
the appropriate structure. The kernel will be changed later when I add
ELF64 support to it.
* All kernel_args data is now compatible between 32-bit and 64-bit kernels.

Changes to kernel_args to make it identical for x86 and x86_64.

* Added a FixedWidthPointer template class which uses 64-bit storage to hold
a pointer. This is used in place of raw pointers in kernel_args.
* Added __attribute__((packed)) to kernel_args and all structures contained
within it. This is necessary due to different alignment behaviour for
32-bit and 64-bit compilation with GCC.
* With these changes, kernel_args will now come out the same size for both
the x86_64 kernel and the loader, excluding the preloaded_image structure
which has not yet been changed.
* Tested both an x86 GCC2 and GCC4 build, no problems caused by these changes.

[Sorry, couldn't split this one up any further.]
* Images preloaded by the boot loader had to be modules to be of any use
to the kernel. Extended the mechanism so that any images not accepted
by the module code would later be tried to be added as drivers by the
devfs. This is a little hacky ATM, since the devfs manages the drivers
using a hash map keyed by the drivers inode ID, which those drivers
obviously don't have.
* The devfs emulates read_pages() using read(), if the device driver
doesn't implement the former (all old-style drivers), thus making it
possible to BFS, which uses the file cache which in turn requires
read_pages(), on the device. write_pages() emulation is still missing.
* Replaced the kernel_args::boot_disk structure by a KMessage, which can
more flexibly be extended and deals more gracefully with
arbitrarily-size data. The disk_identifier structure still exists,
though. It is added as message field in cases where needed (non net
boot). Moved the boot_drive_number field of the bios_ia32 platform
specific args into the message.
* Made the stage 1 PXE boot loader superfluous. Moved the relevant
initialization code into the stage 2 loader, which can now be loaded
directly via PXE.
* The PXE boot loader does now download a boot tgz archive via TFTP. It
does no longer use the RemoteDisk protocol (it could actually be
removed from the boot loader). It also parses the DHCP options in the
DHCPACK packet provided by PXE and extracts the root path to be
mounted by the kernel.
* Reorganized the boot volume search in the kernel (vfs_boot.cpp) and
added support for network boot. In this case the net stack is
initialized and the network interface the boot loader used is brought
up and configured. Since NBD and RemoteDisk are our only options for
net boot (and those aren't really configurable dynamically) ATM, the
the boot device is found automatically by the disk device manager.

Booting via PXE does work to some degree now. The most grievous problem
is that loading certain drivers or kernel modules (or related activity)
causes a reboot (likely a triple fault, though one wonders where our
double fault handler is on vacation). Namely the keyboard and mouse input
server add-ons need to be deactivated as well as the media server.
A smaller problem is the net server, which apparently tries to
(re-)configure the network interface we're using to boot, which
obviously doesn't work out that well. So, if all this stuff is disabled
Haiku does fully boot, when using the RemoteDisk protocol (not being
able to use keyboard or mouse doesn't make this a particular fascinating
experience, though ;-)). I had no luck with NBD -- it seemed to have
protocol problems with the servers I tried.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21611 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced new build system variables
{HAIKU,HOST,TARGET}_KERNEL_PIC_{CC,LINK}FLAGS which define the
compiler/linker flags specifying the kind of position independence
the kernel shall have. For x86 we had and still have -fno-pic, but the
PPC kernel has -fPIE (position independent executable) now, as we
need to relocate it.
* The boot loader relocates the kernel now. Mostly copied the relocation
code from the kernel ELF loader. Almost completely rewrote the PPC
specific relocation code, though. It's more correct and more complete now
(some things are still missing though).
* Added boot platform awareness to the kernel. Moved the generic
Open Firmware code (openfirmware.c/h) from the boot loader to the kernel.
* The kernel PPC serial debug output is sent to the console for the time
being.
* The PPC boot loader counts the CPUs now and allocates the kernel stacks
(made OF device iteration a bit more flexible on the way -- the search
can be restricted to subtree). Furthermore we really enter the kernel...
(Yay! :-) ... and crash in the first dprintf() (in the atomic_set()
called by acquire_spinlock()). kprintf() works, though.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15756 a95241bf-73f2-0310-859d-f6bbb57e9c96

Now stores an inode number along with the other preloaded image data.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7985 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved the addr_range struct out of kernel_args.h to a separate file.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7703 a95241bf-73f2-0310-859d-f6bbb57e9c96

For debugging purposes, the boot loader is now able to load an additional
symbol table into memory and hand it over to the kernel.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7695 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added an ID field that connects the real kernel image with the preloaded image structure.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7530 a95241bf-73f2-0310-859d-f6bbb57e9c96

Actually, elf_region already contains an area_id field...


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7517 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added two fields that ease the kernel initialization of the preloaded images.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7503 a95241bf-73f2-0310-859d-f6bbb57e9c96

Is now also usable from plain C.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7329 a95241bf-73f2-0310-859d-f6bbb57e9c96

We also need the size of the dynamic section.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7275 a95241bf-73f2-0310-859d-f6bbb57e9c96

Replaced addr with addr_t.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@7262 a95241bf-73f2-0310-859d-f6bbb57e9c96

Changes for the ELF loader in the boot loader.
Removed the arch_mmu_alloc() function, added (empty) arch_mmu_free() function.
Added platform prototypes for allocating and freeing a region.


git-svn-id: file:///srv/svn/repos/haiku/trunk/current@5044 a95241bf-73f2-0310-859d-f6bbb57e9c96