libbarrelfish: setting arch specific fields when creating new dispatchers

Refactoring the domain_new_dispatcher function and extract arch specific
functions to
- set the segment selectors on x86
- set the got base on armv7/armv8
in the respective arch specific dispatchers structs.

Signed-off-by: Reto Achermann <reto.achermann@inf.ethz.ch>

replacing umlaute and fixing address in headers Haldeneggsteig -> Universitaetsstrasse

Signed-off-by: Reto Achermann <reto.achermann@inf.ethz.ch>

dispatcher: remove DISPATCHER_FRAME_BITS, and redefine DISPATCHER_FRAME_SIZE as 512kB.

This is necessary with the new contents struct dispatcher_<arch> when
using array-based pmap on larger machines, as we otherwise run out of
room for LMP channel receive buffers in the SKB.

Signed-off-by: Simon Gerber <simon.gerber@inf.ethz.ch>

libbarrelfish: heap: explicitly store and propagate heap pagesize instead of relying on heap alignment to infer pagesize

Signed-off-by: Simon Gerber <simon.gerber@inf.ethz.ch>

T82: checkpoint. Guest domain up. Need to implement hypervisor service.

Signed-off-by: Simon Gerber <simon.gerber@inf.ethz.ch>

fpu: unifing the fpu's context switching
x86_64: fixing the stack's alignment

Signed-off-by: Adam Turowski <adam.turowski@inf.ethz.ch>

fpu: unifing the fpu's context switching
x86_64: fixing the stack's alignment

Signed-off-by: Adam Turowski <adam.turowski@inf.ethz.ch>

Clean up and document the process management code.

* clean up benchmarking code from spawnd and the process manager;
* document the process management functions in usr/proc_mgmt;
* remove dangling prints from lib/barrelfish/domain.c and threads.c.

Signed-off-by: Razvan Damachi <razvan.damachi@gmail.com>

Fix bug in libdomain/domain_cap_hash()

Signed-off-by: Razvan Damachi <razvan.damachi@gmail.com>

Extract a through-monitor-only cap_identify call from debug_cap_identify.

The function debug_cap_identify in debug.c used to try to identify the given
capref by first invoking the kernel, then performing an RPC with the monitor,
if the kernel invocation failed. The kernel invocation was there for domains
that hold the cap_kernel, which is necessary for the invocation to succeed.
Therefore, having the invocation there for domains which do not hold cap_kernel
is redundant, incurring unnecessary extra work.

A monitor_cap_identify_remote function is now part of monitor_client.c, through
which callers can identify caprefs through the monitor directly. The old
debug_cap_identify function in debug.c now resorts to this new function if the
kernel invocation fails.

Moreover, userspace apps such as the process manager or spawnd need to identify
caps involved in their RPCs (e.g. domain caps), however they do not hold the
cap_kernel reference. They can thus now use the new function to query the
monitor directly, without attempting to drop in the local kernel first.

Signed-off-by: Razvan Damachi <razvan.damachi@gmail.com>

Implement kill_request_handler in spawnd, using domain caps.

Domain cap hashing is now used by spawnd in a manner similar to the process
manager's, for looking up domains to kill as per incoming kill(domain_cap)
requests.

Note that with the new API and implementation, spawnd no longer tracks domain
exit code or waiters (although some backwards-compatibility is attempted), this
task being now attributed to the process manager (to be implemented).

Signed-off-by: Razvan Damachi <razvan.damachi@gmail.com>

Add spawn, spawn_with_caps and span calls to the Process Manager API.

The process manager keeps local domain state and forwards valid requests to
spawnds, acting as a proxy for spawning & spanning. Requests are validated as
per the Span-Stop-Cleanup state machine at https://goo.gl/6JuJj2.

Spawnd has been enriched with several asynchronous API calls, in order for
the process manager to serve requests non-blockingly. Specifically, clients of
the process management service issue RPCs to the process manager server
(blockingly), but the latter forwards requests to spawnd asynchronously, hence
it is able to serve multiple clients at the same time.

The client code for spanning uses libbarrelfish/domain.h to create the new
dispatcher for the remote core.

Bug: if the first dispatcher exits main(), dispatchers spanned on other cores
will pagefault.

Signed-off-by: Razvan Damachi <razvan.damachi@gmail.com>

Refactor process manager to enable spawnd discovery.

The process manager now allocates a special LMP endpoint for monitor.0 and
gives said endpoint to monitor.0 as part of the export_cb. After monitor.0
spawns spawnd, it uses that endpoint to send the new spawnd's iref to the
process manager. The latter then binds with spawnd, to keep track of its state.

For app cores, a similar workflow applies, except inter-monitor communication
is now performed. Namely, the new app core's monitor sends the new app core's
spawnd to monitor.0, which forwards it to the process manager. The latter then
binds with the app core's spawnd just the same.

Diagrams for the discovery protocol can be found at https://goo.gl/eJE37u.

Signed-off-by: Razvan Damachi <razvan.damachi@gmail.com>

network: replacing net_queue_manager with devif as a data path, still WIP so it'll probably break the network stack

Signed-off-by: Adam Turowski <adam.turowski@inf.ethz.ch>

flounder: merging rpc client with a binding layer

Signed-off-by: Adam Turowski <adam.turowski@inf.ethz.ch>

Variable renamings, fixing event_mutex locking and thread_unblock_one.

Signed-off-by: Adam Turowski <adam.turowski@inf.ethz.ch>

[T102] Implement CPU_SET and pthread_attr_setaffinity_np. Closes T102.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

Conflicts:
lib/newlib/newlib/libc/include/sys/types.h
lib/posixcompat/pthreads.c
usr/tests/posixcompat/Tupfile.lua

Merge of upstream arrakis code.

This merge includes the following:
* VT-d (IOMMU) support
* VMX (Intel CPU virtualization) support
* Parallel hake build
* MegaRAID driver
* Subways fast packet forwarding application
* Identify physical address of VNodes
* libstorage (VSIC & VSA support)
* POSIX support: poll, uname, nanosleep, fsync, poll, pthread attributes, syslog, wait3, ...
* TenaciousD (persistent data structures: log & queue)
* Hack to skip boot RAM when it's too large
* Arranet: Support for raw IP sockets, interrupts
* Removal of colorful debug output
* MegaRAID blockdevfs support
* Arrakis compile wrappers for Autoconf
* Patches to binutils & GCC to build Arrakis C++ applications

Signed-off-by: Simon Gerber <simon.gerber@inf.ethz.ch>
Signed-off-by: Simon Peter <simpeter@cs.washington.edu>
Signed-off-by: Oleg Godunok <ogodunok@gmail.com>
Signed-off-by: Antoine Kaufmann <antoinek@cs.washington.edu>
Signed-off-by: Jialin Li <lijl@cs.washington.edu>
Signed-off-by: Dalton Black <dalton.m.black@gmail.com>
Signed-off-by: Taesoo Kim <taesoo@uw.edu>

Handing over alignment paramenter of morecore when spanning

Spanning domains resulted in a NULL-pointer dereferencing
in when initializing libbarrelfish and parsing arguments
for the morecore alignment. Added new parameter to pass
the alignment of the spanning domain to the new dispatcher
on the remote core.

Signed-off-by: Reto Achermann <reto.achermann@inf.ethz.ch>

32bit: libbarrelfish: fix casts in new x-core thread functionality.

Signed-off-by: Simon Gerber <simon.gerber@inf.ethz.ch>

T65: implement proper x-core join.

Signed-off-by: Simon Gerber <simon.gerber@inf.ethz.ch>

T65: properly implement x-core thread creation.

Signed-off-by: Simon Gerber <simon.gerber@inf.ethz.ch>

Added current core_id so we can properly reroute interrupts after moving kcb.

disabling creation of default_waitset dispatcher thread in domain.c

libcxx: exception handling

added missing code to enable exception handling of C++ programs on
barrelfish:

+ adding .eh_frame and .eh_frame_hdr section information to the
dispatcher when the new domain is spawned.

+ adding two new functions to obtain the eh_frame and eh_frame_hdr
information from userspace (used by libcxx for exception handling)

The Arrakis backport to Barrelfish.

Here are the macro changes:
* Made lwIP an optional network stack
* Moved lwIP headers out of the general include directory
* Parallel hake
* CPU-virtualized ("arrakis") applications
* libarrakis is a new libOS that uses CPU features directly
* arrakismon is a new monitor for Arrakis apps
* QEMU Multiboot support instead of GRUB (enables KVM)
* e10k virtual function driver
* Software virtual->physical mapping (substitutes IOMMU)
* To be replaced soon with an Intel IOMMU driver
* Better GDB and stack tracing support
* New POSIX support
* pthreads, inet_*(), usleep(), syslimits.h, sigaction(), mman.h,
* resource.h, sendmsg(), recvmsg(), mkfifo(), syslog.h, sysexits.h,
* fcntl(), getegid(), getgid(), pipe(), rlimits
* epoll support
* Separate waitset for interrupts
* Thread TLS key and ID support
* Moved PCRE headers to include/, where they usually are
* Intel performance counters support
* Handle boot-time RAM overflow (for machines with >=64GB RAM)
* Arranet network stack
* Fix for build of lib/barrelfish/arch/x86_64/dispatch.c without -O2
* New comandline options to pci daemon:
* maxvfs=x, x is number of virtual functions to create for each
* device supporting it
* skb_bridge_program=x, x is SKB bridge programming script to use
* bridge_bios is a script that uses the BIOS pre-set values, like Linux
* Correctly build machine-optimized newlib parts
* Library versions of e1000 and e10k drivers
* Support for virtualization in e10k driver
* e10k datacenter control plane daemon
* Modified hellotest to be both a Barrelfish and an Arrakis application

I've decided not to include the lwIP 1.4.1 code, as the port is
not finished and who wants to use lwIP anyway with the Arrakis
network stack and mTCP being released soon. If you want this,
let me know (it's hugely superior to lwIP 1.3.1).

I've made the code backwards-compatible with Barrelfish. The
default should do what Barrelfish does. One can specify
commandline options to the PCI daemon and e10k driver to
configure I/O virtualization and the optional arrakismon allows
loading CPU-virtualized "arrakis" programs.

There are more pending changes, such as support for hardware
rate limiting and weighted round robin packet scheduling, but
I'd like these to go in first, before I push the rest.

Cleaned up code and removed all the debugging related code

Webserver working for full website with LMP.

This version of code still does not work for UMP,
and also it does not work for httperf test.

Also, there is lot of commented out code which was helpful for me
in debugging the issue of randome page-fault.

Minor fixes to error handling.

The octopus rpc client binding is used by the nameservice and the terminal
subsystem. Rename {get,set}_nameservice_rpc_client to
{get,set}_octopus_rpc_client.

libbarrelfish: Simplified logic and fixed infinite loop on cap_destroy error in span_domain_reply().

libbarrelfish: Moved cap_destroy() into error case in span_domain_reply.

libbarrelfish: Rename the header lib/barrelfish/include/threads.h to
lib/barrelfish/include/threads_priv.h.

To avoid confusion between the public and the private thread header, rename
the libbarrelfish-private thread header to threads_priv.h.

--HG--
rename : lib/barrelfish/include/threads.h => lib/barrelfish/include/threads_priv.h

libbarrelfish: Spilt private waitset_chan.h header into a public and private
header.

* Check all files that included the old header, if the need the private or the
public header or both or none at all. Some files included the header but
didn't use any of the functions provided by the header.

* Change libahci so that it no longer uses the libbarrelfish-private header.

rename dist2 interface to octopus

--HG--
branch : octopus.renaming
rename : if/dist2.if => if/octopus.if

Changes for nameservice. Use RPC client for NS functionality.

remove support for Beehive

see the mailing list discussion at the end of December 2011 for motivation

Merge from ASPLOS tree.

Initial version of public stable barrelfish repository.