[devmgr] Use C++ enums for message ops

Test: cq

Change-Id: I2e32fd14b1be11b5e238b5460d6a5c2c6cc3d981

[devmgr] Use C++ only for internal headers

Test: cq

Change-Id: I9f8f0b0f7eed557c9234d061e7bb60f8c08d6a73

[core] Mark core header declarations with CDECLs

Test: no functional change, so just CQ
Change-Id: I377759224b3b9656efb2db13a25e4674cddb5b6b

[devmgr][devcoord] load system drivers ahead of time

Using a dedicated thread and caching their DSOs should avoid
deadlocks against filesystem + blockdevice devhosts we can
otherwise see under certain conditions.

Refactors find_loadable_drivers() to take a callback rather than
hardcoding dc_driver_added() and adjusts call sites.

To be completely safe system drivers must not link against
libraries hosted in /system

Should mostly resolve ZX-2385. BLD-224 is necessary for complete
safety (ensuring no linking against libraries in /system).

Testing: verified system boot in zircon and garnet

Change-Id: Idd0fb110b8ea27e970617f1908cd9a29cf00009b

[core][devmgr] Don't add ASan drivers after the first devhost

An ASan driver needs an ASan devhost, so when we know there are
ASan drivers we launch devhost.asan. But if we already launched
a devhost and then discover new ASan drivers, it's too late
because they might need to be loaded in the non-ASan devhost
already launched.

ZX-2083 #comment Refuse to load ASan drivers after non-ASan devhosts exist

Change-Id: Ie286ea40d589dd2f402674039ffdc882e3b9f537

[ddk] Add support for device metadata

Device metadata can be used to provide extra information to drivers.
For example, MAC addresses for drivers implementing network devices.

device_add_metadata() adds metadata to an existing zx_device_t.

device_publish_metadata() publishes metadata to be added to a future device
with a specified device path.

device_get_metadata() returns metadata with a specified type for a device.
Parent devices are searched if no matching metadata is found on the device itself.

New header file <ddk/metadata.h> contains definitions for metadata types.

Change-Id: I7654942f8a9bb1ebc55fd04f50a857234f811587

[fidl] Rename fidl2 things to just fidl

Change-Id: I17780c327272c554b4c80a7c5fc64df78d0a5601

[devhost][devcoord] migrate firmware loading to the device coordinator

Soon devhosts will not have direct access to the filesystem.
Later firmware load locations may be device-specific.
In both cases the devcoord is better equipped to locate the fw.

Change-Id: I47fea624b51fa8fc070d9f48e89031af2cd2a922

[ddk][block] change device_rebind() semantics

Previously device_rebind() would call device_remove() on all of
the children of the device it was called on, *without* invoking
their unbind() hook, in violation of the device model.

Change this so that it instead causes all children to be unbound
and, then, when all those children are finally removed and eventually
released, *then* cause the rebind attempt.

This preserves the device model and further ensures that new children
do not get bound until the existing children are fully destructed.

Since all block protocol devices are accessed from userspace via the
shared block driver, instead of having the shared block driver forward
the BLOCK_RR_PART ioctl() to its parent, it fields this ioctl() itself,
causing its children to become unbound and the rebound.

One interesting change is that it no longer changes name -- if you RR_PART
against /dev/class/block/123, that device will persist (since it *is*
the same device, it's just that you asked for it to remove its children
and try again because its partition table may have changed, etc).

Currently we synthesize a REMOVE and ADD directory watch event for a
device being rebound. We should instead just generate a MODIFY event,
but that will require changes to various clients (block-watcher, tests, etc)
That depend on the old behavior.

Handling of BLOCK_RR_PART is removed from all block core drivers.

Change-Id: I9fd40ec11c216e3d2e0e88ef65ccd23e031eb245

[devhost][devcoord] adjust opcode differentiator

devfs forwards OPEN messages to devhosts over the same channel
used for devcoordinator messages. Previously the fact that the
high bit was set on DC messages differentiated them from RIO
messages, but the high bit will be used for FIDL system messages.

This moves the differentiator bit to bit 28, adds a static assert
to verify it is non-conflicting at build time, and fixes the
check for RIO_OPEN in devhost.c to check the entire opcode field.

Change-Id: I60d72cf2f35c6b9d87103be2a41edeeea9fecb8f

[devmgr] Fix device_remove() for devices added with DEVICE_ADD_MUST_ISOLATE

If a driver adds a device with DEVICE_ADD_MUST_ISOLATE, the new device
is added in a new devhost.
Currently when this driver calls device_remove() on this device,
the device and its children are removed from the devcoordinator.
But the devhost containing the device is not informed, so drivers in
that devhost are not informed and are left still actively running.
This change adds an RPC from the coordinator to the devhost to tell
it to remove the device.

This showed up on the Hikey960. The "usbctl mode device" command
failed because the xhci driver did not shutdown so the dwc3 driver
failed to open an interrupt handle for the interrupt it shares with xhci.

Change-Id: I5c20cb072a2e3703ac028f146e216db12360cf05

[mexec] invoke mexec syscall from devmgr

Change-Id: I16d8e1962d44dcf0b0142c586e53545b8ed7e5df

[devmgr] invoke suspend() on devices on reboot or poweroff

Devcoordinator invokes suspend() on each devhost, and each
devhost invokes the suspend() op on each bound device. The
children devices are suspended before their parent. If
any device failed to suspend, the system aborts the operation.
Currently, unimplemented suspend() hooks are considered
successful.

Change-Id: I0ec15fe9392767bf123877418e70d8c5f04d6923

[fdio] Align zxrio_msg header with FIDL2

Assumes 32-bit txid (gated on an assertion, in case
we change our minds).

Change-Id: Idfad946c5556c95cc31c4f56a85d8662d9c9bbe5

[devmgr] remove acpi from devcoordinator

Implement the ACPI bus driver's suspend() hook to reboot
or poweroff, and add a devcoordinator op to invoke suspend.
Remove ACPI client from devcoordinator.

Change-Id: I84532a35f32a482a3dc35eda070ce46c02746abf

[devcoord][devhost] stop tracking a per-device resource handle

This is no longer used, superceded by rpc between proxy device
and the device it proxies.

Change-Id: I824250280e305d3079155a0964daa329502185fb

[ddk][devcoord] support adding "invisible" devices

These will not appear in devfs or be subject to binding rules until
device_make_visible(dev) is called on them. Once made visible it is
not possible to make a device invisible again.

Change-Id: I3d072395b17c5a05233a8dbd9e19c67a3913be1e

[devmgr][fshost] ability to compile fs glue into fshost

Not enabled by default since we can't quite boot in this mode
yet (need a little more plumbing and chicken-and-egg-juggling),
but this takes us one step closer to a separate filesystem
host process.

Change-Id: Ic667e4979b7aed53d040702ad294b6dad7db2a36

[ddk][devcoord][devhost] BusDev becomes MustIsolate

Drivers that *require* that children are placed in a new devhost
behind a proxy object now use flags _MUST_ISOLATE instead of the
older, less clear _BUSDEV.

Later we will introduce _MAY_ISOLATE for drivers that do not
require process isolation, but support it -- allowing for more
flexibly policies on isolation.

Change-Id: Ie827f24cbcdcae69b146164d4246c400d6ff7e3f

[devmgr][devhost] rename shadow to proxy throughout

Change-Id: Iffce11212c28ee14e81abcc9d77e9c5c127469a4

[devmgr] Use ASanified devhosts in the presence of ASanified drivers

Drivers built with ASan can only be loaded by a devhost that was
also built with ASan. But this can be independent of whether
devmgr itself was built with ASan or not.

In the ASan build, install "devhost.asan" instead of "devhost" to
identify it as the host that can load drivers built with ASan.

Put a flag into the driver identification note indicating a
driver built with ASan. Make devmgr notice the flag and use
devhost.asan if there are any ASan drivers.

Change-Id: I25d745b09033cf8b54928e7767bd03fd222bc3c8

[ddk][devhost] plumbing for shadow->busdev rpc

- the resource that we pass to shadows is becoming the shadow
rpc handle (it's still used as a resource for topside
platform devices -- will disentangle that later)
- new device op rxrpc() enables busdevs to receive rpcs from
their shadows
- new api device_get_shadow_channel() enables shadow devices
to get at the rpc channel to send rpcs
- new devcoord->devhost rpc DC_OP_CONNECT_SHADOW

Change-Id: I9064a807e1ab5e817bc6244442c674ade4f50233

[zx] Magenta -> Zircon

The Great Renaming is here!

Change-Id: I3229bdeb2a3d0e40fb4db6fec8ca7d971fbffb94

[devmgr] enumerate /system/driver/... once /system is mounted

For ramdisks this already worked, but for /system on regular
block devices that get mounted after the dev coordinator starts
up we were not loading drivers.

Change-Id: I296d98f49704f0a3034ce590b70cbfe8151c4bfa

[dmctl][devmgr] protocol to observe devices (add/remove/changed)

This is experimental.

Change-Id: Ie571ff9305125883ee2ff9b68101bed3396db668

[devmgr] fix string handling in driver load message processing

Also fix up dmctl driver to return length written on successful
write so stuff like echo -n > /dev/misc/dmctl does not run forever.

Change-Id: Ia91e3b28f9ea5192f542b39194f1cda4a249dd6a

[devmgr] support adding new drivers at runtime

This is a placeholder while we sort out the longer time driver
management API.

Sending "add-driver:/path/to/driver.so" to /dev/misc/dmctl will
request that a driver be loaded. If any unbound devices have
properties compatible with the driver's binding program, the
devmgr will attempt to bind it.

Change-Id: I36bfdd381bfdddbd7b5f08a08d53cc2b6a587df8

[gfxconsole][run-vc][dmctl] support run-vc in new virtcon service

Change-Id: I3ea3502e2d55dc9cefbde8561993803a6ead8d46

[devhost][device] provide IOCTL_DEVICE_GET_TOPO_PATH

It is now possible to obtain the topological path of an open device.

For example /dev/class/block/123 might have the topological path
of /dev/pci/00:01:00/ahci/sata0/block

Change-Id: I243fc4646be602fe50bca24ab54643e0e37c180b

[ulib][port] pull the little port library out of devmgr

This will be useful in gfxconsole and some other places
where it will replace mxio's dispatcher, which will be
going away.

Change-Id: I93028e012bf0d596824d682670510d2a0b28bee8

[devmgr] Add platform bus support

The platform-bus is a special driver that is able to instantiate devices
based on information in the MDI.
The devmgr special cases this driver and launches it in a separate devhost
and passes it a copy of the MDI data from the bootdata passed in from the kernel.

When the platform-bus driver binds, it creates devices corresponding to
the "device" nodes contained in the top level "platform-bus" node in the MDI.
Other drivers can bind to these nodes based on a vid/pid/did protocol.
These devices can optionally have children specified in the MDI.

This change also adds temporary support for platform devices to discover
each other's protocols. This will be replaced by a new mechanism in the future.

Currently platform-bus only supports instantiating a list of devices from the MDI.
In upcoming changes we will add support for specifying IRQs, MMIO ranges, etc
in the MDI and passing these to drivers via resources.

Change-Id: Icf57e486b4c93b1cf68910886f57744a5392cc7d

[devmgr] devcoord now uses internal devfs implementation

This allows tighter coupling with the underlying device_t tree
and lifecycle, and will allow further reduction in duplicate fields
between devnode_t's and device_t's in later changes.

memfs will no longer need to support the quirks of devfs. The only
remaining API is devfs_mount(mx_handle_t) which installs the remote
handle in /dev during startup.

Change-Id: Ib3cc0eeac2c8d38bf3e9e4f03efac3a333b3f194

Revert "[devmgr] devcoord now uses internal devfs implementation"

This breaks /dev/socket, sadly, so roll it back for the moment.

This reverts commit ddca75b2027cb09b434fec5f38531ecdf3c51e0f.

Change-Id: I7c81a481794899b9d07fd28778e7418c01f1c2bf

[devmgr] devcoord now uses internal devfs implementation

This allows tighter coupling with the underlying device_t tree
and lifecycle, and will allow further reduction in duplicate fields
between devnode_t's and device_t's in later changes.

memfs will no longer need to support the quirks of devfs. The only
remaining API is devfs_mount(mx_handle_t) which installs the remote
handle in /dev during startup.

Change-Id: I4cd9ba5b5e2a9dd3d9967fa1bc99ad21f813e658

[devmgr] stop using deprecated name driver_ctx_t

Change-Id: I23bcb1c5413abc9b9c547d4cc458fc295ee07f3b

[devmgr] remove devhost v1

Change-Id: I3c3c81173d16b39f559178ac600e5252bf88e98b

[devmgr] improve device loading a bit

- devcoord now passes the device library vmo to the devhost
- devhost uses dlopen_vmo() to open from the vmo
- this ensures devhost binds the *exact* driver devcoord specifies
- track full device binary pathnames
- adjust devcoord rpc protocol for the above

Change-Id: I5e3b9a749d415742206d2d36b24c9c82fa356f43

[devmgr] phase 4: continue to honor the '!' version prio hack

Change-Id: Ie806bb990fe359b294d001b697b3a78c6693c6f8

[devmgr] phase 4: track driver library name

and display it in dm dump output

Change-Id: I302641b766f3645d649693a8ed4bc7b7d6cd2f34

[devmgr] phase 4: fix issues with device removal

- Rename UNDEAD to ZOMBIE, since devices in this state are
similar to zombie processes (dead but still hanging around)
- Adjust the removal logic so the rpc dispatch always does
the final remove, ensuring we're not in a use-after-free
state ever and ensuring that we've always successfully
sent an rpc response *before* closing the device's rpc handle.
- Simplified the code flow a bunch
- Fixes USB hot plug.

Change-Id: I7b1b6669f708f584da510a44701afc0691f23d0b

[devmgr] phase 4: lifecycle improvements

- devcoord: devhost_t tracks all children in a devhost
- devcoord: device removal differentiates between 'forced' (channel closed)
and 'normal' (requested by devhost) removal
- devcoord: forced removal indicates that the devhost is dying and we
remove all other devices on that devhost at the same time
and mark it as dying, to simplify the lifecycle
- devcoord: new "UNDEAD" device_t state tracks the "was removed because
the devhost is dying" state (where we'll get the 'real' remove()
call when the channel finally closes)
- devcoord: when a device becomes unbound we restart the bind process on it

Change-Id: I0355b1b9988dd7107ca4599a72562fe8b6902ea0

[devmgr] phase 4: improved lifecycle management

- shared: debug tracing for port_() functions
- shared: port_queue() to send an event
- shared: port handler callback receives an event (0 for non-queue'd packets)
- devhost: remove iostate lock (not needed for v2)
- devhost: use port_queue() to work around ports v2 port_cancel() limitations
- devhost: correctly tear down the iostate on devhost_remove()
- devcoord: add refcounts to devhosts
- devcoord: reorganize dc_remove_device() actions
- devcoord: correctly use msg txid in reply
- devcoord: remove dc_destroy_device() and clean up teardown
- devcoord: if rpc from a devhost fails, detach and remove

Change-Id: I63f8013b87c6df19716233e0fad9aad1a066cc31

[devmgr] phase 4: support device bind ioctl, and cleanup

Change-Id: I4221034a24d20cbdef781b733f128fd5418055c6

[devmgr] phase 4: hook up dmctl in the new world

- dmctl now portals all commands through to the devcoordinator
- the get loader service ioctl remains but will eventually be
obsoleted by unified service discovery and removed
- bring back "dm dump" to dump the entire device hierarchy

Change-Id: I69291bd49fc0e8ef0b0a3f2cac355a65f410461d

[devmgr] phase 4: factor out common dc rpc call code

Change-Id: I913e39aae9d7381586c4644d7f56aafaaeb001da

[devmgr] phase 4: wire up responses from devhost

These are handled asynchronously by devcoordinator, to avoid deadlocking
with synchronous rpc calls from the devhost and. device_t's maintain
a list of pending actions that they are waiting for a response to.

Change-Id: I6b642ce616a417ffadc3e4827f2e8be758e4890f

[devmgr] phase 4: shadow devices and improved binding

- create shadow devices for busdevs and instantiate those in the
new devhost process
- support binding drivers to devices in the same devhost
(gfxconsole now can bind to the display again, whee!)

Change-Id: Idb02dd22b25afb1115fca6c69e3221381d6b6674

[devmgr] Remove mx_driver_t embedded in driver_ctx_t

mx_driver_t is by drivers to represent themselves to the devmgr
while driver_ctx_t represents a driver within the internals
of the devmgr itself. Keeping the two separate makes the difference
clearer in my opinion.

Change-Id: I3be29042443c92420295ad513d82b39b4e71016f

[devmgr] rename device_ctx_t and devhost_ctx_t

...to just device_t and devhost_t, as the ctx is just kinda clunky.

Also changed hdevice to hrpc.

Change-Id: I038962ef29f2ef3bffbe305584518cf7a4cd7c1b

[devmgr] phase 4: basic bus driver / device mechanisms working

- pci bus devices get instantiated in devhosts which are created
on demand, only when a suitable driver exists to bind to them
- next step: general bus device binding to replace the cheesy
prototype that enables pci devices to bind
- bus device arguments and resource passed through rpc
- logging macros replace printf() for better control over chatter
in the debuglog
- device coordinator now has a unified work queue that's integrated
with its rpc handling. rpc messages are higher priority than
queued work
- devhost_device_install() provides a mechanism to install created
devices into the lifecycle (and properly initialize them)
- various minor fixes and cleanups in rpc glue, etc

Change-Id: Iad9b93b9bbf5e00d525236fb19210904e30e7f26

[devhost] phase 4: host->coordinator RPC

- more descriptive trace output from devhost
- devhost can send ADD / REMOVE rpc messages to devcoord
- misc devices are now published
- remoteio OPEN messages are now received (eg, /dev/misc/console)

Change-Id: Ic753949abf932415588f9ea26dc93ca8ee091386

[devhost] more phase 4 work

- devhost create device now creates device contexts
- devhost device contexts can now receive further messages
- devhost start of bind driver message processing
- devhost loads drivers upon bind driver message
- devmgr start wiring up centralized binding support
- devmgr creation of misc devhost on-demand
- devmgr send bind requests for misc devices
- dc_is_bindable() interface from devcoord to bind engine

Change-Id: Ib09e46e3c1a50e2516af42fee944461e96257ae9

[devmgr] devmgr phase 4 work

- simple, flexible object watch (port_*()) so we can do things like
watch for process exit, resource change, and rpc messages all with
a single shared port
- device_add_etc() provides api surface for bus managers to publish
bus devices
- dc_msg_t and pack/unpack helpers for coordinator v2 messages
- devhost-shared.c for code shared between coordinator and devhost
- build both devhost and devhost2 (makes it easier to verify that
devhost2 changes aren't breaking devhost1)

Change-Id: I2a25c9a75961ec634927c966ea12db293584a7a8

[devmgr] Refactor & Clean devmgr Vnode constructors

- Avoid 'combined' create function. This makes passing arguments
and initializing vnodes with different demands difficult.
- Avoid leaking 'combined' create function to vfs-boot.
- Delete "create_from_buffer" functionality.

Change-Id: I6722b3076dc384338a4e689a4acfcd98400ca4bd

[devmgr] devhost launch, driver ctx, driver scan

- coordinator_launch_devhost() starts a devhost
v2 devhosts are blank slates, and will wait for
instructions on the provided rpc channel (hrpc)
- enumerate_drivers() kicks off driver enumeration
for now this simply scans /boot/lib/driver
- coordinator_new_driver() is called for each
driver located

Change-Id: I12b2c58aaaeae3432dfac8dae0c3351469673ab3

[devmgr] skeleton for devhost v2

Built when ENABLE_DEVHOST_V2 is true. Doesn't do much yet.

This is where we'll build up the new devhost framework which
will replace the old one once it's fully operational.

Change-Id: Ia30ff527eadb2a79643f9b08a9a1d2dec751a6aa

[devmgr][cleanup] call this agent "dev coordinator"

It lives in devmgr (but may move some day), and handles requests
from devhosts (but is not a devhost). It coordinates the top level
tree of devices and arranges to publish them in devfs, so let's call
it coordinator to keep it distinct from other services and components
in devmgr and devhosts.

Change-Id: I48312fd74f11fe07e43b3983db10eb03800fdff3