[fs-management][minfs]: Add option to set min data slices.

Allows the FS to pre-allocate data slices when created, so it will have
a known minimum size, unless resized.

Add mkfs option to set the data slices, and a minfs option to reflect
this, so it can be propagated through fs-mkfs.

TEST=fs-management-test, fs-test

Change-Id: I3d774da9166d2949485709b913100c3113045fd3

[async] Publish constant for async loop configuration.

First step towards disallowing NULL being passed to
async_loop_create() and async::Loop()

This adds a new constant and provides an alternative to the
existing one to be consistent with naming.

It also updates all client calls within zircon.

Test:CQ
Change-Id: I93c53e9b575248293dd5bfeea6269281fc329aa6

[async] Change async_t -> async_dispatcher_t

Change name of dispatcher type from async_t to async_dispatcher_t
Keep typedef of async_t for compatibility

Change async_{get,set}_default() to async_{get,set}_default_dispatcher()
Again, keep compatible apis until other layers have been modified to use
new api.

Bug:ZX-2337
Test:CQ
Change-Id: If0b5d061fd114d60e23078dbb5f2a6ed9ed6f6cc

[musl] Rename zx_get_startup_handle to zx_take_startup_handle

The new name more clearly describes the ownership semantics.

ZX-2235 #comment

Change-Id: I887c1f2ad657e3aff9716b9f1865184cb8cf05f7

[minfs][blobfs] Re-implement filesystem teardown

Rather than simply "sync + exit process", implement
a fully featured filesystem teardown procedure, capable
of:
- Tearing down all active connections.
- Syncing underlying media using a non-blocking dispatch loop.
- Use closures to signal both mounters and unmounters on
filesystem destruction.

As a consequence of the controlled filesystem teardown
procedure, make the "filesystem" objects in minfs and blobfs
no longer refcounted, to clarify that "Shutdown" is the
single point of destruction after mounting.

ZX-1577 #done
ZX-1923 #done
ZX-1959 #done
ZX-1576 #comment In Progress

Change-Id: I3f8d71af76e391c41c665434baf86a245a7cd223

[minfs] Add aggregator metrics

These metrics are, if active, logged at unmount time.

ZX-1942 #done

Change-Id: Idc1d5f999bb23caa8f71ba1083fcc1321e37571a

[async] Move async-loop implementation to its own directory.

People keep getting confused by the relationship between async-loop
and the rest of the library. So let's fix it.

This change includes compatibility includes for loop.h to be removed
after upper layers are fixed to use the new paths.

Change-Id: I95c57be533fbd91468275bb2470890a4f10101eb

[minfs] Internalize Minfs VFS layer, combine with Minfs object

Change-Id: Iff58f994b35c8daccf70c86e47900a1e918a6692

[async] Move async headers to match naming rules

The naming rules for C/C++ header files that are extensively used but
not part of the 'system headers' or 'global headers' as defined by
https://fuchsia.googlesource.com/docs/+/master/development/languages/c-cpp/naming.md#include-paths
states that the include lines for headers in the 'async' library should
be spelled like this:

<lib/async/dispatcher.h>

This moves the header files from system/ulib/async/include/async/dispatcher.h
to system/ulib/async/include/lib/async/dispatcher.h and updates
includes to match.

Change-Id: I85de9f17d67605d59df763be54715a1f2aa9ed77

[minfs] Clean up option parsing

ZX-1770 #done
ZX-1724 #done

Change-Id: Ic1afe498f2a0dab31965f09c32273254a070236c

[async] Separate fbl dependency from async_t

The C++ FIDL support library in Garnet wants to be able to use async_t
without linking against libfbl. Previously, the C declarations needed
create an async_wait_t were in the same header as code that depended on
fbl, which meant C++ clients of async_wait_t were forced to depend on
fbl.

After this change, the C++ wrappers for libasync interfaces are
separated from the core C declarations.

Change-Id: Ic172911a9dc775d5b5e540a33ce87d2dc218a3d9

[minfs] Convert minfs.h into private interface

Change-Id: I3a0fe542871172b591a0f178eb1d855f12742558

[minfs] Split host/target MinFS applications

As we take steps to separate the host/target interface
to MinFS (and reduce ifdefs), making a separate application
for the host-side MinFS is a necessary first step.

ZX-1502 #done

Change-Id: I53d65f6522039b20d869df733d5225023db8d613

[minfs][host] Support recursive copy.

This change extends the minfs cp command to check if the source
is a directory and to copy it recursively if so.

Change-Id: I9eab76855861ef5ac3cf8dea8dfe660f86f43984

[minfs] [host] Allow offset for minfs partition

ZX-1505 #done

Change-Id: Ieb1099c8524bd63221745c293a694ecfb276b390

[uapp][fvm][minfs] Fix wrong usage of S_IFBLK

S_IF* are not single bits, they are values in the S_IFMT mask.
The S_IS* macros exist to make testing st_mode values correctly easy.

Change-Id: I58847c200e58bf1ef8d8a1324a46ff10120c23b4

[fs] Clean usage of fs/trace.h, remove extern 'trace_bits'

- Use "xprintf" (conditionally compiled printfs), allowing
for a debugging printf style which matches the rest of Zircon.
- Remove the trace categories from fs/trace.h (which were
used rarely, and often inconsistently).

Change-Id: Ifd6c91c7dc91bb7d0ba31a42affd5ec13c150761

[minfs] Ensure host-side MinFS creates an image with permissions

Change-Id: If4a50a0b6e4885695012200d77bbf29884b0fe33

[fs][minfs] Add basic support for tracing MinFS

Conditionally define "TRACE_DURATION" (and other
macros which will be used by filesystem tracing)
within ulib/fs, to avoid "ifdef __Fuchsia__" bloat.

ulib/fs's tracing will continue to be modified
in future patches.

ZX-1300 #done

Change-Id: I1e25b5f2feb68e58b41887d8958eafa6ef5b1582

[minfs] always truncate file to the requested size

Fixes usage such as:
minfs data.blk@10m create
fvm fvm.blk create --data data.blk

Change-Id: I74d447911cc15e2c14638ba9dd3ef96a946a4d39

[blobstore] [minfs] On add manifest, copy files from manifest directory

Change-Id: Iffbfb0ff8e7e265f5109553831be5283fc43efef

[minfs] Refactor MinFS headers to be more fine-grained

Separate the following:
- On-Disk Structures
- Bcache (mechanism to interact with disk)
- MinFS Writeback
- MinFS Vnodes
- MinFS fsck

Into separate headers.

Change-Id: I920fe68d5c38ce53ed8e59ecee5003e6c3336577

[minfs] Convert MinFS to a library

ZX-1314 #comment In Progress

Change-Id: I0a7672d8321e0be07dd4c66232a0b4836db777a3

[minfs-blobstore] Respect the block device read-only flag.

At mount time, inspect the underlying device read-only flag and mount the
FS as read-ony if needed.

Adds a test-intended ioctl to ramfs to behave as a read only device.
This doesn't alter writing to the device, it just reports itself as
read-only.

ZX-1337 #done

Change-Id: I21691797c3c37982742d5a66ebbda33a4e87f456

[minfs] Add support for host-side fsck

Change-Id: Ie50df5d28f1cc8f6fbd235c02ee8cf273980d541

[blobstore] [minfs] Update host-side add manifest

For both, ignore lines starting with '#'
For blobstore, allow manifest lines of the form 'dst=src'
For blobstore, ignore duplicate blobs added from manifest

ZX-1354 #done
ZX-1355 #done

Change-Id: Icbbd958b862924f8a556abfab2e38e26b76c06f0

[minfs] Avoid using global VFS for MinFS

To be more specific, avoid using a global VFS for the
target-side MinFS implementation.

The host-side code still relies on a global, but now
ONLY the host-side code.

Change-Id: I8303450c28e00fa4e329e5f92a9586951b989553

[minfs][blobstore] Fix size_t usage in getline

Change-Id: If3a5d87e174e05abb96efb9f4f4da95da79eb5df

[minfs][fs-management] Add support for mounting MinFS as read-only

Change-Id: I5506f22a2c8a4f1c42060ecfbeb406cd2ae3c847

[minfs] Allow host tool to add files from manifest

Change-Id: I84f512675ab12a8f2307e4d66e437c3fcdf32c76

[minfs][fvm][container] Use scoped file descriptors

Change-Id: I57c2e2290786be4c91cb9ccbf1ae52c5bc492e83

[minfs] Remove devfs hack from host-side minfs tool

Change-Id: I8f64062a7a0fda7cef4838d9747db9e3bff6b5ff

[minfs] Add host-side tests

US-332 #done

Change-Id: I26475e93227c2950cfcda707025df5b94179e9bc

[minfs][host] Reduce host-side emu macros

Additionally, make "emu_..." functions execute for
the fs only, without a fallback to call host-side
functions.

Provide a "FileWrapper" class to abstract away
the "emu_..." file descriptors from native file
descriptors.

Change-Id: I27ed154c91847a25052c1b9ddb5da6fd13485d53

[fs] Move VFS RPC code into Connection object.

The RPC functions become members of the Connection object itself
such that the Connection encapsulates both the state and the behavior.

As a result of this change, we no longer need separate AsyncDispatcher
or AsyncHandler objects since all of the work is being done by the
Connection.

Also moves the readable/writable predicates to a common location
and fixed them to use O_ACCMODE, which will make it easier to add
support for O_PATH later.

Change-Id: Ib2940aae1ff03d36d3310d6887ce725069373030

[minfs] Minimize host-side tooling

- Remove bitrotting tests
- Remove "rename" and "unlink" from host-side tools.
These functions should not be necessary for generating
golden images, and they increase the complexity of the
VFS layer (the Vfs::Walk function should be private).

Change-Id: I96599828e6e39db15cce95286481671fa49b2611

[fs] Separate Vnode, Vfs include files

Additionally, fix some headers, remove "ifdef cplusplus"
macros in ulib/fs (it isn't functional for C filesystems),
make the Readdir cookie typed, and make "MountNode" private
to the Vfs class.

MG-1117 #in-progress

Change-Id: Iba2169875b5dd73619f3e8c919fefb998a94a08f

[zx] Magenta -> Zircon

The Great Renaming is here!

Change-Id: I3229bdeb2a3d0e40fb4db6fec8ca7d971fbffb94

[fs][minfs] Replace the vfs-dispatcher with libasync

Change-Id: Icfc3566510ae0e5f29ac2f0615681c00c831ece9

[zircon][mxtl->fbl] Rename mxtl to fbl

Change-Id: Ie21b6498e1bfb0a7fa0315e40b9e5c3ee78646be

[fs] Make VFS non-static object holding filesystem-global state

Change-Id: I60735f33a751e98d1b2324886b59d3144ef49276

[fs] Remove duplicate dispatchers from minfs, blobstore

Change-Id: Iabb3de2e5fcdc7d6bffe472b993c25135255df94

[libfs] Move more common code into libfs and use mx::channel.

Move the check for whether a Vnode really is a directory into libfs
and renamed fs::Vfs::ServeFilesystem to ServeDirectory to more clearly
indicate this responsibility.

Changed the vfs dispatcher contract to use mx::channel to clarify
where handle ownership transfers occur.

Change-Id: I4238518dd6a0d3ac3cf64ea991ed9df9a3eb0e52

[libfs] Plumb fs::Dispatcher through filesystem servers.

Replaced vfs_rpc_serve with Vfs::ServeFilesystem which takes an
fs::Dispatcher. As part of this change, we also make fs::MxioDispatcher
a little more flexible about which thread it runs on.

This will be followed by a change which plumb the calling dispatcher
through as an argument to Serve() instead of requiring each Vnode to
carry it (unless it wants to serve somewhere else).

Change-Id: Iac53e00d8eaf9d1bd781b2f9b56debc87a49fb8b

[cpp][countof] Prefer the typesafe mxtl::count_of in C++ code

Change-Id: I3e475731f6872e6daf764f31964bed9f09186c7c

[minfs] Determine block dev size from BLOCK_GET_INFO, not stat

Change-Id: Ia2e2e8660b5d5f8cd4be68e3aee687b503d3b642

[uapp][status] Use the new MX_OK and MX_ERR_* mx_status_t names

Change-Id: Iaa8c8c12736e49d202c33612b22d2acd528fcf2d

[fs][minfs][blobstore] Don't #define info

Previously, trace.h defined info, warn, and error as macros. These names are
very commonly used and as this header is being included in more places these
macros caused name conflicts.

This patch changes the macros to be in ALL_CAPS and to start with "FS_" to try
to avoid conflicts with other macros.

Change-Id: Idd9e27efac57624db530b0b1b1e81a934da22f3b

[minfs] Convert bcache to unique_ptr

Change-Id: I76be5152694da0b2484061bcb3b95eff608b0a35

[minfs] Significantly simplify block cache

Change-Id: I7da98f2d593e798ea86e941c8a06cbc1437a09b6

[minfs] Remove reliance of fsck on bcache

We're using page-cache structures (VMOs) to
store files on mounted filesystems; we may as
well do the same for fsck. This is necessary
to remove portions of the bcache.

Change-Id: I0ace36d260ce1d0970472d3a28ed0d3387a8a78d

[fs] Make Vnodes refcounted

Rather than using the old "RefAcquire" and "RefRelease"
functions, manually incrementing and decrementing, this
patch updates Vnodes to use "mxtl::RefPtr".

MG-719 #done

Change-Id: Idb998a36c52d44aaf35b6a03f4c46077eaa7709f

[processargs] tidy up naming and numbering

This cleans stuff up ahead of the addition of string tables for
the upcoming PA_NS_DIR and PA_NS_REQ types and the eventual
flattening of PA_MXIO_* to a single type, using subtypes for
protocol.

- Align names with current-day usage
- Organize numbering / ordering around subsystem usage
- Reduce "type" field to 8 bits
- Reserve space for "subtype" field
- Use PA_ namespace and shorter a lot of very long names

Change-Id: I5e59f412c3b9bd79d90f921a248a6d98d4b703bc

[mxio] add multi-thread pool to mxio_dispatcher

. add multithreaded cpp dispatcher to fs/vfs-dispatcher.cpp
. change minfs to use multithread dispatcher (only use, currently)

. add big vfs lock to prevent weirdness as we open up multithreading
. move vfs_iostate private to vfs-rpc.c (narrow locking changes)
. acquire vfs_big_lock before and after dispatch
. acquire vn before and release after dispatch

Change-Id: I3974bc1ad558c57c2878146876dc3100ebe6159f

[fs] Convert Vnodes to C++

- vnode_t struct split into:
- fs::Vnode abstract base class, with default "ERR_NOT_SUPPORTED"
versions of operations.
- MinFS: subclass minfs::VnodeMinfs
- Blobstore: subclass blobstore::VnodeBlob
- MemFS: subclass memfs::VnodeMemfs, which provides some common
functionality to all types of MemFS vnodes, but which is itself
subclassed into "VnodeFile", "VnodeDir", "VnodeVmo", and
"VnodeDevice".

Change-Id: I114acd067dec7849f1591eddedf020102ddf7e67

[minfs] Use ulib/bitmap instead of ad-hoc bitmap

Change-Id: Ie0ef231c058df65c26729312b900e74c72626384

[fs] Use the countof() macro (for array sizes) in some more places

Change-Id: I71a8b246f5f5fc3104c1664649d53b50cd473e1d

[minfs] Clarify MinFS CLI behavior

Change-Id: Idcc7755d94326abbc05dabcde9ac9633fe71c574

[minfs] Add size assertions around bcache

Change-Id: Ic2664b5c076a634fe0081cb88bd416bc5a2ed37a

[devmgr][fs][fs-management] Pass Block Device by fd, not path

Change-Id: Ie04ea7481edb6eb1375338666face5d14c7d879c

[minfs] Continue MinFS conversion to C++

- Convert bcache_t to Bcache class
- Convert block_t to Block class
- Convert bitmap_t to Bitmap class
- Convert minfs_t to Minfs class
- Converted Linked Lists to mxtl doubly linked lists (for O(k) removal)
- Converted ad-hoc hashmap to mxtl hashmap
- Removed extraneous 'output' parameter from block cache "Get"
functions. Parameter is already accessible through the 'Block' return
value.
- Convert all access to "BlockNodes" to use RefPtrs
- Remove 'gbb_ctxt_t'; it's no longer necessary. To distinguish between
'bitmap blocks' and 'other blocks', use the naming convention
'bitmap_blk' where &gbb was used.

Change-Id: I724db3f0a996a92e9968b87bd5e575ce0e5c82a2

[minfs][fs] Convert MinFS to C++

This patch contains the minimal set of changes to compile MinFS as a C++
program -- it does not intend to use "best-style" C++ yet; those changes
will appear in subsequent patches.

Change-Id: Iebdf381f3aece27b863a30a496b05520352757d8