- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Make hastctl(1) ('list' command) output a worker pid.

Reviewed by: pjd
MFC after: 3 days

Get rid of libl dependency. We needed it only to provide yywrap. But
yywrap is not necessary when parsing a single hast.conf file.

Suggested by: kib
Reviewed by: pjd

Remove code duplication.

Add the Clang specific -Wmissing-variable-declarations to WARNS=6.

This compiler flag enforces that that people either mark variables
static or use an external declarations for the variable, similar to how
-Wmissing-prototypes works for functions.

Due to the fact that Yacc/Lex generate code that cannot trivially be
changed to not warn because of this (lots of yy* variables), add a
NO_WMISSING_VARIABLE_DECLARATIONS that can be used to turn off this
specific compiler warning.

Announced on: toolchain@

Rename 'status' command to 'list' and introduce new 'status' which produces
more terse output more observable for both scripts and humans.

Also, it shifts hastctl closer to GEOM utilities with their list/status command
pairs.

Approved by: pjd
MFC after: 4 weeks

Removed redundant includes.

Fix casting.

MFC after: 3 days

Add i/o error counters to hastd(8) and make hastctl(8) display
them. This may be useful for detecting problems with HAST disks.

Discussed with and reviewed by: pjd
MFC after: 1 week

Fixes to man8 groff mandoc style, usage mistakes, or typos.

PR: 168016
Submitted by: Nobuyuki Koganemaru
Approved by: gjb
MFC after: 3 days

For functions that return -1 on failure check exactly for -1 and not for
any negative number.

MFC after: 3 days

Spelling fixes for sbin/

Use NO_WCAST_ALIGN for usr.bin/hastctl and usr.bin/hastd; the alignment
warnings in sbin/hastd/lzf.c are only emitted for i386 and amd64, and
there they can be safely ignored.

MFC after: 1 week

Revert r226726. The line was not duplicated.

Spotted by: pluknet
MFC after: 3 days

Remove duplicated line.

MFC after: 3 days

Prefer PJDLOG_ASSERT()/PJDLOG_ABORT() over assert().

MFC after: 3 days

When exiting with error because of an invalid command line argument
use errx(3), not err(3), and the exit code from sysexits(3).

Approved by: pjd (mentor)

Compile hastd and hastctl with capsicum support.

X-MFC after: capsicum merge

Keep statistics on number of BIO_READ, BIO_WRITE, BIO_DELETE and BIO_FLUSH
requests as well as number of activemap updates.

Number of BIO_WRITEs and activemap updates are especially interesting, because
if those two are too close to each other, it means that your workload needs
bigger number of dirty extents. Activemap should be updated as rarely as
possible.

MFC after: 1 week

Currently we are unable to use capsicum for the primary worker process,
because we need to do ioctl(2)s, which are not permitted in the capability
mode. What we do now is to chroot(2) to /var/empty, which restricts access
to file system name space and we drop privileges to hast user and hast
group.

This still allows to access to other name spaces, like list of processes,
network and sysvipc.

To address that, use jail(2) instead of chroot(2). Using jail(2) will restrict
access to process table, network (we use ip-less jails) and sysvipc (if
security.jail.sysvipc_allowed is turned off). This provides much better
separation.

MFC after: 1 week

Fix assert messages.

Approved by: pjd (mentor)

Remove hast_proto_recv(). It was used only in one place, where
hast_proto_recv_hdr() may be used. This also fixes the issue
(introduced by r220523) with hastctl, which crashed on assert in
hast_proto_recv_data().

Suggested and approved by: pjd (mentor)

hastd(8) maintains a map of dirty extents, not hastctl(8). Fix this.

Approved by: pjd (mentor)
MFC after: 3 days

Forgot to commit this as part of r219873.

MFC after: 1 week

When dropping privileges prefer capsicum over chroot+setgid+setuid.
We can use capsicum for secondary worker processes and hastctl.
When working as primary we drop privileges using chroot+setgid+setuid
still as we need to send ioctl(2)s to ggate device, for which capsicum
doesn't allow (yet).

X-MFC after: capsicum is merged to stable/8

Forgot to commit this as a part of r219818.

MFC after: 1 week

In command line options allow size to be specified using k/M/G/T
suffixes.

Approved by: pjd (mentor)
MFC after: 1 week

Print some of the numbers in human readable form (using %N).

MFC after: 2 weeks

- Turn on printf extentions.
- Load support for %T for pritning time.
- Add support for %N for printing number in human readable form.
- Add support for %S for printing sockaddr structure (currently only AF_INET
family is supported, as this is all we need in HAST).
- Disable gcc compile-time format checking as this will no longer work.

MFC after: 2 weeks

Allow to compress on-the-wire data using two algorithms:
- HOLE - it simply turns all-zero blocks into few bytes header;
it is extremely fast, so it is turned on by default;
it is mostly intended to speed up initial synchronization
where we expect many zeros;
- LZF - very fast algorithm by Marc Alexander Lehmann, which shows
very decent compression ratio and has BSD license.

MFC after: 2 weeks

Allow to checksum on-the-wire data using either CRC32 or SHA256.

MFC after: 2 weeks

Drop privileges after connecting to hastd, but before sending or receiving
anything.

MFC after: 1 week

Add missing argument after r218192.

Add functions to initialize/finalize pjdlog. This allows to open/close log
file at will.

MFC after: 1 week

Fix some more warnings found by clang.

Fix typos, spelling, formatting and mdoc mistakes found by Nobuyuki while
translating these manual pages. Minor corrections by me.

Submitted by: Nobuyuki Koganemaru <n-kogane@syd.odn.ne.jp>

Update the arguments to yy_config_parse() to match r210883.

Choose the more conservative option ('yes' to exit on error) to match
the equivalent code in hastd.

Fix typo.

PR: docs/149033
Submitted by: Kolar <hsn@sendmail.cz>
MFC after: 3 days

mdoc: move remaining sections into consistent order

This pertains mostly to FILES, HISTORY, EXIT STATUS and AUTHORS sections.

Found by: mdocml lint run
Reviewed by: ru

Fix compilation with WITHOUT_CRYPT or WITHOUT_OPENSSL options.

Reported by: Andrei V. Lavreniyuk <andy.lavr@reactor-xg.kiev.ua>
MFC after: 3 days

Remove redundant WARNS?=6 overrides and inherit the WARNS setting from
the toplevel directory.

This does not change any WARNS level and survives a make universe.

Approved by: ed (co-mentor)

Please welcome HAST - Highly Avalable Storage.

HAST allows to transparently store data on two physically separated machines
connected over the TCP/IP network. HAST works in Primary-Secondary
(Master-Backup, Master-Slave) configuration, which means that only one of the
cluster nodes can be active at any given time. Only Primary node is able to
handle I/O requests to HAST-managed devices. Currently HAST is limited to two
cluster nodes in total.

HAST operates on block level - it provides disk-like devices in /dev/hast/
directory for use by file systems and/or applications. Working on block level
makes it transparent for file systems and applications. There in no difference
between using HAST-provided device and raw disk, partition, etc. All of them
are just regular GEOM providers in FreeBSD.

For more information please consult hastd(8), hastctl(8) and hast.conf(5)
manual pages, as well as http://wiki.FreeBSD.org/HAST.

Sponsored by: FreeBSD Foundation
Sponsored by: OMCnet Internet Service GmbH
Sponsored by: TransIP BV