- Copy stable/10@296371 to releng/10.3 in preparation for 10.3-RC1
builds.
- Update newvers.sh to reflect RC1.
- Update __FreeBSD_version to reflect 10.3.
- Update default pkg(8) configuration to use the quarterly branch.

Approved by: re (implicit)

MFC r290406:

Fix g_eli error loss conditions

Sponsored by: Multiplay

MFC r273489 (cperciva):
Populate the GELI passphrase cache with the kern.geom.eli.passphrase
variable (if any) provided in the boot environment. Unset it from
the kernel environment after doing this, so that the passphrase is
no longer present in kernel memory once we enter userland.

This will make it possible to provide a GELI passphrase via the boot
loader.

Note: head and stable/10 differ as a result of r273174, which renames
the getenv(), setenv(), and unsetenv() functions with kern_getenv(),
kern_setenv(), and kern_unsetenv(), which was reverted in the relevant
parts of this change in 10-STABLE.

PR: 200448
Approved by: re (kib)
Sponsored by: The FreeBSD Foundation

Revert r285249, pending further investigation on how the build
broke.

Approved by: re (implicit)
Pointyhat: gjb (self)
Sponsored by: The FreeBSD Foundation

MFC r273489 (cperciva):
Populate the GELI passphrase cache with the kern.geom.eli.passphrase
variable (if any) provided in the boot environment. Unset it from
the kernel environment after doing this, so that the passphrase is
no longer present in kernel memory once we enter userland.

This will make it possible to provide a GELI passphrase via the boot
loader.

PR: 200448
Approved by: re (kib)
Sponsored by: The FreeBSD Foundation

MFC r271664:
Cache GELI passphrases entered at the console during the boot process,
in order to improve user-friendliness when a system has multiple disks
encrypted using the same passphrase.

Relnotes: yes
Approved by: re (gjb)

MFC: r267145

Fix the keyfile being cleared prematurely after r259428 (MFCed to stable/10
in r266749).

PR: 185084
Submitted by: fk@fabiankeil.de
Reviewed by: pjd

MFC: r259428

Clear content of keyfiles loaded by the loader after processing them.

MFC: r259429

Clear some more places with potentially sensitive data.

MFC r257965:
Add missing line breaks.

PR: 181900

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Make ELI destruction (including orphanization) less aggressive, making it
always wait for provider close. Old algorithm was reported to cause NULL
dereference panic on attempt to close provider after softc destruction.
If not global workaroung in GEOM, that could even cause destruction with
requests still in flight.

Use correct part of the Master-Key for generating encryption keys.
Before this change the IV-Key was used to generate encryption keys,
which was incorrect, but safe - for the XTS mode this key was unused
anyway and for CBC mode it was used differently to generate IV
vectors, so there is no risk that IV vector collides with encryption
key somehow.

Bump version number and keep compatibility for older versions.

MFC after: 2 weeks

replace uses of libkern gets with cngets

MFC after: 2 months

Before this change when GELI detected hardware crypto acceleration it will
start only one worker thread. For software crypto it will start by default
N worker threads where N is the number of available CPUs.

This is not optimal if hardware crypto is AES-NI, which uses CPU for AES
calculations.

Change that to always start one worker thread for every available CPU.
Number of worker threads per GELI provider can be easly reduced with
kern.geom.eli.threads sysctl/tunable and even for software crypto it
should be reduced when using more providers.

While here, when number of threads exceeds number of CPUs avilable don't
reduce this number, assume the user knows what he is doing.

Reported by: Yuri Karaban <dev@dev97.com>
MFC after: 3 days

Add support for creating GELI devices with older metadata version for use
with older FreeBSD versions:
- Add -V option to 'geli init' to specify version number. If no -V is given
the most recent version is used.
- If -V is given don't allow to use features not supported by this version.
- Print version in 'geli list' output.
- Update manual page and add table describing which GELI version is
supported by which FreeBSD version, so one can use it when preparing GELI
device for older FreeBSD version.

Inspired by: Garrett Cooper <yanegomi@gmail.com>
MFC after: 3 days

Prefer G_ELI_VERSION_* defines for version numbers over plain digits.

MFC after: 3 days

Fit lines into 80 chars.

MFC after: 3 days

Include sys/sbuf.h directly.

Reviewed by: pjd

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

Export GELI class version via sysctl kern.geom.eli.version.

MFC after: 1 week

When support for multiple encryption keys was committed, GELI integrity mode
was not updated to pass CRD_F_KEY_EXPLICIT flag to opencrypto. This resulted in
always using first key.

We need to support providers created with this bug, so set special
G_ELI_FLAG_FIRST_KEY flag for GELI provider in integrity mode with version
smaller than 6 and pass the CRD_F_KEY_EXPLICIT flag to opencrypto only if
G_ELI_FLAG_FIRST_KEY doesn't exist.

Reported by: Anton Yuzhaninov <citrin@citrin.ru>
MFC after: 1 week

Remove "for a moment" assignment. struct g_geom zeroed when allocated.

MFC after: 1 week

Instead of allocating memory for all the keys at device attach,
create reasonably large cache for the keys that is filled when
needed. The previous version was problematic for very large providers
(hundreds of terabytes or serval petabytes). Every terabyte of data
needs around 256kB for keys. Make the default cache limit big enough
to fit all the keys needed for 4TB providers, which will eat at most
1MB of memory.

MFC after: 2 weeks

In g_eli_read_done() and g_eli_write_done(), for a bio with
bio_children > 1, g_destroy_bio() is never called and the bio
leaks. Fix this by calling g_destroy_bio() earlier, before the check.

Submitted by: Victor Balada Diaz <victor@bsdes.net> (initial version)
Approved by: pjd (mentor)
MFC after: 1 week

Add some FEATURE macros for various GEOM classes.

No FreeBSD version bump, the userland application to query the features will
be committed last and can serve as an indication of the availablility if
needed.

Sponsored by: Google Summer of Code 2010
Submitted by: kibab
Reviewed by: silence on geom@ during 2 weeks
X-MFC after: to be determined in last commit with code from this project

Use the preload_fetch_addr() and preload_fetch_size() convenience
functions to obtain the address and size of the preloaded key files.

Sponsored by: Juniper Networks.

Add support for asterisk characters when filling in the GELI password
during boot.

Change the last argument of gets() to indicate a visibility flag and add
definitions for the numerical constants. Except for the value 2, gets()
will behave exactly the same, so existing consumers shouldn't break. We
only use it in two places, though.

Submitted by: lme (older version)

Add State tag, so 'geli status' will report active/suspended status, eg:

# geli status
Name Status Components
da0.eli SUSPENDED da0
da1.eli ACTIVE da1

Encryption keys array might be NULL if device is suspended. Check for this, so
we don't panic when we detach suspended device.

Move sc_akeyctx and sc_ivctx initialization to the g_eli_mkey_propagate()
function which eliminates code duplication and will ensure proper order
of operation.

Free opencrypto sessions on suspend, as they also might keep encryption keys.

Fix a bug introduced in r213067 where we use authentication key before
initializing it.

Bring in geli suspend/resume functionality (finally).

Before this change if you wanted to suspend your laptop and be sure that your
encryption keys are safe, you had to stop all processes that use file system
stored on encrypted device, unmount the file system and detach geli provider.

This isn't very handy. If you are a lucky user of a laptop where suspend/resume
actually works with FreeBSD (I'm not!) you most likely want to suspend your
laptop, because you don't want to start everything over again when you turn
your laptop back on.

And this is where geli suspend/resume steps in. When you execute:

# geli suspend -a

geli will wait for all in-flight I/O requests, suspend new I/O requests, remove
all geli sensitive data from the kernel memory (like encryption keys) and will
wait for either 'geli resume' or 'geli detach'.

Now with no keys in memory you can suspend your laptop without stopping any
processes or unmounting any file systems.

When you resume your laptop you have to resume geli devices using 'geli resume'
command. You need to provide your passphrase, etc. again so the keys can be
restored and suspended I/O requests released.

Of course you need to remember that 'geli suspend' won't clear file system
cache and other places where data from your geli-encrypted file system might be
present. But to get rid of those stopping processes and unmounting file system
won't help either - you have to turn your laptop off. Be warned.

Also note, that suspending geli device which contains file system with geli
utility (or anything used by 'geli resume') is not very good idea, as you won't
be able to resume it - when you execute geli(8), the kernel will try to read it
and this read I/O request will be suspended.

- Add missing comments.
- Make a comment consistent with others.

Change g_eli_debug to int, so one can turn off any GELI output by setting
kern.geom.eli.debug sysctl to -1.

MFC after: 2 weeks

Update copyright years.

MFC after: 1 week

Add support for AES-XTS. This will be the default now.

MFC after: 1 week

Implement switching of data encryption key every 2^20 blocks.
This ensures the same encryption key won't be used for more than
2^20 blocks (sectors). This will be the default now.

MFC after: 1 week

Define default overwrite count, so that userland can use it.

MFC after: 1 week

Correct offset conversion to little endian. It was implemented in version 2,
but because of a bug it was a no-op, so we were still using offsets in native
byte order for the host. Do it properly this time, bump version to 4 and set
the G_ELI_FLAG_NATIVE_BYTE_ORDER flag when version is under 4.

MFC after: 2 weeks

Use lower priority for GELI worker threads. This improves system
responsiveness under heavy GELI load.

MFC after: 3 days

Detach GELI providers on shutdown/reboot, which will allow providers underneath
to close properly.

Reported, reviewed and tested by: guido
MFC after: 1 week

Backout this commit whil a better solution is developed

When attaching a geli on boot make sure that it is detached
upon last close. (needed for a gmirror to properly shutdown
upon reboot when a geli is on top the gmirror)

Style(9).

Clear passphrase buffer after use.

Submitted by: Fabian Keil <fk@fabiankeil.de> (a bit different version)

Rename the kthread_xxx (e.g. kthread_create()) calls
to kproc_xxx as they actually make whole processes.
Thos makes way for us to add REAL kthread_create() and friends
that actually make theads. it turns out that most of these
calls actually end up being moved back to the thread version
when it's added. but we need to make this cosmetic change first.

I'd LOVE to do this rename in 7.0 so that we can eventually MFC the
new kthread_xxx() calls.

Commit 14/14 of sched_lock decomposition.
- Use thread_lock() rather than sched_lock for per-thread scheduling
sychronization.
- Use the per-process spinlock rather than the sched_lock for per-process
scheduling synchronization.

Tested by: kris, current@
Tested on: i386, amd64, ULE, 4BSD, libthr, libkse, PREEMPTION, etc.
Discussed with: kris, attilio, kmacy, jhb, julian, bde (small parts each)

Use root_mounted().

Overhaul driver/subsystem api's:
o make all crypto drivers have a device_t; pseudo drivers like the s/w
crypto driver synthesize one
o change the api between the crypto subsystem and drivers to use kobj;
cryptodev_if.m defines this api
o use the fact that all crypto drivers now have a device_t to add support
for specifying which of several potential devices to use when doing
crypto operations
o add new ioctls that allow user apps to select a specific crypto device
to use (previous ioctls maintained for compatibility)
o overhaul crypto subsystem code to eliminate lots of cruft and hide
implementation details from drivers
o bring in numerous fixes from Michale Richardson/hifn; mostly for
795x parts
o add an optional mechanism for mmap'ing the hifn 795x public key h/w
to user space for use by openssl (not enabled by default)
o update crypto test tools to use new ioctl's and add cmd line options
to specify a device to use for tests

These changes will also enable much future work on improving the core
crypto subsystem; including proper load balancing and interposing code
between the core and drivers to dispatch small operations to the s/w
driver as appropriate.

These changes were instigated by the work of Michael Richardson.

Reviewed by: pjd
Approved by: re

Fix geli after last commit for UP systems that are running SMP kernel.

Submitted by: Hyo geol, Lee <hyogeollee@gmail.com>
MFC after: 1 week

It is possible that GEOM taste provider before SMP is started.
We can't bind to a CPU which is not yet on-line, so add code that wait for
CPUs to go on-line before binding to them.

Reported by: Alin-Adrian Anton <aanton@spintech.ro>
MFC after: 2 weeks

I want CPU number here.

Noticed by: ru

Skip disabled CPU, because after we sched_bind() to a disabled CPU,
we won't be able to exit from the thread.

Function g_eli_cpu_is_disabled() stoled from kern_pmc.c.

PR: 104669
Reported by: Nikolay Mirin <nik@optim.com.ru>
MFC after: 1 week

Implement BIO_FLUSH handling by simply passing it down to the components.

Sponsored by: home.pl

Remove trailing spaces.

Before using byte offset for IV creation, covert it to little endian.
This way one will be able to use provider encrypted on eg. i386 on
eg. sparc64. This doesn't really buy us much today, because UFS isn't
endian agnostic.

We retain backward compatibility by setting G_ELI_FLAG_NATIVE_BYTE_ORDER
flag on devices with version number less than 2 and not converting the
offset.

Allow geli to operate on read-only providers.

Initial patch from: vd
MFC after: 2 weeks

Fix what looks like a typo: MODULE_DEPEND() takes module names,
not KLD file names; and GELI module's name is g_eli, not geom_eli.

Approved by: pjd (silence)
MFC after: 5 days

Implement data integrity verification (data authentication) for geli(8).

Supported by: Wheel Sp. z o.o. (http://www.wheel.pl)

Make kern.geom.eli.overwrites sysctl a tunable as well.

geli(8) provides keys on newsession time, so remove CRD_F_KEY_EXPLICIT flag
as HW crypto drivers don't support it.

Correct debug: we are sending child bio here, not parent bio.

MFC after: 1 week

Pass BIO_GETATTR requests down.

MFC after: 1 week

Typos.

Teach geli how to load keyfiles before root file system is mounted.
An example entries for loader.conf to make it possible:

geli_da0_keyfile0_load="YES"
geli_da0_keyfile0_type="da0:geli_keyfile0"
geli_da0_keyfile0_name="/boot/keys/da0.key0"
geli_da0_keyfile1_load="YES"
geli_da0_keyfile1_type="da0:geli_keyfile1"
geli_da0_keyfile1_name="/boot/keys/da0.key1"
geli_da0_keyfile2_load="YES"
geli_da0_keyfile2_type="da0:geli_keyfile2"
geli_da0_keyfile2_name="/boot/keys/da0.key2"

geli_da1s3a_keyfile0_load="YES"
geli_da1s3a_keyfile0_type="da1s3a:geli_keyfile0"
geli_da1s3a_keyfile0_name="/boot/keys/da1s3a.key"

Thanks for jhb and kan who showed me the right direction.

MFC after: 3 days

Check rootvnode variable to see if we still want to ask for passphrase on
boot. Other methods just don't work properly.

MFC after: 3 days

Clean up some sysctl descriptions, debug messages etc.

Approved by: pjd
MFC after: 3 days

Remove trailing spaces.

Remove dead code.

Found by: Coverity Prevent(tm)
MFC after: 3 days

Remove unused value.

Found by: Coverity Prevent(tm)
MFC after: 3 days

Check for g_read_data(9) errors properly:

o The only indication of error condition is NULL value returned by
the function;

o value pointed to by error argument is undefined in the case when
operation completes successfully.

Discussed with: phk

Fix copy&paste typo.

MFC after: 3 days

Don't forget to initialize crp_etype field.

Reported by: Nick Evans <nevans@syphen.net>
MFC after: 3 days

By default, when doing crypto work in software, start as many threads
as we have active CPUs and bind each thread to its own CPU.

MFC after: 3 days

Remove stale comment (we now always start worker thread).

MFC after: 3 days

Always run dedicated kernel thread (even when we have hardware support).
There is no performance impact, but allows to allocate memory with
M_WAITOK flag.
As a side effect this simplify code a bit.

MFC after: 3 days

We should now return 0.

Even if crypto_dispatch() return an error, request is not canceled and
our callback will still be called, just to tell us that requested
failed...

Reported by: Mike Tancsa <mike@sentex.net>
MFC after: 3 days

We don't need to clear allocated memory. This will speed-up things a bit.

MFC after: 3 days

Because code paths for I/O requests are quite complex, add comments above
the functions which participate in I/O paths.

MFC after: 1 day

GELI doesn't need cryptodev.

MFC after: 3 days

MFp4: Export more informations about encrypted providers.

MFC after: 1 week

Reduce default debug level to 0.

MFC after: 1 week

Add GEOM_ELI class which provides GEOM providers encryption.
For features list and usage see manual page: geli(8).

Sponsored by: Wheel Sp. z o.o.
http://www.wheel.pl
MFC after: 1 week