MFC r362130:
Control for Special Register Buffer Data Sampling mitigation.

MFC r358315:
Fix IBRS for machines with IBRS_ALL capability.

MFC r354759:
TSX Asynchronous Abort mitigation for Intel CVE-2019-11135.
This CVE has already been announced in FreeBSD SA-19:26.mcu.

Mitigation for TAA involves either turning off TSX or turning on the
VERW mitigation used for MDS. Some CPUs will also be self-mitigating
for TAA and require no software workaround.

Control knobs are:
machdep.mitigations.taa.enable:
0 - no software mitigation is enabled
1 - attempt to disable TSX
2 - use the VERW mitigation
3 - automatically select the mitigation based on processor
features.

machdep.mitigations.taa.state:
inactive - no mitigation is active/enabled
TSX disable - TSX is disabled in the bare metal CPU as well as
- any virtualized CPUs
VERW - VERW instruction clears CPU buffers
not vulnerable - The CPU has identified itself as not being
vulnerable

Nothing in the base FreeBSD system uses TSX. However, the instructions
are straight-forward to add to custom applications and require no kernel
support, so the mitigation is provided for users with untrusted
applications and tenants.

Reviewed by: emaste, imp, kib, scottph
Sponsored by: Intel
Differential Revision: 22374

MFC r354649:
Workaround for Intel SKL002/SKL012S errata.

Security: CVE-2018-12207

MFC r347368:
x86: Put other CPUs into tight loop when updating Intel microcode from
loaded OS.

MFC r337715, r337751, r337754, r337758, r337813, r338354, r338687,
r339124, r341821:
Add support for boot-time Intel microcode loading.

MFC r347566:
Mitigations for Microarchitectural Data Sampling.

Reference: https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00233.html
Security: CVE-2018-12126, CVE-2018-12127, CVE-2018-12130, CVE-2019-11091
Security: FreeBSD-SA-19:07.mds
Reviewed by: jhb
Tested by: emaste, lwhsu
Approved by: so (gtetlow)

MFC r334050, r334051:
Flush caches before initiating a microcode update on Intel CPUs.

Approved by: re (gjb, kib)

MFC r334004:
Add Intel Spec Store Bypass Disable control.

This also includes the i386/include/pcpu.h part of the r334018.

Security: CVE-2018-3639
Approved by: re (gjb)

Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

MFC r328083,328096,328116,328119,328120,328128,328135,328153,328157,
328166,328177,328199,328202,328205,328468,328470,328624,328625,328627,
328628,329214,329297,329365:

Meltdown mitigation by PTI, PCID optimization of PTI, and kernel use of IBRS
for some mitigations of Spectre.

Tested by: emaste, Arshan Khanifar <arshankhanifar@gmail.com>
Discussed with: jkim
Sponsored by: The FreeBSD Foundation

MFC r327963:
When re-evaluating cpu_features, also re-print CPU identification.

MFC r327597:
Make it possible to re-evaluate cpu_features.

MFC r315588:
Update the list of cpudev ioctls which require write access.

MFC r315586:
Style.

MFC r308538:
Increase the max allowed size of the microcode update blob for x86.

MFC r308218: Add support for microcode update on newer AMD CPUs (10h+)

MFC r308225: dev/cpuctl: put debug output under CPUCTL_DEBUG rather than DEBUG

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Replace a number of conflations of mp_ncpus and mp_maxid with either
mp_maxid or CPU_FOREACH() as appropriate. This fixes a number of places in
the kernel that assumed CPU IDs are dense in [0, mp_ncpus) and would try,
for example, to run tasks on CPUs that did not exist or to allocate too
few buffers on systems with sparse CPU IDs in which there are holes in the
range and mp_maxid > mp_ncpus. Such circumstances generally occur on
systems with SMT, but on which SMT is disabled. This patch restores system
operation at least on POWER8 systems configured in this way.

There are a number of other places in the kernel with potential problems
in these situations, but where sparse CPU IDs are not currently known
to occur, mostly in the ARM machine-dependent code. These will be fixed
in a follow-up commit after the stable/11 branch.

PR: kern/210106
Reviewed by: jhb
Approved by: re (glebius)

Always allow loading of cpuctl(4). When a CPU feature is not
supported, e.g. CPUID or MSR, return ENODEV from the ioctl which needs
that feature.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Approved by: re (hrs)

Improve panic message by specifying on which cpu it really is.

Increase allowed size of the microcode blob to 32KB. Some Intel CPU's
updates weight 28KB.

PR: 179523
MFC after: 1 week

Make cpuctl_do_cpuid() and cpuctl_do_cpuid_count() return void.
There is no error to report.

Requested by: attilio
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Restore the ABI of the cpuctl(4) ioctl request CPUCTL_CPUID, use
separate argument structure with added level_type field for
CPUID_CPUID_COUNT request.

Reviewed by: attilio (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Following comments in r242565 add the possibility to specify ecx when
performing cpuid calls.
Add also a new way to specify the level type to cpucontrol(8) as
reported in the manpage.

Sponsored by: EMC / Isilon storage division
Reviewed by: bdrewery, gcooper
Testerd by: bdrewery

Use correct types for sizeof() in the calculations for the malloc(9) sizes [1].
While there, remove unneeded checks for failed allocations with M_WAITOK flag.

Submitted by: Conrad Meyer <cemeyer@uw.edu> [1]
MFC after: 1 week

Call free() on the pointer returned from malloc().

Reported and tested by: Oliver Pinter <oliver.pntr@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Approved by: re (delphij)

Match malloc(9) calls with free(9), not contigfree(9). Also remove
unneeded checks for NULL, free(9) can handle NULL pointers on its own,
and the regions were allocated with M_WAITOK flag as well.

Reported and tested by: Larry Rosenman <ler@lerctr.org>
MFC after: 1 week

Increase the microcode max size to 16K to accomodate more recent Intel
firmware.

Typos in comments.

cpuctl_do_cpuid: explicitly use ecx=0 for cpuid call

... instead of whatever random value may happen to be in the register.
ecx is important to some cpuid leaves.

To do: extend cpuctl interface to provide for ecx value parameter.

MFC after: 5 days

Add VIA microde update support to cpuctl(4) and cpucontrol(8).

Support have been tested with X2 CPU and QuadCore CPU.

MFC after: 1 month

- Add support to atomically set/clear individual bits of a MSR register
via cpuctl(4) driver. Two new CPUCTL_MSRSBIT and CPUCTL_MSRCBIT ioctl(2)
calls treat the data field of the argument struct passed as a mask
and set/clear bits of the MSR register according to the mask value.
- Allow user to perform atomic bitwise AND and OR operaions on MSR registers
via cpucontrol(8) utility. Two new operations ("&=" and "|=") have been
added. The first one applies bitwise AND operaion between the current
contents of the MSR register and the mask, and the second performs bitwise
OR. The argument can be optionally prefixed with "~" inversion operator.
This allows one to mimic the "clear bit" behavior by using the command
like this:
cpucontrol -m 0x10&=~0x02 # clear the second bit of TSC MSR

Inversion operator support in all modes (assignment, OR, AND).

Approved by: re (kib)
MFC after: 1 month

- Don't zero data field in case of MSR write operation. Before this change
the value written to MSR register was always 0 regardless of value passed
by user.
- Use proper data pointer when performing AMD microcode update. Previously,
the pointer to user-space data has been provided instead, which is totally
incorrect.

Approved by: re (kib)
MFC after: 1 week

Replace all calls to minor() with dev2unit().

After I removed all the unit2minor()/minor2unit() calls from the kernel
yesterday, I realised calling minor() everywhere is quite confusing.
Character devices now only have the ability to store a unit number, not
a minor number. Remove the confusion by using dev2unit() everywhere.

This commit could also be considered as a bug fix. A lot of drivers call
minor(), while they should actually be calling dev2unit(). In -CURRENT
this isn't a problem, but it turns out we never had any problem reports
related to that issue in the past. I suspect not many people connect
more than 256 pieces of the same hardware.

Reviewed by: kib

Remove unneeded D_NEEDMINOR from cpuctl(4).

The D_NEEDMINOR flag was introduced for drivers that do not actually
depend on storing a device unit/minor number, but require the ability to
address the cdevs by this number, which is used by clone_create().

The cpuctl(4) driver sets D_NEEDMINOR, even though it doesn't use the
clone_create() API. Remove the flag, because maybe we want to get rid of
it somewhere in the far future.

- Add cpuctl(4) pseudo-device driver to provide access to some low-level
features of CPUs like reading/writing machine-specific registers,
retrieving cpuid data, and updating microcode.
- Add cpucontrol(8) utility, that provides userland access to
the features of cpuctl(4).
- Add subsequent manpages.

The cpuctl(4) device operates as follows. The pseudo-device node cpuctlX
is created for each cpu present in the systems. The pseudo-device minor
number corresponds to the cpu number in the system. The cpuctl(4) pseudo-
device allows a number of ioctl to be preformed, namely RDMSR/WRMSR/CPUID
and UPDATE. The first pair alows the caller to read/write machine-specific
registers from the correspondent CPU. cpuid data could be retrieved using
the CPUID call, and microcode updates are applied via UPDATE.

The permissions are inforced based on the pseudo-device file permissions.
RDMSR/CPUID will be allowed when the caller has read access to the device
node, while WRMSR/UPDATE will be granted only when the node is opened
for writing. There're also a number of priv(9) checks.

The cpucontrol(8) utility is intened to provide userland access to
the cpuctl(4) device features. The utility also allows one to apply
cpu microcode updates.

Currently only Intel and AMD cpus are supported and were tested.

Approved by: kib
Reviewed by: rpaulo, cokane, Peter Jeremy
MFC after: 1 month