makesyscalls: generate core libsys header

Create a header covering most of the "stable" libsys interfaces.
Specifically __sys_<foo> syscall stubs and __sys_<foo>_t typedefs for
those interfaces.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D44386

sysproto.h: sys/acl.h -> sys/types.h

In sysproto.h, stop including sys/acl.h as syscall defintions now use
__acl* types from sys/_types.h. Add sys/types.h to provide types
previously provided by sys/param.h (via sys/acl.h).

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D44467

libsys: don't try to expose yield

The undocumented yield system call has never been implemented via libc
or libsys (except accidentally for <15 minutes in 1998 between commits
abd529cebab9 and 0db2fac06ab7). Avoid trying to export it now to avoid
failures when linking with --no-undefined-version.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D44236

lib{c,sys}: expose _getlogin consistently

Historically we exposed _getlogin as a private symbol on a per-arch basis
(except on aarch64 and riscv) for no obvious reason. We now need to
expose it for libc's use so remove the special case from
makesyscalls.lua and expose it in the generated syscalls.map.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D44116

makesyscalls: generate private syscall symbols

For libsys we need to expose all the private symbols (_ and __sys_
prefixes) so libsys can replace the libc versions. Rather than trying
to maintain a table, teach makesyscalls to generate it.

There are a small number of "_" prefixed symbols that are exposed as
public interfaces rather than in the private symbol space. Since the
list is short, just hardcode it for now.

If doesn't appear that we need to export freebsd#_foo symbols for compat
system calls explicitly. If it turns out we do, there are probably few
enough of them to handle seperately.

Reviewed by: kib, emaste, imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/908

makesyscalls: add COMPAT14 support

Reviewed by: kevans, imp
Fixes: 84d12f887c91f Add a COMPAT_FREEBSD14 kernel option
Differential Revision: https://reviews.freebsd.org/D42861

makesyscall: Simplify a bit emitting syscall declarations

Reviewed by: kevans, imp
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42168

makesyscalls: don't make syscall.mk by default

We only want to produce syscall.mk for the main syscall table so default
to not producing it (send it to /dev/null) and add a syscalls.conf to
sys/kern to trigger the creation of sys/sys/syscall.mk. This eliminates
the need for entries in other syscalls.conf files and is a cleaner
pattern going forward.

Reviewed by: kevans, imp
Differential Revision: https://reviews.freebsd.org/D42663

sys: Remove $FreeBSD$: two-line lua tag

Remove /^--\n--\s*\$FreeBSD\$.*$\n/

makesyscall: Stop generating $FreeBSD$

With 14 coming, we no longer need to generate the $FreeBSD$. We can
likely MFC that to 13 as well.

MFC After: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D39879

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

makesyscalls.lua: Minor fluff removal

luacheck pointed out two minor issues: line isn't declared as a global,
so declare it local. Also remove an unused parameter.

Suggested by: kevans
Sponsored by: Netflix

makesyscalls.lua: Use "sysxxx" consistently

Find the few places where we use 'sysxxx' and use "sysxxx" instead to be
more consistent.

Sponsored by: Netflix

makesyscalls.lua: Make more luaish

x["y"] can be written as x.y, which looks better and is a more typical
lua idiom.

Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D39709

syscallarg_t: Add a type for system call arguments

This more clearly differentiates system call arguments from integer
registers and return values. On current architectures it has no effect,
but on architectures where pointers are not integers (CHERI) and may
not even share registers (CHERI-MIPS) it is necessiary to differentiate
between system call arguments (syscallarg_t) and integer register values
(register_t).

Obtained from: CheriBSD

Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D33780

Revert "syscallarg_t: Add a type for system call arguments"

Missed issues in truss on at least armv7 and powerpcspe need to be
resolved before recommit.

This reverts commit 3889fb8af0b611e3126dc250ebffb01805152104.
This reverts commit 1544e0f5d1f1e3b8c10a64cb899a936976ca7ea4.

syscallarg_t: Add a type for system call arguments

This more clearly differentiates system call arguments from integer
registers and return values. On current architectures it has no effect,
but on architectures where pointers are not integers (CHERI) and may
not even share registers (CHERI-MIPS) it is necessiary to differentiate
between system call arguments (syscallarg_t) and integer register values
(register_t).

Obtained from: CheriBSD

Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D33780

sysent: move away from allowing all compat options for other ABIs

Notably, the current compat_options only makes sense for native and
freebsd32 ABIs. For the others, it just adds cruft. Switch to having
sets of compat options, and default to the native set. Setup the other
ABIs where it doesn't make sense to opt-out of the native set.

This removes some redundant COMPAT_FREEBSD* stuff from Linuxolator bits.

line_expr in makesyscalls.lua is fixed to allow empty strings to be
specified, since they're harmless.

Reviewed by: brooks, kib (both earlier version)
Differential Revision: https://reviews.freebsd.org/D33356

syscalls: add COMPAT13

Reviewed by: brooks
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33343

makesyscalls: add a new SYSMUX type

This type is for system call multiplexers (syscall(2), __syscall(2))
that don't have a normal handler and instead are handled in the
machine-dependent syscall code.

Reviewed by: kib, imp

makesyscalls: strip _Contains_ argument annotations

Reviewed by: kevans

makesyscalls: add override of ABI change detection

While we can detect most ABI changes through analysis of
syscalls.master with suitable annotations, to cases are handled
in the core implementation and others have changes that can not be
infered. Add two new config variables syscall_abi_change and
syscall_no_abi_change which override the detected value. Both are
space-seperated lists of syscall names.

Reviewed by: kevans

makesyscalls: automate detection of ABI changes

Use pattern matching including matches of _Contains_*_ argument
annotations to (mostly) determine which system calls require
ABI-specific handling. Automatically treat syscalls as NOPROTO
if no ABI changes are present.

Reviewed by: kevans

makesyscalls.lua: Add the ABI function prefix reliably.

The previous code mostly worked, but ended up adding bogus sys_<foo>()
declerations.

Reviewed by: kevans

makesyscalls: allow config to force OBSOL and UNIMPL

The obsol and unimpl config variables are space-seperated lists of
syscalls that should treated as being declared OBSOL and UNIMPL.

The allows an ABI to exclude select system calls listed in
syscalls.master.

Reviewed by: kevans

makesyscalls: handle 64-bit args on 32-bit

On 32-bit architectures, 64-bit arguments are passed in pairs of
registers. On non-x86 architectures these arguments must be in evenly
aligned registers which necessiciates inserting a pad register into the
argument list. This has historically been supported by adding ifdefs
around padded and unpadded syscall defintions in syscalls.master.

In order to enable generation of 32-bit support files from the base
syscalls.master, pull this support in to makesyscalls.lua enabled by
adding pair_64bit to abi_flags.

The changes to sys_proto.h simply add #ifdef PAD64_REQUIRED
around pad arguments in struct <syscall>_args. In systrace_args(),
replace static syscall index values with post-incremented indexs
allowing a simple ifdef around the argument. Under -O1 or higher
code generation is identical. systrace_entry_setargdesc() is a bit
more complicated as we switch on argument indices. Solve this
with some use of define/undef pairs to compute the correct indices.

Reviewed by: kevans

makesyscalls: handle arrays of pointers

When the config variable abi_intptr_t is not "", transform arrays of
pointers to arrays of abi_intptr_t.

Reviewed by: kevans

makesyscalls: handle longs in ABI compat

Replace long-derived types with their abi equivalent where
required by the target ABI. There are two cases:
- All pointers to types that go from 64-bit to 32-bit between the
default ABI and the target ABI.
- Signed arguments that go from 64-bit to 32-bit (these require
sign-extension before passing to general kernel ABIs).

This adds four new config variables: abi_long, semid_t, abi_size_t,
and abi_u_long which default to long, size_t, and u_long respectively.

Reviewed by: kevans

makesyscalls.lua: Allow translation of intptr_t arguments

Translate instances of intptr_t to the config value abi_intptr_t
(defaults to "intptr_t"). Used in CheriABI to translate intptr_t
to intcap_t for hybrid kernels.

Reviewed by: kevans

makesyscalls: intptr_t arguments are pointers

Match intptr_t arguments as pointers. Currently this is a no-op
change to systrace_args.c.

Reviewed by: kevans

makesyscalls: Add a way to include per-ABI headers

When the string %%ABI_HEADERS%% is found in syscalls.master, replace
it with the contents of the abi_headers config variable. This allows
an ABI-specific syscalls.conf to add lines like:

#include <compat/freebsd32/freebsd32.h>

when working from a shared syscalls.master.

Reviewed by: kevans

makesyscalls: allow multiple expressions for each abi change

Limitations in lua patterns means we need to be able to match more
than one possible expression.

Reviewed by: kevans

makesyscalls: Fix expression for time_t_size

Reviewed by: kevans

makesyscalls: Remove _native suffix handling

This supported a hack we used in CheriBSD which has been removed.

Reviewed by: kevans

makesyscalls: make strip_abi_prefix more robust

Allow strip_abi_prefix() to be called with nil and return nil in that
case. This simplifies handling of RESERVED entries.

Reviewed by: kevans

makesyscalls: sprinkle some assert() on standard function calls

Improves our error reporting, ensuring that we aren't just ignoring
errors in the common case.

Note specifically the boundary where we have to change up our error
handling approach. It's fine to error() out up until we create the
tempdir, then the rest should try to handle it gracefully and abort().
A future change will clean this up further by pcall'ing all of the bits
that cannot currently error() without cleaning up.

makesyscalls: rip out arbitrary command execution

This was previously needed only for CloudABI, which used it to generate
its capenabled from syscalls.master. CloudABI was removed in
cf0ee8738e31, so we don't need to support this anymore. Others looking
to do similar things should come up with a more integrated technique,
such as a .conf flag or pattern/glob support. brooks suggests that it
could be done in modern makesyscalls.lua by adding a config flag to
specify always-on/initial flags (CAPENABLED).

Reviewed by: brooks, imp
MFC after: never
Differential Revision: https://reviews.freebsd.org/D32095

makesyscalls: stop trying to remove . and .. in cleanup

lfs.dir() will include these entries, but os.remove() cannot remove them
for obvious reasons.

Drop cloudabi

According to https://github.com/NuxiNL/cloudlibc:
CloudABI is no longer being maintained. It was an awesome experiment,
but it never got enough traction to be sustainable.

There is no reason to keep it in FreeBSD.

Approved by: ed (private mail)
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D31923

makesyscalls.lua: add a CAPENABLED flag

The CAPENABLED flag indicates that the syscall can be used in capsicum
capability mode. It is intended to replace capabilities.conf.

Reviewed by: kevans, emaste
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D31349

makesyscalls.lua: improve generated file style(9) compliance

We generally like to avoid style changes when other changes are not
planned. In this case there are some makesyscalls.lua changes in the
pipeline, and this cleans up style nits in generated files that were
highlighted by experiments with clang-format.

Reviewed by: brooks, kevans
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30235

syscalls.master: Add a new syscall type: RESERVED

RESERVED syscall number are reserved for local/vendor use. RESERVED is
identical to UNIMPL except that comments are ignored.

Reviewed by: kevans
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D27988

Make makesyscalls.lua initialize 'struct sysent' entries using c99
designated initializers. This makes it easier to modify 'struct sysent'
layout.

Reviewed by: kevans
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26530

makesyscalls.lua: improve syscall ordering validation

There were two separate issues here:

1.) #if/#else wasn't taken into account at all for maxsyscall figures, but
2.) We didn't validate contiguous syscall numbers anyways...

This kind of inconsistency is bad as we don't currently ensure explicit
indexing of, e.g., the sysent array if one syscall is unimplemented/missing.
This could be fixed and might be more robust, but it's also good to have the
"documentation" that comes from being explicit as to what the missing
syscalls are.

The new version looks much like the awk version; stash off the current
'last highest syscall seen' if we hit an #if, restore to that if we hit an
#else, and make sure that we're explicitly always defining the next syscall.

The logic at the tail end of process_syscall_def that moves maxsyscall has
been 'cleaned up' a little since we're now ensuring that it's monotonically
increasing earlier in the function. At the moment I think it's unlikely we'd
see range-definitions that are not UNIMPL, but there's no reason to
specifically handle that case for bumping maxsyscall there.

This change was provoked by reading the commit message for r363832 and
realizing that this validation hadn't been included in the initial rewrite
to lua.

Reviewed by: brooks
Differential Revision: https://reviews.freebsd.org/D25945

makesyscalls.lua: generate all files in /tmp, write into place at the end

This makes makesyscalls.lua more parallel-friendly, or at least not as
hostile to the idea. We get into situations where we're running parallel if
we end up with MAKE_JOBS>1 entering any of the sysent targets, since each
output file is recognized a distinct build step that needs to be executed.

Another commit will add some .ORDER to further improve the situation.

Reported by: jhb
Reviewed by: brooks
Differential Revision: https://reviews.freebsd.org/D23098

makesyscalls.lua: trim trailing spaces/commas from args

These are insignificant as far as declarations go, and we've historically
allowed it. fhlinkat in ^/sys/kern/syscalls.master, for example, currently
has a trailing comma after its final argument that this version of
makesyscalls is ignoring (not by conscious decision).

Fix it for now by actively stripping off trailing whitespace/commas until
we decide to actively prohibit it.

makesyscalls.lua: improve config processing

The current version will strip out #include directives appearing inside strings, which is clearly wrong. Improve the processing entirely in the following ways:

- Strip only whole-line comments on every single iteration
- Abort if we see a malformed line that doesn't match the key=value format
- For quoted (backtick or double quote) strings, we'll advance to the end of
the key=value pair and make sure there's not extra stuff left over
- For unquoted key=value pairs, we'll strip any trailing comments and verify
there's no internal whitespace

This has revealed the caveat that key/value pairs can't even include escaped quotes (and haven't been able to). I don't know if this is actually problematic, as we're usually looking at cases like "#include <foo>" or raw identifiers.The current version will strip out #include directives appearing inside strings, which is clearly wrong. Improve the processing entirely in the following ways:

Reviewed and noticed by: brooks
Differential Revision: https://reviews.freebsd.org/D22698

Add makesyscalls.lua, a rewrite of makesyscalls.sh

This currently requires a suitable lua + luafilesystem + luaposix from the
ports tree to build. Discussion is underway in D21893 to add a suitable lua
to the base system, cleverly disguised and out of the way of normal
consumers.

makesyscalls.sh is a good target for rewrite into lua as it's currently a
sh+sed+awk script that can be difficult to add on to, at times. For
instance, adding a new COMPAT* option (that mimicks the behaivor of most
other COMPAT* options) requires a fairly substantial amount of copy/paste;
see r352693 for instance. Attempts to generate part of the awk script for
COMPAT* handling was (very kindly) rejected with a desire to just rewrite
the script in a single language that can handle all of it.

Reviewed by: brooks
Differential Revision: https://reviews.freebsd.org/D21775