MFC r355206, r355212, r355257, r355271: tty nits

r355206: tty_pts: don't rely on tty header pollution for sys/mutex.h
r355212: tty_rel_gone: add locking assertion
r355257: usb: remove some extraneous tty.h includes
r355271: Remove more needless <sys/tty.h> includes

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

sys/kern: spelling fixes in comments.

No functional change.

Implement lockless resource limits.

Use the same scheme implemented to manage credentials.

Code needing to look at process's credentials (as opposed to thred's) is
provided with *_proc variants of relevant functions.

Places which possibly had to take the proc lock anyway still use the proc
pointer to access limits.

fd: remove filedesc argument from fdclose

Just accept a thread instead. This makes it consistent with fdalloc.

No functional changes.

Implement FIODTYPE for master ptys.

Requested and reviewed by: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Add a new fo_fill_kinfo fileops method to add type-specific information to
struct kinfo_file.
- Move the various fill_*_info() methods out of kern_descrip.c and into the
various file type implementations.
- Rework the support for kinfo_ofile to generate a suitable kinfo_file object
for each file and then convert that to a kinfo_ofile structure rather than
keeping a second, different set of code that directly manipulates
type-specific file information.
- Remove the shm_path() and ksem_info() layering violations.

Differential Revision: https://reviews.freebsd.org/D775
Reviewed by: kib, glebius (earlier version)

Fix various issues with invalid file operations:
- Add invfo_rdwr() (for read and write), invfo_ioctl(), invfo_poll(),
and invfo_kqfilter() for use by file types that do not support the
respective operations. Home-grown versions of invfo_poll() were
universally broken (they returned an errno value, invfo_poll()
uses poll_no_poll() to return an appropriate event mask). Home-grown
ioctl routines also tended to return an incorrect errno (invfo_ioctl
returns ENOTTY).
- Use the invfo_*() functions instead of local versions for
unsupported file operations.
- Reorder fileops members to match the order in the structure definition
to make it easier to spot missing members.
- Add several missing methods to linuxfileops used by the OFED shim
layer: fo_write(), fo_truncate(), fo_kqfilter(), and fo_stat(). Most
of these used invfo_*(), but a dummy fo_stat() implementation was
added.

Make sendfile() a method in the struct fileops. Currently only
vnode backed file descriptors have this method implemented.

Reviewed by: kib
Sponsored by: Nginx, Inc.
Sponsored by: Netflix

Allow O_CLOEXEC in posix_openpt() flags.

PR: kern/162374
Reviewed by: ed

Fix possible fp reference leak in posix_openpt

Reviewed by: ed
Approved by: trasz (mentor)
MFC after: 3 days

In order to maximize the re-usability of kernel code in user space this
patch modifies makesyscalls.sh to prefix all of the non-compatibility
calls (e.g. not linux_, freebsd32_) with sys_ and updates the kernel
entry points and all places in the code that use them. It also
fixes an additional name space collision between the kernel function
psignal and the libc function of the same name by renaming the kernel
psignal kern_psignal(). By introducing this change now we will ease future
MFCs that change syscalls.

Reviewed by: rwatson
Approved by: re (bz)

Fix a deficiency in the selinfo interface:
If a selinfo object is recorded (via selrecord()) and then it is
quickly destroyed, with the waiters missing the opportunity to awake,
at the next iteration they will find the selinfo object destroyed,
causing a PF#.

That happens because the selinfo interface has no way to drain the
waiters before to destroy the registered selinfo object. Also this
race is quite rare to get in practice, because it would require a
selrecord(), a poll request by another thread and a quick destruction
of the selrecord()'ed selinfo object.

Fix this by adding the seldrain() routine which should be called
before to destroy the selinfo objects (in order to avoid such case),
and fix the present cases where it might have already been called.
Sometimes, the context is safe enough to prevent this type of race,
like it happens in device drivers which installs selinfo objects on
poll callbacks. There, the destruction of the selinfo object happens
at driver detach time, when all the filedescriptors should be already
closed, thus there cannot be a race.
For this case, mfi(4) device driver can be set as an example, as it
implements a full correct logic for preventing this from happening.

Sponsored by: Sandvine Incorporated
Reported by: rstone
Tested by: pluknet
Reviewed by: jhb, kib
Approved by: re (bz)
MFC after: 3 weeks

Add the fo_chown and fo_chmod methods to struct fileops and use them
to implement fchown(2) and fchmod(2) support for several file types
that previously lacked it. Add MAC entries for chown/chmod done on
posix shared memory and (old) in-kernel posix semaphores.

Based on the submission by: glebius
Reviewed by: rwatson
Approved by: re (bz)

Fix whitespace inconsistencies in the TTY layer and its drivers owned by me.

Add accounting for RACCT_NPTS.

After the r219999 is merged to stable/8, rename fallocf(9) to falloc(9)
and remove the falloc() version that lacks flag argument. This is done
to reduce the KPI bloat.

Requested by: jhb
X-MFC-note: do not

Make "struct pts_softc" point to ucred instead of uidinfo. This is no-op,
required for resource containers.

Reviewed by: kib (as part of a larger patch), ed

Do not leak master pty or ptmx vnode.

Report and test case by: Petr Salinger <Petr.Salinger seznam cz>
Reviewed by: ed
MFC after: 1 week

Rename st_*timespec fields to st_*tim for POSIX 2008 compliance.

A nice thing about POSIX 2008 is that it finally standardizes a way to
obtain file access/modification/change times in sub-second precision,
namely using struct timespec, which we already have for a very long
time. Unfortunately POSIX uses different names.

This commit adds compatibility macros, so existing code should still
build properly. Also change all source code in the kernel to work
without any of the compatibility macros. This makes it all a less
ambiguous.

I am also renaming st_birthtime to st_birthtim, even though it was a
local extension anyway. It seems Cygwin also has a st_birthtim.

Remove the 1000 pseudo terminal limit from pts(4).

Even with the old utmp format, we could in fact go to pts/9999, because
ut_line wasn't guaranteed to be null terminated there.

Make TIOCSTI work again.

It looks like I didn't implement this when I imported MPSAFE TTY.
Applications like mail(1) still use this. I think it's conceptually bad.

Tested by: Pete French <petefrench ticketswitch com>
MFC after: 2 weeks

Make the wchan names of pts(4) fit in top(1).

Just like a similar change we made to the TTY code about half a year
ago, make these strings look similar.

Suggested by: Jille Timmermans <jille@quis.cx>

Use C99 initialization for struct filterops.

Obtained from: Mac OS X
Sponsored by: Apple Inc.
MFC after: 3 weeks

Move ptmx into pty(4).

Now that pty(4) is a loadable kernel module, I'd better move /dev/ptmx
in there as well. This means that pty(4) now provides almost all
pseudo-terminal compatibility code. This means it's very easy to test
whether applications use the proper library interfaces when allocating
pseudo-terminals (namely posix_openpt and openpty).

Allow pty(4) to be loaded as a kld.

Unfortunately, the wrappers that are present in pts(4) don't have the
mechanics to allow pty(4) to be unloaded safely, so I'm forcing this kld
to return EBUSY. This also means we have to enable some extra code in
pts(4) unconditionally.

Proposed by: rwatson

Add ttydisc_rint_simple().

I noticed several drivers in our tree don't actually care about parity
and framing, such as pts(4), snp(4) (and my partially finished console
driver). Instead of duplicating a lot of code, I think we'd better add a
utility function for those drivers to quickly process a buffer of input.

Also change pts(4) and snp(4) to use this function.

Fix two bugs related to TTY input:

- fix write() on pseudo-terminal masters to return the amount of bytes
passed to the TTY, not the amount of bytes read from user.

- fix ttydisc_rint_bypass() to set the high watermark when it cannot
write all input, just like ttydisc_rint() itself.

Approved by: re (kib)

Fix regressions in return events of poll() on TTYs.

As pointed out, POLLHUP should be generated, even if it hasn't been
specified on input. It is also not allowed to return both POLLOUT and
POLLHUP at the same time.

Reported by: jilles
Approved by: re (kib)

Adapt vfs kqfilter to the shared vnode lock used by zfs write vop. Use
vnode interlock to protect the knote fields [1]. The locking assumes
that shared vnode lock is held, thus we get exclusive access to knote
either by exclusive vnode lock protection, or by shared vnode lock +
vnode interlock.

Do not use kl_locked() method to assert either lock ownership or the
fact that curthread does not own the lock. For shared locks, ownership
is not recorded, e.g. VOP_ISLOCKED can return LK_SHARED for the shared
lock not owned by curthread, causing false positives in kqueue subsystem
assertions about knlist lock.

Remove kl_locked method from knlist lock vector, and add two separate
assertion methods kl_assert_locked and kl_assert_unlocked, that are
supposed to use proper asserts. Change knlist_init accordingly.

Add convenience function knlist_init_mtx to reduce number of arguments
for typical knlist initialization.

Submitted by: jhb [1]
Noted by: jhb [2]
Reviewed by: jhb
Tested by: rnoland

Last minute TTY API change: remove mutex argument from tty_alloc().

I don't want people to override the mutex when allocating a TTY. It has
to be there, to keep drivers like syscons happy. So I'm creating a
tty_alloc_mutex() which can be used in those cases. tty_alloc_mutex()
should eventually be removed.

The advantage of this approach, is that we can just remove a function,
without breaking the regular API in the future.

Turn MAXPTSDEVS into a sysctl tunable.

This allows users to increase the maximum amount of pseudo-terminals
without changing any source code. Users must increase UT_LINESIZE before
attempting to increase kern.pts_maxdev.

Improve my previous changes to the TTY code: also remove memcpy().

It's better to just use internal language constructs, because it is
likely the compiler has a better opinion on whether to perform inlining,
which is very likely to happen to struct winsize.

Submitted by: Christoph Mallon <christoph mallon gmx de>

Replace bcopy() calls inside the TTY layer with memcpy()/strlcpy().

In all these cases the buffers never overlap. Program names are also
likely to be shorter, so use a regular strlcpy() to copy p_comm.

Squash some small bugs in pts(4).

- Don't return a negative errno when using an unknown ioctl() on a
pseudo-terminal master device. Be sure to convert ENOIOCTL to ENOTTY,
just like the TTY layer does.

- Even though we should return st_rdev of the master device node when
emulating pty(4) devices, FIODGNAME should still return the name of
the slave device. Otherwise ptsname(3) and ttyname(3) return an
invalid device name.

Set PTS_FINISHED before waking up any threads.

Inside ptsdrv_{in,out}wakeup() we call KNOTE_LOCKED() to wake up any
kevent(2) users. Because the kqueue handlers are executed synchronously,
we must set PTS_FINISHED before calling ptsdrv_{in,out}wakeup().

Discovered by: nork

Add FIONREAD to pseudo-terminal master devices.

All ioctl()'s that aren't implemented by pts(4) are forwarded to the TTY
itself. Unfortunately this is not correct for FIONREAD, because it will
give the wrong amount of bytes that are available to read.

Tested by: keramida
Reminded by: keramida

Add kqueue()-support to pseudo-terminal master devices.

One thing I didn't expect many applications to use, was kqueue() on
pseudo-terminal master devices. There are applications that use kqueue()
on the TTY itself (rtorrent, etc). That doesn't mean we shouldn't
implement this. Libraries like libevent use kqueue() by default, which
means they wouldn't be able to use kqueue().

The old TTY layer implements a very broken version of kqueue() by
performing the actual polling on the TTY device.

Discussed with: peter

Remove redundant return value tests.

There is no need to test whether the return value is non-zero here. Just
return the error number directly.

Track state to determine if the associated TTY device node has been used.

It turns out our old TTY layer (and other implementations) block when
you read() on a PTY master device of which the slave device node has not
been opened yet. Our new implementation just returned 0. This caused
applications like telnetd to die in a very subtle way (when child
processes would open the TTY later than the first call to select()).

Introduce a new flag called PTS_FINISHED, which indicates whether we
should block or bail out of a read() or write() occurs.

Reported by: Claude Buisson <clbuisson orange fr>

Make fstat() on a pseudo-terminal master return sane timestamps.

Because pseudo-terminal master file descriptors no longer have a vnode
underneath, we have to fill in fstat() values ourselves. Make our
implementation somewhat sane by returning the timestamps of the TTY
device node that corresponds with our file descriptor.

Obtained from: //depot/projects/mpsafettty/...

Implement pts(4) packet mode.

As reported by several users on the mailing lists, applications like
screen(1) fail to properly handle ^S and ^Q characters. This was because
MPSAFE TTY didn't implement packet mode (TIOCPKT) yet. Add basic packet
mode support to make these applications work again.

Obtained from: //depot/projects/mpsafetty/...

Fix pts(4) error codes when slave device is closed.

Unlike pre-MPSAFE TTY, the pts(4) driver always returned ENXIO when a
read() or write() was performed on a pseudo-terminal master device when
the slave device was not opened. The old implementation had different
semantics:

- When the slave device had not been opened yet, read() and write() just
blocked.
- When the slave device had been closed, a read() call would return 0
bytes length.
- When the slave device had been closed, a write() call would return
EIO.

Change the new implementation to return 0 and EIO as well. We don't
implement the first rule, but I suspect this is not needed, because
routines like openpty() also open the slave device node. posix_openpt()
users also do similar things.

Reported by: rink
Tested by: rink

Integrate the new MPSAFE TTY layer to the FreeBSD operating system.

The last half year I've been working on a replacement TTY layer for the
FreeBSD kernel. The new TTY layer was designed to improve the following:

- Improved driver model:

The old TTY layer has a driver model that is not abstract enough to
make it friendly to use. A good example is the output path, where the
device drivers directly access the output buffers. This means that an
in-kernel PPP implementation must always convert network buffers into
TTY buffers.

If a PPP implementation would be built on top of the new TTY layer
(still needs a hooks layer, though), it would allow the PPP
implementation to directly hand the data to the TTY driver.

- Improved hotplugging:

With the old TTY layer, it isn't entirely safe to destroy TTY's from
the system. This implementation has a two-step destructing design,
where the driver first abandons the TTY. After all threads have left
the TTY, the TTY layer calls a routine in the driver, which can be
used to free resources (unit numbers, etc).

The pts(4) driver also implements this feature, which means
posix_openpt() will now return PTY's that are created on the fly.

- Improved performance:

One of the major improvements is the per-TTY mutex, which is expected
to improve scalability when compared to the old Giant locking.
Another change is the unbuffered copying to userspace, which is both
used on TTY device nodes and PTY masters.

Upgrading should be quite straightforward. Unlike previous versions,
existing kernel configuration files do not need to be changed, except
when they reference device drivers that are listed in UPDATING.

Obtained from: //depot/projects/mpsafetty/...
Approved by: philip (ex-mentor)
Discussed: on the lists, at BSDCan, at the DevSummit
Sponsored by: Snow B.V., the Netherlands
dcons(4) fixed by: kan

- Close a race with concurrent open's of a pts master device which could
result in leaked tty structures.
- When constructing a new pty, allocate it's tty structure before adding
it to the list.

MFC after: 1 week

In keeping with style(9)'s recommendations on macros, use a ';'
after each SYSINIT() macro invocation. This makes a number of
lightweight C parsers much happier with the FreeBSD kernel
source, including cflow's prcc and lxr.

MFC after: 1 month
Discussed with: imp, rink

Fix bad function type passed to destroy_dev_sched_cb().

Approved by: re (rwatson)

Use make_dev_credf(MAKEDEV_REF) instead of make_dev() from the clone handler.
Lock Giant in the clone handler.
Use destroy_dev_sched() explicitely from pty_maybecleanup() and postpone
pty_release() until both master and slave cdevs are destroyed by setting
it as callback for destroy_dev_sched().

Debugging help and testing by: Peter Holm
Approved by: re (kensmith)

Canonicalize copyrights in some files I hold copyrights on:

- Sort by date in license blocks, oldest copyright first.
- All rights reserved after all copyrights, not just the first.
- Use (c) to be consistent with other entries.

MFC after: 3 days

Sweep kernel replacing suser(9) calls with priv(9) calls, assigning
specific privilege names to a broad range of privileges. These may
require some future tweaking.

Sponsored by: nCircle Network Security, Inc.
Obtained from: TrustedBSD Project
Discussed on: arch@
Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri,
Alex Lyashkov <umka at sevcity dot net>,
Skip Ford <skip dot ford at verizon dot net>,
Antoine Brodin <antoine dot brodin at laposte dot net>

Increment nb_allocated while holding the pt_mtx lock to avoid races.

Comment and style tweak.

Free tty struct after last close. This should fix the pty-leak by numbers.
Remove workarounds for tty_refcount beeing 0, this will be fixed differently
later.

Fix our ioctl(2) implementation when the argument is "int". New
ioctls passing integer arguments should use the _IOWINT() macro.
This fixes a lot of ioctl's not working on sparc64, most notable
being keyboard/syscons ioctls.

Full ABI compatibility is provided, with the bonus of fixing the
handling of old ioctls on sparc64.

Reviewed by: bde (with contributions)
Tested by: emax, marius
MFC after: 1 week

Also check use_pty in the ptmx clone lookup; this means that when ptmx
support is turned off using the sysctl, we no longer even allow the
ptmx device to be looked up.

Foot provided by: peter

Allow root to open prison pts devices too.

Pointed out by: rwatson

Move pts master devices into /dev/pty/ instead of littering /dev with them;
this is more consistent with the placement of slaves in /dev/pts. The
actual name doesn't matter as it's not part of the exposed API or used by
libc. In some sense, it would be nice if these device nodes didn't have to
have names in devfs at all.

Suggested by: Stephen McKay <smckay at internode dot on dot net>

Rename use_old_pty variable to use_pts, as this more accurately reflects
the sense of the variable.

Suggested by: dwhite

Merge a bunch of changes that where done in tty_pty.c after tty_pts.c was
forked from it, but missed from some reason.

Take into account that bits 0x0000ff00 can't be used for minor.

Don't attempt to re-create the /dev entry for the slave part if it already
exist when opening the master. This can happen if one open the master, then
open the slave, then close and re-open the master.

Reported by: Peter Holm

Bring in a sysv-style pts implementation, as found in the rwatson_pts perforce branch. It works the same as its SysV/linux counterpart : You obtain a fd to the master pseudo terminal by opening /dev/ptmx, which craetes a node for the master as /dev/pty[num] and a node for the slave as /dev/pts/[num].
It should play nicely with the existing BSD ptys.
By default, the system will use the BSD ptys, one can set the sysctl
kern.pts.enable to 1 to make it use the new pts system.
The max number of pty that can be allocated on a system can be changed with the
sysctl kern.pts.max. It defaults to 1000, and can be increased, but it is not
recommanded, as any pty with a number > 999 won't be handled by whatever uses
utmp(5).