- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

It was reported via email that the cu_sent field used by the
krpc client side UDP was observed as way out of range and
caused the rpc.lockd daemon to hang trying to do an RPC.
Inspection of the code found two places where the RPC request
is re-queued, but the value of cu_sent was not incremented.
Since cu_sent is always decremented when the RPC request is
dequeued, I think this could have caused cu_sent to go out of
range. This patch adds lines to increment cu_sent for these
two cases.

Reported by: dwhite@ixsystems.com
Discussed with: dwhite@ixsystems.com
MFC after: 2 weeks

Add support for host-based (Kerberos 5 service principal) initiator
credentials to the kernel rpc. Modify the NFSv4 client to add
support for the gssname and allgssname mount options to use this
capability. Requires the gssd daemon to be running with the "-h" option.

Reviewed by: jhb

Fix a potential socket leak in the NFS server. If a client closes its
connection after it was accepted by the userland nfsd process but before
it was handled off to svc_vc_create() in the kernel, then svc_vc_create()
would see it as a new listen socket and try to listen on it leaving a
dangling reference to the socket. Instead, check for disconnected sockets
and treat them like a connected socket. The call to pru_getaddr() should
fail and cause svc_vc_create() to fail. Note that we need to lock the
socket to get a consistent snapshot of so_state since there is a window
in soisdisconnected() where both flags are clear.

Reviewed by: dfr, rmacklem
MFC after: 1 week

Improve error handling when unwrapping received data.

Submitted by: Rick Macklem
MFC after: 1 week

Revert 195703 and 195821 as this special stop handling in NFS is now
implemented via VFCF_SBDRY rather than passing PBDRY to individual
sleep calls.

Use m_get(), m_gethdr() and m_getcl() instead of historic macros.

Sponsored by: Nginx, Inc.

Add support for backchannels to the kernel RPC. Backchannels
are used by NFSv4.1 for callbacks. A backchannel is a connection
established by the client, but used for RPCs done by the server
on the client (callbacks). As a result, this patch mixes some
client side calls in the server side and vice versa. Some
definitions in the .c files were extracted out into a file called
krpc.h, so that they could be included in multiple .c files.
This code has been in projects/nfsv4.1-client for some time.
Although no one has given it a formal review, I believe kib@
has taken a look at it.

Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags within sys.

Exceptions:

- sys/contrib not touched
- sys/mbuf.h edited manually

Modify the comment to take out the names and URL.

Requested by: kib
MFC after: 3 days

Add a comment describing why r241097 was done.

Suggested by: rwatson
MFC after: 1 week

rpc: convert all uid and gid variables to u_int.

After further discussion, instead of pretending to use
uid_t and gid_t as upstream Solaris and linux try to, we
are better using u_int, which is in fact what the code
can handle and best approaches the range of values used
by uid and gid.

Discussed with: bde
Reviewed by: bde

libtirpc: be sure to free cl_netid and cl_tp

When creating a client with clnt_tli_create, it uses strdup to copy
strings for these fields if nconf is passed in. clnt_dg_destroy frees
these strings already. Make sure clnt_vc_destroy frees them in the same
way.

This change matches the reference (OpenSolaris) implementation.

Tested by: David Wolfskill
Obtained from: Bull GNU/Linux NFSv4 Project (libtirpc)
MFC after: 2 weeks

RPC: Convert all uid and gid variables of the type uid_t and gid_t.

This matches what upstream (OpenSolaris) does.

Tested by: David Wolfskill
Obtained from: Bull GNU/Linux NFSv4 project (libtirpc)
MFC after: 3 days

Attila Bogar and Herbert Poeckl both reported similar problems
w.r.t. a Linux NFS client doing a krb5 NFS mount against the
FreeBSD server. We determined this was a Linux bug:
http://www.spinics.net/lists/linux-nfs/msg32466.html, however
the mount failed to work, because the Destroy operation with a
bogus encrypted checksum destroyed the authenticator handle.
This patch changes the rpcsec_gss code so that it doesn't
Destroy the authenticator handle for this case and, as such,
the Linux mount will work.

Tested by: Attila Bogar and Herbert Poeckl
MFC after: 2 weeks

Complete revert of r239963:

The attempt to merge changes from the linux libtirpc caused
rpc.lockd to exit after startup under unclear conditions.

After many hours of selective experiments and inconsistent results
the conclusion is that it's better to just revert everything and
restart in a future time with a much smaller subset of the
changes.
____

MFC after: 3 days
Reported by: David Wolfskill
Tested by: David Wolfskill

Partial revert of r239963:

The following change caused rpc.lockd to exit after startup:
____

libtirpc: be sure to free cl_netid and cl_tp

When creating a client with clnt_tli_create, it uses strdup to copy
strings for these fields if nconf is passed in. clnt_dg_destroy frees
these strings already. Make sure clnt_vc_destroy frees them in the
same way.
____

MFC after: 3 days
Reported by: David Wolfskill
Tested by: David Wolfskill

Fix RPC headers for C++

C++ mangling will cause trouble with variables like __rpc_xdr
in xdr.h so rename this to XDR.
While here add proper C++ guards to RPC headers.

PR: 137443
MFC after: 2 weeks

Bring some changes from Bull's NFSv4 libtirpc implementation.

We especifically ignored the glibc compatibility changes
but this should help interaction with Solaris and Linux.
____

Fixed infinite loop in svc_run()
author Steve Dickson
Tue, 10 Jun 2008 12:35:52 -0500 (13:35 -0400)
Fixed infinite loop in svc_run()
____

__rpc_taddr2uaddr_af() assumes the netbuf to always have a
non-zero data. This is a bad assumption and can lead to a
seg-fault. This patch adds a check for zero length and returns
NULL when found.
author Steve Dickson
Mon, 27 Oct 2008 11:46:54 -0500 (12:46 -0400)
____

Changed clnt_spcreateerror() to return clearer
and more concise error messages.
author Steve Dickson
Thu, 20 Nov 2008 08:55:31 -0500 (08:55 -0500)
____

Converted all uid and gid variables of the type uid_t and gid_t.
author Steve Dickson
Wed, 28 Jan 2009 12:44:46 -0500 (12:44 -0500)
____

libtirpc: set r_netid and r_owner in __rpcb_findaddr_timed

These fields in the rpcbind GETADDR call are being passed uninitialized
to CLNT_CALL. In the case of x86_64 at least, this usually leads to a
segfault. On x86, it sometimes causes segfaults and other times causes
garbage to be sent on the wire.

rpcbind generally ignores the r_owner field for calls that come in over
the wire, so it really doesn't matter what we send in that slot. We just
need to send something. The reference implementation from Sun seems to
send a blank string. Have ours follow suit.
author Jeff Layton
Fri, 13 Mar 2009 11:44:16 -0500 (12:44 -0400)
____

libtirpc: be sure to free cl_netid and cl_tp

When creating a client with clnt_tli_create, it uses strdup to copy
strings for these fields if nconf is passed in. clnt_dg_destroy frees
these strings already. Make sure clnt_vc_destroy frees them in the same
way.

author Jeff Layton
Fri, 13 Mar 2009 11:47:36 -0500 (12:47 -0400)

Obtained from: Bull GNU/Linux NFSv4 Project
MFC after: 3 weeks

Both a crash reported on freebsd-current on Oct. 18 under the
subject heading "mtx_lock() of destroyed mutex on NFS" and
PR# 156168 appear to be caused by clnt_dg_destroy() closing
down the socket prematurely. When to close down the socket
is controlled by a reference count (cs_refs), but clnt_dg_create()
checks for sb_upcall being non-NULL to decide if a new socket
is needed. I believe the crashes were caused by the following race:
clnt_dg_destroy() finds cs_refs == 0 and decides to delete socket
clnt_dg_destroy() then loses race with clnt_dg_create() for
acquisition of the SOCKBUF_LOCK()
clnt_dg_create() finds sb_upcall != NULL and increments cs_refs to 1
clnt_dg_destroy() then acquires SOCKBUF_LOCK(), sets sb_upcall to
NULL and destroys socket

This patch fixes the above race by changing clnt_dg_destroy() so
that it acquires SOCKBUF_LOCK() before testing cs_refs.

Tested by: bz
PR: 156168
Reviewed by: dfr
MFC after: 2 weeks

Remove an extraneous "already" from a comment introduced by r226081.

Submitted by: bf1783 at googlemail.com
MFC after: 3 days

A crash reported on freebsd-fs@ on Sep. 23, 2011 under the subject
heading "kernel panics with RPCSEC_GSS" appears to be caused by a
corrupted tailq list for the client structure. Looking at the code, calls
to the function svc_rpc_gss_forget_client() were done in an SMP unsafe
manner, with the svc_rpc_gss_lock only being acquired in the function
and not before it. As such, when multiple threads called
svc_rpc_gss_forget_client() concurrently, it could try and remove the
same client structure from the tailq lists multiple times.
The patch fixes this by moving the critical code into a separate
function called svc_rpc_gss_forget_client_locked(), which must be
called with the lock held. For the one case where the caller would
have no interest in the lock, svc_rpc_gss_forget_client() was retained,
but a loop was added to check that the client structure is still in
the tailq lists before removing it, to make it safe for multiple
concurrent calls.

Tested by: clinton.adams at gmail.com (earlier version)
Reviewed by: zkirsch
MFC after: 3 days

Make sure RPC calls over UDP return RPC_INTR status is the process has
been interrupted in a restartable syscall. Otherwise we could end up
in an (almost) endless loop in clnt_reconnect_call().

PR: kern/160198
Reviewed by: rmacklem
Approved by: re (kib), avg (mentor)
MFC after: 1 week

Fix the kgssapi so that it can be loaded as a module. Currently
the NFS subsystems use five of the rpcsec_gss/kgssapi entry points,
but since it was not obvious which others might be useful, all
nineteen were included. Basically the nineteen entry points are
set in a structure called rpc_gss_entries and inline functions
defined in sys/rpc/rpcsec_gss.h check for the entry points being
non-NULL and then call them. A default value is returned otherwise.
Requested by rwatson.

Reviewed by: jhb
MFC after: 2 weeks

This patch is believed to fix a problem in the kernel rpc for
non-interruptible NFS mounts, where a kernel thread will seem
to be stuck sleeping on "rpccon". The msleep() in clnt_vc_create()
that was waiting to a TCP connect to complete would return ERESTART,
since PCATCH was specified. Then the tsleep() in clnt_reconnect_call()
would sleep for 1 second and then try again and again and...
The patch changes the msleep() in clnt_vc_create() so it only sets
the PCATCH flag for interruptible cases.

Tested by: pho
Reviewed by: jhb
MFC after: 2 weeks

Fix a couple of mbuf leaks introduced by r217242. I do
not believe that these leaks had a practical impact,
since the situations in which they would have occurred
would have been extremely rare.

MFC after: 2 weeks

Mfp4 CH=177274,177280,177284-177285,177297,177324-177325

VNET socket push back:
try to minimize the number of places where we have to switch vnets
and narrow down the time we stay switched. Add assertions to the
socket code to catch possibly unset vnets as seen in r204147.

While this reduces the number of vnet recursion in some places like
NFS, POSIX local sockets and some netgraph, .. recursions are
impossible to fix.

The current expectations are documented at the beginning of
uipc_socket.c along with the other information there.

Sponsored by: The FreeBSD Foundation
Sponsored by: CK Software GmbH
Reviewed by: jhb
Tested by: zec

Tested by: Mikolaj Golub (to.my.trociny gmail.com)
MFC after: 2 weeks

sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly.

Commit the kernel changes.

Fix a bug in the client side krpc where it was, sometimes
erroneously, assumed that 4 bytes of data were in the first
mbuf of a list by replacing the bcopy() with m_copydata().
Also, replace the uses of m_pullup(), which can fail for
reasons other than not enough data, with m_copydata().
For the cases where it isn't known that there is enough
data in the mbuf list, check first via m_len and m_length().
This is believed to fix a problem reported by dpd at dpdtech.com
and george+freebsd at m5p.com.

Reviewed by: jhb
MFC after: 8 days

Fix the krpc so that it can handle NFSv3,UDP mounts with a read/write
data size greater than 8192. Since soreserve(so, 256*1024, 256*1024)
would always fail for the default value of sb_max, modify clnt_dg.c
so that it uses the calculated values and checks for an error return
from soreserve(). Also, add a check for error return from soreserve()
to clnt_vc.c and change __rpc_get_t_size() to use sb_max_adj instead of
the bogus maxsize == 256*1024.

PR: kern/150910
Reviewed by: jhb
MFC after: 2 weeks

Make the RPC specific __rpc_inet_ntop() and __rpc_inet_pton() general
in the kernel (just as inet_ntoa() and inet_aton()) are and sync their
prototype accordingly with already mentioned functions.

Sponsored by: Sandvine Incorporated
Reviewed by: emaste, rstone
Approved by: dfr
MFC after: 2 weeks

Remove unnecessary weak reference that was apparently copied from the
version of this function in lib/libc/inet/inet_pton.c

MFC after: 1 week

- Check the result of malloc(M_NOWAIT) in replay_alloc(). The caller
(replay_alloc()) knows how to handle replay_alloc() failure.
- Eliminate 'freed_one' variable, it is not needed - when no entry is found
rce will be NULL.
- Add locking assertions where we expect a rc_lock to be held.

Reviewed by: rmacklem
MFC after: 2 weeks

Add mutex locking for the call to replay_prune() in
replay_setsize(), since replay_prune() expects the
rc_lock to be held when it is called.

MFC after: 2 weeks

If the first iteration of the do loop in replay_prune()
succeeded and a subsequent interation failed to find an
entry to prune, it could loop infinitely, since the
"freed" variable wasn't reset to FALSE. This patch moves
setting freed FALSE to inside the loop to fix the problem.

Tested by: alan.bryan at yahoo.com
MFC after: 2 weeks

When the regular NFS server replied to a UDP client out of the replay
cache, it did not free the request argument mbuf list, resulting in a leak.
This patch fixes that leak.

Tested by: danny AT cs.huji.ac.il
PR: kern/144330
Submitted by: to.my.trociny AT gmail.com (earlier version)
Reviewed by: dfr
MFC after: 2 weeks

Replace the static NGROUPS=NGROUPS_MAX+1=1024 with a dynamic
kern.ngroups+1. kern.ngroups can range from NGROUPS_MAX=1023 to
INT_MAX-1. Given that the Windows group limit is 1024, this range
should be sufficient for most applications.

MFC after: 1 month

Make options KGSSAPI build and add it to NOTES.

rpcsec_gss_prot.c:
Use kernel printf and headers.

vc_rpcsec_gss.c:
Use a local RPCAUTH_UNIXGIDS definition for 16 instead of using NGROUPS.

Remove extraneous semicolons, no functional changes.

Submitted by: Marc Balmer <marc@msys.ch>
MFC after: 1 week

(S)LIST_HEAD_INITIALIZER takes a (S)LIST_HEAD as an argument.
Fix some wrong usages.
Note: this does not affect generated binaries as this argument is not used.

PR: 137213
Submitted by: Eygene Ryabinkin (initial version)
MFC after: 1 month

Add a check for the connection being shut down to the krpc
client just before queuing a request for the connection. The
code already had a check for the connection being shut down
while the request was queued, but not one for the shut down
having been initiated by the server before the request was
in the queue. This appears to fix the problem of slow reconnects
against an NFS server that drops inactive connections reported
by Olaf Seibert, but does not fix the case
where the FreeBSD client generates RST segments at about the
same time as ACKs. This is still a problem that is being
investigated. This patch does not cause a regression for this
case.

Tested by: Olaf Seibert, Daniel Braniss
Reviewed by: dfr
MFC after: 5 days

Set the prison in NFS anon and GSS SVC creds (as I indended to in r197581).

Reviewed by: marcel

Back out r197581, which replaced this file witk sys/kern/vfs_export.c.

Who knew that "svn export" was an actual command, or that I would have
vfs_export.c stuck in my mind deep enough to type "export" instead of
"commit"?

Pointy Hat to: jamie

Set the prison in NFS anon and GSS SVC creds.

Reviewed by: marcel
MFC after: 3 days

Fix NFS panics with options VIMAGE kernels by apropriately setting curvnet
context inside the RPC code.

Temporarily set td's cred to mount's cred before calling socreate() via
__rpc_nconf2socket().

Submitted by: rmacklem (in part)
Reviewed by: rmacklem, rwatson
Discussed with: dfr, bz
Approved by: re (rwatson), julian (mentor)
MFC after: 3 days

Use PBDRY flag for msleep(9) in NFS and NLM when sleeping thread owns
kernel resources that block other threads, like vnode locks. The SIGSTOP
sent to such thread (process, rather) shall not stop it until thread
releases the resources.

Tested by: pho
Reviewed by: jhb
Approved by: re (kensmith)

When unmounting an NFS mount using sec=krb5[ip], the umount system
call could get hung sleeping on "gsssta" if the credentials for a user
that had been accessing the mount point have expired. This happened
because rpc_gss_destroy_context() would end up calling itself when the
"destroy context" RPC was attempted, trying to refresh the credentials.
This patch just checks for this case in rpc_gss_refresh() and returns
without attempting the refresh, which avoids the recursive call to
rpc_gss_destroy_context() and the subsequent hang.

Reviewed by: dfr
Approved by: re (Ken Smith), kib (mentor)

Make sure that cr_error is set to ESHUTDOWN when closing the connection.
This is normally done by a loop in clnt_dg_close(), but requests that aren't
in the pending queue at the time of closing, don't get set. This avoids a
panic in xdrmbuf_create() when it is called with a NULL cr_mrep if
cr_error doesn't get set to ESHUTDOWN while closing.

Reviewed by: dfr
Approved by: re (Ken Smith), kib (mentor)

Fix two known problems in clnt_rc.c, plus issues w.r.t. smp noted
during reading of the code. Change the code so that it never accesses
rc_connecting, rc_closed or rc_client when the rc_lock mutex is not held.
Also, it now performs the CLNT_CLOSE(client) and CLNT_RELEASE(client)
calls after the rc_lock mutex has been released, since those calls do
msleep()s with another mutex held. Change clnt_reconnect_call() so that
releasing the reference count is delayed until after the
"if (rc->rc_client == client)" check, so that rc_client cannot have been
recycled.

Tested by: pho
Reviewed by: dfr
Approved by: kib (mentor)

If the initial attempt to refresh credentials in the RPCSEC_GSS client
side fails, the entry in the cache is left with no valid context
(gd_ctx == GSS_C_NO_CONTEXT). As such, subsequent hits on the cache
will result in persistent authentication failure, even after the user has
done a kinit or similar and acquired a new valid TGT. This patch adds a test
for that case upon a cache hit and calls rpc_gss_init() to make another
attempt at getting valid credentials. It also moves the setting of gc_proc
to before the import of the principal name to ensure that, if that case
fails, it will be detected as a failure after going to "out:".

Reviewed by: dfr
Approved by: kib (mentor)

Delete the declaration of an unused variable so that it will build.

Approved by: rwatson (mentor)

Rework the credential code to support larger values of NGROUPS and
NGROUPS_MAX, eliminate ABI dependencies on them, and raise the to 1024
and 1023 respectively. (Previously they were equal, but under a close
reading of POSIX, NGROUPS_MAX was defined to be too large by 1 since it
is the number of supplemental groups, not total number of groups.)

The bulk of the change consists of converting the struct ucred member
cr_groups from a static array to a pointer. Do the equivalent in
kinfo_proc.

Introduce new interfaces crcopysafe() and crsetgroups() for duplicating
a process credential before modifying it and for setting group lists
respectively. Both interfaces take care for the details of allocating
groups array. crsetgroups() takes care of truncating the group list
to the current maximum (NGROUPS) if necessary. In the future,
crsetgroups() may be responsible for insuring invariants such as sorting
the supplemental groups to allow groupmember() to be implemented as a
binary search.

Because we can not change struct xucred without breaking application
ABIs, we leave it alone and introduce a new XU_NGROUPS value which is
always 16 and is to be used or NGRPS as appropriate for things such as
NFS which need to use no more than 16 groups. When feasible, truncate
the group list rather than generating an error.

Minor changes:
- Reduce the number of hand rolled versions of groupmember().
- Do not assign to both cr_gid and cr_groups[0].
- Modify ipfw to cache ucreds instead of part of their contents since
they are immutable once referenced by more than one entity.

Submitted by: Isilon Systems (initial implementation)
X-MFC after: never
PR: bin/113398 kern/133867

Since svc_[dg|vc|tli|tp]_create() did not hold a reference count on the
SVCXPTR structure returned by them, it was possible for the structure
to be free'd before svc_reg() had been completed using the structure.
This patch acquires a reference count on the newly created structure
that is returned by svc_[dg|vc|tli|tp]_create(). It also
adds the appropriate SVC_RELEASE() calls to the callers, except the
experimental nfs subsystem. The latter will be committed separately.

Submitted by: dfr
Tested by: pho
Approved by: kib (mentor)

Replace the global references to "hostid" in svc_rpcsec_gss.c to local
variables set via the getcredhostid() function. I also changed the type
of ci_hostid to "unsigned long" so that it matches what is returned by
getcredhostid(). Although "struct svc_rpc_gss_clientid" goes on the wire
during RPCSEC_GSS, it is just a variable # of opaque bytes to the client,
so it doesn't matter how much storage ci_hostid uses.

Approved by: kib (mentor)

When a Solaris10 client does an NFS mount using krb5i or krb5p, the
server would crash because the Solaris10 client would attempt to use
Sun's NFSACL protocol, which FreeBSD doesn't support. When the server
generated the error reply via svcerr_noprog(), it would cause a crash
because it would try and wrap a NULL reply. According to RFC2203, no
wrapping is required for error cases. This one line change avoids
wrapping of NULL replies.

Reviewed by: dfr
Approved by: kib (mentor)

For the case where another thread was doing a connect and that
connect failed, the thread would be left stuck in msleep()
indefinitely, since it would call msleep() again for the case
where rc_client == NULL. Change the loop criteria and the if just
after the loop, so that this case is handled correctly.

Reviewed by: dfr
Approved by: kib (mentor)

Add a temporary workaround for panics being seen on NFS servers with ZFS,
where an improperly initialized prison field could lead to a panic. This
is not the correct solution, since it fails to address similar problems
for both AUDIT and MAC, which also rely on properly initialized
credentials, but should reduce panic reports while we work that out.

Reported by: ps, kan, others

Add a check to xprt_unregister() to catch the case where another
thread has already unregistered the structure. Also add a KASSERT()
to xprt_unregister_locked() to check that the structure hasn't already
been unregistered.

Reviewed by: jhb
Tested by: pho
Approved by: kib (mentor)

Fix a lockorder reversal I introduced in r193436 when I moved the
mtx_destroy() of the pool mutex to after SVC_RELEASE(), because
the pool mutex was still locked when soclose() was called by svc_dg_destroy().
To fix this, an mtx_unlock() was added where mtx_destroy() was before
r193436.

Reviewed by: jhb
Tested by: pho
Approved by: rwatson (mentor)

Correct MAC compile problems resulting from the new RPC code copying and
pasting code from the general socket code without also bringing along
required opt_mac.h includes.

Fix upcall races in the client side krpc. For the client side upcall,
holding SOCKBUF_LOCK() isn't sufficient to guarantee that there is
no upcall in progress, since SOCKBUF_LOCK() is released/re-acquired
in the upcall. An upcall reference counter was added to the upcall
structure that is incremented at the beginning of the upcall and
decremented at the end of the upcall. As such, a reference count == 0
when holding the SOCKBUF_LOCK() guarantees there is no upcall in
progress. Add a function that is called just after soupcall_clear(),
which waits until the reference count == 0.
Also, move the mtx_destroy() down to after soupcall_clear(), so that
the mutex is not destroyed before upcalls are done.

Reviewed by: dfr, jhb
Tested by: pho
Approved by: kib (mentor)

Fix two races in the server side krpc w.r.t upcalls:
Add a flag so that soupcall_clear() is only called once to cancel
an upcall.
Move the test for xprt_registered in the upcall down to after the
mtx_lock() of the pool mutex, to catch the case where it is
unregistered while the upcall is waiting for the mutex.
Also, move the mtx_destroy() of the pool mutex to after SVC_RELEASE(),
so that it isn't destroyed before the upcalls are disabled.

Reviewed by: dfr, jhb
Tested by: pho
Approved by: kib (mentor)

Add internal 'mac_policy_count' counter to the MAC Framework, which is a
count of the number of registered policies.

Rather than unconditionally locking sockets before passing them into MAC,
lock them in the MAC entry points only if mac_policy_count is non-zero.

This avoids locking overhead for a number of socket system calls when no
policies are registered, eliminating measurable overhead for the MAC
Framework for the socket subsystem when there are no active policies.

Possibly socket locks should be acquired by policies if they are required
for socket labels, which would further avoid locking overhead when there
are policies but they don't require labeling of sockets, or possibly
don't even implement socket controls.

Obtained from: TrustedBSD Project

Rework socket upcalls to close some races with setup/teardown of upcalls.
- Each socket upcall is now invoked with the appropriate socket buffer
locked. It is not permissible to call soisconnected() with this lock
held; however, so socket upcalls now return an integer value. The two
possible values are SU_OK and SU_ISCONNECTED. If an upcall returns
SU_ISCONNECTED, then the soisconnected() will be invoked on the
socket after the socket buffer lock is dropped.
- A new API is provided for setting and clearing socket upcalls. The
API consists of soupcall_set() and soupcall_clear().
- To simplify locking, each socket buffer now has a separate upcall.
- When a socket upcall returns SU_ISCONNECTED, the upcall is cleared from
the receive socket buffer automatically. Note that a SO_SND upcall
should never return SU_ISCONNECTED.
- All this means that accept filters should now return SU_ISCONNECTED
instead of calling soisconnected() directly. They also no longer need
to explicitly clear the upcall on the new socket.
- The HTTP accept filter still uses soupcall_set() to manage its internal
state machine, but other accept filters no longer have any explicit
knowlege of socket upcall internals aside from their return value.
- The various RPC client upcalls currently drop the socket buffer lock
while invoking soreceive() as a temporary band-aid. The plan for
the future is to add a new flag to allow soreceive() to be called with
the socket buffer locked.
- The AIO callback for socket I/O is now also invoked with the socket
buffer locked. Previously sowakeup() would drop the socket buffer
lock only to call aio_swake() which immediately re-acquired the socket
buffer lock for the duration of the function call.

Discussed with: rwatson, rmacklem

fix xdrmem_control to be safe in an if statement
fix zfs to depend on krpc
remove xdr from zfs makefile

Submitted by: dchagin@freebsd.org

Place hostnames and similar information fully under the prison system.
The system hostname is now stored in prison0, and the global variable
"hostname" has been removed, as has the hostname_mtx mutex. Jails may
have their own host information, or they may inherit it from the
parent/system. The proper way to read the hostname is via
getcredhostname(), which will copy either the hostname associated with
the passed cred, or the system hostname if you pass NULL. The system
hostname can still be accessed directly (and without locking) at
prison0.pr_host, but that should be avoided where possible.

The "similar information" referred to is domainname, hostid, and
hostuuid, which have also become prison parameters and had their
associated global variables removed.

Approved by: bz (mentor)

MFdevbranch 192944
- add FreeBSD implementation of xdrmem_control needed by zfs
- have zfs define xdr_ops using FreeBSD's definition
- remove solaris xdr files from zfs compile

Remove the unmaintained University of Michigan NFSv4 client from 8.x
prior to 8.0-RELEASE. Rick Macklem's new and more feature-rich NFSv234
client and server are replacing it.

Discussed with: rmacklem

Added a field to the SVCXPRT structure that the nfsv4 server can
use to identify if the socket is the same one that a cached request
came in on. It is set by nfsrvd_addsock() to a unique value generated
by incrementing an unsigned 64bit static variable for each assignment
and then the value of xp_sockref is tested to see if it is equal to
the value that was saved with the cached reply.

Submitted by: rmacklem
Reviewed by: dfr
Approved by: kib (mentor)

Use the correct creds when reconnecting so that we have enough privilege to
bind reserved ports (if necessary).

Submitted by: Jaakko Heinonen <jh at saualaht dot fi>

Add a missing call to mtx_destroy().

Implement support for RPCSEC_GSS authentication to both the NFS client
and server. This replaces the RPC implementation of the NFS client and
server with the newer RPC implementation originally developed
(actually ported from the userland sunrpc code) to support the NFS
Lock Manager. I have tested this code extensively and I believe it is
stable and that performance is at least equal to the legacy RPC
implementation.

The NFS code currently contains support for both the new RPC
implementation and the older legacy implementation inherited from the
original NFS codebase. The default is to use the new implementation -
add the NFS_LEGACYRPC option to fall back to the old code. When I
merge this support back to RELENG_7, I will probably change this so
that users have to 'opt in' to get the new code.

To use RPCSEC_GSS on either client or server, you must build a kernel
which includes the KGSSAPI option and the crypto device. On the
userland side, you must build at least a new libc, mountd, mount_nfs
and gssd. You must install new versions of /etc/rc.d/gssd and
/etc/rc.d/nfsd and add 'gssd_enable=YES' to /etc/rc.conf.

As long as gssd is running, you should be able to mount an NFS
filesystem from a server that requires RPCSEC_GSS authentication. The
mount itself can happen without any kerberos credentials but all
access to the filesystem will be denied unless the accessing user has
a valid ticket file in the standard place (/tmp/krb5cc_<uid>). There
is currently no support for situations where the ticket file is in a
different place, such as when the user logged in via SSH and has
delegated credentials from that login. This restriction is also
present in Solaris and Linux. In theory, we could improve this in
future, possibly using Brooks Davis' implementation of variant
symlinks.

Supporting RPCSEC_GSS on a server is nearly as simple. You must create
service creds for the server in the form 'nfs/<fqdn>@<REALM>' and
install them in /etc/krb5.keytab. The standard heimdal utility ktutil
makes this fairly easy. After the service creds have been created, you
can add a '-sec=krb5' option to /etc/exports and restart both mountd
and nfsd.

The only other difference an administrator should notice is that nfsd
doesn't fork to create service threads any more. In normal operation,
there will be two nfsd processes, one in userland waiting for TCP
connections and one in the kernel handling requests. The latter
process will create as many kthreads as required - these should be
visible via 'top -H'. The code has some support for varying the number
of service threads according to load but initially at least, nfsd uses
a fixed number of threads according to the value supplied to its '-n'
option.

Sponsored by: Isilon Systems
MFC after: 1 month

Retire the MALLOC and FREE macros. They are an abomination unto style(9).

MFC after: 3 months

Step 1.5 of importing the network stack virtualization infrastructure
from the vimage project, as per plan established at devsummit 08/08:
http://wiki.freebsd.org/Image/Notes200808DevSummit

Introduce INIT_VNET_*() initializer macros, VNET_FOREACH() iterator
macros, and CURVNET_SET() context setting macros, all currently
resolving to NOPs.

Prepare for virtualization of selected SYSCTL objects by introducing a
family of SYSCTL_V_*() macros, currently resolving to their global
counterparts, i.e. SYSCTL_V_INT() == SYSCTL_INT().

Move selected #defines from sys/sys/vimage.h to newly introduced header
files specific to virtualized subsystems (sys/net/vnet.h,
sys/netinet/vinet.h etc.).

All the changes are verified to have zero functional impact at this
point in time by doing MD5 comparision between pre- and post-change
object files(*).

(*) netipsec/keysock.c did not validate depending on compile time options.

Implemented by: julian, bz, brooks, zec
Reviewed by: julian, bz, brooks, kris, rwatson, ...
Approved by: julian (mentor)
Obtained from: //depot/projects/vimage-commit2/...
X-MFC after: never
Sponsored by: NLnet Foundation, The FreeBSD Foundation

Rename RPC's 'struct pmap' to 'struct portmap' to avoid confusing it with
the other 'struct pmap'.

Pointed out by: kmacy
MFC after: 2 weeks

Rename the static M_RPC defined here to M_RPCCLNT, since a global M_RPC
now optionally exists.

Reviewed by: dfr
MFC after: 3 days

Commit step 1 of the vimage project, (network stack)
virtualization work done by Marko Zec (zec@).

This is the first in a series of commits over the course
of the next few weeks.

Mark all uses of global variables to be virtualized
with a V_ prefix.
Use macros to map them back to their global names for
now, so this is a NOP change only.

We hope to have caught at least 85-90% of what is needed
so we do not invalidate a lot of outstanding patches again.

Obtained from: //depot/projects/vimage-commit2/...
Reviewed by: brooks, des, ed, mav, julian,
jamie, kris, rwatson, zec, ...
(various people I forgot, different versions)
md5 (with a bit of help)
Sponsored by: NLnet Foundation, The FreeBSD Foundation
X-MFC after: never
V_Commit_Message_Reviewed_By: more people than the patch

Add a missing call to mtx_destroy() in clnt_reconnect_destroy().

Submitted by: zachary.loafman at isilon.com
MFC after: 2 weeks

Re-work the code slightly to avoid a possible livelock.

MFC after: 2 weeks

Move the NFS/RPC code away from lbolt.

The kernel has a special wchan called `lbolt', which is triggered each
second. It doesn't seem to be used a lot and it seems pretty redundant,
because we can specify a timeout value to the *sleep() routines. In an
attempt to eventually remove lbolt, make the NFS/RPC code use a timeout
of `hz' when trying to reconnect.

Only the TTY code (not MPSAFE TTY) and the VFS syncer seem to use lbolt
now.

Reviewed by: attilio, jhb
Approved by: philip (mentor), alfred, dfr

Introduce a new lock, hostname_mtx, and use it to synchronize access
to global hostname and domainname variables. Where necessary, copy
to or from a stack-local buffer before performing copyin() or
copyout(). A few uses, such as in cd9660 and daemon_saver, remain
under-synchronized and will require further updates.

Correct a bug in which a failed copyin() of domainname would leave
domainname potentially corrupted.

MFC after: 3 weeks

It may be #if 0'd out code, but change a varname to not shadow a global.

Include <sys/pcpu.h> for curthread.

Re-implement the client side of rpc.lockd in the kernel. This implementation
provides the correct semantics for flock(2) style locks which are used by the
lockf(1) command line tool and the pidfile(3) library. It also implements
recovery from server restarts and ensures that dirty cache blocks are written
to the server before obtaining locks (allowing multiple clients to use file
locking to safely share data).

Sponsored by: Isilon Systems
PR: 94256
MFC after: 2 weeks

Fix some issues that showed up during Kris' testing.

Reported by: kris
MFC after: 3 days

Minor changes to improve compatibility with older FreeBSD releases.

Add kernel module support for nfslockd and krpc. Use the module system
to detect (or load) kernel NLM support in rpc.lockd. Remove the '-k'
option to rpc.lockd and make kernel NLM the default. A user can still
force the use of the old user NLM by building a kernel without NFSLOCKD
and/or removing the nfslockd.ko module.

Add the new kernel-mode NFS Lock Manager. To use it instead of the
user-mode lock manager, build a kernel with the NFSLOCKD option and
add '-k' to 'rpc_lockd_flags' in rc.conf.

Highlights include:

* Thread-safe kernel RPC client - many threads can use the same RPC
client handle safely with replies being de-multiplexed at the socket
upcall (typically driven directly by the NIC interrupt) and handed
off to whichever thread matches the reply. For UDP sockets, many RPC
clients can share the same socket. This allows the use of a single
privileged UDP port number to talk to an arbitrary number of remote
hosts.

* Single-threaded kernel RPC server. Adding support for multi-threaded
server would be relatively straightforward and would follow
approximately the Solaris KPI. A single thread should be sufficient
for the NLM since it should rarely block in normal operation.

* Kernel mode NLM server supporting cancel requests and granted
callbacks. I've tested the NLM server reasonably extensively - it
passes both my own tests and the NFS Connectathon locking tests
running on Solaris, Mac OS X and Ubuntu Linux.

* Userland NLM client supported. While the NLM server doesn't have
support for the local NFS client's locking needs, it does have to
field async replies and granted callbacks from remote NLMs that the
local client has contacted. We relay these replies to the userland
rpc.lockd over a local domain RPC socket.

* Robust deadlock detection for the local lock manager. In particular
it will detect deadlocks caused by a lock request that covers more
than one blocking request. As required by the NLM protocol, all
deadlock detection happens synchronously - a user is guaranteed that
if a lock request isn't rejected immediately, the lock will
eventually be granted. The old system allowed for a 'deferred
deadlock' condition where a blocked lock request could wake up and
find that some other deadlock-causing lock owner had beaten them to
the lock.

* Since both local and remote locks are managed by the same kernel
locking code, local and remote processes can safely use file locks
for mutual exclusion. Local processes have no fairness advantage
compared to remote processes when contending to lock a region that
has just been unlocked - the local lock manager enforces a strict
first-come first-served model for both local and remote lockers.

Sponsored by: Isilon Systems
PR: 95247 107555 115524 116679
MFC after: 2 weeks

Replaced the misleading uses of a historical artefact M_TRYWAIT with M_WAIT.
Removed dead code that assumed that M_TRYWAIT can return NULL; it's not true
since the advent of MBUMA.

Reviewed by: arch

There are ongoing disputes as to whether we want to switch to directly using
UMA flags M_WAITOK/M_NOWAIT for mbuf(9) allocation.

Remove the now-unused NET_{LOCK,UNLOCK,ASSERT}_GIANT() macros, which
previously conditionally acquired Giant based on debug.mpsafenet. As that
has now been removed, they are no longer required. Removing them
significantly simplifies error-handling in the socket layer, eliminated
quite a bit of unwinding of locking in error cases.

While here clean up the now unneeded opt_net.h, which previously was used
for the NET_WITH_GIANT kernel option. Clean up some related gotos for
consistency.

Reviewed by: bz, csjp
Tested by: kris
Approved by: re (kensmith)

Check for a NULL return from rpcclnt_buildheader- it can fail if
the passed in auth_type is unacceptable to rpcauth_buildheader-
this avoids a null pointer panic. Clean up allocations if this
happens. This also quiets a gcc 4.2 complaint about ussing mheadend
without it being initialized.

Reviewed by: alfred

Move rpc/types.h under sys/, as this is used by ZFS kernel module.

Repo-copied by: simon

Replace GIANT_REQUIRED's present for socket locking with NET_LOCK_GIANT().
If/when someone does the necessary MPSAFEty locking for the NFSv4 client,
the socket code is generally MPSAFE now.

Spotted by: kris

Fix up some cut-n-paste damage and some out-of-date comments.
No code changes.

Submitted by: cel@citi.umich.edu

Use thread0 instead of user's thread for sobind.
This fixes reconnect after, for example, tcp idle disconnection.
Previously this would fail if a normal user tried to bind to a privileged
port.

Submitted by: cel@citi.umich.edu
MFC after: 1 week

Fix -Wundef.

Add GIANT_REQUIRED and WITNESS sleep warnings to uprintf() and tprintf(),
as they both interact with the tty code (!MPSAFE) and may sleep if the
tty buffer is full (per comment).

Modify all consumers of uprintf() and tprintf() to hold Giant around
calls into these functions. In most cases, this means adding an
acquisition of Giant immediately around the function. In some cases
(nfs_timer()), it means acquiring Giant higher up in the callout.

With these changes, UFS no longer panics on SMP when either blocks are
exhausted or inodes are exhausted under load due to races in the tty
code when running without Giant.

NB: Some reduction in calls to uprintf() in the svr4 code is probably
desirable.

NB: In the case of nfs_timer(), calling uprintf() while holding a mutex,
or even in a callout at all, is a bad idea, and will generate warnings
and potential upset. This needs to be fixed, but was a problem before
this change.

NB: uprintf()/tprintf() sleeping is generally a bad ideas, as is having
non-MPSAFE tty code.

MFC after: 1 week

- Don't call rpcclnt_realign() if we don't have any mbufs to realign.
- Remove a bogus and unneeded null pointer check.

Found by: Coverity Prevent analysis tool
Approved by: alfred

/* -> /*- for license, minor formatting changes

Prefer C99's __func__ over GCC's __FUNCTION__.

Approved by: alfred

fix array index out of bounds in rpc->rc_srtt[], rpc->rc_sdrtt[]

Noticed by: tedu
Approved by: alfred

Constify 'rpcclnt_backoff'.

Remove advertising clause from University of California Regent's
license, per letter dated July 22, 1999 and email from Peter Wemm,
Alan Cox and Robert Watson.

Approved by: core, peter, alc, rwatson

Reset callout if in nfs_timeout and rpcclnt_timeout functions. Timer
are supposed to continue firing as long as there is work to do, not
stop after the first invocation.

This is damage control after a patch that has been committed prematurely.

Tested by: kris

only do nfs rpc callouts if there is work to do.

Submitted by: kan
Approved by: alfred

Calculate NFS timeouts in units of 10ms, not 5ms. This matches the default
clock precision on i386. This is a NOP change on i386. But this stops
the mount_nfs units from suddenly changing to units of 1/20 of a second
(vs the normal 1/10 of a second) if HZ is increased.

Convert from timeout to callout API.

Don't panic because of RPC proto mismatches. Whitespace cleanup.

Submitted by: Jim Rees <rees@umich.edu>

Prevent a panic when mounting a v2/v3 only server with mount_nfs4.

Submitted by: Jim Rees <rees@umich.edu>
Reported/testing: Florian C. Smeets <flo@kasimir.com>

Fix a panic when attempting a v4 op against a v3/v2-only server.
It happens because rpcclnt_request is incorrectly returning 0 in the case
of an rpc mismatch or auth error.

Submitted by: Jim Rees <rees@umich.edu>

Change the definition of NULL on ia64 (for LP64 compilations) from
an int constant to a long constant. This change improves consistency
in the following two ways:
1. The first 8 arguments are always passed in registers on ia64, which
by virtue of the generated code implicitly widens ints to longs and
allows the use of an 32-bit integral type for 64-bit arguments.
Subsequent arguments are passed onto the memory stack, which does
not exhibit the same behaviour and consequently do not allow this.
In practice this means that variadic functions taking pointers
and given NULL (without cast) work as long as the NULL is passed
in one of the first 8 arguments. A SIGSEGV is more likely the
result if such would be done for stack-based arguments. This is
due to the fact that the upper 4 bytes remain undefined.
2. All 64-bit platforms that FreeBSD supports, with the obvious
exception of ia64, allow 32-bit integral types (specifically NULL)
when 64-bit pointers are expected in variadic functions by way of
how the compiler generates code. As such, code that works correctly
(whether rightfully so or not) on any platform other than ia64, may
fail on ia64.

To more easily allow tweaking of the definition of NULL, this commit
removes the 12 definitions in the various headers and puts it in a
new header that can be included whenever NULL is to be made visible.

This commit fixes GNOME, emacs, xemacs and a whole bunch of ports
that I don't particularly care about at this time...

Remove unneeded file. (could be repo removed as nothing ever referenced it.)

Submitted by: Jim Rees <rees@umich.edu>

Use %zu to printf a size_t instead of an int cast.

Requested by: jmallett, wollman

Fix compilation warnings on sparc.
Cast sizeof to int for printing with %d.

University of Michigan's Citi NFSv4 kernel client code.

Submitted by: Jim Rees <rees@umich.edu>

Bring in a hybrid of SunSoft's transport-independent RPC (TI-RPC) and
associated changes that had to happen to make this possible as well as
bugs fixed along the way.

Bring in required TLI library routines to support this.

Since we don't support TLI we've essentially copied what NetBSD
has done, adding a thin layer to emulate direct the TLI calls
into BSD socket calls.

This is mostly from Sun's tirpc release that was made in 1994,
however some fixes were backported from the 1999 release (supposedly
only made available after this porting effort was underway).

The submitter has agreed to continue on and bring us up to the
1999 release.

Several key features are introduced with this update:
Client calls are thread safe. (1999 code has server side thread
safe)
Updated, a more modern interface.

Many userland updates were done to bring the code up to par with
the recent RPC API.

There is an update to the pthreads library, a function
pthread_main_np() was added to emulate a function of Sun's threads
library.

While we're at it, bring in NetBSD's lockd, it's been far too
long of a wait.

New rpcbind(8) replaces portmap(8) (supporting communication over
an authenticated Unix-domain socket, and by default only allowing
set and unset requests over that channel). It's much more secure
than the old portmapper.

Umount(8), mountd(8), mount_nfs(8), nfsd(8) have also been upgraded
to support TI-RPC and to support IPV6.

Umount(8) is also fixed to unmount pathnames longer than 80 chars,
which are currently truncated by the Kernel statfs structure.

Submitted by: Martin Blapp <mb@imp.ch>
Manpage review: ru
Secure RPC implemented by: wpaul

$Id$ -> $FreeBSD$

Resolve conflicts.

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

First commit of a series of cleanups for the libc rpc code which has been
suffering a bad case neglect for the last few years.

- Add full prototypes, including to function pointers.
- Make the wire protocols 64-bit type safe, eg: 32 bit quantities are
int32_t, not long. The orginal rpc code was implemented when an int
could be 16 bits.

Obtained from: a diff of FreeBSD vs. OpenBSD/NetBSD rpc code.

Fix a bunch of spelling errors in the comment fields
of a bunch of system include files.

Remove trailing whitespace.

Use the header files that are compatible with the code just moved over
from 1.1.5.

Install RPC headers from include, like they always should have been.