Add/fix statx, readahead, close_range. From GSoC 2023 by Theodore Preduta

- Add compat_linux statx(2) syscall.
- The AT_EMPTY_PATH processing from the modification of
sys/compat/linux/common/linux_file64.c r1.63 has been separated, and made
common to linux_statat(), so that it can be used not only by
linux32_sys_fstatat64() but also by other *statat() variants.

rename bsd_to_linux_stat() to bsd_to_linux_stat64() to avoid confusion.
There is a static function with the same name in linux/common/linux_file.c.

NFC.

rename bsd_to_linux_stat() to bsd_to_linux_stat64() to avoid confusion.
There is a static function with the same name in linux/common/linux_file.c.

NFC.

add support COMPAT_LINUX for aarch64

linux_sys_fstatat64: Add support for AT_EMPTY_PATH flag.

This is required for rtld shipped with glibc 2.33.

Now, userland of Fedora 34 is working fine on NetBSD/amd64.

Fix info leaks.

Fix info leak.

branches: 1.60.4;
Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

branches: 1.59.2; 1.59.4;
Fail, don't panic, on bad dirents from file system.

Controllable via puffs from userland.

From Ilja Van Sprundel.

branches: 1.58.6;
copy the terminating NUL (njoly)

branches: 1.57.2;
fix typo

convert strcpy to a bounded copy to avoid compiler warnings, although the
reclen test prevents overflow.

branches: 1.55.4; 1.55.6; 1.55.8; 1.55.10; 1.55.14;
Add struct linux_stat64 definition for alpha, and enable syscalls that
use it.

implement the *at() syscalls.
bring the unimplemented syscall list up to date.

branches: 1.53.2; 1.53.8; 1.53.12; 1.53.14; 1.53.16; 1.53.22;
Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock. Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.

Make linux_sys_getdents64 fails with ENOTDIR instead of EINVAL, when fd
does not refer to a directory.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

branches: 1.50.2;
When implementing "read directory", when there are too many empty entries
in a row, and we need to try to read the next block, and have passed a
non-NULL cookie pointer to VOP_READDIR, ensure that we free the cookie
buffer before re-doing VOP_READDIR, so that we don't leak memory.
This fix is similar to nfs_serv.c revisions 1.115 + 1.124.

This should fix the long-standing problem observed by e.g. using Linux-
emulated programs to take backup of servers, which is one of the problems
which were reported in PR#42661.

Thanks to pooka@ for the hints for traversing the VOP* layer.

branches: 1.49.2;
pad -> PAD

branches: 1.48.6; 1.48.10; 1.48.14;
Replace references to getsock/getvnode.

branches: 1.47.2; 1.47.4;
Remove clause 3 and 4 from TNF licenses

branches: 1.46.2; 1.46.4;
Catch up with descriptor handling changes. See kern_descrip.c revision
1.173 for details.

branches: 1.45.6;
Extract the fcntl64() suppoort from linux_file64.c into linux_fcntl64.c
Unravel some of the knots that caused linux_file64.c to be compiled twice
for an amd64 kernel (once for linux and once for linux32) with different
parts being skipped each time.

Convert all the system call entry points from:
int foo(struct lwp *l, void *v, register_t *retval)
to:
int foo(struct lwp *l, const struct foo_args *uap, register_t *retval)
Fixup compat code to not write into 'uap' and (in some cases) to actually
pass a correctly formatted 'uap' structure with the right name to the
next routine.
A few 'compat' routines that just call standard ones have been deleted.
All the 'compat' code compiles (along with the kernels required to test
build it).
98% done by automated scripts.

branches: 1.43.4;
ANSIfy most of the function definitions in sys/compat (but not ndis).
All by the magic of sed ...

Remove all the __P

branches: 1.41.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.40.2;
Add compat_linux and exec_linux_elf lkm support for amd64:
- Add needed COMPAT_OSSAUDIO to GENERIC.
- Add missing includes needed by linux_syscallargs.h.
- Add lkm building.

branches: 1.39.6; 1.39.8; 1.39.12;
Split the fcntl locking code out from its copyin/out.
Use to avoid all the stackgap stuff in compat code.

Change the way that emulations locate files within the emulation root to
avoid having to allocate space in the 'stackgap'
- which is very LWP unfriendly.
The additional code for non-emulation namei() is trivial, the reduction for
the emulations is massive.
The vnode for a processes emulation root is saved in the cwdi structure
during process exec.
If the emulation root the TRYEMULROOT flag are set, namei() will do an initial
search for absolute pathnames in the emulation root, if that fails it will
retry from the normal root.
".." at the emulation root will always go to the real root, even in the middle
of paths and when expanding symlinks.
Absolute symlinks found using absolute paths in the emulation root will be
relative to the emulation root (so /usr/lib/xxx.so -> /lib/xxx.so links
inside the emulation root don't need changing).
If the root of the emulation would be returned (for an emulation lookup), then
the real root is returned instead (matching the behaviour of emul_lookup,
but being a cheap comparison here) so that programs that scan "../.."
looking for the root dircetory don't loop forever.
The target for symbolic links is no longer mangled (it used to get the
CHECK_ALT_xxx() treatment, so could get /emul/xxx prepended).
CHECK_ALT_xxx() are no more. Most of the change is deleting them, and adding
TRYEMULROOT to the flags to NDINIT().
A lot of the emulation system call stubs could now be deleted.

branches: 1.37.2;
Update all the compat stuff to not use the 'stackgap' for processing
sys_stat() and friends, instead use do_sys_stat() and do_sys_fstat()
that write the answer into a kernel buffer (on stack) that can be
converted to the correct form and written the userspace.
I've test compiled a few kernels, and tested i386 netbsd1.6 ls.
Given I think I've fixed some bugs, it might be 50-50 with new ones.

branches: 1.36.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.35.2;
Merge newlock2 to head.

branches: 1.34.4; 1.34.8;
Use the LWP cached credentials where sane.

integrate kauth.

branches: 1.32.2; 1.32.4; 1.32.6;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.

Add initial (but unfinished) COMPAT_LINUX32 for amd64. This is good enough so
that the i386 license manager part of amd64 version of Fluent works.

While I'm here, add SysV IPC to COMPAT_LINUX/amd64

branches: 1.30.2; 1.30.4; 1.30.6;
merge ktrace-lwp.

64 bit inode changes.

branches: 1.28.2;
- sprinkle const.
- add XXXUNCONST to the emul_find() pbuf argument free'ing. XXX: this needs
an api change.
- avoid variable shadowing.

First work on COMPAT_LINUX/amd64
Process startup and dynamiclinking work, but processes hang due to
Linux arch_prctl(2) not being really supported yet.

PR/29696: Joel Carnat: NetBSD freezes when accessing smbfs mounted FS with
firefox/linux due to compat getdents() call assumption that all filesystems
support cookies.

branches: 1.25.2;
Add nanosecond handling on the stat and stat64 code for the i386.

nuke trailing whitespace

branches: 1.23.4; 1.23.6;
There's no need to pass a proc value when using UIO_SYSSPACE with
vn_rdwr(9) and uiomove(9).

OK'd by Jason Thorpe

add Linux compat arg wrapper for ftruncate64(), and change linux compat
truncate64() wrapper to translate args structure
NetBSD truncate() and ftrucate() have hidden 'pad' argument, so we have
to do the argument translation

Problem found and patch supplied in PR kern/22360 by Ales Krenek

This is the last of syscalls with hidden 'pad' arg we didn't have
wrapper for; all the others (lseek, mmap, pwrite, pread) already had
wrapper before.

branches: 1.21.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Merge the nathanw_sa branch.

branches: 1.18.2;
linux_sys_getdents64(): remove the oldcall bug workaround cruft,
it's not needed for this call, and update comment accordingly

linux_sys_getdents64(): make the d_off member point to next entry, rather
than current one; this is expected (even documented) behaviour of the
new system call
This fixes the endless loop when reading directories on NFS, though
applications won't see all directory entries due to different problem.
Fix by Matthias Scheler and Charles Hannum.

On archs with broken linux struct stat64 st_ino, rename linux_stat64's
lst_ino to __lst_ino and lst_ino64 to lst_ino, and define
LINUX_STAT64_HAS_BROKEN_ST_INO. Only assign __lst_ino if this define
is present.

This fixes compile on powerpc archs, and hence port-powerpc/16893.

branches: 1.15.2;
Make COMPAT_LINUX compile again on Alpha. linux_getdents64 moved but
files.linux_alpha wasn't updated. common/linux_file64.c contains a lot
of syscalls which aren't on Linux/alpha so #ifdef them for alpha.

Remove an unreachable "return error;".

hook in remaining 64bit fs syscalls
make most of stuff in linux_file64.c available unconditionally,
with exception of fcntl64, which is not present on Linux m68k

Remove check for size of field "d_off" in Linux "dirent" structure because
this problem cannot happen with the 64Bit version of the getdents().

Move implementation of linux_sys_getdents64() to source file for
Linux 64bit filesystem calls.

gc debugging printfs.

implement the locking calls for linux_file64.

make the stackgap_{init,alloc} functions MP friendly (i.e. pass struct proc *
in, instead of using curproc). While there add an optional size argument to
stackgap_init.

- add a real linux_sys___sysctl because the mib numbers are different.
implement only the kernel version getting entries.
- make the signal conversion functions consistent; dest is arg 1.
- make linux_fakedev deal with block and char devices differently.
- add linux_sys_ugetrlimit, and flesh out the regular get/setrlimit so
that they work properly.
- add linux_mmap2 [untested].
- bump kernel to 2.4.18, and make the date be valentine's day :-)
- linux_sys_*stat64 family was totally busted. Fix it.

tested only on i386.

Status:
gdb, telnet, work
jdk-1.4.0 extractor works, jdk still coredumps.

christos

add RCSIDs (including regeneration of files as appropriate)

Fixed a minor problem so that it builds

Fixes my last commit: only mips really need fcntl64 to work now. i386 and
powerpc could use it later if needed. m68k and alpha should never need it.

Added initial support for fcntl64 system call (file locking is not
implemented)

branches: 1.2.2; 1.2.4; 1.2.6; 1.2.8;
add linux_sys_truncate64()

add emulation of Linux truncate64(2), ftruncate64(2), *stat64(2)

- Add compat_linux statx(2) syscall.
- The AT_EMPTY_PATH processing from the modification of
sys/compat/linux/common/linux_file64.c r1.63 has been separated, and made
common to linux_statat(), so that it can be used not only by
linux32_sys_fstatat64() but also by other *statat() variants.

rename bsd_to_linux_stat() to bsd_to_linux_stat64() to avoid confusion.
There is a static function with the same name in linux/common/linux_file.c.

NFC.

rename bsd_to_linux_stat() to bsd_to_linux_stat64() to avoid confusion.
There is a static function with the same name in linux/common/linux_file.c.

NFC.

add support COMPAT_LINUX for aarch64

linux_sys_fstatat64: Add support for AT_EMPTY_PATH flag.

This is required for rtld shipped with glibc 2.33.

Now, userland of Fedora 34 is working fine on NetBSD/amd64.

Fix info leaks.

Fix info leak.

branches: 1.60.4;
Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

branches: 1.59.2; 1.59.4;
Fail, don't panic, on bad dirents from file system.

Controllable via puffs from userland.

From Ilja Van Sprundel.

branches: 1.58.6;
copy the terminating NUL (njoly)

branches: 1.57.2;
fix typo

convert strcpy to a bounded copy to avoid compiler warnings, although the
reclen test prevents overflow.

branches: 1.55.4; 1.55.6; 1.55.8; 1.55.10; 1.55.14;
Add struct linux_stat64 definition for alpha, and enable syscalls that
use it.

implement the *at() syscalls.
bring the unimplemented syscall list up to date.

branches: 1.53.2; 1.53.8; 1.53.12; 1.53.14; 1.53.16; 1.53.22;
Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock. Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.

Make linux_sys_getdents64 fails with ENOTDIR instead of EINVAL, when fd
does not refer to a directory.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

branches: 1.50.2;
When implementing "read directory", when there are too many empty entries
in a row, and we need to try to read the next block, and have passed a
non-NULL cookie pointer to VOP_READDIR, ensure that we free the cookie
buffer before re-doing VOP_READDIR, so that we don't leak memory.
This fix is similar to nfs_serv.c revisions 1.115 + 1.124.

This should fix the long-standing problem observed by e.g. using Linux-
emulated programs to take backup of servers, which is one of the problems
which were reported in PR#42661.

Thanks to pooka@ for the hints for traversing the VOP* layer.

branches: 1.49.2;
pad -> PAD

branches: 1.48.6; 1.48.10; 1.48.14;
Replace references to getsock/getvnode.

branches: 1.47.2; 1.47.4;
Remove clause 3 and 4 from TNF licenses

branches: 1.46.2; 1.46.4;
Catch up with descriptor handling changes. See kern_descrip.c revision
1.173 for details.

branches: 1.45.6;
Extract the fcntl64() suppoort from linux_file64.c into linux_fcntl64.c
Unravel some of the knots that caused linux_file64.c to be compiled twice
for an amd64 kernel (once for linux and once for linux32) with different
parts being skipped each time.

Convert all the system call entry points from:
int foo(struct lwp *l, void *v, register_t *retval)
to:
int foo(struct lwp *l, const struct foo_args *uap, register_t *retval)
Fixup compat code to not write into 'uap' and (in some cases) to actually
pass a correctly formatted 'uap' structure with the right name to the
next routine.
A few 'compat' routines that just call standard ones have been deleted.
All the 'compat' code compiles (along with the kernels required to test
build it).
98% done by automated scripts.

branches: 1.43.4;
ANSIfy most of the function definitions in sys/compat (but not ndis).
All by the magic of sed ...

Remove all the __P

branches: 1.41.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.40.2;
Add compat_linux and exec_linux_elf lkm support for amd64:
- Add needed COMPAT_OSSAUDIO to GENERIC.
- Add missing includes needed by linux_syscallargs.h.
- Add lkm building.

branches: 1.39.6; 1.39.8; 1.39.12;
Split the fcntl locking code out from its copyin/out.
Use to avoid all the stackgap stuff in compat code.

Change the way that emulations locate files within the emulation root to
avoid having to allocate space in the 'stackgap'
- which is very LWP unfriendly.
The additional code for non-emulation namei() is trivial, the reduction for
the emulations is massive.
The vnode for a processes emulation root is saved in the cwdi structure
during process exec.
If the emulation root the TRYEMULROOT flag are set, namei() will do an initial
search for absolute pathnames in the emulation root, if that fails it will
retry from the normal root.
".." at the emulation root will always go to the real root, even in the middle
of paths and when expanding symlinks.
Absolute symlinks found using absolute paths in the emulation root will be
relative to the emulation root (so /usr/lib/xxx.so -> /lib/xxx.so links
inside the emulation root don't need changing).
If the root of the emulation would be returned (for an emulation lookup), then
the real root is returned instead (matching the behaviour of emul_lookup,
but being a cheap comparison here) so that programs that scan "../.."
looking for the root dircetory don't loop forever.
The target for symbolic links is no longer mangled (it used to get the
CHECK_ALT_xxx() treatment, so could get /emul/xxx prepended).
CHECK_ALT_xxx() are no more. Most of the change is deleting them, and adding
TRYEMULROOT to the flags to NDINIT().
A lot of the emulation system call stubs could now be deleted.

branches: 1.37.2;
Update all the compat stuff to not use the 'stackgap' for processing
sys_stat() and friends, instead use do_sys_stat() and do_sys_fstat()
that write the answer into a kernel buffer (on stack) that can be
converted to the correct form and written the userspace.
I've test compiled a few kernels, and tested i386 netbsd1.6 ls.
Given I think I've fixed some bugs, it might be 50-50 with new ones.

branches: 1.36.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.35.2;
Merge newlock2 to head.

branches: 1.34.4; 1.34.8;
Use the LWP cached credentials where sane.

integrate kauth.

branches: 1.32.2; 1.32.4; 1.32.6;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.

Add initial (but unfinished) COMPAT_LINUX32 for amd64. This is good enough so
that the i386 license manager part of amd64 version of Fluent works.

While I'm here, add SysV IPC to COMPAT_LINUX/amd64

branches: 1.30.2; 1.30.4; 1.30.6;
merge ktrace-lwp.

64 bit inode changes.

branches: 1.28.2;
- sprinkle const.
- add XXXUNCONST to the emul_find() pbuf argument free'ing. XXX: this needs
an api change.
- avoid variable shadowing.

First work on COMPAT_LINUX/amd64
Process startup and dynamiclinking work, but processes hang due to
Linux arch_prctl(2) not being really supported yet.

PR/29696: Joel Carnat: NetBSD freezes when accessing smbfs mounted FS with
firefox/linux due to compat getdents() call assumption that all filesystems
support cookies.

branches: 1.25.2;
Add nanosecond handling on the stat and stat64 code for the i386.

nuke trailing whitespace

branches: 1.23.4; 1.23.6;
There's no need to pass a proc value when using UIO_SYSSPACE with
vn_rdwr(9) and uiomove(9).

OK'd by Jason Thorpe

add Linux compat arg wrapper for ftruncate64(), and change linux compat
truncate64() wrapper to translate args structure
NetBSD truncate() and ftrucate() have hidden 'pad' argument, so we have
to do the argument translation

Problem found and patch supplied in PR kern/22360 by Ales Krenek

This is the last of syscalls with hidden 'pad' arg we didn't have
wrapper for; all the others (lseek, mmap, pwrite, pread) already had
wrapper before.

branches: 1.21.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Merge the nathanw_sa branch.

branches: 1.18.2;
linux_sys_getdents64(): remove the oldcall bug workaround cruft,
it's not needed for this call, and update comment accordingly

linux_sys_getdents64(): make the d_off member point to next entry, rather
than current one; this is expected (even documented) behaviour of the
new system call
This fixes the endless loop when reading directories on NFS, though
applications won't see all directory entries due to different problem.
Fix by Matthias Scheler and Charles Hannum.

On archs with broken linux struct stat64 st_ino, rename linux_stat64's
lst_ino to __lst_ino and lst_ino64 to lst_ino, and define
LINUX_STAT64_HAS_BROKEN_ST_INO. Only assign __lst_ino if this define
is present.

This fixes compile on powerpc archs, and hence port-powerpc/16893.

branches: 1.15.2;
Make COMPAT_LINUX compile again on Alpha. linux_getdents64 moved but
files.linux_alpha wasn't updated. common/linux_file64.c contains a lot
of syscalls which aren't on Linux/alpha so #ifdef them for alpha.

Remove an unreachable "return error;".

hook in remaining 64bit fs syscalls
make most of stuff in linux_file64.c available unconditionally,
with exception of fcntl64, which is not present on Linux m68k

Remove check for size of field "d_off" in Linux "dirent" structure because
this problem cannot happen with the 64Bit version of the getdents().

Move implementation of linux_sys_getdents64() to source file for
Linux 64bit filesystem calls.

gc debugging printfs.

implement the locking calls for linux_file64.

make the stackgap_{init,alloc} functions MP friendly (i.e. pass struct proc *
in, instead of using curproc). While there add an optional size argument to
stackgap_init.

- add a real linux_sys___sysctl because the mib numbers are different.
implement only the kernel version getting entries.
- make the signal conversion functions consistent; dest is arg 1.
- make linux_fakedev deal with block and char devices differently.
- add linux_sys_ugetrlimit, and flesh out the regular get/setrlimit so
that they work properly.
- add linux_mmap2 [untested].
- bump kernel to 2.4.18, and make the date be valentine's day :-)
- linux_sys_*stat64 family was totally busted. Fix it.

tested only on i386.

Status:
gdb, telnet, work
jdk-1.4.0 extractor works, jdk still coredumps.

christos

add RCSIDs (including regeneration of files as appropriate)

Fixed a minor problem so that it builds

Fixes my last commit: only mips really need fcntl64 to work now. i386 and
powerpc could use it later if needed. m68k and alpha should never need it.

Added initial support for fcntl64 system call (file locking is not
implemented)

branches: 1.2.2; 1.2.4; 1.2.6; 1.2.8;
add linux_sys_truncate64()

add emulation of Linux truncate64(2), ftruncate64(2), *stat64(2)

add support COMPAT_LINUX for aarch64

linux_sys_fstatat64: Add support for AT_EMPTY_PATH flag.

This is required for rtld shipped with glibc 2.33.

Now, userland of Fedora 34 is working fine on NetBSD/amd64.

Fix info leaks.

Fix info leak.

branches: 1.60.4;
Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

branches: 1.59.2; 1.59.4;
Fail, don't panic, on bad dirents from file system.

Controllable via puffs from userland.

From Ilja Van Sprundel.

branches: 1.58.6;
copy the terminating NUL (njoly)

branches: 1.57.2;
fix typo

convert strcpy to a bounded copy to avoid compiler warnings, although the
reclen test prevents overflow.

branches: 1.55.4; 1.55.6; 1.55.8; 1.55.10; 1.55.14;
Add struct linux_stat64 definition for alpha, and enable syscalls that
use it.

implement the *at() syscalls.
bring the unimplemented syscall list up to date.

branches: 1.53.2; 1.53.8; 1.53.12; 1.53.14; 1.53.16; 1.53.22;
Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock. Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.

Make linux_sys_getdents64 fails with ENOTDIR instead of EINVAL, when fd
does not refer to a directory.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

branches: 1.50.2;
When implementing "read directory", when there are too many empty entries
in a row, and we need to try to read the next block, and have passed a
non-NULL cookie pointer to VOP_READDIR, ensure that we free the cookie
buffer before re-doing VOP_READDIR, so that we don't leak memory.
This fix is similar to nfs_serv.c revisions 1.115 + 1.124.

This should fix the long-standing problem observed by e.g. using Linux-
emulated programs to take backup of servers, which is one of the problems
which were reported in PR#42661.

Thanks to pooka@ for the hints for traversing the VOP* layer.

branches: 1.49.2;
pad -> PAD

branches: 1.48.6; 1.48.10; 1.48.14;
Replace references to getsock/getvnode.

branches: 1.47.2; 1.47.4;
Remove clause 3 and 4 from TNF licenses

branches: 1.46.2; 1.46.4;
Catch up with descriptor handling changes. See kern_descrip.c revision
1.173 for details.

branches: 1.45.6;
Extract the fcntl64() suppoort from linux_file64.c into linux_fcntl64.c
Unravel some of the knots that caused linux_file64.c to be compiled twice
for an amd64 kernel (once for linux and once for linux32) with different
parts being skipped each time.

Convert all the system call entry points from:
int foo(struct lwp *l, void *v, register_t *retval)
to:
int foo(struct lwp *l, const struct foo_args *uap, register_t *retval)
Fixup compat code to not write into 'uap' and (in some cases) to actually
pass a correctly formatted 'uap' structure with the right name to the
next routine.
A few 'compat' routines that just call standard ones have been deleted.
All the 'compat' code compiles (along with the kernels required to test
build it).
98% done by automated scripts.

branches: 1.43.4;
ANSIfy most of the function definitions in sys/compat (but not ndis).
All by the magic of sed ...

Remove all the __P

branches: 1.41.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.40.2;
Add compat_linux and exec_linux_elf lkm support for amd64:
- Add needed COMPAT_OSSAUDIO to GENERIC.
- Add missing includes needed by linux_syscallargs.h.
- Add lkm building.

branches: 1.39.6; 1.39.8; 1.39.12;
Split the fcntl locking code out from its copyin/out.
Use to avoid all the stackgap stuff in compat code.

Change the way that emulations locate files within the emulation root to
avoid having to allocate space in the 'stackgap'
- which is very LWP unfriendly.
The additional code for non-emulation namei() is trivial, the reduction for
the emulations is massive.
The vnode for a processes emulation root is saved in the cwdi structure
during process exec.
If the emulation root the TRYEMULROOT flag are set, namei() will do an initial
search for absolute pathnames in the emulation root, if that fails it will
retry from the normal root.
".." at the emulation root will always go to the real root, even in the middle
of paths and when expanding symlinks.
Absolute symlinks found using absolute paths in the emulation root will be
relative to the emulation root (so /usr/lib/xxx.so -> /lib/xxx.so links
inside the emulation root don't need changing).
If the root of the emulation would be returned (for an emulation lookup), then
the real root is returned instead (matching the behaviour of emul_lookup,
but being a cheap comparison here) so that programs that scan "../.."
looking for the root dircetory don't loop forever.
The target for symbolic links is no longer mangled (it used to get the
CHECK_ALT_xxx() treatment, so could get /emul/xxx prepended).
CHECK_ALT_xxx() are no more. Most of the change is deleting them, and adding
TRYEMULROOT to the flags to NDINIT().
A lot of the emulation system call stubs could now be deleted.

branches: 1.37.2;
Update all the compat stuff to not use the 'stackgap' for processing
sys_stat() and friends, instead use do_sys_stat() and do_sys_fstat()
that write the answer into a kernel buffer (on stack) that can be
converted to the correct form and written the userspace.
I've test compiled a few kernels, and tested i386 netbsd1.6 ls.
Given I think I've fixed some bugs, it might be 50-50 with new ones.

branches: 1.36.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.35.2;
Merge newlock2 to head.

branches: 1.34.4; 1.34.8;
Use the LWP cached credentials where sane.

integrate kauth.

branches: 1.32.2; 1.32.4; 1.32.6;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.

Add initial (but unfinished) COMPAT_LINUX32 for amd64. This is good enough so
that the i386 license manager part of amd64 version of Fluent works.

While I'm here, add SysV IPC to COMPAT_LINUX/amd64

branches: 1.30.2; 1.30.4; 1.30.6;
merge ktrace-lwp.

64 bit inode changes.

branches: 1.28.2;
- sprinkle const.
- add XXXUNCONST to the emul_find() pbuf argument free'ing. XXX: this needs
an api change.
- avoid variable shadowing.

First work on COMPAT_LINUX/amd64
Process startup and dynamiclinking work, but processes hang due to
Linux arch_prctl(2) not being really supported yet.

PR/29696: Joel Carnat: NetBSD freezes when accessing smbfs mounted FS with
firefox/linux due to compat getdents() call assumption that all filesystems
support cookies.

branches: 1.25.2;
Add nanosecond handling on the stat and stat64 code for the i386.

nuke trailing whitespace

branches: 1.23.4; 1.23.6;
There's no need to pass a proc value when using UIO_SYSSPACE with
vn_rdwr(9) and uiomove(9).

OK'd by Jason Thorpe

add Linux compat arg wrapper for ftruncate64(), and change linux compat
truncate64() wrapper to translate args structure
NetBSD truncate() and ftrucate() have hidden 'pad' argument, so we have
to do the argument translation

Problem found and patch supplied in PR kern/22360 by Ales Krenek

This is the last of syscalls with hidden 'pad' arg we didn't have
wrapper for; all the others (lseek, mmap, pwrite, pread) already had
wrapper before.

branches: 1.21.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Merge the nathanw_sa branch.

branches: 1.18.2;
linux_sys_getdents64(): remove the oldcall bug workaround cruft,
it's not needed for this call, and update comment accordingly

linux_sys_getdents64(): make the d_off member point to next entry, rather
than current one; this is expected (even documented) behaviour of the
new system call
This fixes the endless loop when reading directories on NFS, though
applications won't see all directory entries due to different problem.
Fix by Matthias Scheler and Charles Hannum.

On archs with broken linux struct stat64 st_ino, rename linux_stat64's
lst_ino to __lst_ino and lst_ino64 to lst_ino, and define
LINUX_STAT64_HAS_BROKEN_ST_INO. Only assign __lst_ino if this define
is present.

This fixes compile on powerpc archs, and hence port-powerpc/16893.

branches: 1.15.2;
Make COMPAT_LINUX compile again on Alpha. linux_getdents64 moved but
files.linux_alpha wasn't updated. common/linux_file64.c contains a lot
of syscalls which aren't on Linux/alpha so #ifdef them for alpha.

Remove an unreachable "return error;".

hook in remaining 64bit fs syscalls
make most of stuff in linux_file64.c available unconditionally,
with exception of fcntl64, which is not present on Linux m68k

Remove check for size of field "d_off" in Linux "dirent" structure because
this problem cannot happen with the 64Bit version of the getdents().

Move implementation of linux_sys_getdents64() to source file for
Linux 64bit filesystem calls.

gc debugging printfs.

implement the locking calls for linux_file64.

make the stackgap_{init,alloc} functions MP friendly (i.e. pass struct proc *
in, instead of using curproc). While there add an optional size argument to
stackgap_init.

- add a real linux_sys___sysctl because the mib numbers are different.
implement only the kernel version getting entries.
- make the signal conversion functions consistent; dest is arg 1.
- make linux_fakedev deal with block and char devices differently.
- add linux_sys_ugetrlimit, and flesh out the regular get/setrlimit so
that they work properly.
- add linux_mmap2 [untested].
- bump kernel to 2.4.18, and make the date be valentine's day :-)
- linux_sys_*stat64 family was totally busted. Fix it.

tested only on i386.

Status:
gdb, telnet, work
jdk-1.4.0 extractor works, jdk still coredumps.

christos

add RCSIDs (including regeneration of files as appropriate)

Fixed a minor problem so that it builds

Fixes my last commit: only mips really need fcntl64 to work now. i386 and
powerpc could use it later if needed. m68k and alpha should never need it.

Added initial support for fcntl64 system call (file locking is not
implemented)

branches: 1.2.2; 1.2.4; 1.2.6; 1.2.8;
add linux_sys_truncate64()

add emulation of Linux truncate64(2), ftruncate64(2), *stat64(2)

linux_sys_fstatat64: Add support for AT_EMPTY_PATH flag.

This is required for rtld shipped with glibc 2.33.

Now, userland of Fedora 34 is working fine on NetBSD/amd64.

Fix info leaks.

Fix info leak.

branches: 1.60.4;
Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

branches: 1.59.2; 1.59.4;
Fail, don't panic, on bad dirents from file system.

Controllable via puffs from userland.

From Ilja Van Sprundel.

branches: 1.58.6;
copy the terminating NUL (njoly)

branches: 1.57.2;
fix typo

convert strcpy to a bounded copy to avoid compiler warnings, although the
reclen test prevents overflow.

branches: 1.55.4; 1.55.6; 1.55.8; 1.55.10; 1.55.14;
Add struct linux_stat64 definition for alpha, and enable syscalls that
use it.

implement the *at() syscalls.
bring the unimplemented syscall list up to date.

branches: 1.53.2; 1.53.8; 1.53.12; 1.53.14; 1.53.16; 1.53.22;
Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock. Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.

Make linux_sys_getdents64 fails with ENOTDIR instead of EINVAL, when fd
does not refer to a directory.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

branches: 1.50.2;
When implementing "read directory", when there are too many empty entries
in a row, and we need to try to read the next block, and have passed a
non-NULL cookie pointer to VOP_READDIR, ensure that we free the cookie
buffer before re-doing VOP_READDIR, so that we don't leak memory.
This fix is similar to nfs_serv.c revisions 1.115 + 1.124.

This should fix the long-standing problem observed by e.g. using Linux-
emulated programs to take backup of servers, which is one of the problems
which were reported in PR#42661.

Thanks to pooka@ for the hints for traversing the VOP* layer.

branches: 1.49.2;
pad -> PAD

branches: 1.48.6; 1.48.10; 1.48.14;
Replace references to getsock/getvnode.

branches: 1.47.2; 1.47.4;
Remove clause 3 and 4 from TNF licenses

branches: 1.46.2; 1.46.4;
Catch up with descriptor handling changes. See kern_descrip.c revision
1.173 for details.

branches: 1.45.6;
Extract the fcntl64() suppoort from linux_file64.c into linux_fcntl64.c
Unravel some of the knots that caused linux_file64.c to be compiled twice
for an amd64 kernel (once for linux and once for linux32) with different
parts being skipped each time.

Convert all the system call entry points from:
int foo(struct lwp *l, void *v, register_t *retval)
to:
int foo(struct lwp *l, const struct foo_args *uap, register_t *retval)
Fixup compat code to not write into 'uap' and (in some cases) to actually
pass a correctly formatted 'uap' structure with the right name to the
next routine.
A few 'compat' routines that just call standard ones have been deleted.
All the 'compat' code compiles (along with the kernels required to test
build it).
98% done by automated scripts.

branches: 1.43.4;
ANSIfy most of the function definitions in sys/compat (but not ndis).
All by the magic of sed ...

Remove all the __P

branches: 1.41.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.40.2;
Add compat_linux and exec_linux_elf lkm support for amd64:
- Add needed COMPAT_OSSAUDIO to GENERIC.
- Add missing includes needed by linux_syscallargs.h.
- Add lkm building.

branches: 1.39.6; 1.39.8; 1.39.12;
Split the fcntl locking code out from its copyin/out.
Use to avoid all the stackgap stuff in compat code.

Change the way that emulations locate files within the emulation root to
avoid having to allocate space in the 'stackgap'
- which is very LWP unfriendly.
The additional code for non-emulation namei() is trivial, the reduction for
the emulations is massive.
The vnode for a processes emulation root is saved in the cwdi structure
during process exec.
If the emulation root the TRYEMULROOT flag are set, namei() will do an initial
search for absolute pathnames in the emulation root, if that fails it will
retry from the normal root.
".." at the emulation root will always go to the real root, even in the middle
of paths and when expanding symlinks.
Absolute symlinks found using absolute paths in the emulation root will be
relative to the emulation root (so /usr/lib/xxx.so -> /lib/xxx.so links
inside the emulation root don't need changing).
If the root of the emulation would be returned (for an emulation lookup), then
the real root is returned instead (matching the behaviour of emul_lookup,
but being a cheap comparison here) so that programs that scan "../.."
looking for the root dircetory don't loop forever.
The target for symbolic links is no longer mangled (it used to get the
CHECK_ALT_xxx() treatment, so could get /emul/xxx prepended).
CHECK_ALT_xxx() are no more. Most of the change is deleting them, and adding
TRYEMULROOT to the flags to NDINIT().
A lot of the emulation system call stubs could now be deleted.

branches: 1.37.2;
Update all the compat stuff to not use the 'stackgap' for processing
sys_stat() and friends, instead use do_sys_stat() and do_sys_fstat()
that write the answer into a kernel buffer (on stack) that can be
converted to the correct form and written the userspace.
I've test compiled a few kernels, and tested i386 netbsd1.6 ls.
Given I think I've fixed some bugs, it might be 50-50 with new ones.

branches: 1.36.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.35.2;
Merge newlock2 to head.

branches: 1.34.4; 1.34.8;
Use the LWP cached credentials where sane.

integrate kauth.

branches: 1.32.2; 1.32.4; 1.32.6;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.

Add initial (but unfinished) COMPAT_LINUX32 for amd64. This is good enough so
that the i386 license manager part of amd64 version of Fluent works.

While I'm here, add SysV IPC to COMPAT_LINUX/amd64

branches: 1.30.2; 1.30.4; 1.30.6;
merge ktrace-lwp.

64 bit inode changes.

branches: 1.28.2;
- sprinkle const.
- add XXXUNCONST to the emul_find() pbuf argument free'ing. XXX: this needs
an api change.
- avoid variable shadowing.

First work on COMPAT_LINUX/amd64
Process startup and dynamiclinking work, but processes hang due to
Linux arch_prctl(2) not being really supported yet.

PR/29696: Joel Carnat: NetBSD freezes when accessing smbfs mounted FS with
firefox/linux due to compat getdents() call assumption that all filesystems
support cookies.

branches: 1.25.2;
Add nanosecond handling on the stat and stat64 code for the i386.

nuke trailing whitespace

branches: 1.23.4; 1.23.6;
There's no need to pass a proc value when using UIO_SYSSPACE with
vn_rdwr(9) and uiomove(9).

OK'd by Jason Thorpe

add Linux compat arg wrapper for ftruncate64(), and change linux compat
truncate64() wrapper to translate args structure
NetBSD truncate() and ftrucate() have hidden 'pad' argument, so we have
to do the argument translation

Problem found and patch supplied in PR kern/22360 by Ales Krenek

This is the last of syscalls with hidden 'pad' arg we didn't have
wrapper for; all the others (lseek, mmap, pwrite, pread) already had
wrapper before.

branches: 1.21.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Merge the nathanw_sa branch.

branches: 1.18.2;
linux_sys_getdents64(): remove the oldcall bug workaround cruft,
it's not needed for this call, and update comment accordingly

linux_sys_getdents64(): make the d_off member point to next entry, rather
than current one; this is expected (even documented) behaviour of the
new system call
This fixes the endless loop when reading directories on NFS, though
applications won't see all directory entries due to different problem.
Fix by Matthias Scheler and Charles Hannum.

On archs with broken linux struct stat64 st_ino, rename linux_stat64's
lst_ino to __lst_ino and lst_ino64 to lst_ino, and define
LINUX_STAT64_HAS_BROKEN_ST_INO. Only assign __lst_ino if this define
is present.

This fixes compile on powerpc archs, and hence port-powerpc/16893.

branches: 1.15.2;
Make COMPAT_LINUX compile again on Alpha. linux_getdents64 moved but
files.linux_alpha wasn't updated. common/linux_file64.c contains a lot
of syscalls which aren't on Linux/alpha so #ifdef them for alpha.

Remove an unreachable "return error;".

hook in remaining 64bit fs syscalls
make most of stuff in linux_file64.c available unconditionally,
with exception of fcntl64, which is not present on Linux m68k

Remove check for size of field "d_off" in Linux "dirent" structure because
this problem cannot happen with the 64Bit version of the getdents().

Move implementation of linux_sys_getdents64() to source file for
Linux 64bit filesystem calls.

gc debugging printfs.

implement the locking calls for linux_file64.

make the stackgap_{init,alloc} functions MP friendly (i.e. pass struct proc *
in, instead of using curproc). While there add an optional size argument to
stackgap_init.

- add a real linux_sys___sysctl because the mib numbers are different.
implement only the kernel version getting entries.
- make the signal conversion functions consistent; dest is arg 1.
- make linux_fakedev deal with block and char devices differently.
- add linux_sys_ugetrlimit, and flesh out the regular get/setrlimit so
that they work properly.
- add linux_mmap2 [untested].
- bump kernel to 2.4.18, and make the date be valentine's day :-)
- linux_sys_*stat64 family was totally busted. Fix it.

tested only on i386.

Status:
gdb, telnet, work
jdk-1.4.0 extractor works, jdk still coredumps.

christos

add RCSIDs (including regeneration of files as appropriate)

Fixed a minor problem so that it builds

Fixes my last commit: only mips really need fcntl64 to work now. i386 and
powerpc could use it later if needed. m68k and alpha should never need it.

Added initial support for fcntl64 system call (file locking is not
implemented)

branches: 1.2.2; 1.2.4; 1.2.6; 1.2.8;
add linux_sys_truncate64()

add emulation of Linux truncate64(2), ftruncate64(2), *stat64(2)

Fix info leaks.

Fix info leak.

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

branches: 1.59.2; 1.59.4;
Fail, don't panic, on bad dirents from file system.

Controllable via puffs from userland.

From Ilja Van Sprundel.

branches: 1.58.6;
copy the terminating NUL (njoly)

branches: 1.57.2;
fix typo

convert strcpy to a bounded copy to avoid compiler warnings, although the
reclen test prevents overflow.

branches: 1.55.4; 1.55.6; 1.55.8; 1.55.10; 1.55.14;
Add struct linux_stat64 definition for alpha, and enable syscalls that
use it.

implement the *at() syscalls.
bring the unimplemented syscall list up to date.

branches: 1.53.2; 1.53.8; 1.53.12; 1.53.14; 1.53.16; 1.53.22;
Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock. Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.

Make linux_sys_getdents64 fails with ENOTDIR instead of EINVAL, when fd
does not refer to a directory.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

branches: 1.50.2;
When implementing "read directory", when there are too many empty entries
in a row, and we need to try to read the next block, and have passed a
non-NULL cookie pointer to VOP_READDIR, ensure that we free the cookie
buffer before re-doing VOP_READDIR, so that we don't leak memory.
This fix is similar to nfs_serv.c revisions 1.115 + 1.124.

This should fix the long-standing problem observed by e.g. using Linux-
emulated programs to take backup of servers, which is one of the problems
which were reported in PR#42661.

Thanks to pooka@ for the hints for traversing the VOP* layer.

branches: 1.49.2;
pad -> PAD

branches: 1.48.6; 1.48.10; 1.48.14;
Replace references to getsock/getvnode.

branches: 1.47.2; 1.47.4;
Remove clause 3 and 4 from TNF licenses

branches: 1.46.2; 1.46.4;
Catch up with descriptor handling changes. See kern_descrip.c revision
1.173 for details.

branches: 1.45.6;
Extract the fcntl64() suppoort from linux_file64.c into linux_fcntl64.c
Unravel some of the knots that caused linux_file64.c to be compiled twice
for an amd64 kernel (once for linux and once for linux32) with different
parts being skipped each time.

Convert all the system call entry points from:
int foo(struct lwp *l, void *v, register_t *retval)
to:
int foo(struct lwp *l, const struct foo_args *uap, register_t *retval)
Fixup compat code to not write into 'uap' and (in some cases) to actually
pass a correctly formatted 'uap' structure with the right name to the
next routine.
A few 'compat' routines that just call standard ones have been deleted.
All the 'compat' code compiles (along with the kernels required to test
build it).
98% done by automated scripts.

branches: 1.43.4;
ANSIfy most of the function definitions in sys/compat (but not ndis).
All by the magic of sed ...

Remove all the __P

branches: 1.41.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.40.2;
Add compat_linux and exec_linux_elf lkm support for amd64:
- Add needed COMPAT_OSSAUDIO to GENERIC.
- Add missing includes needed by linux_syscallargs.h.
- Add lkm building.

branches: 1.39.6; 1.39.8; 1.39.12;
Split the fcntl locking code out from its copyin/out.
Use to avoid all the stackgap stuff in compat code.

Change the way that emulations locate files within the emulation root to
avoid having to allocate space in the 'stackgap'
- which is very LWP unfriendly.
The additional code for non-emulation namei() is trivial, the reduction for
the emulations is massive.
The vnode for a processes emulation root is saved in the cwdi structure
during process exec.
If the emulation root the TRYEMULROOT flag are set, namei() will do an initial
search for absolute pathnames in the emulation root, if that fails it will
retry from the normal root.
".." at the emulation root will always go to the real root, even in the middle
of paths and when expanding symlinks.
Absolute symlinks found using absolute paths in the emulation root will be
relative to the emulation root (so /usr/lib/xxx.so -> /lib/xxx.so links
inside the emulation root don't need changing).
If the root of the emulation would be returned (for an emulation lookup), then
the real root is returned instead (matching the behaviour of emul_lookup,
but being a cheap comparison here) so that programs that scan "../.."
looking for the root dircetory don't loop forever.
The target for symbolic links is no longer mangled (it used to get the
CHECK_ALT_xxx() treatment, so could get /emul/xxx prepended).
CHECK_ALT_xxx() are no more. Most of the change is deleting them, and adding
TRYEMULROOT to the flags to NDINIT().
A lot of the emulation system call stubs could now be deleted.

branches: 1.37.2;
Update all the compat stuff to not use the 'stackgap' for processing
sys_stat() and friends, instead use do_sys_stat() and do_sys_fstat()
that write the answer into a kernel buffer (on stack) that can be
converted to the correct form and written the userspace.
I've test compiled a few kernels, and tested i386 netbsd1.6 ls.
Given I think I've fixed some bugs, it might be 50-50 with new ones.

branches: 1.36.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.35.2;
Merge newlock2 to head.

branches: 1.34.4; 1.34.8;
Use the LWP cached credentials where sane.

integrate kauth.

branches: 1.32.2; 1.32.4; 1.32.6;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.

Add initial (but unfinished) COMPAT_LINUX32 for amd64. This is good enough so
that the i386 license manager part of amd64 version of Fluent works.

While I'm here, add SysV IPC to COMPAT_LINUX/amd64

branches: 1.30.2; 1.30.4; 1.30.6;
merge ktrace-lwp.

64 bit inode changes.

branches: 1.28.2;
- sprinkle const.
- add XXXUNCONST to the emul_find() pbuf argument free'ing. XXX: this needs
an api change.
- avoid variable shadowing.

First work on COMPAT_LINUX/amd64
Process startup and dynamiclinking work, but processes hang due to
Linux arch_prctl(2) not being really supported yet.

PR/29696: Joel Carnat: NetBSD freezes when accessing smbfs mounted FS with
firefox/linux due to compat getdents() call assumption that all filesystems
support cookies.

branches: 1.25.2;
Add nanosecond handling on the stat and stat64 code for the i386.

nuke trailing whitespace

branches: 1.23.4; 1.23.6;
There's no need to pass a proc value when using UIO_SYSSPACE with
vn_rdwr(9) and uiomove(9).

OK'd by Jason Thorpe

add Linux compat arg wrapper for ftruncate64(), and change linux compat
truncate64() wrapper to translate args structure
NetBSD truncate() and ftrucate() have hidden 'pad' argument, so we have
to do the argument translation

Problem found and patch supplied in PR kern/22360 by Ales Krenek

This is the last of syscalls with hidden 'pad' arg we didn't have
wrapper for; all the others (lseek, mmap, pwrite, pread) already had
wrapper before.

branches: 1.21.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Merge the nathanw_sa branch.

branches: 1.18.2;
linux_sys_getdents64(): remove the oldcall bug workaround cruft,
it's not needed for this call, and update comment accordingly

linux_sys_getdents64(): make the d_off member point to next entry, rather
than current one; this is expected (even documented) behaviour of the
new system call
This fixes the endless loop when reading directories on NFS, though
applications won't see all directory entries due to different problem.
Fix by Matthias Scheler and Charles Hannum.

On archs with broken linux struct stat64 st_ino, rename linux_stat64's
lst_ino to __lst_ino and lst_ino64 to lst_ino, and define
LINUX_STAT64_HAS_BROKEN_ST_INO. Only assign __lst_ino if this define
is present.

This fixes compile on powerpc archs, and hence port-powerpc/16893.

branches: 1.15.2;
Make COMPAT_LINUX compile again on Alpha. linux_getdents64 moved but
files.linux_alpha wasn't updated. common/linux_file64.c contains a lot
of syscalls which aren't on Linux/alpha so #ifdef them for alpha.

Remove an unreachable "return error;".

hook in remaining 64bit fs syscalls
make most of stuff in linux_file64.c available unconditionally,
with exception of fcntl64, which is not present on Linux m68k

Remove check for size of field "d_off" in Linux "dirent" structure because
this problem cannot happen with the 64Bit version of the getdents().

Move implementation of linux_sys_getdents64() to source file for
Linux 64bit filesystem calls.

gc debugging printfs.

implement the locking calls for linux_file64.

make the stackgap_{init,alloc} functions MP friendly (i.e. pass struct proc *
in, instead of using curproc). While there add an optional size argument to
stackgap_init.

- add a real linux_sys___sysctl because the mib numbers are different.
implement only the kernel version getting entries.
- make the signal conversion functions consistent; dest is arg 1.
- make linux_fakedev deal with block and char devices differently.
- add linux_sys_ugetrlimit, and flesh out the regular get/setrlimit so
that they work properly.
- add linux_mmap2 [untested].
- bump kernel to 2.4.18, and make the date be valentine's day :-)
- linux_sys_*stat64 family was totally busted. Fix it.

tested only on i386.

Status:
gdb, telnet, work
jdk-1.4.0 extractor works, jdk still coredumps.

christos

add RCSIDs (including regeneration of files as appropriate)

Fixed a minor problem so that it builds

Fixes my last commit: only mips really need fcntl64 to work now. i386 and
powerpc could use it later if needed. m68k and alpha should never need it.

Added initial support for fcntl64 system call (file locking is not
implemented)

branches: 1.2.2; 1.2.4; 1.2.6; 1.2.8;
add linux_sys_truncate64()

add emulation of Linux truncate64(2), ftruncate64(2), *stat64(2)

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

branches: 1.59.2;
Fail, don't panic, on bad dirents from file system.

Controllable via puffs from userland.

From Ilja Van Sprundel.

branches: 1.58.6;
copy the terminating NUL (njoly)

branches: 1.57.2;
fix typo

convert strcpy to a bounded copy to avoid compiler warnings, although the
reclen test prevents overflow.

branches: 1.55.4; 1.55.6; 1.55.8; 1.55.10; 1.55.14;
Add struct linux_stat64 definition for alpha, and enable syscalls that
use it.

implement the *at() syscalls.
bring the unimplemented syscall list up to date.

branches: 1.53.2; 1.53.8; 1.53.12; 1.53.14; 1.53.16; 1.53.22;
Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock. Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.

Make linux_sys_getdents64 fails with ENOTDIR instead of EINVAL, when fd
does not refer to a directory.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

branches: 1.50.2;
When implementing "read directory", when there are too many empty entries
in a row, and we need to try to read the next block, and have passed a
non-NULL cookie pointer to VOP_READDIR, ensure that we free the cookie
buffer before re-doing VOP_READDIR, so that we don't leak memory.
This fix is similar to nfs_serv.c revisions 1.115 + 1.124.

This should fix the long-standing problem observed by e.g. using Linux-
emulated programs to take backup of servers, which is one of the problems
which were reported in PR#42661.

Thanks to pooka@ for the hints for traversing the VOP* layer.

branches: 1.49.2;
pad -> PAD

branches: 1.48.6; 1.48.10; 1.48.14;
Replace references to getsock/getvnode.

branches: 1.47.2; 1.47.4;
Remove clause 3 and 4 from TNF licenses

branches: 1.46.2; 1.46.4;
Catch up with descriptor handling changes. See kern_descrip.c revision
1.173 for details.

branches: 1.45.6;
Extract the fcntl64() suppoort from linux_file64.c into linux_fcntl64.c
Unravel some of the knots that caused linux_file64.c to be compiled twice
for an amd64 kernel (once for linux and once for linux32) with different
parts being skipped each time.

Convert all the system call entry points from:
int foo(struct lwp *l, void *v, register_t *retval)
to:
int foo(struct lwp *l, const struct foo_args *uap, register_t *retval)
Fixup compat code to not write into 'uap' and (in some cases) to actually
pass a correctly formatted 'uap' structure with the right name to the
next routine.
A few 'compat' routines that just call standard ones have been deleted.
All the 'compat' code compiles (along with the kernels required to test
build it).
98% done by automated scripts.

branches: 1.43.4;
ANSIfy most of the function definitions in sys/compat (but not ndis).
All by the magic of sed ...

Remove all the __P

branches: 1.41.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.40.2;
Add compat_linux and exec_linux_elf lkm support for amd64:
- Add needed COMPAT_OSSAUDIO to GENERIC.
- Add missing includes needed by linux_syscallargs.h.
- Add lkm building.

branches: 1.39.6; 1.39.8; 1.39.12;
Split the fcntl locking code out from its copyin/out.
Use to avoid all the stackgap stuff in compat code.

Change the way that emulations locate files within the emulation root to
avoid having to allocate space in the 'stackgap'
- which is very LWP unfriendly.
The additional code for non-emulation namei() is trivial, the reduction for
the emulations is massive.
The vnode for a processes emulation root is saved in the cwdi structure
during process exec.
If the emulation root the TRYEMULROOT flag are set, namei() will do an initial
search for absolute pathnames in the emulation root, if that fails it will
retry from the normal root.
".." at the emulation root will always go to the real root, even in the middle
of paths and when expanding symlinks.
Absolute symlinks found using absolute paths in the emulation root will be
relative to the emulation root (so /usr/lib/xxx.so -> /lib/xxx.so links
inside the emulation root don't need changing).
If the root of the emulation would be returned (for an emulation lookup), then
the real root is returned instead (matching the behaviour of emul_lookup,
but being a cheap comparison here) so that programs that scan "../.."
looking for the root dircetory don't loop forever.
The target for symbolic links is no longer mangled (it used to get the
CHECK_ALT_xxx() treatment, so could get /emul/xxx prepended).
CHECK_ALT_xxx() are no more. Most of the change is deleting them, and adding
TRYEMULROOT to the flags to NDINIT().
A lot of the emulation system call stubs could now be deleted.

branches: 1.37.2;
Update all the compat stuff to not use the 'stackgap' for processing
sys_stat() and friends, instead use do_sys_stat() and do_sys_fstat()
that write the answer into a kernel buffer (on stack) that can be
converted to the correct form and written the userspace.
I've test compiled a few kernels, and tested i386 netbsd1.6 ls.
Given I think I've fixed some bugs, it might be 50-50 with new ones.

branches: 1.36.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.35.2;
Merge newlock2 to head.

branches: 1.34.4; 1.34.8;
Use the LWP cached credentials where sane.

integrate kauth.

branches: 1.32.2; 1.32.4; 1.32.6;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.

Add initial (but unfinished) COMPAT_LINUX32 for amd64. This is good enough so
that the i386 license manager part of amd64 version of Fluent works.

While I'm here, add SysV IPC to COMPAT_LINUX/amd64

branches: 1.30.2; 1.30.4; 1.30.6;
merge ktrace-lwp.

64 bit inode changes.

branches: 1.28.2;
- sprinkle const.
- add XXXUNCONST to the emul_find() pbuf argument free'ing. XXX: this needs
an api change.
- avoid variable shadowing.

First work on COMPAT_LINUX/amd64
Process startup and dynamiclinking work, but processes hang due to
Linux arch_prctl(2) not being really supported yet.

PR/29696: Joel Carnat: NetBSD freezes when accessing smbfs mounted FS with
firefox/linux due to compat getdents() call assumption that all filesystems
support cookies.

branches: 1.25.2;
Add nanosecond handling on the stat and stat64 code for the i386.

nuke trailing whitespace

branches: 1.23.4; 1.23.6;
There's no need to pass a proc value when using UIO_SYSSPACE with
vn_rdwr(9) and uiomove(9).

OK'd by Jason Thorpe

add Linux compat arg wrapper for ftruncate64(), and change linux compat
truncate64() wrapper to translate args structure
NetBSD truncate() and ftrucate() have hidden 'pad' argument, so we have
to do the argument translation

Problem found and patch supplied in PR kern/22360 by Ales Krenek

This is the last of syscalls with hidden 'pad' arg we didn't have
wrapper for; all the others (lseek, mmap, pwrite, pread) already had
wrapper before.

branches: 1.21.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Merge the nathanw_sa branch.

branches: 1.18.2;
linux_sys_getdents64(): remove the oldcall bug workaround cruft,
it's not needed for this call, and update comment accordingly

linux_sys_getdents64(): make the d_off member point to next entry, rather
than current one; this is expected (even documented) behaviour of the
new system call
This fixes the endless loop when reading directories on NFS, though
applications won't see all directory entries due to different problem.
Fix by Matthias Scheler and Charles Hannum.

On archs with broken linux struct stat64 st_ino, rename linux_stat64's
lst_ino to __lst_ino and lst_ino64 to lst_ino, and define
LINUX_STAT64_HAS_BROKEN_ST_INO. Only assign __lst_ino if this define
is present.

This fixes compile on powerpc archs, and hence port-powerpc/16893.

branches: 1.15.2;
Make COMPAT_LINUX compile again on Alpha. linux_getdents64 moved but
files.linux_alpha wasn't updated. common/linux_file64.c contains a lot
of syscalls which aren't on Linux/alpha so #ifdef them for alpha.

Remove an unreachable "return error;".

hook in remaining 64bit fs syscalls
make most of stuff in linux_file64.c available unconditionally,
with exception of fcntl64, which is not present on Linux m68k

Remove check for size of field "d_off" in Linux "dirent" structure because
this problem cannot happen with the 64Bit version of the getdents().

Move implementation of linux_sys_getdents64() to source file for
Linux 64bit filesystem calls.

gc debugging printfs.

implement the locking calls for linux_file64.

make the stackgap_{init,alloc} functions MP friendly (i.e. pass struct proc *
in, instead of using curproc). While there add an optional size argument to
stackgap_init.

- add a real linux_sys___sysctl because the mib numbers are different.
implement only the kernel version getting entries.
- make the signal conversion functions consistent; dest is arg 1.
- make linux_fakedev deal with block and char devices differently.
- add linux_sys_ugetrlimit, and flesh out the regular get/setrlimit so
that they work properly.
- add linux_mmap2 [untested].
- bump kernel to 2.4.18, and make the date be valentine's day :-)
- linux_sys_*stat64 family was totally busted. Fix it.

tested only on i386.

Status:
gdb, telnet, work
jdk-1.4.0 extractor works, jdk still coredumps.

christos

add RCSIDs (including regeneration of files as appropriate)

Fixed a minor problem so that it builds

Fixes my last commit: only mips really need fcntl64 to work now. i386 and
powerpc could use it later if needed. m68k and alpha should never need it.

Added initial support for fcntl64 system call (file locking is not
implemented)