MFC r289172,r290254:

r289172:

Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

r290254:

Remove unused variable (SRCDIR)

MFC: r285552
Use strtonum(3) instead of atoi(3)

MFC: r286289
Introduce -P0, creating as many concurrent processes as possible

Relnotes: yes

MFC: r286461
Fix regression in xargs -Px, add more regression tests

MFC: r287004
MFC: r287005
Fix the racy xargs -P0 -n2 test added in r286289

PR: 199976
Submitted by: Nikolai Lifanov <lifanov@mail.lifanov.com>

MFC various moves of tools/regressions/ tests to the new infrastructure.

- r263220 Migrate tools/regression/sbin/ to the new tests layout.
- r263222 Add Makefile missed in r263220.
- r263226 Migrate tools/regression/{usr.bin/lastcomm,usr.sbin}/ to the new tests layout.
- r263227 Migrate most of tools/regression/usr.bin/ to the new tests layout.
- r263345 Expand tabs that sneaked in into spaces.
- r263346 Migrate tools/regression/usr.bin/make/ to the new tests layout.
- r263348 Add Makefiles missed in r263346.
- r263351 Migrate tools/regression/usr.bin/pkill/ to the new tests layout.
- r263388 Mark multi_test as requiring /usr/share/dict/words.
- r263814 Fix path to the run.pl script to let these tests run.
- r264742 Prevent building tests when bootstrapping make.

This is 'make tinderbox' clean.

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Style(9) changes fo xargs.c

Follow calloc convention in other code, this is functionally identical
to its previous form.

xargs: Before exiting, wait for all invocations of the utility.

This only has an effect with -P, otherwise errors are only detected when the
utility is not running.

Submitted by: Matthew Story

xargs: Remove an unclear comment that only tried to repeat what the code did

Reported by: bde

xargs: Fix comma splice in error message.

Reported by: bde

xargs: If a utility exits with 255 or a signal, write an error message.

If a utility called by xargs exits with status 255 or because of a signal,
POSIX requires writing an error message.

PR: 165155
Submitted by: Matthew Story matthewstory gmail com

Discourage from using "cp -r".

Remove the advertising clause from UCB copyrighted files in usr.bin. This
is in accordance with the information provided at
ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change

Also add $FreeBSD$ to a few files to keep svn happy.

Discussed with: imp, rwatson

Various syntactic tweaks to satisfy style(9). No change to execution.

Submitted by: gcooper@

xargs can be fooled by exiting children that it did not start, causing
it to kick off a new command before the previous has finished, resulting
in corrupted (interleaved) output. It is also fooled by non-exiting
children it did not start, failing to exit until all extraneous children
have exited.

This patch makes xargs keep track of children it starts, ignoring
pre-existing ones.

Build usr.bin/ with WARNS=6 by default.

Also add some missing $FreeBSD$ to keep svn happy.

Revive '-opt' flags which I accidentally removed.

Noticed by: simon

Document the no-op -r option of BSD xargs(1).

PR: docs/106416
Submitted by: Pete Slagle, freebsd-stable at voidcaptain.com
MFC after: 3 days

Markup fixes.

NUL terminate buffer from fgetln(3). From fgsch@OpenBSD.

Whitespace after switch.

Inspired by: OpenBSD

Check the entire length of the current argument for the EOF string with -E,
but don't expect a proper ASCII string to exist right here right now, don't
use strcmp(3) which checks for a NUL. As we're still building the argument
up, the next character might be garbage. It would probably be just as safe to
temporarily write a NUL there, but if we've reached the end of argument memory
that might not be the best idea, I think. It's unclear.

Doing it this way seems to meet the most with the original intent.

PR: 85696
Prodded by: stefanf

Add a FreeBSD-specific -S flag which controls the maximum size of an argument
having replacements done in it via -I.

Handle maxsize==0 in such a way that we don't modify the string.

Wording tweaks.

PR: not quite docs/84620
MFC after: 2 weeks

Add -r option for GNU compatibility.

MFC after: 2 weeks

Sync program's usage() with manpage's SYNOPSIS.

MFC 1.22: Fix -0 vs -L/-I processing, mainly so that 'xargs -0 -I []' will
do something sensible (namely: treat then '\0' as the EOL character, when
deciding what "a line" is for -N). Note that -I implies -N.

MFC after: 3 days

Added the EXIT STATUS section where appropriate.

Modify behaviour of `xargs -I` in order to:
1. Conform to IEEE Std 1003.1-2004, which state that "Constructed
arguments cannot grow larger than 255 bytes", and
2. Avoid a buffer overflow.

Unfortunately the standard doesn't indicate how xargs is supposed to
handle arguments which (with the appropriate substitutions) would grow
larger than 255 bytes; this solution handles those by making as many
substitutions as possible without overflowing the buffer.

OpenBSD's xargs resolves this in a different direction, by making
all the substitutions and then silently truncating the resulting string.

Since this change may break existing scripts which rely upon the buffer
overflow (255 bytes isn't really all that long...) it will not be MFCed.

Document incorrect handling of multibyte characters with -I and -J options.

Call setlocale() with category LC_ALL instead of LC_MESSAGES. We need
LC_CTYPE and LC_COLLATE to correctly interpret regular expressions
returned by nl_langinfo(YESEXPR), and it doesn't hurt to include the
rest.

Fix spelling error in my own paragraph.

- Use _PATH_TTY and _PATH_DEVNULL macros.
- Don't fail if we can't open /dev/null since this can happen if
xargs is jail'ed or chroot'ed.

These fixes were submitted by Todd Miller from the OpenBSD project.
There was one problem in those fixes that broke -o, which is corrected
here and should be committed to the OpenBSD repo by Todd soon.

MFC in: 3 days

Use waitpid, instead of wait3, which is more portable.

Submitted by: "Todd C. Miller" <Todd.Miller@courtesan.com>
Obtained from: OpenBSD
Reviewed by: tjr

Remove redundant return; from void function.

Submitted by: "Todd C. Miller" <Todd.Miller@courtesan.com>
Obtained from: OpenBSD

Markup bits.

Approved by: re (blanket)

Don't shaddow exp(3).

Migrate to a new way of dealing with building from old revisions of
FreeBSD. This method attempts to centralize all the necessary hacks
or work arounds in one of two places in the tree (src/Makefile.inc1
and src/tools/build). We build a small compatibility library
(libbuild.a) as well as selectively installing necessary include
files. We then include this directory when building host binaries.

This removes all the past release compatibilty hacks from various
places in the tree. We still build on tip of stable and current. I
will work with those that want to support more, although I anticipate
it will just work.

Many thanks to ru@, obrien@ and jhb@ for providing valuable input at
various stage of implementation, as well as for working together to
positively effect a change for the better.

Simplify compatibility ifdef.

xargs(1) is WARNS=6 clean.

Tested on: i386, sparc64

When -o isn't specified, open /dev/null as stdin.

Suggested by: jhb

Add a new -o option to tell xargs(1) to reopen /dev/tty as stdin in
the child process, before executing the command. This is very useful
when you do stuff like ``find ... | xargs interactive_application''.
Without -o, the application would inherit the pipe as its stdin, and
you thus lose any control over it.

This flag has been carefully chosen to not conflit with other options
of other xargs utilities like GNU xargs.

Reviewed by: jmallett

Extend our -R extension which sets the number of arguments in which -I will
replace to support magic values less than zero, which mean to just go nuts
and expand as many as we want.

MFC after: 2 weeks

De-typo usage string.

A few changes for clarity.

PR: 47170
Submitted by: Gary W. Swearingen <swear@attbi.com> (original version)

english(4) police.

mdoc(7) police: most -mdoc macros can take multiple arguments.

Add the -P option which executes multiple copies of the specified utility
in parallel. Idea from GNU xargs.

Take __FreeBSD_version into account when BOOTSTRAPPING.

Fix typos; each file has at least one s/seperat/separat/
(I skipped those in contrib/, gnu/ and crypto/)
While I was at it, fixed a lot more found by ispell that I
could identify with certainty to be errors. All of these
were in comments or text, not in actual code.

Suggested by: bde
MFC after: 3 days

Declare environ as char **environ like in environ(7), not char *environ[].
This corrects a problem whereby xargs could not walk the environment table
to count the amount of space it used, and treated it as if it were empty.
This problem was introduced in rev 1.15.

MFC after: 2 days

Stylistic nit:

main()'s argv argument is char*[], and functions that may inherit that arg
should use char*[] as well.

Check for results of repeated calls to strnsubst(), as well as for the
behaviour with NULL match string, as that has changed over time.

When giving an example that relies on shell expansion/globbing, don't use a
replstr for -J that will be interpereted by the shell.

MFC after: 1 day

Note that this appeared at least as early as PWB UNIX.

Use the literal string 'PWB UNIX', as we still have no .At macro for it.

Err, duh, free(3) doesn't set its argument to NULL... Fix realloc of a freed
chunk.

Restructure sanity checks so that -I and -J just override eachother, rather
than triggering a usage(). Allow -R and -I to be specified in any order, and
thus change how -R checks for -I not being given and triggering a usage().

Partially requested by: gad

Allow the input line to be NULL, and teach strnsubst() that NULL means to use
a nil-string.

mdoc(7) police: punctuation, miscellaneous.

Simplify prerun() in the case where there are no arguments. Can't do -I or -J
substitution if `utility' is not specified and we're using the buildin echo;
It has no arguments for us to abuse.

Fix a "tiny and squeeby and little" markup mistake.

Submitted by: fenner

xargs.c,v 1.33 broke the upgrade path from old versions of 4.x and 5.0.

-I and -J both set the same variable, replstr, to the string to replace with
a line of input, and both work differently, so prevent them from both being
passed to xargs(1).

Modify the -p implementation to use a user's locale, so they can respond to
the prompt in their native language.

Also make the prompt fit what POSIX asks for (?...).

This should not affect use of -p with yes(1) [as every locale I know of matches
'y' as YESEXPR as well], but that's what -t is for anyway. -p is meant to be
really used interactively.

Submitted by: tjr, jmallett

Fix a typo.

Submitted by: Carl Schmidt <cschmidt@slackerbsd.org>

Wrap an obscenely long line while I'm here.

In an effort to make this utility easier to work with at a source level, move
out the parse loop to a seperate function, and move local variables around as
is needed. To keep the state of some variables and keep from having huge arg
lists to functions, make a bunch global.

Change use of err(3) in situations where malloc(3) will be setting errno to
errx(3) since assuming malloc(3) sets errno is non-portable, and provides no
more useful information in the context of FreeBSD.

Submitted by: bde (err/errx), Carl Schmidt <cschmidt@slackerbsd.org> (some
of the movement of the input loop to a function)
Approved by: src/tools/regression/usr.bin/xargs

Grouping changes.

Submitted by: Carl Schmidt <cschmidt@slackerbsd.org>

Don't use pointers as booleans.

Replace calloc(3) with malloc(3), and set the tail of the
argument list to NULL, to terminate the arguments passed
to execvp(2).

Thanks to: bde (for pointing out some missing parens)

And with apologies to Ozzy Osbourne:

On his way to dinner
It took him by surprise
When an email from bde
Said his code was full of lies

Style (indentation)
And his bracing were all wrong
He casted to size_t
When he should have cast to long

I don't mind

Single letter identifiers
Unwrapped Line
Over 80 chars
Far over 80 chars

Who can we get to send diffs
We need Bruce D. Evans
Shows you comparason to his
We need Bruce D. Evans again

Fix a bug whereby we were getting ~0 and comparing it to maxsize, i.e. if
s1 was 0 length, and replstr was 0 length, etc., we would end up subtracting
one from zero and seeing if it was greater than the size_t (unsigned) var
maxsize... This would cause us to return a string consisting of essentially
only match, which is not the right behaviour if we have 0 length inpline.

Move substitution out to a function, prerun(), and replace use of run() with
it. It handles everything (right now) that needs done before run(), except
the -J case, because while that would be easy, I don't quite understand -J.

Reviewed by: src/tools/regression/usr.bin/xargs

Get rid of bogus holder for realloc(3).

Rewrite the loop that handles substitution in the -I case, and try to make
it easier to understand. Making it easy to understand isn't all that easy,
so litter the code with comments so some other poor soul can come along some
day and work on this if they see fit to do so. Avoid calling strlen(3) to
check for a nil-string, when we can just check for *str=='\0'.

Approved by: src/tools/regression/usr.bin/xargs/

Rename `lflag' to `Lflag' to match the option's case.

Revert revision 1.22. I incorrectly modified the Berkeley identifiers.

Requested by: bde, mike

__COPYRIGHT() and __SCCSID().

An explicit cast to size_t for an inline integer, since the two are not the
same on Alpha and lint(1) pointed that out.

lint(1) on the same architecture pointed out how silly a cast to (u_int) to
malloc(3)'s argument was. Change that to size_t.

A markup fix, and document -R as non-standard.

Make -J a proper flag internal to the code (rather than just check for
use of replstr and lack of Iflag), and add -R, which when given with
-I controls the number of arguments on which replacement will be done.

Some people happen to think it's idiotic to limit to 5 arguments, so
let the user override it if they like.

*str is spelled 's1' inside the body of the code.

Make 'this' not a local variable, since it isn't necessary or complex enough
to warrant such.

Rework some of the -I support so I can't find a way to make xargs(1) core or
corrupt memory. Simplifies the code in one or two places, also removes some
code that looks like it was bogus or incomplete.

Update strnsubst to have one or two extra conditions which maybe would make
it more efficient, or at least more versatile. This is likely a no-op.

In the case where we can't open /dev/tty, fall back to -t behaviour.

Submitted by: fenner

Bump WARNS to 4.

Compound two fprintf(3)s into 1 using string concatenation.

After 3 months...

Merge xargs(1) with that of xMach.

Bring in xargs(1) changes to add -L and -I as per the Single Unix Specification
version 3. Proper exit status numbers are implemented, and the manual page has
been updated to reflect reality.

The code has been ANSIfied, and a new file has been added to xargs(1) to do the
substring substitution as SUSv3 requires.

Traditional behaviour should not be affected, use of -J should be deprecated
in favor of the more portable -I (though -J has been left, for now).

Submitted by: me, tjr (the exit status stuff)
Obtained from: xMach

ANSIfy: Function declarations and prototypes, use of environ(7).

Reviewed by: mike
Approved by: mike

Change back to using vfork() now that execvp() is vfork()-safe. If execvp()
fails, errno is saved to a volatile variable that the parent later inspects.

PR: bin/34898
Submitted by: Tim J. Robbins <tim@robbins.dropbear.id.au>
MFC after: 1 week

Leave the `copyright' variable exposed, since the main purpose of it
is to put a copyright in the object file.

Submitted by: bde

Fix vendor ID (mostly obtained from Rev 1.1).

WARNS=2 fixes, remove register keyword, use __FBSDID()

Spelling police: extention -> extension.

Add a -J replstr option that allows the user to tell xargs to insert
the data read from standard input at a specific point in the command
line arguments rather than at the end.

Submitted by: dd, gad
Reviewed by: gad, brian

Alphabetize command-line options in the synopsis.

mdoc(7) police: Fixed typo and markup in rev.1.11.

o The -s limit is ARG_MAX - 4K, not ARG_MAX - 2K.
o Mention that the current environment is part of the -s calculation.
o Add a BUGS section that warns against executing a program that increases
the size of the argument list or the size of the environment.

I have wondered for a while what the difference is between

get a big list | xargs sudo command

which fails and

get a big list | sudo xargs command

which succeeds. The answer is that in the first case, sudo expands
the environment and pushes the amount of data passed into execve over
the E2BIG threshold.

Prepare for mdoc(7)NG.

Prepare for mdoc(7)NG.

mdoc(7) police: use the new features of the Nm macro.

Use .Xr

$Id$ -> $FreeBSD$

Add $Id$, to make it simpler for members of the translation teams to
track.

The $Id$ line is normally at the bottom of the main comment block in the
man page, separated from the rest of the manpage by an empty comment,
like so;

.\" $Id$
.\"

If the immediately preceding comment is a @(#) format ID marker than the
the $Id$ will line up underneath it with no intervening blank lines.
Otherwise, an additional blank line is inserted.

Approved by: bde

xargs was spliting the input in a very different way from everyone else,
and was also a bit inconsistent: leading blanks, or any double blanks
generated empty arguments, but a trailing blank did not.

PR: bin/2630, bin/10914
Submitted by: Arne Henrik Juul <arnej@imf.unit.no>

Calls one or more of malloc(), warn(), err(), syslog(), execlp() or
execvp() in the child branch of a vfork(). Changed to use fork()
instead.

Some of these (mv, find, apply, xargs) might benefit greatly from
being rewritten to use vfork() properly.

PR: Loosely related to bin/8252
Approved by: jkh and bde

Remove compile time dependency on ARG_MAX.

This fix only removes the dependency on compile time constants. The code
has other (old) problems that need to be addressed.

PR: 1791
Reviewed-by: bde, tegge

Submitted by: Peter Hawkins <peter@clari.net.au>
Spelling police.

Use err(3) instead of local redefinition.

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

Reviewed by: Warner Losh <imp@village.org>
Submitted by: Marc Slemko <marcs@znep.com>
Obtained from: OpenBSD

Add -0 for reading the results of find -0.

Make xargs take into account the bytes occupied by the environment.

Original by: peter

This commit was generated by cvs2svn to compensate for changes in r1590,
which included commits to RCS files with non-trunk default branches.