Add libxo support to du

Convert du to use libxo enabling structured output.

[[ minor style fixes by imp ]]

Signed-off-by: Nathan Huff <nhuff@acm.org>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1145

Remove copyright strings ifdef'd out

We've ifdef'd out the copyright strings for some time now. Go ahead and
remove the ifdefs. Plus whatever other detritis was left over from other
recent removals. These copyright strings are present in the comments and
are largely from CSRG's attempt at adding their copyright to every
binary file (which modern interpretations of the license doesn't
require).

Sponsored by: Netflix

usr.bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

fts_read: Handle error from a NULL return better.

This is addressing cases such as fts_read(3) encountering an [EIO]
from fchdir(2) when FTS_NOCHDIR is not set. That would otherwise be
seen as a successful traversal in some of these cases while silently
discarding expected work.

As noted in r264201, fts_read() does not set errno to 0 on a successful
EOF so it needs to be set before calling it. Otherwise we might see
a random error from one of the iterations.

gzip is ignoring most errors and could be improved separately.

Reviewed by: vangyzen
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D27184

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

du(1): Add --si option to display in terms of powers of 1000

Reviewed by: cem (earlier version), emaste
Approved by: emaste (mentor)
Differential Revision: https://reviews.freebsd.org/D11748

du(1): trivial whitespace cleanup

MFC after: 1 month

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

Fix improper use of "its".

Sponsored by: Dell EMC Isilon

du(1): replace malloc + memset with calloc.

Document -g option in the usage string.

Add -g (gigabyte) flag to complement -k (kilobyte) and -m (megabyte).

MFC after: 1 week

du: Allow multiple -HLP options, the last one wins.

This matches 4.4BSD tradition and other utilities with these options and is
required by POSIX (POSIX does not specify -P, only -HL).

MFC after: 2 weeks

Update du(1):

- Sort arguments alphabetically where appropriate
- '-B blocksize' is not mutually exclusive of '-h|-k|-m'
- Mention '-t' in synopsis
- Other wording improvements
- Update usage() output to reflect the new synopsis [1]
- Other miscellaneous improvements

PR: 162438
Submitted by: arundel
Reviewed by: Benjamin Kaduk (kaduk ! mit.edu), jhb[1] (original version)
MFC after: 1 week

Simplify getopt switch parsing.

Only one of these flags can be set. Just add them together and check the
value. Also, get rid of the listall variable. The code is already filled
with direct *flag-comparisons.

Obtained from: Alexander Best (though modified)

Add missing static keywords to du(1)

Remove the advertising clause from UCB copyrighted files in usr.bin. This
is in accordance with the information provided at
ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change

Also add $FreeBSD$ to a few files to keep svn happy.

Discussed with: imp, rwatson

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Add a -t switch for masking output that's above or below certain thresholds.
This switch makes it a lot easier to locate problem areas when a process
is threatening to consume all of your disk space.

PR: 144192
Submitted by: gk
MFC after: 3 weeks

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Report the next directory being scanned when a ^T is pressed (or any
SIGINFO). Provides some progress report for the impatient. This
won't report that we're blocking in our walk due to disk/network
problems, however. There's no really good way to report that
condition that I'm aware of...

Restore (intmax_t) casts I lost during the last change & unbreak the build.

Add two new options to du(1):
-A Display the apparent size instead of the disk usage. This can be
helpful when operating on compressed volumes or sparse files.

-B blocksize
Calculate block counts in blocksize byte blocks. This is differ-
ent from the -k, -m options or setting BLOCKSIZE and gives an
estimate of how much space the examined file hierachy would
require on a filesystem with the given blocksize. Unless in -A
mode, blocksize is rounded up to the next multiple of 512.

The former is similar to GNU's du(1) --apparent-size. The latter is
different from what GNU's du(1) -B does, which is equivalent to setting
BLOCKSIZE in our implementation and is rather pointless as it doesn't add
any real value (i.e. you can achieve the same with a simple awk-script).

No change in the normal output or processing.

Reviewed by: keramida@, Peter French
Otherwise silience from: freebsd-hackers@

Declare functions and variables static and save a few byte. This is a
style(9)-change, too. Separate commit as it changes the object.

style(9): mostly avoiding line wrap by not indenting cases. No obj change.

- Import the HEAD csup code which is the basis for the cvsmode work.

Implement from scratch a -l option for du(1), to match the same option
of the GNU utility. The default behavior of our original `du' is to
count hardlinked files only once for each invocation of the utility.
With the new -l option they count towards the final size every time
they are found.

PR: bin/117944
Submitted by: keramida
Reviewed by: des, obrien
MFC after: 2 weeks

setenv(3) sets errno on failure, use warn() instead

Pointed out by: ru

Check return value for setenv()

Significantly reduce the memory leak as noted in BUGS section for
setenv(3) by tracking the size of the memory allocated instead of using
strlen() on the current value.

Convert all calls to POSIX from historic BSD API:
- unsetenv returns an int.
- putenv takes a char * instead of const char *.
- putenv no longer makes a copy of the input string.
- errno is set appropriately for POSIX. Exceptions involve bad environ
variable and internal initialization code. These both set errno to
EFAULT.

Several patches to base utilities to handle the POSIX changes from
Andrey Chernov's previous commit. A few I re-wrote to use setenv()
instead of putenv().

New regression module for tools/regression/environ to test these
functions. It also can be used to test the performance.

Bump __FreeBSD_version to 700050 due to API change.

PR: kern/99826
Approved by: wes
Approved by: re (kensmith)

Back out all POSIXified *env() changes.

Not because I admit they are technically wrong and not because of bug
reports (I receive nothing). But because I surprisingly meets so
strong opposition and resistance so lost any desire to continue that.

Anyone who interested in POSIX can dig out what changes and how
through cvs diffs.

Eliminate error with -W* strict flags and make putenv() calls conforming to
standard and portable in the same way as f.e. gcc internal portable code does.

o Implement "-n" flag: ignore files and directories with user "nodump"
flag set. Useful for calculation dump -h dump size.

PR: bin/96864
Submitted by: Dmitry Kazarov
Obtained from: NetBSD (man page)
MFC after: 1 month

Remove unused variables.

Use 64bit 'fts_bignum' field instead of 32bit (on 32bit archs) field
'fts_number' to remember number of blocks.
This makes du(1) 64bit-clean.

This work is part of the BigDisk project:

http://www.FreeBSD.org/projects/bigdisk/

Discussed on: arch@
MFC after: 5 days

Use the length modifier 'll' instead of 'q' to print long longs.

Respect locale settings from the environment.

Give du(1) a -m option to report in megabytes.

Submitted by: Vasily Korytov <deskpot@msk.yell.ru>
PR: 66976

Use humanize_number(3) to format sizes into a human readable form.

More style fixes, per bde.

Style fixes, most suggested by bde.

Speed up hardlink detection by using a self-sizing hash
table rather than the old linear list search.

On my "hardlink detection torture test", this reduced
user time from 4700 seconds down to 4.2 seconds
and wallclock time from 1:24:48 down to 1:08.
(Yes, that's over one THOUSAND times reduction in user time. ;-)
In the worst case, the new code doubles peak memory usage,
though it could actually reduce memory usage in many cases.

MFC after: 1 week
PR: misc/42167, bin/51151

-{h,k} are mutually exclisive. So only pay attention to the last of the
two when both are given.

Back out rev 1.27; getbsize(3)'s original interface has been restored.

Approved by: markm

Change the type of an unused variable to appease a warning.

ANSIify function definitions to avoid a warning.

Consistently use FBSDID

remove __P

GRR. This was not WARNS=2 clean. You cannot printf a 'long' with
'%qd'.

usr.bin/du/du.c:288: warning: long long int format, long int arg (arg 2)

Pointy hat to: markm

WARNS=2 fixup

Use the correct blocksize when invoked with both -h and -k
options.

PR: 30275
Reviewed by: jake

Add '-I mask' cmdline flag to ignore/skip files and subdirectories
matching a specified shell-glob mask.

Reviewed by: no serious objections on -arch and -audit over
the last few months
MFC after: 1 month

Use warnx(), -Wall cleaning and remove unused #include
Better use of .Nm and .Ar

fix bug with du -hd

Submitted by: dcs

Add human-readable output flag, -h

Reviewed by: obrien

Tweaks to allow compiling -Wall (mostly adding "const" to char rcsid[]).

Add missing '\n'; remove redundant 'h' case in getopt switch.

Add -c option to usage string.

PR: 8568
Submitted by: Ryan Younce <ryany@pobox.com>

Print correct total for -c option (previously, the total for the last
directory scanned was printed instead of the real total)

PR: bin/7896
Submitted by: Zach Heilig <zach@gaffaneys.com>

Don't try to dereference a pointer to freed memory.

PR: bin/7393

* add a -c option which displays the grand total of all files
counted.

* re-word parts of the man page which I felt were badly worded
or ambiguous.

* change the behaviour of argument processing so that when more
than one of the -P, -H and -L options are specified it will
print an error message, rather than choosing the last option
specified, this behaviour is more logical and consistent with
other utilities.

* change the behaviour of argument processing so that negative
arguments to the -d option are not allowed.

PR: 5388
Submitted by: Niall Smart <rotel@indigo.ie>

du(1) always exit with value zero.

$ du /foobar
du: /foobar: No such file or directory
$ echo $?
0

Document -k flag.

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

Merge from Lite2

Import some parts of CSRG 4.4BSD-Lite2 usr.bin sources to fix tree build.

Add a 'depth (-d#)' flag to du

patched (context diff), compiled (w/ -Wall) and tested

Submitted by: John-Mark Gurney <jmg@nike.efn.org>

Remove trailing whitespace.

The last commit was bogus...the putenv doesn't affect the parent process,
so the BLOCKSIZE doesn't need to be preserved.

Implemented the -k flag more properly...don't destroy the user's setting
of the BLOCKSIZE environment variable.

Gimme my `-k' option back!

BSD 4.4 Lite Usr.bin Sources