Revert r337826: MFC of ls(1) COLORTERM honoring

It was not ready, and was much-objected-to.

MFC r337506: ls(1): Enable colors with COLORTERM is set in the environment

COLORTERM is the de facto standard, while CLICOLOR is generally specific to
FreeBSD and ls(1).

PR: 230101

Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Fix some issues with the application of libxo to ls(1)

* Add whitespace trimming to some fields (username, group, size, inode, blocks) to avoid whitespace in JSON strings
* fix -m mode, was invalid JSON (repeated keys), and was missing outer array container
* in -n mode, numeric uids and gids were returned as strings

Approved by: eadler (mentor)
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D2854

Prevent potential integer overflow

PR: 192971
Submitted by: David Carlier <david.carlier@hardenedbsd.org>

Convert ls(1) to use libxo(3).

Obtained from: Phil Shafer <phil@juniper.net>
Sponsored by: Juniper Networks, Inc.

It has been long time that when doing 'ls -G /path/to/a/symlink', instead of
using the color of symbolic link, the color is determined by the link target.
This behavior was quite confusing.

Looking at the file history, it looks like that r203665 intends to fix this
but the issue was never actually fixed.

Fix this by not setting FTS_COMFOLLOW when color is requested like what was
done in r203665.

MFC after: 2 weeks

Make -f set -a, as required by the standard.

From the original OpenBSD commit message:

restore the traditional behavior of -f implying -a; apparently Keith
Bostic forgot to restore it when the -f flag was put back on 2nd of
September 1989, after being removed on 16th of August as a
consequence of issues getting it working over NFS, so deviation from
traditional UNIX behavior in all BSDs looks like an historical
accident; as a side effect, this change accommodates behavior of
this option to IEEE Std 1003.1-2008 (``POSIX.1'').

joint work with jmc@ (who found the inaccuracy in our
implementation), schwarze@ (who provided a detailed tracking of
historical facts) and millert@

Submitted by: Igor Sobrado
Discussed with: mckusick
Obtained from: OpenBSD project
MFC after: 2 weeks

Undo over-aggressive conversion of spaces to tabs. ie: those within
format strings, "period, space, space" in comment text, etc.

Replace spaces with tabs where appropriate.

Reminded by: jh@

Add y flag and environment variable LS_SAMESORT to specify the same
sorting order for time and name with the -t option. IEEE Std 1003.2
(POSIX.2) mandates that the -t option sort in descending order, and
that if two files have the same timestamp, they should be sorted in
ascending order of their names. The -r flag reverses both of these
sort orders, so they're never the same. This creates significant
problems for sequentially named files stored on FAT file systems,
where it can be impossible to list them in the order in which they
were created.

Add , (comma) option to print file sizes grouped and separated by
thousands using the non-monetary separator returned by localeconv(3),
typically a comma or period.

MFC after: 14 days

Sort option parsing as far as practical.

Fix bin/ build with a 64-bit ino_t.

Original code by: Gleb Kurtsou

If ls was invoked with -i but neither -l nor -s, blocksize was used in
display() to calculate column widths, but was not initialized in
main(). This resulted in a division by zero.

Noticed by: Michael Butler <imb@protected-networks.net>

When calculating the width of the blocksize column, ls(1) used 512-byte
units (as returned by stat(2)) instead of BLOCKSIZE units.

Submitted by: Paul Schenkeveld
MFC after: 3 weeks

Get rid of major/minor number distinction.

As of FreeBSD 6, devices can only be opened through devfs. These device
nodes don't have major and minor numbers anymore. The st_rdev field in
struct stat is simply based a copy of st_ino.

Simply display device numbers as hexadecimal, using "%#jx". This is
allowed by POSIX, since it explicitly states things like the following
(example taken from ls(1)):

"If the file is a character special or block special file, the
size of the file may be replaced with implementation-defined
information associated with the device in question."

This makes the output of these commands more compact. For example, ls(1)
now uses approximately four columns less. While there, simplify the
column length calculation from ls(1) by calling snprintf() with a NULL
buffer.

Don't be afraid; if needed one can still obtain individual major/minor
numbers using stat(1).

Make sure that FTS_COMFOLLOW is not set when the -P option is in effect.
Otherwise the -i option will show the inode number of the referenced file
for symbolic links given on the command line. Similarly, the file color
was printed according to the link target in colorized output.

PR: bin/102394
Reviewed by: jilles
MFC after: 2 weeks

Fixes for ls(1) long format (-l) output:

- Allow -h option to work if the listing contains at least one device
file.
- Align major and minor device numbers correctly to the size field.

PR: bin/125678
Approved by: trasz (mentor)
MFC after: 1 month

Print full path in the error message. It's possible that fts(3)
provides an empty fts_name and reporting the full path is more
appropriate especially with the -R option.

PR: bin/107515
Submitted by: bde
Approved by: trasz (mentor)
MFC after: 1 week

ls: Make -p not inhibit following symlinks.

According to the man page, when neither -H/-L nor -F/-d/-l are given, -H is
implied. This agrees with POSIX, GNU ls and Solaris ls. This means that -p,
although it is very similar to -F, does not prevent the implicit following
of symlinks.

PR: standards/128546

Add -D option to specify exact format of date and time output with ls -l.

Change the -S and -t options to override each other so that the last one
specified wins to make their interaction less confusing.

Fix a bug such that if you enabled sorting by size (-S) and enabled a
flag to use a time other than modtime (-c, -u, or -U), the output would
actually be sorted by the specified time rather than size. This does
alter the behavior in the case where both -S and -t are specified. Now,
-S is always preferred. Previously, -t was preferred if one of -c, -u, or
-U was specified, and -S was preferred otherwise. Perhaps -S and -t should
override each other (last one specified wins).

Add a new -U flag to instruct ls to use the birthtime for printing or
sorting.

Submitted by: Andrzej Tobola ato at iem dot pw dot edu dot pl
MFC after: 1 week

Having three options (-a, -A, -I) controlling the output of dotted
files is too much and hard to follow. Instead, make the -I option
just mean "do not automatically set -A for root". That is, if -A
is explicitly set, -I is ignored. Also, document -I in usage().
(The ls.c diff is better viewed relative to rev. 1.80.)

No objection: mux
Silence from: mnag
MFC after: 3 days

Add a -I option to disable the automatic -A flag for the super-user.

PR: bin/86710
Submitted by: Marcus Alves Grando
MFC after: 3 days

Add the -S option to sort files by size. NetBSD and OpenBSD already
have this option with identical semantics (sorting large files first).
-r can be used to reverse the sort if that is desired.

PR: 81625
Submitted by: Kostas Blekos <mplekos@physics.upatras.gr>, keramida

/*- or .\"- or #- to begin license clauses.

If we are asked to print the total number of blocks, do so even if we
have no entries to print (either due to an empty directory or an
error). This makes the -l and -s options more consistent, like
Solaris and (Debian) Linux. To make this happen, tweak two
optimizations on the second call to display():

- Don't skip display() altogether, even if list == NULL.
- Don't skip the call to the printfn in display() if we
need to print the total.

PR: 45723

Plug small memory leak.

PR: bin/67392
Submitted by: Matthew Emmerton <matt@gsicomp.on.ca>
MFC in: 1 week

Remove clause 3 from the UCB licenses.

OK'ed by: imp, core

Do something sensible if both -h and -k are given.

Approved by: re(scottl)

Fix a bazillion warnings. This makes almost the whole of src/bin/*
WARNS=6, std=c99 clean.

Tested on: i386, alpha

When mac_from_text() fails with -Z, print "-" rather than "" so that
scripts parsing ls(1) output can still count columns.

Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories

Improve handling of symlink targets when listing MAC labels: don't
do the wrong thing when the symlink doesn't have a target, by
considering !f_label in the construction of ch_options.

Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories

Use warn() instead of perror() or fprintf() where appropriate.

Do not include <sys/syslimits.h> directly; it is not intended for general
consumption.

Attempt improved use of fts results: use the correct path to the
object to retrieve label information on, rather than directly
consuming the fts-provided paths (none of which are quite right).
This is based on the similar readlink() code, and may contain
the same bugs.

Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories

Teach "ls -Z" to use the policy-agnostic MAC label interfaces rather
than the LOMAC-specific interfaces for listing MAC labels. This permits
ls to view MAC labels in a manner similar to getfmac, when ls is used
with the -l argument. Next generation LOMAC will use the MAC Framework
so should "just" work with this and other policies. Not the prettiest
code in the world, but then, neither is ls(1).

Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories

Print non-printing characters in directory names, as well as file names,
as `?' or `\ooo', depending on whether the -b or -B flags were used.

PR: 43995
MFC after: 1 month

Make the threatened fts(3) ABI fix. FTSENT now avoids the use of the struct
hack, thereby allowing future extensions to the structure (e.g., for extended
attributes) without rebreaking the ABI. FTSENT now contains a pointer to the
parent stream, which fts_compar() can then take advantage of, avoiding the
undefined behavior previously warned about. As a consequence of this change,
the prototype of the comparison function passed to fts_open() has changed
to reflect the required amount of constness for its use. All callers in the
tree are updated to use the correct prototype.

Comparison functions can now make use of the new parent pointer to access
the new stream-specific private data pointer, which is intended to assist
creation of reentrant library routines which use fts(3) internally.

Not objected to in spirit by: -arch

Replace various spellings with FALLTHROUGH which is lint()able

Fix typos; each file has at least one s/seperat/separat/
(I skipped those in contrib/, gnu/ and crypto/)
While I was at it, fixed a lot more found by ispell that I
could identify with certainty to be errors. All of these
were in comments or text, not in actual code.

Suggested by: bde
MFC after: 3 days

err() is documented as allowing NULL for the format string but GCC isn't
happy about it any more so change the usage to make buildworld work again.

Consistently use __FBSDID

Ignore empty COLUMNS environment variable. COLUMNS should take precedence
over TTY width found via ioctl() (SUSv3)

Add missing options required by SUSv3:
-m List files across the page, separated by commas.
-p Print a slash after directory names
-x Same as -C but sort across the columns rather than down

Submitted by: Kyle Martin <mkm@ieee.org>

restore missing default case removed in ls.c:r1.57

add break statements to default cases where missing.

Submitted by: bde

Make 'user' and 'group' const as to not discard the qualifier from the
functions we use to assign them. Doesn't seem to be anything else that
relies on these being non-const.

Remove an empty default case to make this syntactically correct. Not there
is as good as blaknk.

Fixed divots that I created when I moved prototypes of group_from_gid
and user_from_uid to grp.h and pwd.h. Update the man pages.

Submitted by: David Malone
Pointy hat to: imp

Use __FBSDID() and clean up the vendor tags.

WARNS=4 fixes, plus a healthy dose of fixes inspired by lint.

Fix unused variable.

o __P has been reoved
o Old-style K&R declarations have been converted to new C89 style
o register has been removed
o prototype for main() has been removed (gcc3 makes it an error)
o int main(int argc, char *argv[]) is the preferred main definition.
o Attempt to not break style(9) conformance for declarations more than
they already are.

Force raw printing of non-printable characters via the -w option.

PR: bin/28007

Restore these files to shiny KNF.

Add a new flag, -h which when combined with the -l option causes
file sizes to be displayed with unit suffixes; Byte, Kilobyte,
Megabyte, Gigabyte, Terabyte and Petabyte in order to reduce the
number of digits to three or less.

Submitted by: nik

Mop up some warnings.

Revamp the colour support to allow for bold characters. Colours
are now defined using the characters a-h and A-H for the bold
variants. The old way using 0-7 for the colours still works, but
prints a message asking the user to switch.

PR: bin/27374

Add LOMAC options (the "Z" flag in both cases) to display extra information
in ls(1) and ps(1).

Sponsored by: DARPA, NAI Labs

Un-deprecate the -G flag at obrien's request. Colour sequences are still
dependent upon the output being directed to a terminal however.
(Use the CLICOLOR_FORCE variable to force output).

Correct spelling: depricated -> deprecated.

A change to the way that colours are switched on in ls. The -G
flag has been depricated, although it still works with a warning
message, and replaced with an environment variable CLICOLOR (command
line interface colour). This could be used by other tools that
want to be able to control colour output.

In addition if the environment variable CLICOLOR_FORCE is defined
colour sequences are output irrespective of whether the output is
directed to a terminal (as long as TERM references a colour capable
terminal of course ;)

PR: bin/20291 and bin/20483

make sure we do not write out non-printable characters in file names
and symbolic links (by default)

PR: bin/19354
Reviewed by: silence on -current

Switch over to using the new fflagstostr and strtofflags library calls.

Fix one conditionalization in my prev. commit

Greately simplify oxtabs fix by simple setting f_notabs for f_color

Move colorquit() prototype to extern.h
Add "extern" to variables declarations in extern.h to not make them
commons into each compiled file.

Do SIGINT cleanup for SIGQUIT as well.

Don't use curses includes, include termcap.h instead
Add ^C reaction set

Don't look up the ANSI sequences each time a colour is changed,
this is extremely inefficient, instead write them all down at the
beginning.

The correct sequence to switch colours off is to first use 'op' if
it exists, otherwise use 'oc'. If neither of these exist then we
shouldn't be doing colour with this terminal.

Reviewed by: ache

* Re-implement colour support using termcap's AF and AB capabilities
to manage the ANSI colour sequences. Colour support is disabled
unless the TERM environment variable references a valid termcap.

* Allow optional compilation of the colour support in the Makefile,
defaulting to yes. This allows us to switch it off for fixit
floppies and other mediums where space is an issue and the extra
bloat of statically linking with ncurses isn't acceptable.

* Display a warning if colour is requested with '-G' but support
for it isn't compiled in.

Add colour support to /bin/ls (at a cost of 1056 bytes on my system).

It is not switched on by default and must be enabled with the -G
flag. When using ls -G the output behaviour is modified with ANSI
colour sequences wrapped around filenames to help distinguish file
types. (Colours can be redefined in the LSCOLORS environment
variable as described in the manual page.)

Colour support is silently disabled (if switched on) if stdout
isn't a tty.

Based on: asami's colorls port.
PR: bin/18900 && ports/18616.

Revert part of the last commit, remove {g|s}etflags from the libc
interface, and statically link them to the programs using them.
These functions, upon reflection and discussion, are too generically
named for a library interface with such specific functionality.
Also the api that they use, whilst ok for private use, isn't good
enough for a libc function.

Additionally there were complications with the build/install-world
process. It depends heavily upon xinstall, which got broken by
the change in api, and caused bootstrap problems and general mayhem.

There is work in progress to address future problems that may be
caused by changes in install-chain tools, and better names for
{g|s}etflags can be derived when some future program requires them.
For now the code has been left in src/lib/libc/gen (it started off
in src/bin/ls).

It's important to provide library functions for manipulating file
flag strings if we ever want this interface to be adopted outside
of the source tree, but now isn't necessarily the right moment
with 4.0-release just around the corner.

Approved: jkh

Historically file flags (schg, uschg, etc) have been converted from
string to u_long and back using two functions, flags_to_string and
string_to_flags, which co-existed with 'ls'. As time has progressed
more and more other tools have used these private functions to
manipulate the file flags.

Recently I moved these functions from /usr/src/bin/ls to libutil,
but after some discussion with bde it's been decided that they
really ought to go in libc.

There are two already existing libc functions for manipulating file
modes: setmode and getmode. In keeping with these flags_to_string
has been renamed getflags and string_to_flags to setflags.

The manual page could probably be improved upon ;)

$Id$ -> $FreeBSD$

Backed out my -n change to imply -l by request of sheldonh.

Make -n flag compliant to the Single Unix Specification.

To quote their ls(1) specification:

-n
The same as -l, except that the owner's UID and GID numbers are
written, rather than the associated character strings.

Reviewed by: green

Take integer rounding into account in the buffer size approximation
macro. So now it's (1 +) for the sign and (+ 1) for rounding.

Reported by: bde

Style issues in previous commit:

Use an upward approximation of the number of characters required
for decimal representations of uid_t, gid_t and u_quad_t, intead
of arbitrary values that may not be safe in the future.

Fix disordering.

Requested by: bde

Add -n option to print numeric user and group IDs instead of names
in a long (-l) listing.

MFC-jockies should make sure that bde's concerns regarding the number
of digits required to represent a uid_t and the use of snprintf
on the associated PR have been addressed before going wild.

PR: 12866
Reported by: Philip Kizer <pckizer@nostrum.com>
Obtained from: NetBSD

Various spelling/formatting changes.

Submitted by: Philippe Charnier <charnier@xp11.frmug.org>

Add missing "else", without whom the prev. commit is a null change.
Fix a 2nd level indentation style bug.

Allow env. variable LS_COLWIDTHS to specify minimum column widths,
effectively overriding the dynamically-sized-column feature. This
is mostly useful for non-interactive use, where it may be necessary
to ensure that listings taken at different times have columns that
line-up correctly. I have been assured that at least one large,
well-known program will soon be taking advantage of this. :-)

PR: bin/7011
Submitted by: Joel Ray Holveck <joelh@gnu.org>

Remove the bogus -? option after reading the getopt() manual :)

o Removed two unused variables (f_dirname and f_newline) in ls.c

o Added the -H and -P options for treatment of symbolic links.

o Removed the #ifdef BSD4_4_LITE, since it does not seem to do anything
useful

o Fixed up prn_octal() so its output looks more like that of AT&T Unices
when -b is given.

The next two lines apply only to the first two changes above:

PR: bin/6140
Submitted by: Max Euston

o Renamed '-b' (show unprintables in octal) to '-B'

o Added a new '-b' which behaves as in AT&T Unices (show unprintables in
octal, using C escape codes when possible)

o Added '?' to the getopt() string, since the code in the switch considers
it as a valid option.

Added -b option to display unprintables in octal.
PR: 1315

Fix sorting of directories when doing '-d'.

PR: bin/4558
Submitted by: Keith Bostic

Observe precedence set by Phillippe Charnier in adding an
rcsid.

Remove #if(n)def BSD_4_4_LITE cruft and sccsid -> rcsid.

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Merge Lite2 mods, and -Wall cleaning. undelete(2) cruft
not yet implemented is protected by a define (BSD4_4_LITE)
that should be removed when this call is supported by the
kernel.

set error status on exit, to be consistent with manpage and standard commands.

Change locale to LC_ALL, there no bitmask

Change LC_ALL to LC_CTYPE|LC_TIME - small optimization

Add setlocale LC_ALL

Having BLOCKSIZE=K and using "-k" option ls would use 2K block size...

You will find enclosed some changes to make gcc -Wall more happy in
/usr/src/bin. Note that some patches are still needed in that directory.

I (Joerg) finished most of Philippe's cleanup. /bin/sh will still
need *allot* of work, however.

Submitted by: charnier@lirmm.fr (Philippe Charnier)

Make ls recognize the COLUMNS environmental variable, even
in cases where it does not run on a terminal. This is important
e.g. for commands like

env COLUMNS=132 ls -CR | lpr -Psome-wide-printer

Added $Id$

Add support for '-k' option to print file allocation space in 'K' instead of
system blocks.

This is semi-original code, not the same way this crufty option was handled
in FreeBSD 1.x.

This commit was generated by cvs2svn to compensate for changes in r1556,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite bin Sources