Revert r337826: MFC of ls(1) COLORTERM honoring

It was not ready, and was much-objected-to.

MFC r337506: ls(1): Enable colors with COLORTERM is set in the environment

COLORTERM is the de facto standard, while CLICOLOR is generally specific to
FreeBSD and ls(1).

PR: 230101

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Add more text to explain --libxo flag.

Fix whitespace error in ls(1) detected by igor

Approved by: bcr (mentor)
Sponsored by: EuroBSDCon DevSummit

Use one fewer made up words in the ls(1) man page

PR: 203337
Submitted by: Mike Dame <damemi@gmail.com>
Approved by: bcr (mentor)
Sponsored by: EuroBSDCon DevSummit

Convert ls(1) to use libxo(3).

Obtained from: Phil Shafer <phil@juniper.net>
Sponsored by: Juniper Networks, Inc.

Bump Dd, missed as part of r264098 and related commits.

Sponsored by: The FreeBSD Foundation

-k option is compliant with POSIX.

Typo.

Reported by: Igor Sobrado.

Add information on standards compliance of many options.

MFC after: 2 weeks

Make -f set -a, as required by the standard.

From the original OpenBSD commit message:

restore the traditional behavior of -f implying -a; apparently Keith
Bostic forgot to restore it when the -f flag was put back on 2nd of
September 1989, after being removed on 16th of August as a
consequence of issues getting it working over NFS, so deviation from
traditional UNIX behavior in all BSDs looks like an historical
accident; as a side effect, this change accommodates behavior of
this option to IEEE Std 1003.1-2008 (``POSIX.1'').

joint work with jmc@ (who found the inaccuracy in our
implementation), schwarze@ (who provided a detailed tracking of
historical facts) and millert@

Submitted by: Igor Sobrado
Discussed with: mckusick
Obtained from: OpenBSD project
MFC after: 2 weeks

Expand the use of stat(2) flags to allow storing some Windows/DOS
and CIFS file attributes as BSD stat(2) flags.

This work is intended to be compatible with ZFS, the Solaris CIFS
server's interaction with ZFS, somewhat compatible with MacOS X,
and of course compatible with Windows.

The Windows attributes that are implemented were chosen based on
the attributes that ZFS already supports.

The summary of the flags is as follows:

UF_SYSTEM: Command line name: "system" or "usystem"
ZFS name: XAT_SYSTEM, ZFS_SYSTEM
Windows: FILE_ATTRIBUTE_SYSTEM

This flag means that the file is used by the
operating system. FreeBSD does not enforce any
special handling when this flag is set.

UF_SPARSE: Command line name: "sparse" or "usparse"
ZFS name: XAT_SPARSE, ZFS_SPARSE
Windows: FILE_ATTRIBUTE_SPARSE_FILE

This flag means that the file is sparse. Although
ZFS may modify this in some situations, there is
not generally any special handling for this flag.

UF_OFFLINE: Command line name: "offline" or "uoffline"
ZFS name: XAT_OFFLINE, ZFS_OFFLINE
Windows: FILE_ATTRIBUTE_OFFLINE

This flag means that the file has been moved to
offline storage. FreeBSD does not have any special
handling for this flag.

UF_REPARSE: Command line name: "reparse" or "ureparse"
ZFS name: XAT_REPARSE, ZFS_REPARSE
Windows: FILE_ATTRIBUTE_REPARSE_POINT

This flag means that the file is a Windows reparse
point. ZFS has special handling code for reparse
points, but we don't currently have the other
supporting infrastructure for them.

UF_HIDDEN: Command line name: "hidden" or "uhidden"
ZFS name: XAT_HIDDEN, ZFS_HIDDEN
Windows: FILE_ATTRIBUTE_HIDDEN

This flag means that the file may be excluded from
a directory listing if the application honors it.
FreeBSD has no special handling for this flag.

The name and bit definition for UF_HIDDEN are
identical to the definition in MacOS X.

UF_READONLY: Command line name: "urdonly", "rdonly", "readonly"
ZFS name: XAT_READONLY, ZFS_READONLY
Windows: FILE_ATTRIBUTE_READONLY

This flag means that the file may not written or
appended, but its attributes may be changed.

ZFS currently enforces this flag, but Illumos
developers have discussed disabling enforcement.

The behavior of this flag is different than MacOS X.
MacOS X uses UF_IMMUTABLE to represent the DOS
readonly permission, but that flag has a stronger
meaning than the semantics of DOS readonly permissions.

UF_ARCHIVE: Command line name: "uarch", "uarchive"
ZFS_NAME: XAT_ARCHIVE, ZFS_ARCHIVE
Windows name: FILE_ATTRIBUTE_ARCHIVE

The UF_ARCHIVED flag means that the file has changed and
needs to be archived. The meaning is same as
the Windows FILE_ATTRIBUTE_ARCHIVE attribute, and
the ZFS XAT_ARCHIVE and ZFS_ARCHIVE attribute.

msdosfs and ZFS have special handling for this flag.
i.e. they will set it when the file changes.

sys/param.h: Bump __FreeBSD_version to 1000047 for the
addition of new stat(2) flags.

chflags.1: Document the new command line flag names
(e.g. "system", "hidden") available to the
user.

ls.1: Reference chflags(1) for a list of file flags
and their meanings.

strtofflags.c: Implement the mapping between the new
command line flag names and new stat(2)
flags.

chflags.2: Document all of the new stat(2) flags, and
explain the intended behavior in a little
more detail. Explain how they map to
Windows file attributes.

Different filesystems behave differently
with respect to flags, so warn the
application developer to take care when
using them.

zfs_vnops.c: Add support for getting and setting the
UF_ARCHIVE, UF_READONLY, UF_SYSTEM, UF_HIDDEN,
UF_REPARSE, UF_OFFLINE, and UF_SPARSE flags.

All of these flags are implemented using
attributes that ZFS already supports, so
the on-disk format has not changed.

ZFS currently doesn't allow setting the
UF_REPARSE flag, and we don't really have
the other infrastructure to support reparse
points.

msdosfs_denode.c,
msdosfs_vnops.c: Add support for getting and setting
UF_HIDDEN, UF_SYSTEM and UF_READONLY
in MSDOSFS.

It supported SF_ARCHIVED, but this has been
changed to be UF_ARCHIVE, which has the same
semantics as the DOS archive attribute instead
of inverse semantics like SF_ARCHIVED.

After discussion with Bruce Evans, change
several things in the msdosfs behavior:

Use UF_READONLY to indicate whether a file
is writeable instead of file permissions, but
don't actually enforce it.

Refuse to change attributes on the root
directory, because it is special in FAT
filesystems, but allow most other attribute
changes on directories.

Don't set the archive attribute on a directory
when its modification time is updated.
Windows and DOS don't set the archive attribute
in that scenario, so we are now bug-for-bug
compatible.

smbfs_node.c,
smbfs_vnops.c: Add support for UF_HIDDEN, UF_SYSTEM,
UF_READONLY and UF_ARCHIVE in SMBFS.

This is similar to changes that Apple has
made in their version of SMBFS (as of
smb-583.8, posted on opensource.apple.com),
but not quite the same.

We map SMB_FA_READONLY to UF_READONLY,
because UF_READONLY is intended to match
the semantics of the DOS readonly flag.
The MacOS X code maps both UF_IMMUTABLE
and SF_IMMUTABLE to SMB_FA_READONLY, but
the immutable flags have stronger meaning
than the DOS readonly bit.

stat.h: Add definitions for UF_SYSTEM, UF_SPARSE,
UF_OFFLINE, UF_REPARSE, UF_ARCHIVE, UF_READONLY
and UF_HIDDEN.

The definition of UF_HIDDEN is the same as
the MacOS X definition.

Add commented-out definitions of
UF_COMPRESSED and UF_TRACKED. They are
defined in MacOS X (as of 10.8.2), but we
do not implement them (yet).

ufs_vnops.c: Add support for getting and setting
UF_ARCHIVE, UF_HIDDEN, UF_OFFLINE, UF_READONLY,
UF_REPARSE, UF_SPARSE, and UF_SYSTEM in UFS.
Alphabetize the flags that are supported.

These new flags are only stored, UFS does
not take any action if the flag is set.

Sponsored by: Spectra Logic
Reviewed by: bde (earlier version)

Add a few examples.

Obtained from: OpenBSD

Add y flag and environment variable LS_SAMESORT to specify the same
sorting order for time and name with the -t option. IEEE Std 1003.2
(POSIX.2) mandates that the -t option sort in descending order, and
that if two files have the same timestamp, they should be sorted in
ascending order of their names. The -r flag reverses both of these
sort orders, so they're never the same. This creates significant
problems for sequentially named files stored on FAT file systems,
where it can be impossible to list them in the order in which they
were created.

Add , (comma) option to print file sizes grouped and separated by
thousands using the non-monetary separator returned by localeconv(3),
typically a comma or period.

MFC after: 14 days

Get rid of major/minor number distinction.

As of FreeBSD 6, devices can only be opened through devfs. These device
nodes don't have major and minor numbers anymore. The st_rdev field in
struct stat is simply based a copy of st_ino.

Simply display device numbers as hexadecimal, using "%#jx". This is
allowed by POSIX, since it explicitly states things like the following
(example taken from ls(1)):

"If the file is a character special or block special file, the
size of the file may be replaced with implementation-defined
information associated with the device in question."

This makes the output of these commands more compact. For example, ls(1)
now uses approximately four columns less. While there, simplify the
column length calculation from ls(1) by calling snprintf() with a NULL
buffer.

Don't be afraid; if needed one can still obtain individual major/minor
numbers using stat(1).

Update sticky(7) cross references.

PR: docs/124468
X-MFC with: r218998

Add reference to strmode(3).

- Add -D to usage().
- Bump document date for the addition of the -D option.
- Reformat a sentence to look like a real sentence.

Add -D option to specify exact format of date and time output with ls -l.

- Improve description of the -A option.
- Document how whiteouts look in the long output. [1]
- Sort entry types.
- Fix description of the socket type.

PR: docs/51921 [1]

Markup fixes.

Change the -S and -t options to override each other so that the last one
specified wins to make their interaction less confusing.

Add a few more references to -U.

Add a new -U flag to instruct ls to use the birthtime for printing or
sorting.

Submitted by: Andrzej Tobola ato at iem dot pw dot edu dot pl
MFC after: 1 week

-mdoc sweep.

Having three options (-a, -A, -I) controlling the output of dotted
files is too much and hard to follow. Instead, make the -I option
just mean "do not automatically set -A for root". That is, if -A
is explicitly set, -I is ignored. Also, document -I in usage().
(The ls.c diff is better viewed relative to rev. 1.80.)

No objection: mux
Silence from: mnag
MFC after: 3 days

Mention the non-standardness of the -I option in ls(1).

Spotted by: ru
MFC after: 3 days

Add a -I option to disable the automatic -A flag for the super-user.

PR: bin/86710
Submitted by: Marcus Alves Grando
MFC after: 3 days

fix typos: decribed -> described, preceeded -> preceded

Improved descriptions of block size handling.

PR: docs/84765
Submitted by: garys
Approved by: keramida
MFC after: 3 days

Remove the EXAMPLES section that describes how to sort by size using
sort(1). This functionality is provided by the -S option now, and it
is useful even though a similar effect is achievable with sort(1),
since the latter doesn't work in combination with -h. This option is
also present in NetBSD, OpenBSD, and GNU fileutils, so there's clearly
a demand for it.

Noticed by: asmodai

Improve wording: A sort is "in" a particular order, not "by" a
particular order.

Add the -S option to sort files by size. NetBSD and OpenBSD already
have this option with identical semantics (sorting large files first).
-r can be used to reverse the sort if that is desired.

PR: 81625
Submitted by: Kostas Blekos <mplekos@physics.upatras.gr>, keramida

Expand *n't contractions.

Add the new standard EXIT STATUS section where appropriate.
Sort standard sections in the (documented) preferred order.

The total sum of blocks for the -l and -s option is printed always,
regardless whether the output is to a terminal or not. As this is
consistent with the SUSPv2 specification (even though we do not
otherwise fully implement SUSPv2's ls(1) options), document this as it
is now, rather than trying to change the behaviour itself.

PR: docs/76072
Submitted by: Sebastian Rey <Sebastian.rey@gmx.net>
MFC after: 1 week

/*- or .\"- or #- to begin license clauses.

Mechanically kill hard sentence breaks.

Punctuation.

Treat filenames as multibyte character strings (according to the current
LC_CTYPE setting) when determining which characters are printable.
This is an often-requested feature.

Use wcwidth() to determine the number of column positions a character
takes up, although there are still a few places left where we assume
1 byte = 1 column position, e.g. line-wrapping when handling the -m option.

The error handling here is somewhat more complicated than usual: we do
our best to show what we can of a filename in the presence of conversion
errors, instead of simply aborting.

Remove clause 3 from the UCB licenses.

OK'ed by: imp, core

Document incorrect handling of multibyte characters in filenames.

Do something sensible if both -h and -k are given.

Approved by: re(scottl)

Note in STANDARDS that -gno are not (yet?) POSIX conformant.

Spotted by: harti
MFC after: 3 days

-l prints all permissions, not just owner and group

PR: 54294
Submitted by: Per Hedeland <per@hedeland.org>
MFC after: 3 days

df(1) and ls(1) print units in 'four or fewer' not 'three or less'.

PR: 35523
Submitted by: Tomas Svensson <tsn@gbdev.net>

Reference maclabel(7).

Sponsored by: DARPA, Network Associates Laboratories

mdoc(7) police: markup fixes.

Approved by: re

Print a `+' character after the standard UNIX permission fields in long
listings if the file has an extended ACL (more than the required 3 entries).
This is what Solaris and IRIX do, and what the withdrawn POSIX.2c standard
required.

Reviewed by: rwatson (an earlier version of the patch)

Teach "ls -Z" to use the policy-agnostic MAC label interfaces rather
than the LOMAC-specific interfaces for listing MAC labels. This permits
ls to view MAC labels in a manner similar to getfmac, when ls is used
with the -l argument. Next generation LOMAC will use the MAC Framework
so should "just" work with this and other policies. Not the prettiest
code in the world, but then, neither is ls(1).

Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories

s/filesystem/file system/ as discussed on -developers

mdoc(7) police: fixed the document date.

Submitted by: iedowse

mdoc(7) police: markup nits.

Add missing options required by SUSv3:
-m List files across the page, separated by commas.
-p Print a slash after directory names
-x Same as -C but sort across the columns rather than down

Submitted by: Kyle Martin <mkm@ieee.org>

Consistancy check s/file system/filesystem/

Reviewed by: brian

ls(1) is utility not function

Document option to option dependancy. Fix typo.

MFC after: 1 week

mdoc(7) police: tidy up.

Remove part of a diff committed at the end of the file.

Force raw printing of non-printable characters via the -w option.

PR: bin/28007

Add a new flag, -h which when combined with the -l option causes
file sizes to be displayed with unit suffixes; Byte, Kilobyte,
Megabyte, Gigabyte, Terabyte and Petabyte in order to reduce the
number of digits to three or less.

Submitted by: nik

Revamp the colour support to allow for bold characters. Colours
are now defined using the characters a-h and A-H for the bold
variants. The old way using 0-7 for the colours still works, but
prints a message asking the user to switch.

PR: bin/27374

Make a bit more clear the types of characters that -B option will
print in octal.

Reviewed by: roam, silence on -doc

mdoc(7) police: s/LKM/KLD/, sort xrefs.

Add LOMAC options (the "Z" flag in both cases) to display extra information
in ls(1) and ps(1).

Sponsored by: DARPA, NAI Labs

mdoc(7) police: utilize the new .Ex macro.

Don't use the pronoun "you".

Reviewed by: sheldonh, mpp

Don't put a shell prompt in the example since none of the other man
pages do it. Also, don't use brackets to indicate optional options in
the example; instead, add a paragraph below the example explaining it.

Add missing closing parenthesis.

Show an example of how to sort a file listing by size. Hopefully this
will reduce the number of people calling for this functionality to be
added to ls(1).

Remove whitespace at EOL.

mdoc(7) police: use the default ``file ...'' feature of the .Ar macro.

mdoc(7) police: split punctuation characters + misc fixes.

grammar gnit: "relationships (plural) _are_ complex"

Prepare for mdoc(7)NG.

mdoc(7) police: use the new features of the Nm macro.

Un-deprecate the -G flag at obrien's request. Colour sequences are still
dependent upon the output being directed to a terminal however.
(Use the CLICOLOR_FORCE variable to force output).

Correct spelling: depricated -> deprecated.

A change to the way that colours are switched on in ls. The -G
flag has been depricated, although it still works with a warning
message, and replaced with an environment variable CLICOLOR (command
line interface colour). This could be used by other tools that
want to be able to control colour output.

In addition if the environment variable CLICOLOR_FORCE is defined
colour sequences are output irrespective of whether the output is
directed to a terminal (as long as TERM references a colour capable
terminal of course ;)

PR: bin/20291 and bin/20483

".It" -> ".It Ev" in a couple of places.

Explain the dependence of colour support on the capabilities of the
terminal emulator.

As pointed out by jhb, a more scalable solution would be preferable
when multiple applications in the base system begin linking against
libh.

Submitted by: Doug Barton <DougB@gorean.org>

Fix changes from 1.34 through 1.37:

Remove newly added hard sentence breaks.
Mark ANSI up as a type name (Tn).
Avoid parenthesized sentences and paragraphs.

Order the ENVIRONMENT section alphabetically.

I broke locale sensitive ordering of date and month in
the long -l output format with the last commit. Fix it
by replacing the "%b %e" strftime format with "%Ef".

Make a note in the manual page that the LANG environment
variable affects the running of ls.

Reviewed by: ache

Reflect reality:
yellow -> brown
white -> light grey

Add colour support to /bin/ls (at a cost of 1056 bytes on my system).

It is not switched on by default and must be enabled with the -G
flag. When using ls -G the output behaviour is modified with ANSI
colour sequences wrapped around filenames to help distinguish file
types. (Colours can be redefined in the LSCOLORS environment
variable as described in the manual page.)

Colour support is silently disabled (if switched on) if stdout
isn't a tty.

Based on: asami's colorls port.
PR: bin/18900 && ports/18616.

Remove more single-space hard sentence breaks.

Remove single-space hard sentence breaks. These degrade the quality
of the typeset output, tend to make diffs harder to read and provide
bad examples for new-comers to mdoc.

Add `n' to the synopsis.

Forgotten by: sheldonh

$Id$ -> $FreeBSD$

Add -n option to print numeric user and group IDs instead of names
in a long (-l) listing.

MFC-jockies should make sure that bde's concerns regarding the number
of digits required to represent a uid_t and the use of snprintf
on the associated PR have been addressed before going wild.

PR: 12866
Reported by: Philip Kizer <pckizer@nostrum.com>
Obtained from: NetBSD

Various spelling/formatting changes.

Submitted by: Philippe Charnier <charnier@xp11.frmug.org>

.Xr chflags 1 ,

changable -> changeable

PR: docs/8697
Submitted by: Sascha Blank <blank@fox.uni-trier.de>

Null commit, just to test something. Please ignore.

Allow env. variable LS_COLWIDTHS to specify minimum column widths,
effectively overriding the dynamically-sized-column feature. This
is mostly useful for non-interactive use, where it may be necessary
to ensure that listings taken at different times have columns that
line-up correctly. I have been assured that at least one large,
well-known program will soon be taking advantage of this. :-)

PR: bin/7011
Submitted by: Joel Ray Holveck <joelh@gnu.org>

Correct use of .Nm. Typo.

.Sh ENVIRONMENT VARIABLES -> .Sh ENVIRONMENT

PR: 6599
Reviewed by: phk
Submitted by: Josh Gilliam <josh@quick.net>

Add note about BLOCKSIZE variable to -s flag.

Remove the bogus -? option after reading the getopt() manual :)

o Removed two unused variables (f_dirname and f_newline) in ls.c

o Added the -H and -P options for treatment of symbolic links.

o Removed the #ifdef BSD4_4_LITE, since it does not seem to do anything
useful

o Fixed up prn_octal() so its output looks more like that of AT&T Unices
when -b is given.

The next two lines apply only to the first two changes above:

PR: bin/6140
Submitted by: Max Euston

o Renamed '-b' (show unprintables in octal) to '-B'

o Added a new '-b' which behaves as in AT&T Unices (show unprintables in
octal, using C escape codes when possible)

o Added '?' to the getopt() string, since the code in the switch considers
it as a valid option.

Added -b option to display unprintables in octal.
PR: 1315

Use consistent spelling,
writeable -> writable (recall prior debate over this? :-)
initialise -> initialize
recognise -> recognize

Merry Christmas! :)

environmental -> environment.

Enable whiteout code since we now have the lite2 support for them.

make usage() and SYNOPSIS agree with each other and add missing options
documented in the DESCRIPTION section.

Typo fix.

Add a missing period in the -o option description.

Document the depreciated -g option.

Inspired by: OpenBSD PR# 119

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Minor mdoc cleanup.

Add a blurb describing the fact that ls will print the
year in place of the hour/minute fields if the time is
more than 6 months in the past or future.

Also some minor mdoc cleanup.

Merge Lite2 mods, and -Wall cleaning. undelete(2) cruft
not yet implemented is protected by a define (BSD4_4_LITE)
that should be removed when this call is supported by the
kernel.

[HISTORY] command appeared in Version 1 AT&T UNIX
Obtained from: A Quarter Century of UNIX, Peter H. Salus, page 41

Uncomment out references to FIFOs for the -F and -l options.
Also mention sockets in the text for -F.

Partially obtained from the NetBSD commit mail list.

Added $Id$

Add support for '-k' option to print file allocation space in 'K' instead of
system blocks.

This is semi-original code, not the same way this crufty option was handled
in FreeBSD 1.x.

This commit was generated by cvs2svn to compensate for changes in r1556,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite bin Sources