Remove copyright strings ifdef'd out

We've ifdef'd out the copyright strings for some time now. Go ahead and
remove the ifdefs. Plus whatever other detritis was left over from other
recent removals. These copyright strings are present in the comments and
are largely from CSRG's attempt at adding their copyright to every
binary file (which modern interpretations of the license doesn't
require).

Sponsored by: Netflix

bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

dd(1): neutralize SIGINT while non-async-signal safe code is executing

making the SIGINT handler (the terminate() function) safe to execute at
any interruption moment. This fixes a race in
5807f35c541c26bbd91a3ae12506cd8dd8f20688 where SIGINT delivered right
after the check_terminate() but before a blocking syscall would not
cause abort.

Do it by setting the in_io flag around potentially blocking io syscalls.
If handler sees the flag, it terminates the program. Otherwise,
termination is delegated to the before_io/after_io fences.

Reviewed by: Andrew Gierth <andrew@tao146.riddles.org.uk>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D40281

dd: Fix SIGINT handling.

Currently, we handle SIGINT by calling summary() and _exit() directly from the signal handler, which we install after setup(). There are several issues with this:

* summary() is not signal safe;
* the parent is not informed about the signal;
* setup() can block on open(), and catching SIGINT at that stage will produce the correct exit status but will not print anything to stderr as POSIX demands.

Fix this by making SIGINT non-restartable, changing our signal handler to only set a flag, installing it before setup(), and checking the termination flag before and after every blocking operation, i.e. open(), read(), write().

Also add two test cases, one for catching SIGINT while opening the input and one for catching it while reading. I couldn't think of an easy way to test catching SIGINT while writing (it's certainly feasible, but perhaps not from a shell script).

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: cracauer, ngie, imp
Differential Revision: https://reviews.freebsd.org/D39641

Add O_DIRECT flag to DD for cache bypass

FreeBSD DD utility has not had support for the O_DIRECT flag, which
is useful to bypass local caching, e.g. for unconditionally issuing
NFS IO requests during testing.

Reviewed by: rgrimes (mentor)
Approved by: rgrimes (mentor, blanket)
MFC after: 3 weeks
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D25066

Add iflag=fullblock to dd

Normally, count=n means read(2) will be called n times on the input to dd. If
the read() returns short, as may happen when reading from a pipe, fewer bytes
will be copied from the input. With conv=sync the buffer is padded with zeros
to fill the rest of the block.

iflag=fullblock causes dd to continue reading until the block is full, so that
count=n means n full blocks are copied. This flag is compatible with illumos
and GNU dd and is used in the ZFS test suite.

Submitted by: Ryan Moeller
Reviewed by: manpages, mmacy@
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D21441

Add oflag=fsync and oflag=sync capability to dd

Sets the O_FSYNC flag on the output file. oflag=fsync and oflag=sync are
synonyms just as O_FSYNC and O_SYNC are synonyms. This functionality is
intended to improve portability of dd commands in the ZFS test suite.

Submitted by: Ryan Moeller
Reviewed by: manpages, mmacy@
MFC after: 1 week
Sponsored by: iXsytems, Inc.
Differential Revision: https://reviews.freebsd.org/D21422

dd: Check result of close(2) for errors

close(2) can return errors from previous operations which should not be ignored.

PR: 229616
Submitted by: Thomas Hurst
Reported by: Thomas Hurst
Reviewed by: mmacy@
Obtained from: Ryan Moeller
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21376

Add conv=fdatasync flag to dd

The fdatasync flag performs an fdatasync(2) on the output file before closing it.
This will be useful for the ZFS test suite.

Submitted by: Ryan Moeller
Reviewed by: manpages, mmacy@
MFC after: 1 week
Sponsored by: iXSystems, Inc.
Differential Revision: https://reviews.freebsd.org/D21373

Add conv=fsync flag to dd

The fsync flag performs an fsync(2) on the output file before closing it.
This will be useful for the ZFS test suite.

Submitted by: ryan@ixsystems.com
Reviewed by: jilles@, imp@
MFC after: 1 week
Sponsored by: iXsystems, Inc.

Replace hand-crafted naive byte-by-byte zero block detection routine
with macro based around memcmp(). The latter is expected to be some
8 times faster on a modern 64-bit architectures.

In practice, throughput of doing conv=sparse from /dev/zero to /dev/null
went up some 5-fold here from 1.9GB/sec to 9.7GB/sec with this change
(bs=128k).

MFC after: 2 weeks

capsicum: use a new capsicum helpers in tools

Use caph_{rights,ioctls,fcntls}_limit to simplify the code.

dd: Incorporate some changes from imp for status=progress

Notable changes from what landed in r337505:
- sigalarm handler isn't setup unless we're actually using it
- Humanized versions of the amount of data transferred in the progress
update

Submitted by: imp
Reviewed by: kevans
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D16642

dd: add status=progress support

This reports the current status on a single line every second, mirroring
similar functionality in GNU dd, and carefully interacts with SIGINFO.

PR: 229615
Submitted by: Thomas Hurst <tom@hur.st> (modified for style(9) nits by me)
MFC after: 1 week

Convert `cap_enter() < 0 && errno != ENOSYS` to `caph_enter() < 0`.

No functional change intended.

dd(1): Use a local swapbytes() function.

swab(3) has restrict qualifiers for src and dst.
Avoid relying on undefined overlapping swab behavior.

Obtained From: OpenBSD

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

dd(1): Incorrect casting of arguments

dd(1) casts many of its numeric arguments from uintmax_t to intmax_t and
back again to detect whether or not the original arguments were negative.
This is not correct, and causes problems with boundary cases, for example
when count is SSIZE_MAX-1.

PR: 191263
Submitted by: will@worrbase.com
Reviewed by: pi, asomers
MFC after: 3 weeks

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

Fix regression when stdin/out/err fds are are overridden by shell.

Submitted by: Pawel Biernacki <pawel.biernacki@gmail.com>
Reported by: ngie
Approved by: ngie
Sponsored by: Mysterious Code Ltd.
Differential Revision: https://reviews.freebsd.org/D8543

Capsicum support for dd(1)

Adds Capsicum sandboxing to dd utility.

Submitted by: Pawel Biernacki <pawel.biernacki@gmail.com>
Reviewed by: allanjude, emaste, oshogbo
Approved by: oshogbo
Sponsored by: Mysterious Code Ltd.
Differential Revision: https://reviews.freebsd.org/D8543

[dd] Revert the capsicum bits for now until it gets fixed.

dd is a bootstrap tool and that header isn't installed as part of the
bootstrap environment for previous releases (eg freebsd-10.)

We'll figure it out in post and then re-commit it.

Restore dd changes included accidentally in r308857.

Bring in support for Ingenic XBurst JZ4780 and
X1000 systems on chips.

Imgtec CI20 and Ingenic CANNA boards supported.

Submitted by: Alexander Kabaev <kan@FreeBSD.org>
Reviewed by: Ruslan Bukin <br@FreeBSD.org>
Sponsored by: DARPA, AFRL

Capsicum support for dd(1)

Adds Capsicum sandboxing to dd utility.

Submitted by: Pawel Biernacki <pawel.biernacki@gmail.com>
Reviewed by: allanjude, emaste, oshogbo
Approved by: oshogbo
Sponsored by: Mysterious Code Ltd.
Differential Revision: https://reviews.freebsd.org/D8543

Add speed limit to dd(1). This is useful for testing RCTL disk io limits
(when they actually get committed, that is), and might also come in handy
in other situations.

Reviewed by: wblock@ (man page)
MFC after: 1 month
Sponsored by: The FreeBSD Foundation

Reorganize the handling all-zeroes terminal block in sparse mode

The intent of the previous code in that case was to force
an explicit write, but the implementation was incorrect, and
as a result the write was never performed. This new implementation
instead uses ftruncate(2) to extend the file with a trailing hole.

Also introduce regression tests for these cases.

PR: 189284
(original PR whose fix introduced this bug)

PR: 207092

Differential Revision: D5248
Reviewed by: sobomax,kib
MFC after: 2 weeks

bin/dd: revert 273734, as it fails on 32bit platforms

Revert: insufficient testing on 32bit platforms

PR: 191263

bin/dd: Fix incorrect casting of arguments

dd(1) casts many of its numeric arguments from uintmax_t to intmax_t
and back again to detect whether or not the original arguments were
negative. This caused wrong behaviour in some boundary cases:

$ dd if=/dev/zero of=/dev/null count=18446744073709551615
dd: count cannot be negative

After the fix:

$ dd if=/dev/zero of=/dev/null count=18446744073709551615
dd: count: Result too large

PR: 191263
Submitted by: will@worrbase.com
Approved by: cognet@

Incorporate feedback from bde and jilles regarding r265472 to dd(1).

* Don't use sysexits.h. Just exit 1 on error and 0 otherwise.
* Don't sacrifice precision by converting the output of clock_gettime() to a
double and then comparing the results. Instead, subtract the values of
the two clock_gettime() calls, then convert to double.
* Don't use CLOCK_MONOTONIC_PRECISE. It's an unportable synonym for
CLOCK_MONOTONIC.
* Use more appropriate names for some local variables.
* In the summary message, round elapsed time to the nearest microsecond.

Reported by: bde, jilles
MFC after: 3 days
X-MFC-With: 265472

(dd_out): Fix handling of all-zeroes block at end of input with
conv=sparse.

This change fixes two separate issues observed when the last output
block is all zeroes, and conv=sparse is in use. In this case, care
must be taken to roll back the last seek and write the entire last zero
block at the original offset where it should have occurred: when the
destination file is a block device, it is not possible to roll back
by just one character as the write would then not be properly aligned.

Furthermore, the buffer used to write this last all-zeroes block
needs to be properly zeroed-out. This was not the case previously,
resulting in a junk data byte appearing instead of a zero in the
output stream.

PR: bin/189174
PR: bin/189284
Reviewed by: kib
MFC after: 2 weeks

dd(1) uses gettimeofday(2) to compute the throughput statistics. However,
gettimeofday returns the system clock, which may jump forward or back,
especially if NTP is in use. If the time jumps backwards, then dd will see
negative elapsed time, round it up to 1usec, and print an absurdly fast
transfer rate.

The solution is to use clock_gettime(2) with CLOCK_MONOTONIC_PRECISE as the
clock_id. That clock advances steadily, regardless of changes to the system
clock.

Reviewed by: delphij
MFC after: 3 days
Sponsored by: Spectra Logic

Make C_* contants fit in 32 bits again by using 4 unused bits.

Noticed by: bde

Implement GNU's extension of 'status' operand. The GNU syntax is
borrowed where syntax status=noxfer means no transfer statistics
and status=none means no status information at all.

This feature is useful because the statistics information can
sometimes be annoying, and redirecting stderr to /dev/null would
mean error messages also gets silenced.

Obtained from: OpenBSD
MFC after: 2 weeks

Make dd's signal handler async safe.

PR: bin/75258
Submitted by: "Oleg V. Nauman" <oleg@reis.zp.ua>
Arrival Date: Sun Dec 19 14:50:21 GMT 2004
Reviewed by: mjg, jhb
Reviewed by: jilles (earlier version)
MFC after: 1 week

Literally follow POSIX:
If the bs= expr operand is specified and no conversions other than sync,
noerror, or notrunc are requested, the data returned from each input
block shall be written as a separate output block.

In particular, when both bs=size and conv=sparce were specified, the
resulted file was fully filled, instead of sparce.

PR: standards/177742
Submitted by: Matthew Rezny <mrezny@hexaneinc.com>
MFC after: 2 weeks

IFP4 change 222074.

Introduce an explicit close of the output descriptor so that work done
on close is accounted for in the summary output triggered at exit
(implicit close()s occur after atexit() hooks).

This is useful because some devices such as cfi(4) may perform
signficant work after a close occurs (e.g. erasing and rewriting a
block of flash).

Rework all non-contributed files that use `struct timezone'.

This structure is not part of POSIX. According to POSIX, gettimeofday()
has the following prototype:

int gettimeofday(struct timeval *restrict tp, void *restrict tzp);

Also, POSIX states that gettimeofday() shall return 0 (as long as tzp is
not used). Remove dead error handling code. Also use NULL for a
nul-pointer instead of integer 0.

While there, change all pieces of code that only use tv_sec to use
time(3), as this provides less overhead.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

- Import the HEAD csup code which is the basis for the cvsmode work.

Add a "fillchar" command line argument to dd(1) that permits the user
to specify an alternative padding character when using a conversion
mode, or when using noerror with sync and an input error occurs. This
facilities reading old and error-prone media by allowing the user to
more effectively mark error blocks in the output stream.

Remove clause 3 from the UCB licenses.

OK'ed by: imp, core

Teach dd(1) about parity bits.

Quiet warnings about copyright[].

WARNS=4 fixes. This would be WARNS=9 if we were -std=99 instead of
-ansi, due to 'long long'.

Reviewed by: green (slightly earlier version)

Don't call DIOCWLABEL on disks, it is not implemented, and calling it
like this negated any practical value of the feature.

Consistently use __FBSDID

1) Rev.1.35 of dd.c has a more serious regression. It backs out rev.1.31,
thus breaking systems with unpolluted <sys/stat.h>'s.

2) Back out an initialisation of a variable in BSS.

Reported by: bde (1), many(2)

Fix warnings inspired by lint, a commercial lint and WARNS=4.

o __P has been reoved
o Old-style K&R declarations have been converted to new C89 style
o register has been removed
o prototype for main() has been removed (gcc3 makes it an error)
o int main(int argc, char *argv[]) is the preferred main definition.
o Attempt to not break style(9) conformance for declarations more than
they already are.

Approved by: arch@, new style(9)

Commit general cleanups (separate get_num() and get_off_t() functions to
debogosify some of the command-line string-number conversions into
an unsigned and signed variant.)

Use __unused for non-used variables.

Submitted by: Mike Barcroft <mike@q9media.com>

Don't depend on <sys/stat.h> bogusly including <sys/time.h> (and thereby
<time.h>).

Make the comment regarding ftruncate() correct.

Various cleanups are made to reduce warnings and make code prettier :)
Also, check for ftruncate() return value and die on failure, but only
try to ftruncate() when the file is a regular file.

Fix ``dd if=/dev/zero of=/dev/daN'' on the Alpha by allowing the label
to be overwritten.

Submitted by: green

Nuke the FIODTYPE compatibility bits. It's time.

Make count=0 set cpy_cnt to -1, which is slight overloading, but makes
what I was trying to do work much better (ie at all. I could have sworn
it was working...) Fix a SEEK_SET to be SEEK_CUR, and make Bruce's
lseek() test work correctly.

Even more dd(1) cleanups! Thanks to Bruce for staying on my case until
we're done (not yet!) :)

ISDISK -> ISSEEK

Allow a device type of D_DISK or D_MEM to be ISSEEK.

Even more cleanups to dd(1). This is probably the culmination of the
BDEification process of dd(1). Most of the changes are from BDE's archive.
Support for negative offsets is gone again, but the case where you
lseek() onto byte -1 of something from a negative offset using seek/skip
is fixed; if you end up on -1, you won't get a false positive lseek failure.
The biggest changes are to data types (more size_t, for instance) and
argument parsing. skip/seek on /dev/{,k}mem now occurs (instead of "read
until you reach the offset") due to mem devices now being D_DISK. Some
const things are now correctly declared as such, and the "case table"
building is better. The only thing that seems to be left to make dd(1)
everything TOG wants it to be is l10n.

Relax things a bit. Not having FIODTYPE will be a warning for now.

Pointy hat: green
Pointed out by: peter

$Id$ -> $FreeBSD$

Use FIODTYPE to unbogosify much of the file type checking in dd.

Implement seekability for disk devices (not just regular files).
Also, fix pos_out() to do the same checks pos_in() did.

Done for: jdp, luigi, the good of the world

This is the second round of dd(1) changes. Some changes made/reversed by
request of Bruce. More changes may follow later. 'g' multiplier has
been added (i.e. dd seek=5g if=bigfile.) Some minor corrections were made
as well.

Noticed by: bde

Miscellaneous dd(1) changes: mainly fixing variable types (size_t, ssize_t,
off_t, int, u_int64_t, etc.). dd(1) should now work properly with REALLY
big amounts of data.

First set of fixes to keep egcs happy. These include {} around single
statement if blocks[*] when the else could be ambiguous, not defaulting
to int type and removal of some unused variables.

[*] This is explicitly allowed by style(9) when the single statement
spans more than one line.

Reviewed by: obrien, chuckr

Restore Lite-2 sccsid. Restore include of sys/types.h.

Correct use of .Nm and other cosmetics. Add rcsid. Remove unused #inc.

Use the correct value in the call to swab(3) with conv=swab. Previously,
dd if=/dev/zero of=/dev=null obs=23520 conv=swab
coredumped.

Please somebody review it, i'm not 105 % sure i'm understanding all
this mess correctly.

Detected by: Holm Tiffe <holm@geophysik.tu-freiberg.de>

Teach dd(1) about an option to write sparse files. Can be useful for
things like diskless clients' swap files etc.

Submitted by: pascal@zuo.dec.com (Pascal Pederiva) (ages ago, with many
stylistic changes by me)

Pad the input buffer whenever sync is used, not just if the noerror flag
is also set.
Change osync to not to tack on an empty block if the input buffer is null,
or an even multiple of the blocksize.
Also change osync to pad the output with nulls/spaces depending whether
this is a block-oriented conversion or not (same as sync).

PR: 3818

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

-Wall cleaning.

Bruce says: "You have been programming in the kernel for too long :-)."

and he's right ... I forgot about this floating point stuff you can
use in user-land :-)

Increase precision of duration to microseconds.
No heuristics to avoid overflow in calculation needed - just depend
on DBL_MAX being a bit larger than LONG_MAX.

Use double instead of `struct timeval' in dd.h so that everything
doesn't have to include <sys/time.h>.

Fixed style bugs in recent and old FreeBSD changes.

Reviewed by: phk
Submitted by: bde

Increase precision of duration to milliseconds.
Some heuristics to avoid overflow in calculation attempted.

Add setlocale LC_CTYPE

More koshering [ul]case fix, don't use pre-initialized tables at all,
treat 0xFF as valid character.

Make conv=[lu]case works with localized ctype (8bit)

Added $Id$

BSD 4.4 Lite bin Sources