MFC r326276:

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

MFC r314659,r314676:

r314659:

usr.sbin: normalize paths using SRCTOP-relative paths or :H when possible

This simplifies make logic/output

r314676:

Fix build after r314656

Some of the changes I introduced to use .ALLSRC were correct in spirit,
but incorrect in reality -- in particular, ../Makefile.inc hadn't been
pulled in via bsd.init.mk (via bsd.lib.mk, bsd.prog.mk), so the value
of .ALLSRC (evaluated immediately) was empty. .include bsd.init.mk
explicitly so we can be certain that the values used as dependencies in
the targets are defined when the target recipe has been evaluated.

Reminder: thou shalt separate out separate functional changes before
committing them.

(YUGE) Pointyhat to: ngie
In collaboration with: bdrewery

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Remove a link to the CTM section of the Handbook, which no longer exists.

MFC after: 1 week

Flesh out the SEE ALSO section.

PR: 202929
Submitted by: jhs@berklix.com
MFC after: 1 week

Add META_MODE support.

Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision: D2796
Reviewed by: brooks imp

Convert usr.sbin to LIBADD
Reduce overlinking

With the move away from GNATS, drop the CTM config file for downloading
the GNATS database, it will be going away soon.

use .Mt to mark up email addresses consistently (part2)

PR: 191174
Submitted by: Franco Fichtner <franco@lastsummer.de>

NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.

In usr.sbin/ctm/ctm_dequeue/ctm_dequeue.c, cast a printf field width
parameter to int.

MFC after: 1 week

mdoc: order prologue macros consistently by Dd/Dt/Os

Although groff_mdoc(7) gives another impression, this is the ordering
most widely used and also required by mdocml/mandoc.

Reviewed by: ru
Approved by: philip, ed (mentors)

Shrink encoded line length from 76 to 72 characters.

Some Exchange systems wrap lines over 75 characters long while converting
messages to quoted-printable, preventing ctm_rmail from reassembling
emailed deltas. For a negligible loss of encoding efficiency, this change
allows ctm deltas to once more pass through Exchange undamaged.

The last big commit: let usr.sbin/ use WARNS=6 by default.

Expand *n't contractions.

Line up license text.

Let bsd.prog.mk set SRCS and MAN to their default values.

Sort sections.

Added the EXIT STATUS section where appropriate.

Start the dreaded NOFOO -> NO_FOO conversion.

OK'ed by: core

For variables that are only checked with defined(), don't provide
any fake value.

Mechanically kill hard sentence breaks.

Use malloc() instead of alloca() to allocate a (potentially) large buffer
for storing the "diff -n" output. Some files (eg ports/INDEX,v) are too
big nowadays to fit on the stack.

Submitted by: Stephen Montgomery-Smith <stephen@math.missouri.edu>

The log file is more useful when line buffered.

Submitted by: Stephen Montgomery-Smith <stephen@math.missouri.edu>

No need to define externs for getopt interface.

Make the example "too large" message in the manual match the updated one
actually sent by ctm_smail since rev 1.13.

From the PR:

I am the maintainer of CTM. There is a problem that when very large deltas
are created, that the program ctm_smail, which is responsible for mailing
the deltas out, will instead create a single message that says the delta
is too large. However, if the -q option is set, instead of placing this
message in the queue (as it would have done with the deltas), it mails it
out directly. This conflicts with the current working of CTM in that the
email address is set as %%REPLACE-ME%% so that the created mailing pieces
can be signed by gnu-pgp, and then have the mailing address changed.

This fix means that if the -q option is set, and the delta is too large,
the "too large" message is placed in the queue.

Also, I made the "too large" message a little more up to date.

Submitted by: Stephen Montgomery-Smith <stephen@math.missouri.edu>
PR: bin/50328
MFC After: 2 weeks

Remove the maximum patch size limit. It was intended as a check against
applying corrupt deltas, but has never (to my knowledge) caught any sort
of corruption, but instead has caused failures on correct deltas several
times. I don't see any way to make the check useful, so it's gone.

Submitted by: Stephen Montgomery-Smith <stephen@math.missouri.edu>
PR: 50461
MFC after: 7 days

style.Makefile(5)

Correct typos, mostly s/ a / an / where appropriate. Some whitespace cleanup,
especially in troff files.

mdoc(7) police: Deal with self-xrefs.

Make the threatened fts(3) ABI fix. FTSENT now avoids the use of the struct
hack, thereby allowing future extensions to the structure (e.g., for extended
attributes) without rebreaking the ABI. FTSENT now contains a pointer to the
parent stream, which fts_compar() can then take advantage of, avoiding the
undefined behavior previously warned about. As a consequence of this change,
the prototype of the comparison function passed to fts_open() has changed
to reflect the required amount of constness for its use. All callers in the
tree are updated to use the correct prototype.

Comparison functions can now make use of the new parent pointer to access
the new stream-specific private data pointer, which is intended to assist
creation of reentrant library routines which use fts(3) internally.

Not objected to in spirit by: -arch

Usage style sweep: spell "usage" with a small 'u'.
Also change one case of blatant __progname abuse (several more remain)
This commit does not touch anything in src/{contrib,crypto,gnu}/.

Use `The .Nm utility'

Modernize my email address.

Modernize my email address

The mode of files created by ctm_rmail was always 0600, even if the
umask was less restrictive. This was caused by the use of mkstemp()
which internally passes a mode of 0600 to open(). Fix this by
explicitly chmod'ing the files to (0666 & ~umask).

PR: bin/16119
Submitted by: Sascha Blank <blank@uni-trier.de>

Spell "FreeBSD" with "F" and "BSD" in uppercase.

mdoc(7) police:

Avoid using parenthesis enclosure macros (.Pq and .Po/.Pc) with plain text.
Not only this slows down the mdoc(7) processing significantly, but it also
has an undesired (in this case) effect of disabling hyphenation within the
entire enclosed block.

Use STD{ERR,IN,OUT}_FILENO instead of their numeric values. The
definitions are more readable, and it's possible that they're
more portable to pathalogical platforms.

Submitted by: David Hill <david@phobia.ms>

Perform a major cleanup of the usr.sbin Makefiles.
These are not perfectly in agreement with each other style-wise, but they
are orders of orders of magnitude more consistent style-wise than before.

Remove GCC'isms in CFLAGS.

Remove whitespace at EOL.

mdoc(7) police: remove extraneous .Pp before and/or after .Sh.

Fix the type of the NULL arg to execl()

Idea from: Theo de Raadt <deraadt@openbsd.org>

* fix ctm(8) to use mkstemp() instead of tempnam() for tempfile
creation.

* Tag the internal err() function with __printflike to allow checking
for non-constant format string arguments (none exist)

* Use fmtcheck() to sanitize the tar command obtained via -t to make
sure it doesn't contain extraneous format operators.

Reviewed by: mikeh
MFC after: 1 week

- Backout botched attempt to introduce MANSECT feature.
- MAN[1-9] -> MAN.

Set the default manual section for usr.sbin/ to 8.

There is an arbitrary file size limit in ctm of 20MB. By my reckoning,
ports/INDEX,v is currently 19.97MB and will blow this limit on the next
update. Let's try doubling the limit again, to give us time to get
around to removing the limit altogether.

mdoc(7) police: split punctuation characters + misc fixes.

Prepare for mdoc(7)NG.

mdoc(7) police: use canonical form of .Dd macro.

Add `_PATH_DEVZERO'.
Use _PATH_* where where possible.

mdoc(7) police: use the new features of the Nm macro.

remove trailing periods from SEE ALSO.

Use Fx macro wherever possible.

Avoid use of direct troff requests in mdoc(7) manual pages.

Complete migration of aliases file to /etc/mail/aliases.
The maintainers of share/examples/diskless/README.TEMPLATING and mergemaster
have been contacted so those may be updated as well.

Change use of perror() to warn()

Submitted by: bde

Doubling the size of MAXSIZE because of large delta indigestion.

Submitted by: Stefan Esser.

Mark "name" as being an argument, so it is formatted correctly.

PR: docs/16834
Submitted by: Greg Lewis <glewis@trc.adelaide.edu.au>

Correcting SECURITY warning.

Submitted by: Kris Kennaway
Reviewed by: Warner Losh

Fix insecure tempfile handling.

Reviewed by: audit@freebsd.org

|Since I had suffered from partially applied CTM deltas, some time ago,
|I made ctm ignore deltas for files that match the "after edit" MD5.
|(In one case, I had the compiler fill all temporary space while CTM
|was editing files.)

Submitted by: se
Reviewed by: phk

Document the (in)security features of CTM, especially ctm_rmail.

Close buffer overflows. This is partially based on an old version of
the fix from OpenBSD.

Reviewed by: markm

Correct the email address that folk are to bug if they need
help; also point them in the direction of the source, not
PHK (or any other human) if they need to make CTM deltas
of their own.

Remove comments left since mdoc.template time or just useless comments

$Id$ -> $FreeBSD$

Some man page cleanup. Make the NAME and SYNOPSIS sections agree so
that whatis(1) will produce the expected results.

Pointed-out-by: Jesus Monroy <jesus.monroy@usa.net> in freebsd-doc

Use 4 digit year in log file, rather than 2.

Paranoia: use mkstemp instead of mktemp.
PR: bin/3211
Reported by: Mark Pritchard <mpp@FreeBSD.ORG>

Spellint fixes.

PR: 6903 (not the contrib bits).
Reviewed by: phk
Submitted by: Josh Gillam <josh@quick.net>

Spelling/typo fix: (proccessed).

Spelling/typo fixes: (proccessed, og).

(evil) hackers -> crackers

.Sh AUTHOR -> .Sh AUTHORS. Use .An/.Aq.

Delete ctm_scan program. There is no manpage for ctm_scan
and nobody use it.

Reviewed by: phk & Richard Wackerbarth

Use err(3). Use An/Aq for author name.

fix misspelling

Submitted-by: Josh Gilliam

Closes PR:4424

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

Revert $FreeBSD$ to $Id$

Sort cross references.

Sweep through the tree fixing mmap() usage:

- Use MAP_FAILED instead of the constant -1 to indicate
failure (required by POSIX).
- Removed flag arguments of '0' (required by POSIX).
- Fixed code which expected an error return of 0.
- Fixed code which thought any address with the high bit set
was an error.
- Check for failure where no checks were present.

Discussed with: bde

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Be a little less fatalistic in case of problems.

Reviewed by: phk
Submitted by: wosch

Correct MLINKS, now that I know how it actually works.

Documented ctm_dequeue and the new feature of ctm_smail that goes with it.
Expanded the ctm_rmail example usage section.

Simplified. Some fts related bugs removed. Made less verbose. The default
number of mail messages sent per run was lowered from 2 to 1. Why? Well,
some numbers just give you the warm fuzzies, like zero and one. Zero isn't
much use here, so I picked my all time favourite, one.

Removed unnecessary locking. Simplified. Tidied.

Don't dump core on zero-size files.

Latest reality.

Make the case where there is no changes look less fatal.

Check bogus and ignore against the relative name, not the absolute.

Improve mkctm.c so we can use it, and start using it.
This should help quite a bit on the load. Notice that
some minor changes in config files will be needed.
Contact me for details.

fix .Xr macro

aarghh! what is it with this CVSROOT/val-tags regexp anyway? how
many times do I have to do this to get it right? :-]

Add dequeue script for the record.

make it slightly less verbose while creating queue entries..

make the "-l logfile" option actually do something..

After queueing deltas, send out an initial two mails since that will cover
99.9% of the cases at with out delay as before.

fts_children() returns NULL if there are no files. If there was a failure
then errno != 0.

- resync with configs running on freefall
- add ctm_conf.gnats from freefall
- add support for doing both the immediate mailout and the queued mailout.
- use "sendmail -odq -t" rather than "sendamil -t" to make it queue to
the mailqueue rather than immediately begin transmission. This allows
us to take advantage of our ordered dequeueing system without blowing
WC's T1 to hell with a 50 part mailout in parallel.
- bump the max ctm size from 3MB to 10MB.... This is mainly for the fast
list.

Some new options, useful for restoring single files or subtrees from the
deltas.

Submitted by: A JOSEPH KOSHY <koshy@fakir.india.hp.com>

Add some explanation on TMPDIR.
Submitted by: Chuck Robey <chuckr@Glue.umd.edu>

Small changes so that this actually stands a chance of doing the
right thing...

Add ctm_dequeue to the SUBDIR list now that I've checked that it
compiles cleanly on 2.2

Add a facility for a ``slow'' CTM delta queue, allowing `x' number
of delta's to be mailed out every hour (or however often you schedule
the cron job).

ctm_dequeue is the cron job which takes the stuff from the
queue directory and punts it into sendmail. The chunks of
the deltas (and the complete deltas if they are that small)
are sorted into order before being dispatched, so the people
subscribing should still get the bits in the right order.

The changes to ctm_smail should be fairly safe as they won't be
activated unless you go for the new queue directory option.

update SEE ALSO section

Commit the right version of mkCTM, <:-)

Noticed by: "Philippe Charnier" <charnier@lirmm.fr>

Update to current reality.
mkctm.c can replace the guts of mkCTM if anybody feels like it...

Support for "-u" which sets the timestamp from the delta on the file.
Intended for sup mirrors etc. Not well tested yet.

Reviewed by: phk
Submitted by: Giles Lean <giles@nemeton.com.au>
Submitted by: John Hay <jhay@mikom.csir.co.za>

Updated howmany() to be once again identical with the namespace-polluting
one in <sys/types.h>. Lite2 changed the white space in the latter.

Updated format strings to match Lite2's fixed-width type changes.

uids and gids aren't unreasonably long any more, so we get to
change all the format strings that were fixed to use %lu back
to %u.

Add support for local modifications to the tree, by using FOO.ctm instead
of FOO if present. Various other tweaks.

Submitted by: Christian Haury <Christian.Haury@sagem.fr>

ctm(5) has been written, so remove comment about it not existing. :)

typo in my last commit.

A trivial enhancement to ctm_rmail to support the -v option and pass it
through to cvs when unpacking the delta.. (I read the output.. :-)

Remove all trailing white space on the lines.
Suggested by: Phil Taylor <phil@zipmail.co.uk>

Fix a core-dump.

Submitted by: Simon Marlow <simonm@dcs.gla.ac.uk>

Forgot this one in the MD5 reshuffle.

Noticed by: Faried Nawaz <fn@pain.csrv.uidaho.edu>

Remove -g from CFLAGS.

Fix to match new MD5 api. Faster, fixes memory leak.

Remove trailing whitespace.

Check for i/o errors in fclose() so that a full disk doesn't almost
guarantee truncation of the file being edited.

The latest round of bugfixes here...

Fixed a bug so that a delta#0 can be applied.
Fixed a typo in the generation of temp filenames.
"Blame" Joerg for the man-pages (and clarify a couple of minor points).
Most appreceiated Joerg!

Call open(2) and mkdir(2) with 0666/0777 as the `mode' argument (as
opposed to 0644 or 0755). It's finally still masked by the process'
umask(2), and it does not make sense to restrict it further than that.

This (especially for mkdir(2)) was causing major headaches for the CVS
tree, since a member of group cvs was later not able to get cvs
checkout permission for the mirrored tree failed to write the lock file).

Added a man page for the ctm(1) command, as well as a format description
for the CTM deltas.

Largely based on Poul-Henning's README, and the source code.

Make sure exit() is sensible;

Modes on dirs were decimal by mistake.
Submitted by: joerg

Make extra arguments be subdirs to go through.

Finally applying my own fix. :-)

>Description:

ctm(1) sometimes did not free up all used resources (open pipes and
processes, heap memory). This happened whenever one of the passes
ended prematurely, and it became very apparent when running it on
a bunch of already applied deltas, resulting in a ``gunzip: resource
temporarily unavailable'' due to the maxproc # exhausted.

Add a missing "\" in a printf before the "n".

Make the +%d-%d part +%03d-%03d so ls(1) sorts the files more sensibly.

Look at the .ctm_status file and ignores all patches already applied.

Reviewed by: phk
Submitted by: J Wunsch <j@uriah.heep.sax.de>

Added lock-files, so we should know if the machine panic'ed in the middle
of an "apply".

Latest updates to this.

(Not tested yet. I may insist that ctm be invoked with absolute path. /phk)

This patch fixes the concurrency problem, and adds a possibly useful -f switch
(which you can read about in the man page :-) ). It also removes the absolute
path from the invocation of ctm. I'll write a note about how to use a script
with sendmail and procmail or some such, and people can fix their PATH there.

BTW, this patch changes ctm_rmail.1, ctm_rmail.c and error.c in the ctm_rmail
directory.

Stephen.

Reviewed by: phk
Submitted by: Stephen McKay <syssgm@devetir.qld.gov.au>

Just a precaution: ctm will not accept '..' in paths anymore.
I'm never going to generate one, so this is a guard against hackers mostly.

Reviewed by: phk
Submitted by: Stephen McKay <syssgm@devetir.qld.gov.au>
Obtained from:

I (phk) blundered with the last change. Make it right & better now.

Reviewed by: phk
Submitted by: John Hay <jhay@mikom.csir.co.za>

Make us a little safer, by writing to a tempfile, and rename when done.
Still needs more locking I belive.

Be more exclusive about cvs-locks.

Use absolute path for the "ctm" program.

The new and improved mkCTM. Learning from this experience it has been
improved on a couple of accounts. Amongst these are "damage control"
more than 100 files removed and it will bail out...

Some recent small changes. Probably about time I start writing this in C.

Various nitpicking from Stephen.

Reviewed by: phk
Submitted by: Stephen McKay <syssgm@devetir.qld.gov.au>

Clean up some details, to make it clear to gcc that we do not try to do
something stupid.

Stephen lost a -D, now it's back.

A couple of sensible changes from down under...

Reviewed by: phk
Submitted by: Stephen McKay <syssgm@devetir.qld.gov.au>

CTM email tools.

Reviewed by: phk
Submitted by: Stephen McKay <syssgm@devetir.qld.gov.au>

Update to current reality.

Cannot use rmdir() for 'FR' because mkCTM doesn't sort the directories
in -depth order for us. cvs-cur.0018.gz is a good example.

Fix this to work under 2.0 also.

Fixed a typo. Poited out by Bruce & Phillipe.

Fix syntax errors (0 = foo()).

Stefan Esser proved that I couldn't program, and Ollivier Robert that I
couldn't spell :-)
Submitted by: Stefan Esser, Ollivier Robert

These patches fix some lesser problems:

1) malloc.h doesn't exits in 2.0.
2) Makefile.inc wasn't picked up so one of the build steps (install?)
failed.
3) LIBMD wasn't depended on.
4) "ctm foo" dumped core because "foo" doesn't have a '.' in it.

Bruce

I updated the mkCTM stuff while I was at it anyway. /phk

Reviewed by: phk
Submitted by: bde

A couple of nit-picks.

Added a '-c' option: This will warn you if any file in the directory-tree
you run it on are unsuitable food for diff/patch. Can you use it as it
is now Rod ?

Various cleanup and fixes.

Add BINDIR=/usr/sbin for install, fails in other case

Add BINDIR=/usr/sbin for install, fails in other case
Remove -g from CFLAGS

The next batch of refinements. Now it no longer needs to call on ed(1) to
fix our files, it has a builtin "diff -n" editor.

A bunch of improvements. Still far to go.

This commit was generated by cvs2svn to compensate for changes in r2886,
which included commits to RCS files with non-trunk default branches.