MFC r326276:

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Cleanup unnecessary semicolons from utilities we all love.

Set SO_NOSIGPIPE on sockets used by phttpget. Without this, if
(1) phttpget is attempting to download enough files that it can't send
all the requests at once, and
(2) the remote server forcibly closes the connection, resulting in RST
packets being sent,
phttpget will receive a SIGPIPE and terminate without downloading all
of the files.

This is probably responsible for a number of hard-to-reproduce errors
with portsnap and freebsd-update.

MFC after: 3 days

Improve conformance to the HTTP specification by using case-insensitive
comparisons for header keywords. Apparently some proxies use creative
capitalization.

Weird proxy found by: brooks
MFC after: 3 days

Add support for HTTP/1.0 Persistent Connections to phttpget. Requests are
be marked as HTTP/1.1 but "Connection: Keep-Alive" is added; this convinces
HTTP/1.0 servers and proxies to hold the TCP connection open despite not
being able to use HTTP pipelining.

This dramatically cuts down on the number of TCP connections (and thus port
numbers) used by portsnap when talking to an HTTP/1.0 proxy (e.g., squid),
and has the side benefit of improving performance in those cases.

Tested by: simon
Approved by: re (kensmith)
MFC After: 1 week

Add support for the HTTP_TIMEOUT environment variable (integer number
of seconds, just like in fetch(1)).

Submitted by: rdivacky

Allow the HTTP_PROXY environment variable to be (mis)spelled as
"http_proxy", since some people apparently do this and fetch(3)
allows it.

Correct handling of HTTP_PROXY_AUTH. A password may have `:'.

Approved by: cperciva
MFC after: 2 days

Merge from accidental commit to RELENG_5:
Correctly identify the host and port values on a failed
getaddrinfo lookup.

Correctly handle a TCP connection being shutdown by the server while
we're reading response headers. (Handle it as a connection-killing
error, rather than entering an infinite loop reading zero bytes.)

Reported by: simon
Discovered thanks to: A not-very-transparent transparent HTTP proxy.
MFC after: 3 days

Add HTTP proxy authentication, via the HTTP_PROXY_AUTH environment
variable.

Tested by: Emil Mikulic
X-MFC-After: 6.0-RELEASE

Style fix: "if (pointer)" -> "if (pointer != NULL)"

Remove unused variable.

Reported by: stefanf

Replace INT_MAX with OFF_MAX. I used INT_MAX originally (in ports)
because OFF_MAX wasn't defined on FreeBSD 4.x.

When parsing the HTTP_PROXY environment variable, strip a trailing /
from the port number (if any exists). This unbreaks
env HTTP_PROXY="http://localhost:3128/" portsnap fetch

While I'm here, list both the host and the port in the error message
output if getaddrinfo() fails, since either of them could be responsible
for the failure.

Add portsnap to the base system. This is a secure, easy to use,
fast, lightweight, and generally good way for users to keep their
ports trees up to date.

This is version 0.9.4 from the ports tree (sysutils/portsnap) with
the following changes:
1. The experimental pipelined http code is enabled. No seatbelts
in -CURRENT. (^_^)
2. The working directory has moved from /usr/local/portsnap to
/var/db/portsnap (as discussed on -arch two days ago).
3. Portsnap now fetches a list of mirrors (distributed as DNS SRV
records) and selects one randomly. This should help to avoid the
uneven loading which plagues the cvsup mirror network.
4. The license is now 2-clause BSD instead of 3-clause BSD.
5. Various incidental changes to make portsnap fit into the base
system's build mechanics.

X-MFC-After: 6.0-RELEASE
X-MFC-Before: 5.5-RELEASE
X-MFC-To: RELENG_6, RELENG_5, ports
discussed on: -arch and several other places
"yes please" from: simon, remko, flz, Diane Bruce
thinks this is a great idea: bsdimp
Hopes he didn't forget any files: cperciva