Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC: r240476

Do not change owner, group, or mode when package database directory
and its contents are created with pkg_add(1). It may happen when the
packing list contains @owner, @group, or @mode.

Approved by: portmgr (bapt)

MFC 236333,236336

Let pkg_add use the ENV variable PACKAGESUFFIX. This
can be used to override the default .tbz package extension to
.txz .tgz or .tar

Submitted by: kmoore
Obtained from: PC-BSD
Sponsored by: iXsystems

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Backout libinstall.a -> libpkg commit.

Discussed with: erwin, brooks, bapt

- Take libinstall.a out of pkg_install and make it a proper shared library.
- Rework the wrapper support to check libpkg version as well as pkg_install
version.
- Add libfetch to _prebuild_libs.
- There are no new features introduced.

Notes: the API is not stable, so basically, do not use libpkg in your
projects for now. Also there's no manpage for libpkg yet, because the API
will change drastically. I repeat, do not use libpkg for now.

Various fixes.

- Replace hardcoded INDEX version. [1]
- Fix a buffer overlap. [2]
- Remove empty package when fetching fails and -K is used. [3]
- Remove useless chmod2() after mkdtemp(3). [4]
- Replace mkdir(1) call with mkdir(2). [5]
- Get rid of some vsystem() calls.
- Switch from lstat(2) to open(2) in fexists().
- Try rename(2) in move_file() first.
- Bump PKG_INSTALL_VERSION to 20100401.

PR: bin/145101 [1], bin/139492 [2], bin/144919 [3]
bin/144920 [4], bin/144921 [5]
Submitted by: gcooper [1,2,3,4,5]

Disable r194497 for now. It doesn't work well with ports-mgmt/tinderbox.

Approved by: re (ken)

When running pkg_add -r, check & install our dependencies for each
package rather than expecting our top level package to get all of
the dependencies correct.

Previously, the code depended on the top level package having all
of the pkgdep lines in +CONTENTS correct and in the right order,
but that doesn't always happen due to code such as this (in
security/gnutls/Makefile):

.if (defined(WITH_LZO) || exists(${LOCALBASE}/lib/liblzo2.so)) && !defined(WITHOUT_LZO)
LIB_DEPENDS+= lzo2:${PORTSDIR}/archivers/lzo2
....

With such conditional dependencies, my 'sophox-packages' package won't
install. The dependency tree looks like this:

sophox-packages
...
x11/gnome2
x11/gnome-applets
net/libgweather
devel/libsoup
security/gnutls
security/libgcrypt
security/libgpg-error
...
x11/gnome2
archivers/file-roller
archivers/gtar
archivers/lzop
archivers/lzo2
...

gnutls doesn't depend on lzo2 initially, but lzo2 is dragged into the
mix via other dependencies and is built by the initial 'make'. The
subsequent package generation for gnutls adds a pkgdep line for lzo2
to gnutls' +CONTENTS but the pkgdeps in sophox-packages' +CONTENTS
has gnutls *before* lzo2.

As a result, sophox-packages cannot install; gnutls fails because lzo2
is missing, 82 more packages fail because gnutls is missing and the
whole thing spirals into a super-confusing mess!

MFC after: 3 weeks

Remove support for RELENG_4 (__FreeBSD_version < 500039).

MFC after: 1 day

- add: Keep dependent packages too if -K is specified.
- updating: terminating '\n' is not part of the package origin.
- bump PKG_INSTALL_VERSION to 20080612.

PR: bin/119368 [1], bin/124459 [2]
Submitted by: gcooper [1], Beat Gatzi <beat@chruetertee.ch> [2]
MFC after: 3 days

- Restore functionality broken in previous commit; we need to be able to report
multiple installed packages with the same PKGORIGIN.

Reported by: marcus
MFC after: 1 month

Optimize package registration/deregistration. Previously, when looking up the
package name for the origin of a dependency, all entries in /var/db/pkg were
traversed for each dependency of added/removed package. Now, gather all the
origins first, then do the lookup in a single pass over /var/db/pkg.

This should provide a major speedup for packages with hundreds of dependencies.

Submitted by: rdivacky (earlier version)
MFC after: 1 month

Improve style a little and remove one always-true condition.

Approved by: portmgr (pav)
Approved by: kib (mentor)

Add -i option for package installation without fetching and
installing its dependencies.

PR: bin/117065
Submitted by: Vladimir Ermakov <samflanker@gmail.com>
MFC after: 14 days

Add -F option to not fail on already installed packages.

Submitted by: Eugene Grosbein <eugen@grosbein.pp.ru>

Add -K option to pkg_add and pkg_info.
This option saves packages to PKGDIR (if defined or current directory by default)
as they are downloaded.
Silent a warning when -n is used and package has a +DISPLAY file.

Approved by: krion
MFC after: 1 week

Add -P flag, it does the same as the -p option, except that the
given prefix is also used recursively for the dependency packages,
if any. If the -P flag appears after any -p flag on the
command line, it overrides it's effect, causing pkg_add to use the
given prefix recursively.

PR: bin/75742
Submitted by: Frerich Raabe <raabe AT kde DOT org>
MFC after: 3 days

Use the length modifier 'll' instead of 'q' to print long longs.

- pkg_add spawns itself as argv[0] when installing dependent packages, to
enable the use as a port on older systems

- use absolute paths in all calls to external programs, to account for strange
PATH settings

- use INDEX or INDEX-5 depending on FreeBSD version, to enable the use on
FreeBSD 4.x as a port.

- conditionalize all 4.x/5.x differences on __FreeBSD_version, so that the
pkg_install tools can be kept in sync on 4.x and 5.x

- Bump PKG_INSTALL_VERSION

Reviewed by: portmgr (bento run)
MFC after: 4 weeks

- pkg_info: flag -r: (show packages this packages depends on (documentation change))

- pkg_info: new flag -j (show the requirements script)

- pkg_info: fix verbose output when used on packages

- better handling of corrupt entries in /var/db/pkg

- differ between corrupt entires and packages not installed

- various small fixes

PR: 56989, 57016, 57029, 26468

* Add explicit conflict-checking to the package tools. Packages can
register a list of other packages with which they conflict (via the
-C option to pkg_create), and they will refuse to install (unless -f is
specified) if one of the listed packages is already present.

* Update documentation for the new feature as well as fleshing out some
existing documentation.

* Bump PKG_INSTALL_VERSION so this feature can be tested for.

Submitted by: Sergey Matveychuk <sem@ciam.ru>
PR: bin/47145
MFC after: 2 weeks

When installing package from a local file assume that all subsequent
autoinstalled dependencies will have the same extension, not just
".tbz".

Pointy hat to: obrien
X-MFC after: -1 day

Find remaining hardcoded gzip bits and change to bzip2 bits.
Mark with XXX so someone that cares about being able to handle either
bziped or gziped packages knows where the remaining nits are.

- Make use of DEPOROGINs (if there are any) when installing package;
- fix few bogosities here and there;
- move some common routines into the library.

MFC after: 2 weeks

DTRT for packages read from STDIN:

$ cat pkg.tgz | pkg_add -

The above command line will fail on -CURRENT or -STABLE, and
therefore, so will sysinstall if you try to install additional
packages through the network (FTP) from a multiuser system. Because
of the different environment during installation (wrt the playpen),
this bug does not manifest itself during initial installs, and users
may install packages from the network just fine at that time.

This bug was fixed in OpenBSD 4 years ago.

----------------------------
revision 1.4
date: 1998/04/07 05:56:13; author: marc; state: Exp; lines: +13 -8
fix package input from standard input -- the program tried to process
stdin twice. Note: it assumes stdin is a compressed tar file.
----------------------------

PR: conf/36606
Obtained from: OpenBSD
MFC after: 2 weeks

Fix SCM ID's.

WARNS=2 cleanup.

Tested on: i386, alpha
MFC after: 2 weeks

(forced)

Forgot to mention: creation of a full-fleged package archive requires some
changes in bsd.ports.mk, because currently it doesn't install some files
that are usually packed into the package archive (+MTREE_DIRS, +DISPLAY
and so on). Support for this will be committed later.

- Allow package archive to be created from a locally installed package. This
allows for an easy way to backup old version of port prior to installing
a new one;
- silence compiler warnings by killing some unused variables and adding
all includes necessary.

MFC after: 2 weeks

Remove duplicate umask(2) call.

PR: dusty/7998
Submitted by: Stefan Eggers <seggers@semyam.dinoco.de>
MFC after: 2 weeks

Reduce private "lesser known" function redirection to improve clarity.

Approved by: jkh

Cosmetics: replace dozen instances of "(tmp = getenv(PKG_DBDIR) ? tmp : DEF_LOG_DIR)"
with macro.

MFC after: 1 month

Usability tweak:

Use '' quotes instead of `' to delimit names of files and packages in
warning and error messages, because it is easier to cut-n-paste name in
question that way (single click) without confusing the shell. And yes,
I know that it is less eye-candy...

MFC after: 1 month

Properly quote recursive pkg_add operations so that filenames with
spaces or special characters in them won't barf.

Style policy: reformat multiline comments to conform to style(9).

Fix a bug arised from the ')' in the wrong place when calling printf()-like
varargs function, which lead to one of the arguments was left out. This resulted
in failure when inwoking mtree, warning message "mtree returned a non-zero
status - continuing" and probably is the reason for zillion mtree errors on
bento.

Add `_PATH_DEVZERO'.
Use _PATH_* where where possible.

Redirect stdout from mtree to /dev/null; we don't really need to know
the list of directories being created when we install a package.

$Id$ -> $FreeBSD$

-Wall fixes. Ambigious if/elses, uninitialized values, missing headers,
missing prototypes, and empty format strings.

Make it possible to have separate install scripts as well as have
an on-delete script.

Submitted by: Rajesh Vaidheeswarran <rv@fore.com>

correct usage of cleanup() everywhere - this became bogus when the warnx/errx
stuff was added and nobody seems to have noticed. :)
Noticed by: Josh MacDonald <jmacd@paris.CS.Berkeley.EDU>

Fail as documented when a dependency cannot be found
PR: 7318
Submitted by: djv@bedford.net

Fix recursive package add problem (for on-disk package files) I introduced
with the change in semantics to make_playpen(). Now package dependencies
more than 2 levels deep should work again. Pass the hat.

Avoid race-conditions on playpen delete.

Eliminate the idea of nested "playpens" entirely - it just obfuscated
the code and, in at least one case, made it more dangerous to no gain.

I must have been smoking crack when I made this change. Using sh -c
in a vsystem() call is just a ludicrous way of calling sh -c "sh -c ..."
Whoops! I don't know my own bloody code here!

MF22: fix to install script handling.

Don't assume install script is directly runnable - use sh -c.

- do string-magic on the right variable when trying
to find dependent packages on the local disk (i.e. package given as
filename, not URL)
- document $PKG_PATH
- use -v on pkg_add for dependent packages

Submitted by: hubertf@NetBSD.ORG
Obtained from: NetBSD

Don't return from cleanup() - if it's removed the playpen then very
bad things will happen if we return to unsuspecting code.
Noted to my great dismay by: Tim Vanderhoek <hoek@hwcn.org>

Use err(3). Cosmetics in usage string and man page.

Add support for @option preserve - moves existing files out of the way
before replacing them (using pkgname to make this hopefully unique).
Delete also moves them back, if they exist, resulting in a package
which can be "backed out" with reasonable safety.

Revert $FreeBSD$ to $Id$

Fix a bogon with playpen size calculation for pkg of "-". It's amazing
this ever worked, actually.

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Correct a problem which broke +DISPLAY file handling.

Fix PR#1557 - chained package dependencies using URLs didn't work.

Make -force work for pkg_add.

Make pkg_install understand and use libftpio instead of its own
copies of the ftp support routines. Also some cosmetic and minor
bug fixes I've been meaning to incorporate for awhile.

fix PR#599
Submitted-By: jdp

Fix a couple of miscellaneous bugs and make pkg_add also support reading
from stdin.

Bring my pkg_install improvements forward from 2.1.

Fix the `pkg_add relative to .' bug. Thank you, Ville!
Submitted by: Ville Eerola <ville@vlsi.fi>

Bring forward my changes from 2.1

Some fixes to make this "TMPDIR agile".
Submitted by: jmacd + some of my own fixes.

Make "out of space in ${PKG_TMPDIR}" handling a bit more robust.
Print the right thing, do the right thing. Back to you, Satoshi!

o Fix PR# bin/643 - always set "where_to" properly.

o Nuke dependence on libftp - it's about to die!

Merge RELENG_2_0_5 into HEAD

Remove trailing whitespace.

Use mtree -U instead of -u and put back error check. Thanks Rod!

Don't bother checking the return status of the mtree command - it returns
non-zero stats when it changes things, too.

Further work on making all this more robust in the face of failure.

Also allow URL specification for a package. This works for things the
package may depend on, too.

Allow PKG_PATH to be used anywhere a package is being searched for.

Check dependencies BEFORE doing final extraction of package.
Fix a bug where direct-extract mode would have still invoked the
packing list reordering commands.

My handling of the missing dependency but !Verbose was wrong. Make
the right message come out.

Second round:

1. Make paths work correctly.
2. Make pkg_add generally more robust in the face of failure.
3. Make the depend messages come out on stderr or stdout, but not both
interspersed! :-)

1. Add an new @option directive. You'll understand it better in a moment.
2. Fix a long-standing bug in pkg_add where the failure of one package in
a multipackage installation (pkg_add *.tgz) would blow you right out of
the water. Ick.

Greatly simplify the calling of mtree - the error handling wasn't working
anyway, at least not with FreeBSD's mtree, and the arguments were being
built incorrectly.

Add support for the ${PKG_DBDIR} environment variable. /var/db/pkg is the
default, and probably not something you'd ever change, but now it's
possible to do so for local/custom installations.

Neither of these tools really told you what you needed to know neatly
and succinctly in verbose mode. I hope people find this more to their
liking.

Add true support for dependency lists.

1. pkg_create now has a -P argument for specifying dependencies on the
command line.
2. pkg_add will honor dependencies and chain-load them automatically if
it finds the required package(s) in the same directory as the package
that is being loaded. For best results, install packages from a directory
containing all the packages you'll possibly need
(like /usr/ports/packages/all).

2 remaining flaws:

1. pkg_add looks in one place (where you were when you loaded the primary
pkg) for depended packages. If you can come up with a search path scheme
that's not a total hack - be my guest!

2. Recursive dependency expansion can result in the name of a dep being
listed more than once. This doesn't bother pkg_add since it checks
for package existance with pkg_info and will skip already-loaded packages.
I don't know how/if pkg_delete handles this yet, however. I need to look
into it.

Many of John T. Kohl's patches from NetBSD. Thanks, John!
Submitted by: jkohl

Protect against old packages with bogus @name fields. Use basename_of()
on the name before recording it in /var/db/pkg. Sorry, Kaleb, you
were right after all! It was my fault! :)

Whoops - variable was in wrong scope. Correct.

Fix bug in home directory calculation that I introduced with the free space
checking code. Fix the free space checking code, while I'm at it! ;)
Thanks to Garrett for spotting the home dir problem.

Add the ability to verify that a package won't extract in the space
available. Thanks to Michael Elbel for pushing me in the right direction.

The environment variable PKG_PREFIX is set to the first @cwd command
in the packing list, or the argument to -p if it is specified, before
the requirements/installation/deinstallation scripts are called. This
enables the scripts to be written to work on the final installation
destination, even if the user uses -p to override the package's default.

Remove find_name(), replace with more generic find_plist() function.

If require script fails, don't try to delete installed package.

Fix problems with executing from current directory

Make sure permissions on registry location are readable by pkg_info.
Add new % sequence expansion for command execution.

Made staging area configurable rather than assuming /tmp (-t)
Made pkg_add -n less bitchy.
Made -p in pkg_add override "default" package location in pkg_create.

Added concept of "slave" and "master" modes, allowing many different things
to be done to packing lists on the fly. This is really something for
front-end drivers to use, but also something for the more expert user.

rename()'ing files before trying to copy them, thus saving
considerable time when targets and staging area are on same filesystem.

Lots of misc tweaks, support for arbitrary separators in pkg_info, more
intelligent name handling in pkg_create. Most of these files are changed
because of rcsid's being different in my cvs tree and freefall's (foo).

This commit was generated by cvs2svn to compensate for changes in r327,
which included commits to RCS files with non-trunk default branches.

The release version of my package install suite. Please see man pages
for info.