put "stub" within grasp of -DREGRESSION_TESTING.
Extend it slightly: do not stub quirks, so that caching mechanisms work
as usual even when using stubs.

use more specific regression testing knob

fix esoteric error message in case PackageName parses stuff to a bogus
fullpkgname, as seen by Matthias Schmidt

add some more fluff to documentation

fix some very minor syntactic nits

tweak fragment handling error messages a bit
don't pass parameters we don't need

move to use v5.36;
tested by me over the last few weeks, and tb@
also fixed a "manual install" bug properly reported by tb@

aside that there should be *no functional change*.
If you see any message like "hey, the number of params is wrong"
it is a fringe case I didn't run into and should be easy to fix.

don't pass state, we get them from the progressmeter

gc unused/old code

move the interface to SharedLibs to be somewhat object oriented

accordingly, load it "just in time" in State.

Most calls get simplified, and we can save more state for later.

stop using old-style prototypes except where strictly necessary
(for try/catch)

signatures will be much more powerful once I move to 5.36

use the new set_destdir interface

change naming convention for the lru "save history" cache, so that
ports like "lang/chicken/core" do generate files like lang.chicken.core.lru
instead of lang.chicken.core (which can create confusion in people's mind)

do so transparently by reading the old file if need be, and removing it
afterwards.

Funny thing noticed by tb@

ok tb@, sthen@

Have -S actually behave like the other introspection options, namely only
eschew building the package is -n is mentionned.

Document that -S -n is heavily optimized for speed since it's mostly used
by dpb -R to figure out what to rebuild.

fix a logic error from 2018: be silent if any of -S, -n, -q are mentioned.

The actual bug reverted in 1.128 was from "make print-plist-libs"
which would invoke pkg_create -n -Q and filter out the libs: but
in that case, pkg_create would not be silent, thus yielding
reading plist|-/usr/local/lib/libpython3.9.so.0.0
to filter, which obviously wouldn't work.

So, turn on silent mode for -Q as well.

if we can't open the user db, don't try to read from a closed fh

some old signing code was still around

have compute_checksum return a status, so that we don't report similar
errors several times (for instance, if a file doesn't exist, its checksum
is not going to match, obviously)

have pkg_create generate the hash for always-update

clean up and document the LRU stuff better
- create a separate path when we don't want history
- add a dummy entry to see the threshold between changed and unchanged
in verbose mode
- document better the various stuff that gets separated
- error out in case we want history (by default) and anything goes wrong

remove the gzip chunk after CONTENTS, it was only needed back when
we signed the packing-list separately. Now that we use signify to
create the signature without decompressing anything, it's completely
irrelevant.

yet another stupid shadowed variable, thx sthen@

explicitly write LRU in a comment so that it's easier to find ;)

allow disabling the @ts tweak, which may come in handy for new fw_update
that doesn't grok them.

fix typo (noticed by sthen@)

add extra support to pass REVISION, EPOCH, FLAVOR_LIST from the ports
infrastructure, so that pkg_create can catch some naming errors.

remove test that's now bogus, as fullpkgpath takes precedence.
thanks bluhm@

- grab fullpkgpath early (to decide: hard error if not provided) ?
- tweak the look-up mechanism for dependencies to NOT look at installed
packages for inter-dependencies between subpackages (and debug packages)
of the SAME pkgpath.

The rationale for looking at installed packages in the usual case
is that it's normally much faster rather than have the ports tree
spew out packing-lists.

In the case of multi-packages updates, the new version may have been built
successfully (and correctly) while the old version is still around.
However, if pkg_create looks at the installed version, it may fail to create
the package because the shared library versions will be different.

More DESC -> DESCR; spotted by espie@

tyop: DESC -> DESCR

ok landry@ robert@

give pkg_create knowledge of WRKOBJDIR, so it can error out if a symlink
points in there.

okay aja@ sthen@

nit, use "fatal" with parameters as intended

have ask_tree be more specific in its error reporting.
make sure the child has the same effective and real ids,
as perl interprets it as "setuid, refuse to debug"
go to the end of the pipe so the child doesn't exit
prematurely (didn't use to happen but now it does)

help from afresh1@ in diagnosing that one.

when asking the ports tree, we fork a child.
instead of ditching stderr entirely, keep it in a temp file, and if the
child exits with an error, we got something to display.
(note that returning and undef'd plist is enough of an error, just we
had no clue what went wrong previously)

aja@ ran into this a few weeks ago.

start re-adding the non-broken changes:
- commonalize try{} catch {} for pkg_add/delete and pkg_create, so that
debug works the same way in both.
- switch printing command name to the catch handler, so that exceptions
are simpler to handle

backout previous commits for now; some of this is shared with dpb and
results in problems. hints from pvk@ about what introduced the breakage
we were seeing, who confirms that this backout fixes things; comitting
early to unbreak package builds.

====
a bunch of changes, all related to error-handling:
- have Handle->register also create a proper END block, so that
individual packages don't have to, and explain the issue
- kill old Unlink/Copy code that migrated to State years ago
- commonalize try{} catch {} for pkg_add/delete and pkg_create, so that
debug works the same way in both.
- switch printing command name to the catch handler, so that exceptions
are simpler to handle

and a few comments for the hairy parts...

Members:
OpenBSD/AddCreateDelete.pm:1.44->1.45
OpenBSD/AddDelete.pm:1.87->1.88
OpenBSD/Error.pm:1.33->1.34
OpenBSD/PackageRepository.pm:1.164->1.165
OpenBSD/PkgCreate.pm:1.160->1.161
OpenBSD/State.pm:1.58->1.59
OpenBSD/Temp.pm:1.34->1.35
====

a bunch of changes, all related to error-handling:
- have Handle->register also create a proper END block, so that
individual packages don't have to, and explain the issue
- kill old Unlink/Copy code that migrated to State years ago
- commonalize try{} catch {} for pkg_add/delete and pkg_create, so that
debug works the same way in both.
- switch printing command name to the catch handler, so that exceptions
are simpler to handle

and a few comments for the hairy parts...

move the display of cmd into fatal, and don't say it's a fatal error, it's
already obvious

tweak the way we report bugs
- the catch/catchall distinction is no longer relevant, strip some info
by default now
- use state->{cmd} to display the command name properly

This makes sure final displays show the command name.

make pkg_create fail properly (return 1 is useless there)

OpenBSD::Temp error handling as well

actually use error, we can delay fatal problems

check ::Temp creation

oops, let cdrom normalize if it's there.
Noticed by sthen@

make PERMIT_CDROM info optional
as discussed with sthen@, code slightly adjusted

actually heed localbase when looking for groff

reorg groff runner so that failures are handled better
do the logic for manpage formatting better, so that we can't miss things
simplify filenames, fullname always has a slash

kill the solve_tags part of pkg_create for now, as it doesn't work at all

get ready to reuse verify_tag

solve_tags not yet functional, I'll reactivate the checks during creation
when it's cleaned up

well, when we read the plist from the ports tree, we need to register
the tag info similarly to registering libraries

PkgCreate has no business knowing details of define-tag

do some more basic checking of tags vs define-tag so that parameter usage
is handled more gracefully

move the tag base code into SolverBase, and use it in PkgCreate, so that
we refuse to create packages with tags without reachable definitions

split the common half of dependencies handling into SolverBase, as PkgCreate
only uses that

kill Frag/NoFrag, they only make sense in update-plist, so they've been
moved there.

Add a -Dstub option to facilitate creating/adding/deleting "stubbed" packages
with just the packing and dependency info with none of the files.

(useful for debugging tricky cases that involve large packages)

if the parser can't find a starting line in user db, it won't look
at any line, so it won't error out.
Add logic to make it error out.

port -u userlist option from update-plist, as suggested by sthen@.
main differences:
- error out on a problem instead of warning
- always parse the userlist if the option is specified. Avoid ticking time
bomb effect on bulk builds, where packages would be fine until the first
@new(user|group) usage in case the user list got corrupted.

Note it's still an option, so if you're using pkg_create outside of the
ports tree, you can do whatever you want.

adjust highly misleading comment to match reality

on the other hand, commenting this out was involuntary.
ask_tree will often say fun things because of (say) pipe closes

the perl debugger tells you everything a perl process exits.
this child is irrelevant, tell the debugger (if present) to not stop

if we create fragments, we must know which files they come from

update-plist wants to ask the ports tree as well, so refactor so that
it can reused that code

need instrumentation for fragments

reorg error handling so it's always passed to class methods that can
be overridden. Useful for nextgen update-plist

sanitize say/print output further
- have confirm go thru f() every time (adding
confirm_defaults_to_no/confirm_defaults_to_yes apis... verbose but effective)
- check DESCR in pkg_create, as suggested by sthen@
- don't do the whole safe dance when not needed (optimization)
- misnamed allowed -> forbidden (schwarze@)

This effectively prevents DESCR from using diacritics. Sorry!
Ingo makes a compelling argument that using UTF8 while installing a new
box is currently fairly dangerous, considering tty setups.

This affects about 10 ports at the moment, all of which were fixed before
this commit.

add the glue to generate future libset, which are currently 100% unused

@libset is going to be part of the dependency information.

switching to _pbuild is tricky, you can't take it back.
in particular, disable future PORTS_PRIVSEP mechanisms under
pkg_create, because we should already be running as _pbuild
by this point.

Revert previous, it breaks ports/infrastructure/bin/resolve-lib (I think)
with at least python and ruby.

ok naddy@

typo, surprised nobody noticed yet

clean-up option handling slightly, some more to be done.

Add a -S option that computes signatures, so that we may remove
code from bsd.port.mk that does the exact same thing, and no longer
work since the addition of @version.

For now, -S is exclusive of other package creation options, as it exits
as soon as it computes what it needs.

better diagnostics

-V option for pkg_create and incorporate version into signatures.

kill creating old signatures.

remove code that allows pkg_create to sign

pass file name thru to Subst for better error messages.
reported by Adam Wolk

kill remnants of non shared arches

streamline the code that checks meta information before extracting files
- verify_modes should only verify modes
- put the whole checking code in one single routine in validate_meta, makes
the size checking code more obvious
- document what's going on
- prevent PkgCreate from creating impossible packages, only a few Ustar
objects are actually usable.

the Solver uses kept_names, so we need these there too
found by aja@

let pkg_create recreate all packages correctly, again.

simplify progressmeter calls to visit_and: we retrieve the state from the
progressmeter object

have visit_with_count and visit_with_size use the same parameter conventions.
In particular, pass state before the other method parameters.

when we run into real errors, the progress meter is distracting, so have
a method ->disable that does something nice to the display, and stops
the progress meter right before we display our first real error.

final bulk shows chunking to happen in all the right places, and signing
works correctly as well. So activate this.

This basically makes package snapshots rsync-friendly with a low-tech
approach. Instead of having a patched gzip, we borrow from other archive
formats, and cut the archive every few files. Since the files are already
sorted in order of date of modification, packages that don't change tend
to end with identical gzip archives.

Note that this will still work when/if we transition to bzip2/xz packages...

need meta stuff that tells me those are "real files" to not create extra
chunks at the beginning of the package...
Also, be very verbose about everything.

.py files no longer need to be special, as the timestamps are stored in
the plist itself, so there's no longer any worry of the timestamp not
getting updated and python throwing a fit.

fix display of comments to use "say" properly.
Prepare scaffolding to be able to chunk archives

store explicit timestamps in generated plists.
this prevents updates from fucking up mtime of new file, thus the
"python exception" no longer prevents python files from being reordered.

Note that this requires newish scaffolding in pkg_add proper. That
scaffolding was committed prior to 5.6, in preparation for this change.

prevent users from adding timestamp themselves

activate "out-of-order" archives based on history
things to watch out for:
- correct handling of multiple files with the same sha
- cwd handling. This is already destated, so we can call set_destdir on
each entry's cwd, instead of relying on @cwd being in the "right" location
as that's no longer the case...

makewhatis being now external, it no longer requires picky/test/check_dir.
It also can't handle too many manpages at once (shell limit) so hand-feed
it (problem spotted by landry@).

Some tests by schwarze@.

Switch to the new makewhatis(8)/apropos(1)/whatis(1) combo.
"commit the switch now" espie@ "go for it" deraadt@

See the apropos(1) manual for a description of what's new.
On machines where you want the full functionality,
run "sudo makewhatis" and put "MAKEWHATISARGS=' '" into weekly.local(8).
Otherwise, when upgrading via source, run "sudo makewhatis -Q".

don't use $_ as "throw-away" variable, proper idiom is "undef"
(and if you're wondering about the unused extra parameter, this is
designed to allow a new makewhatis that shares more code with pkg_create)

remove $_ usage that warns under perl 5.18

refactor solve_from_ports a bit.

do the write to cache thru a temp file, so the end result is atomic,
to be used to enable a global depends cache for parallel building in
the near future.

pseudo handle needs dependency_info too, forgot that one.

tweak the way we create and sign packages significantly:

all modern compression formats support multi-stream archives.
Store the packing-list as a separate stream, and note where the
first stream ends when reading back the packing list.

When pkg_sign sees a split-stream archive, it does not need to unpack/repack
the whole archive: it can just sign the packing-list, close the stream, and
copy the next stream(s) from the source archive verbatim.

This is perfectly transparent to pkg_add, but abuses IO::Uncompress::Gunzip
internals slightly (to be solved with cpan).

also adds explicit option -C for generating a SHA256 list of all packages.

and okay naddy@

clean-up synopsis

zap the sign-only part (moved to PkgSign.pm).
reuse the signer part (move to Signer.pm)

for the SHA256 file, output checksums in base64; ok espie@

involuntary sneak preview, revert

...and sort SHA256 properly

tweak the interface to generating signatures yet again.
- assume key names match, deduce signer from sec key.
e.g., -s signify -s 55pkg.sec
will set signer to 55pkg and look for a pubkey named 55pkg.pub,
either besides 55pkg.sec or in /etc/signify.
- verify there's no mismatch, if possible, by verifying the first package
signed.

- also build a SHA256 on the fly while signing.

say something if we sign an empty repository
tweak mode on tmp signed file

bump copyright to 2014

if we're re-signing, check old sig first. shouldn't sign stuff we don't
trust.

@signer makes sense only for signify, so move it there.

simplify a bit: pass the first SIGNER for @signer.

switch to internal gzip/gunzip.
reduces the number of external processes and the complexity of the code.

tested on a few select arches by tobiasu, naddy. If it breaks somewhere,
tough. This one is simple to revert.

tweak signing yet again. Have pkg_create automatically add signing
identities every time, and make matching identities mandatory.
e.g., pkg_create and pkg_add must have matching -DSIGNER.
by default, signer is derived from uname -r and role (pkg_add/fw_update),
e.g., 54pkg, 54fw...

integrate signer logic and fix resign with -DSIGNER

if HISTORY_DIR is defined, create a new "permanent" file during build.
this file records the sha256 of all elements in the archive, and keeps
track of changes (quite simply: sha256 that were already present end
up at the end of the file).

Start recording these *now*, so that we have enough info to shuffle
packages later.

allow processing of package lists in parallel.
Turns out re-gzipping a signed package is really expensive, so turn
it into several jobs with a -j option.

more forbidden elements

vendor->signer
shorten pubkey -> pub (naddy@)
\\ -> \e (jmc@)

make sure elements are not set manually, namely forbid
@depend (use -P)
@wantlib (use -W)
@name (duh)
@arch
@localbase
@option manual-installation
@comment pkgpath=...
(some of this is actually redundant, but will give more explicit error
messages)

simplify handling of special files some more, that stuff has fixed names
anyways

make the special file creation more explicit, so that the order actually
doesn't matter

be more explicit in why we don't write DESC
return early instead of adding an extra level of test.

zap PKG_PREFIX (undocumented and unused, now)
okay landry@

signify support

@vendor annotation

fix sign-while-creating mode

sign package thru a signer object, instead of hardcoding signature parameters

move ntogo code, so it can be used while copying packages in pkg_create.
add the few lines necessary for that to work.

get rid of "copy_long", we don't really need to recheck names while copying
a package contents.

Allow -o/-S for output/signature dirs for batch operations.
Fix copying of packages while signing, add progressmeter.

If the fake target of a port installed an empty (0-byte) file as a
manual page, pkg_create(1) asked groff to process the empty file,
which failed, and then decided to install the empty file as a source
manual page, which later caused mandoc(1) called from man(1) to
complain to the end-user. Instead, detect the emptiness in time,
skip the formatting attempt, skip the file, process the packing
list to its end, then abort pkg_create(1) unsuccessfully such that
the porter sees the problem and can fix whatever the root cause is.

Problem reported by naddy@ as seen in p5-Carp-Datum during a bulk build,
solution is joint work with espie@, again tested in a bulk by naddy@;
"please commit" espie@.

simple optimization: if we're already erroring out, there's totally no
need to compute those pesky checksums.

activate new pkg_create magic.
Note that, in most cases, PFRAG.shared are probably no longer needed...

put possible fragments in there.
fix buglet in parsing libraries

supplementary magic code, not active yet

less hackish test for LIB*VERSION: after building the element,
redo a check based on the non-subst version, so we can:
- actually parse the library name
- check we depend upon the right variable

well pkg_create itself doesn't care, but a custom ->annotate will get
very confused

make fragment handling yet more customizable

allow a subclass to override that specific error

make the reading of packing-lists amenable to further annotations, such as
those that will be required for update-plist

move code around a bit more

more refactor

just a bit of refactoring, make sure option handling is done in state,
as should be, so we can reuse it without needing to duplicate all of
pkg_create code.

warn when @option no-default-conflict and not explicit @conflict.
(very seldom correct, mostly used for autoconf/automake)

sanitize stuff a bit more...

Backout activation of the new apropos(1)/whatis(1)/makewhatis(8).
In its current state, it causes too much slowdown, in particular
during system builds, and there are other regressions.
That cannot be fixed quickly while it's enabled.

Problems pointed out by espie@, backout requested by deraadt@,
diff "looks good" to espie@.

Enable the new apropos(1), whatis(1), and makewhatis(8).
Unlink the old apropos(1), whatis(1), and makewhatis(8) from the build.
Call the new tools from pkg_create(1) and pkg_add(1).
"Please enable it now." deraadt@

if the portstree gives us a cache directory, use it.
in lots of multi-packages scenarios that all depend on the main package,
this speeds things *a lot*.
case in point: in php 5.2, make package goes down from 1mn40 to 53s.

much better solver: don't try to solve every library, but just the ones
we have in the package we're trying to create.

So, solve one level, and keep solving until we have what we need or we
run out of level. Any package without @wantlib and lots of runtime depends
will build MUCH faster. Other packages MAY build faster.

use print-plist-libs-with-depends (and make sure it does not skip depends)

small optimization: tell pkg_create to skip !libs if LIBS_ONLY (to be
used in make print-plist-libs)

remove hack, ask for SUBDIRs with FULLPATH set

work-around for default vs. no flavor

speed up dependency checking:
- if the ports tree is -current, do not compute
@wantlib during dependencies.

- add an extra cache. The solver caches "exact" depends, we can also cache
pkgpaths when we go to the ports tree.

- make the checking computation progress message less painful by adding the
pkgpath currently investigated...

zap extra try that's no longer needed: print-plist-with-depends should
always succeed since it only depends on the ports tree itself.

change yet again how we handle transparent formatting of manpages.
simplify: actually create a tmp directory hierarchy that exactly mimics
what fake contains. This avoids trouble with makewhatis checks.
Don't even try to remember what files are there, just wipe it all with
rmtree.

don't erase 1st manpage in case two manpages have the first name.
as noticed on texlive, among a host of other issues...

I can remove duplicate flag since that's handled upwards.

run makewhatis during pkg_create in verbose mode.
tweak temp manpage generation a bit, so that it has correct names in its
own directory, because makewhatis will be really pissed off otherwise.

add yet another check: obviously in a @depend line, if the def part doesn't
match the pattern, the package has something wrong.

prevents some bogus tk LIB_DEPENDS from even packaging.

go all the way, put temp pages in /tmp

create tempfile directly under fake, no need to go down.

create formatted manpages as temporary files we remove after the fact.

handle failures from ports tree with more care.
this does avoid infinite loops in case of wrong plists.
necessary since print-plist-with-depends CAN fail...

simplify some more: if it's not installed, ask the ports tree, who knows,
we will catch more mismatches between spec and actual packagenames ?

set status during dependency check

don't check when simply printing the plist, this 'fixes' a bit of
port-lib-depends-check

don't fatal(), error() allows for more stuff to show up

once we read the list, make sure it matches the dependency.
Otherwise, this WILL turn into an infinite loop indeed !

simplify a bit:
- make find_dep_in_installed more regular
- zap the {more} entry, add_dep is okay for that purpose
- don't special case installed_list, provide older_names instead
- make PseudoSets slightly less hackish
- don't read full packinglist, prelinkstuff is exactly what we need

use a better strategy: ask the ports tree for the plist for packages with
the same pkgpath. Thus, PseudoSets need to have a real list, and
pkgcreate needs to loop to solve those depends.

reactivate the check

disable temporarily

new check in pkg_create: walk the dependency tree for @wantlib, thus
preventing packages that won't install (much easier to fix that when
you still have the fake tree around !)

specifically, look into depend that are installed, and possibly in the
fake tree: if the package does build, it requires the libraries to be
around anyways, so pure RUN_DEPENDS are not an issue (and avoid old libraries,
if we find wantlib only in .lib, something really fishy is going on).

Reuse the Dependencies class, so we're sure to have the exact same way
to find libraries.

Also fix exit codes from pkg_create...

use state->defines

tweak the framework so that individual modules don't exist and return
the exit code upstream instead.
this simplifies the task of people who want to reuse it, as noted by
landry@

minor fix for ports which install manpages in other directories: keep
track of all known directories, so when we generate a manpage in a new
catN directory, we add the entry and it gets deleted properly.
-> requires a bump to affected packages.

initial scaffolding for @rcscript: for now, acts like normal files,
but allow absolute pathnames (treat that as an implicit @cwd).

@symlink/@size/@link are generated automatically, make pkg_create(1) error
out if we create them manually, instead of erroring later during pkg_add.

just bail out of producing empty files.
*this will become an error eventually, please fix those*

use newer system for more efficient redirection, and chdir the way man(1)
does, so groff is happier with .so.
(agreed by ingo)

whitespace

pass state to Ustar objects, use it to display those pesky error and
fatal messages.

zap $opt_x from pkg_mklocated, do things like other commands do.

fix pkg_mklocatedb

we can't checksum directories either, so give a better error message

allow say and errsay to work without parameters, as it's ways common.
create verbose_system up in state, because it makes sense without verbose.

whitespace cleanup

pass state along when signing packages, uses repo and fatal directly

use locator object.
rework options into state

display bad symlinks by destination

zap empty message...

don't call working() in -q mode.

no need to fatal(), we can error() and keep going.

warns if packages contain symlinks that point to non-existent places.
actually error out if a symlink points to itself.
this breaks evolution, until someone fixes the bogus symlinks.

make error messages more consistent. Add an extra sanity check which
shouldn't trigger for special files

ui changes: go thru a state object for most printouts

tweak tweak so it works

tweak fragment handling a bit more. In particular, protest when we can't
get a fragment, regardless of whether or not we're going to use it.

display "switching to" with ->set_status

clean-up into smaller functions

add working cartwheel

move use to common part

add ->spec to Conflict and use the same check as for depend during pkg_create.

move code around, so that commands can be used as modules.
pkg becomes the start hub, which does nothing except require the
correct module.
Saner code wrt Add/Delete, and more sharing.

use more specific regression testing knob

fix esoteric error message in case PackageName parses stuff to a bogus
fullpkgname, as seen by Matthias Schmidt

add some more fluff to documentation

fix some very minor syntactic nits

tweak fragment handling error messages a bit
don't pass parameters we don't need

move to use v5.36;
tested by me over the last few weeks, and tb@
also fixed a "manual install" bug properly reported by tb@

aside that there should be *no functional change*.
If you see any message like "hey, the number of params is wrong"
it is a fringe case I didn't run into and should be easy to fix.

don't pass state, we get them from the progressmeter

gc unused/old code

move the interface to SharedLibs to be somewhat object oriented

accordingly, load it "just in time" in State.

Most calls get simplified, and we can save more state for later.

stop using old-style prototypes except where strictly necessary
(for try/catch)

signatures will be much more powerful once I move to 5.36

use the new set_destdir interface

change naming convention for the lru "save history" cache, so that
ports like "lang/chicken/core" do generate files like lang.chicken.core.lru
instead of lang.chicken.core (which can create confusion in people's mind)

do so transparently by reading the old file if need be, and removing it
afterwards.

Funny thing noticed by tb@

ok tb@, sthen@

Have -S actually behave like the other introspection options, namely only
eschew building the package is -n is mentionned.

Document that -S -n is heavily optimized for speed since it's mostly used
by dpb -R to figure out what to rebuild.

fix a logic error from 2018: be silent if any of -S, -n, -q are mentioned.

The actual bug reverted in 1.128 was from "make print-plist-libs"
which would invoke pkg_create -n -Q and filter out the libs: but
in that case, pkg_create would not be silent, thus yielding
reading plist|-/usr/local/lib/libpython3.9.so.0.0
to filter, which obviously wouldn't work.

So, turn on silent mode for -Q as well.

if we can't open the user db, don't try to read from a closed fh

some old signing code was still around

have compute_checksum return a status, so that we don't report similar
errors several times (for instance, if a file doesn't exist, its checksum
is not going to match, obviously)

have pkg_create generate the hash for always-update

clean up and document the LRU stuff better
- create a separate path when we don't want history
- add a dummy entry to see the threshold between changed and unchanged
in verbose mode
- document better the various stuff that gets separated
- error out in case we want history (by default) and anything goes wrong

remove the gzip chunk after CONTENTS, it was only needed back when
we signed the packing-list separately. Now that we use signify to
create the signature without decompressing anything, it's completely
irrelevant.

yet another stupid shadowed variable, thx sthen@

explicitly write LRU in a comment so that it's easier to find ;)

allow disabling the @ts tweak, which may come in handy for new fw_update
that doesn't grok them.

fix typo (noticed by sthen@)

add extra support to pass REVISION, EPOCH, FLAVOR_LIST from the ports
infrastructure, so that pkg_create can catch some naming errors.

remove test that's now bogus, as fullpkgpath takes precedence.
thanks bluhm@

- grab fullpkgpath early (to decide: hard error if not provided) ?
- tweak the look-up mechanism for dependencies to NOT look at installed
packages for inter-dependencies between subpackages (and debug packages)
of the SAME pkgpath.

The rationale for looking at installed packages in the usual case
is that it's normally much faster rather than have the ports tree
spew out packing-lists.

In the case of multi-packages updates, the new version may have been built
successfully (and correctly) while the old version is still around.
However, if pkg_create looks at the installed version, it may fail to create
the package because the shared library versions will be different.

More DESC -> DESCR; spotted by espie@

tyop: DESC -> DESCR

ok landry@ robert@

give pkg_create knowledge of WRKOBJDIR, so it can error out if a symlink
points in there.

okay aja@ sthen@

nit, use "fatal" with parameters as intended

have ask_tree be more specific in its error reporting.
make sure the child has the same effective and real ids,
as perl interprets it as "setuid, refuse to debug"
go to the end of the pipe so the child doesn't exit
prematurely (didn't use to happen but now it does)

help from afresh1@ in diagnosing that one.

when asking the ports tree, we fork a child.
instead of ditching stderr entirely, keep it in a temp file, and if the
child exits with an error, we got something to display.
(note that returning and undef'd plist is enough of an error, just we
had no clue what went wrong previously)

aja@ ran into this a few weeks ago.

start re-adding the non-broken changes:
- commonalize try{} catch {} for pkg_add/delete and pkg_create, so that
debug works the same way in both.
- switch printing command name to the catch handler, so that exceptions
are simpler to handle

backout previous commits for now; some of this is shared with dpb and
results in problems. hints from pvk@ about what introduced the breakage
we were seeing, who confirms that this backout fixes things; comitting
early to unbreak package builds.

====
a bunch of changes, all related to error-handling:
- have Handle->register also create a proper END block, so that
individual packages don't have to, and explain the issue
- kill old Unlink/Copy code that migrated to State years ago
- commonalize try{} catch {} for pkg_add/delete and pkg_create, so that
debug works the same way in both.
- switch printing command name to the catch handler, so that exceptions
are simpler to handle

and a few comments for the hairy parts...

Members:
OpenBSD/AddCreateDelete.pm:1.44->1.45
OpenBSD/AddDelete.pm:1.87->1.88
OpenBSD/Error.pm:1.33->1.34
OpenBSD/PackageRepository.pm:1.164->1.165
OpenBSD/PkgCreate.pm:1.160->1.161
OpenBSD/State.pm:1.58->1.59
OpenBSD/Temp.pm:1.34->1.35
====

a bunch of changes, all related to error-handling:
- have Handle->register also create a proper END block, so that
individual packages don't have to, and explain the issue
- kill old Unlink/Copy code that migrated to State years ago
- commonalize try{} catch {} for pkg_add/delete and pkg_create, so that
debug works the same way in both.
- switch printing command name to the catch handler, so that exceptions
are simpler to handle

and a few comments for the hairy parts...

move the display of cmd into fatal, and don't say it's a fatal error, it's
already obvious

tweak the way we report bugs
- the catch/catchall distinction is no longer relevant, strip some info
by default now
- use state->{cmd} to display the command name properly

This makes sure final displays show the command name.

make pkg_create fail properly (return 1 is useless there)

OpenBSD::Temp error handling as well

actually use error, we can delay fatal problems

check ::Temp creation

oops, let cdrom normalize if it's there.
Noticed by sthen@

make PERMIT_CDROM info optional
as discussed with sthen@, code slightly adjusted

actually heed localbase when looking for groff

reorg groff runner so that failures are handled better
do the logic for manpage formatting better, so that we can't miss things
simplify filenames, fullname always has a slash

kill the solve_tags part of pkg_create for now, as it doesn't work at all

get ready to reuse verify_tag

solve_tags not yet functional, I'll reactivate the checks during creation
when it's cleaned up

well, when we read the plist from the ports tree, we need to register
the tag info similarly to registering libraries

PkgCreate has no business knowing details of define-tag

do some more basic checking of tags vs define-tag so that parameter usage
is handled more gracefully

move the tag base code into SolverBase, and use it in PkgCreate, so that
we refuse to create packages with tags without reachable definitions

split the common half of dependencies handling into SolverBase, as PkgCreate
only uses that

kill Frag/NoFrag, they only make sense in update-plist, so they've been
moved there.

Add a -Dstub option to facilitate creating/adding/deleting "stubbed" packages
with just the packing and dependency info with none of the files.

(useful for debugging tricky cases that involve large packages)

if the parser can't find a starting line in user db, it won't look
at any line, so it won't error out.
Add logic to make it error out.

port -u userlist option from update-plist, as suggested by sthen@.
main differences:
- error out on a problem instead of warning
- always parse the userlist if the option is specified. Avoid ticking time
bomb effect on bulk builds, where packages would be fine until the first
@new(user|group) usage in case the user list got corrupted.

Note it's still an option, so if you're using pkg_create outside of the
ports tree, you can do whatever you want.

adjust highly misleading comment to match reality

on the other hand, commenting this out was involuntary.
ask_tree will often say fun things because of (say) pipe closes

the perl debugger tells you everything a perl process exits.
this child is irrelevant, tell the debugger (if present) to not stop

if we create fragments, we must know which files they come from

update-plist wants to ask the ports tree as well, so refactor so that
it can reused that code

need instrumentation for fragments

reorg error handling so it's always passed to class methods that can
be overridden. Useful for nextgen update-plist

sanitize say/print output further
- have confirm go thru f() every time (adding
confirm_defaults_to_no/confirm_defaults_to_yes apis... verbose but effective)
- check DESCR in pkg_create, as suggested by sthen@
- don't do the whole safe dance when not needed (optimization)
- misnamed allowed -> forbidden (schwarze@)

This effectively prevents DESCR from using diacritics. Sorry!
Ingo makes a compelling argument that using UTF8 while installing a new
box is currently fairly dangerous, considering tty setups.

This affects about 10 ports at the moment, all of which were fixed before
this commit.

add the glue to generate future libset, which are currently 100% unused

@libset is going to be part of the dependency information.

switching to _pbuild is tricky, you can't take it back.
in particular, disable future PORTS_PRIVSEP mechanisms under
pkg_create, because we should already be running as _pbuild
by this point.

Revert previous, it breaks ports/infrastructure/bin/resolve-lib (I think)
with at least python and ruby.

ok naddy@

typo, surprised nobody noticed yet

clean-up option handling slightly, some more to be done.

Add a -S option that computes signatures, so that we may remove
code from bsd.port.mk that does the exact same thing, and no longer
work since the addition of @version.

For now, -S is exclusive of other package creation options, as it exits
as soon as it computes what it needs.

better diagnostics

-V option for pkg_create and incorporate version into signatures.

kill creating old signatures.

remove code that allows pkg_create to sign

pass file name thru to Subst for better error messages.
reported by Adam Wolk

kill remnants of non shared arches

streamline the code that checks meta information before extracting files
- verify_modes should only verify modes
- put the whole checking code in one single routine in validate_meta, makes
the size checking code more obvious
- document what's going on
- prevent PkgCreate from creating impossible packages, only a few Ustar
objects are actually usable.

the Solver uses kept_names, so we need these there too
found by aja@

let pkg_create recreate all packages correctly, again.

simplify progressmeter calls to visit_and: we retrieve the state from the
progressmeter object

have visit_with_count and visit_with_size use the same parameter conventions.
In particular, pass state before the other method parameters.

when we run into real errors, the progress meter is distracting, so have
a method ->disable that does something nice to the display, and stops
the progress meter right before we display our first real error.

final bulk shows chunking to happen in all the right places, and signing
works correctly as well. So activate this.

This basically makes package snapshots rsync-friendly with a low-tech
approach. Instead of having a patched gzip, we borrow from other archive
formats, and cut the archive every few files. Since the files are already
sorted in order of date of modification, packages that don't change tend
to end with identical gzip archives.

Note that this will still work when/if we transition to bzip2/xz packages...

need meta stuff that tells me those are "real files" to not create extra
chunks at the beginning of the package...
Also, be very verbose about everything.

.py files no longer need to be special, as the timestamps are stored in
the plist itself, so there's no longer any worry of the timestamp not
getting updated and python throwing a fit.

fix display of comments to use "say" properly.
Prepare scaffolding to be able to chunk archives

store explicit timestamps in generated plists.
this prevents updates from fucking up mtime of new file, thus the
"python exception" no longer prevents python files from being reordered.

Note that this requires newish scaffolding in pkg_add proper. That
scaffolding was committed prior to 5.6, in preparation for this change.

prevent users from adding timestamp themselves

activate "out-of-order" archives based on history
things to watch out for:
- correct handling of multiple files with the same sha
- cwd handling. This is already destated, so we can call set_destdir on
each entry's cwd, instead of relying on @cwd being in the "right" location
as that's no longer the case...

makewhatis being now external, it no longer requires picky/test/check_dir.
It also can't handle too many manpages at once (shell limit) so hand-feed
it (problem spotted by landry@).

Some tests by schwarze@.

Switch to the new makewhatis(8)/apropos(1)/whatis(1) combo.
"commit the switch now" espie@ "go for it" deraadt@

See the apropos(1) manual for a description of what's new.
On machines where you want the full functionality,
run "sudo makewhatis" and put "MAKEWHATISARGS=' '" into weekly.local(8).
Otherwise, when upgrading via source, run "sudo makewhatis -Q".

don't use $_ as "throw-away" variable, proper idiom is "undef"
(and if you're wondering about the unused extra parameter, this is
designed to allow a new makewhatis that shares more code with pkg_create)

remove $_ usage that warns under perl 5.18

refactor solve_from_ports a bit.

do the write to cache thru a temp file, so the end result is atomic,
to be used to enable a global depends cache for parallel building in
the near future.

pseudo handle needs dependency_info too, forgot that one.

tweak the way we create and sign packages significantly:

all modern compression formats support multi-stream archives.
Store the packing-list as a separate stream, and note where the
first stream ends when reading back the packing list.

When pkg_sign sees a split-stream archive, it does not need to unpack/repack
the whole archive: it can just sign the packing-list, close the stream, and
copy the next stream(s) from the source archive verbatim.

This is perfectly transparent to pkg_add, but abuses IO::Uncompress::Gunzip
internals slightly (to be solved with cpan).

also adds explicit option -C for generating a SHA256 list of all packages.

and okay naddy@

clean-up synopsis

zap the sign-only part (moved to PkgSign.pm).
reuse the signer part (move to Signer.pm)

for the SHA256 file, output checksums in base64; ok espie@

involuntary sneak preview, revert

...and sort SHA256 properly

tweak the interface to generating signatures yet again.
- assume key names match, deduce signer from sec key.
e.g., -s signify -s 55pkg.sec
will set signer to 55pkg and look for a pubkey named 55pkg.pub,
either besides 55pkg.sec or in /etc/signify.
- verify there's no mismatch, if possible, by verifying the first package
signed.

- also build a SHA256 on the fly while signing.

say something if we sign an empty repository
tweak mode on tmp signed file

bump copyright to 2014

if we're re-signing, check old sig first. shouldn't sign stuff we don't
trust.

@signer makes sense only for signify, so move it there.

simplify a bit: pass the first SIGNER for @signer.

switch to internal gzip/gunzip.
reduces the number of external processes and the complexity of the code.

tested on a few select arches by tobiasu, naddy. If it breaks somewhere,
tough. This one is simple to revert.

tweak signing yet again. Have pkg_create automatically add signing
identities every time, and make matching identities mandatory.
e.g., pkg_create and pkg_add must have matching -DSIGNER.
by default, signer is derived from uname -r and role (pkg_add/fw_update),
e.g., 54pkg, 54fw...

integrate signer logic and fix resign with -DSIGNER

if HISTORY_DIR is defined, create a new "permanent" file during build.
this file records the sha256 of all elements in the archive, and keeps
track of changes (quite simply: sha256 that were already present end
up at the end of the file).

Start recording these *now*, so that we have enough info to shuffle
packages later.

allow processing of package lists in parallel.
Turns out re-gzipping a signed package is really expensive, so turn
it into several jobs with a -j option.

more forbidden elements

vendor->signer
shorten pubkey -> pub (naddy@)
\\ -> \e (jmc@)

make sure elements are not set manually, namely forbid
@depend (use -P)
@wantlib (use -W)
@name (duh)
@arch
@localbase
@option manual-installation
@comment pkgpath=...
(some of this is actually redundant, but will give more explicit error
messages)

simplify handling of special files some more, that stuff has fixed names
anyways

make the special file creation more explicit, so that the order actually
doesn't matter

be more explicit in why we don't write DESC
return early instead of adding an extra level of test.

zap PKG_PREFIX (undocumented and unused, now)
okay landry@

signify support

@vendor annotation

fix sign-while-creating mode

sign package thru a signer object, instead of hardcoding signature parameters

move ntogo code, so it can be used while copying packages in pkg_create.
add the few lines necessary for that to work.

get rid of "copy_long", we don't really need to recheck names while copying
a package contents.

Allow -o/-S for output/signature dirs for batch operations.
Fix copying of packages while signing, add progressmeter.

If the fake target of a port installed an empty (0-byte) file as a
manual page, pkg_create(1) asked groff to process the empty file,
which failed, and then decided to install the empty file as a source
manual page, which later caused mandoc(1) called from man(1) to
complain to the end-user. Instead, detect the emptiness in time,
skip the formatting attempt, skip the file, process the packing
list to its end, then abort pkg_create(1) unsuccessfully such that
the porter sees the problem and can fix whatever the root cause is.

Problem reported by naddy@ as seen in p5-Carp-Datum during a bulk build,
solution is joint work with espie@, again tested in a bulk by naddy@;
"please commit" espie@.

simple optimization: if we're already erroring out, there's totally no
need to compute those pesky checksums.

activate new pkg_create magic.
Note that, in most cases, PFRAG.shared are probably no longer needed...

put possible fragments in there.
fix buglet in parsing libraries

supplementary magic code, not active yet

less hackish test for LIB*VERSION: after building the element,
redo a check based on the non-subst version, so we can:
- actually parse the library name
- check we depend upon the right variable

well pkg_create itself doesn't care, but a custom ->annotate will get
very confused

make fragment handling yet more customizable

allow a subclass to override that specific error

make the reading of packing-lists amenable to further annotations, such as
those that will be required for update-plist

move code around a bit more

more refactor

just a bit of refactoring, make sure option handling is done in state,
as should be, so we can reuse it without needing to duplicate all of
pkg_create code.

warn when @option no-default-conflict and not explicit @conflict.
(very seldom correct, mostly used for autoconf/automake)

sanitize stuff a bit more...

Backout activation of the new apropos(1)/whatis(1)/makewhatis(8).
In its current state, it causes too much slowdown, in particular
during system builds, and there are other regressions.
That cannot be fixed quickly while it's enabled.

Problems pointed out by espie@, backout requested by deraadt@,
diff "looks good" to espie@.

Enable the new apropos(1), whatis(1), and makewhatis(8).
Unlink the old apropos(1), whatis(1), and makewhatis(8) from the build.
Call the new tools from pkg_create(1) and pkg_add(1).
"Please enable it now." deraadt@

if the portstree gives us a cache directory, use it.
in lots of multi-packages scenarios that all depend on the main package,
this speeds things *a lot*.
case in point: in php 5.2, make package goes down from 1mn40 to 53s.

much better solver: don't try to solve every library, but just the ones
we have in the package we're trying to create.

So, solve one level, and keep solving until we have what we need or we
run out of level. Any package without @wantlib and lots of runtime depends
will build MUCH faster. Other packages MAY build faster.

use print-plist-libs-with-depends (and make sure it does not skip depends)

small optimization: tell pkg_create to skip !libs if LIBS_ONLY (to be
used in make print-plist-libs)

remove hack, ask for SUBDIRs with FULLPATH set

work-around for default vs. no flavor

speed up dependency checking:
- if the ports tree is -current, do not compute
@wantlib during dependencies.

- add an extra cache. The solver caches "exact" depends, we can also cache
pkgpaths when we go to the ports tree.

- make the checking computation progress message less painful by adding the
pkgpath currently investigated...

zap extra try that's no longer needed: print-plist-with-depends should
always succeed since it only depends on the ports tree itself.

change yet again how we handle transparent formatting of manpages.
simplify: actually create a tmp directory hierarchy that exactly mimics
what fake contains. This avoids trouble with makewhatis checks.
Don't even try to remember what files are there, just wipe it all with
rmtree.

don't erase 1st manpage in case two manpages have the first name.
as noticed on texlive, among a host of other issues...

I can remove duplicate flag since that's handled upwards.

run makewhatis during pkg_create in verbose mode.
tweak temp manpage generation a bit, so that it has correct names in its
own directory, because makewhatis will be really pissed off otherwise.

add yet another check: obviously in a @depend line, if the def part doesn't
match the pattern, the package has something wrong.

prevents some bogus tk LIB_DEPENDS from even packaging.

go all the way, put temp pages in /tmp

create tempfile directly under fake, no need to go down.

create formatted manpages as temporary files we remove after the fact.

handle failures from ports tree with more care.
this does avoid infinite loops in case of wrong plists.
necessary since print-plist-with-depends CAN fail...

simplify some more: if it's not installed, ask the ports tree, who knows,
we will catch more mismatches between spec and actual packagenames ?

set status during dependency check

don't check when simply printing the plist, this 'fixes' a bit of
port-lib-depends-check

don't fatal(), error() allows for more stuff to show up

once we read the list, make sure it matches the dependency.
Otherwise, this WILL turn into an infinite loop indeed !

simplify a bit:
- make find_dep_in_installed more regular
- zap the {more} entry, add_dep is okay for that purpose
- don't special case installed_list, provide older_names instead
- make PseudoSets slightly less hackish
- don't read full packinglist, prelinkstuff is exactly what we need

use a better strategy: ask the ports tree for the plist for packages with
the same pkgpath. Thus, PseudoSets need to have a real list, and
pkgcreate needs to loop to solve those depends.

reactivate the check

disable temporarily

new check in pkg_create: walk the dependency tree for @wantlib, thus
preventing packages that won't install (much easier to fix that when
you still have the fake tree around !)

specifically, look into depend that are installed, and possibly in the
fake tree: if the package does build, it requires the libraries to be
around anyways, so pure RUN_DEPENDS are not an issue (and avoid old libraries,
if we find wantlib only in .lib, something really fishy is going on).

Reuse the Dependencies class, so we're sure to have the exact same way
to find libraries.

Also fix exit codes from pkg_create...

use state->defines

tweak the framework so that individual modules don't exist and return
the exit code upstream instead.
this simplifies the task of people who want to reuse it, as noted by
landry@

minor fix for ports which install manpages in other directories: keep
track of all known directories, so when we generate a manpage in a new
catN directory, we add the entry and it gets deleted properly.
-> requires a bump to affected packages.

initial scaffolding for @rcscript: for now, acts like normal files,
but allow absolute pathnames (treat that as an implicit @cwd).

@symlink/@size/@link are generated automatically, make pkg_create(1) error
out if we create them manually, instead of erroring later during pkg_add.

just bail out of producing empty files.
*this will become an error eventually, please fix those*

use newer system for more efficient redirection, and chdir the way man(1)
does, so groff is happier with .so.
(agreed by ingo)

whitespace

pass state to Ustar objects, use it to display those pesky error and
fatal messages.

zap $opt_x from pkg_mklocated, do things like other commands do.

fix pkg_mklocatedb

we can't checksum directories either, so give a better error message

allow say and errsay to work without parameters, as it's ways common.
create verbose_system up in state, because it makes sense without verbose.

whitespace cleanup

pass state along when signing packages, uses repo and fatal directly

use locator object.
rework options into state

display bad symlinks by destination

zap empty message...

don't call working() in -q mode.

no need to fatal(), we can error() and keep going.

warns if packages contain symlinks that point to non-existent places.
actually error out if a symlink points to itself.
this breaks evolution, until someone fixes the bogus symlinks.

make error messages more consistent. Add an extra sanity check which
shouldn't trigger for special files

ui changes: go thru a state object for most printouts

tweak tweak so it works

tweak fragment handling a bit more. In particular, protest when we can't
get a fragment, regardless of whether or not we're going to use it.

display "switching to" with ->set_status

clean-up into smaller functions

add working cartwheel

move use to common part

add ->spec to Conflict and use the same check as for depend during pkg_create.

move code around, so that commands can be used as modules.
pkg becomes the start hub, which does nothing except require the
correct module.
Saner code wrt Add/Delete, and more sharing.

fix esoteric error message in case PackageName parses stuff to a bogus
fullpkgname, as seen by Matthias Schmidt

add some more fluff to documentation

fix some very minor syntactic nits

tweak fragment handling error messages a bit
don't pass parameters we don't need

move to use v5.36;
tested by me over the last few weeks, and tb@
also fixed a "manual install" bug properly reported by tb@

aside that there should be *no functional change*.
If you see any message like "hey, the number of params is wrong"
it is a fringe case I didn't run into and should be easy to fix.

don't pass state, we get them from the progressmeter

gc unused/old code

move the interface to SharedLibs to be somewhat object oriented

accordingly, load it "just in time" in State.

Most calls get simplified, and we can save more state for later.

stop using old-style prototypes except where strictly necessary
(for try/catch)

signatures will be much more powerful once I move to 5.36

use the new set_destdir interface

change naming convention for the lru "save history" cache, so that
ports like "lang/chicken/core" do generate files like lang.chicken.core.lru
instead of lang.chicken.core (which can create confusion in people's mind)

do so transparently by reading the old file if need be, and removing it
afterwards.

Funny thing noticed by tb@

ok tb@, sthen@

Have -S actually behave like the other introspection options, namely only
eschew building the package is -n is mentionned.

Document that -S -n is heavily optimized for speed since it's mostly used
by dpb -R to figure out what to rebuild.

fix a logic error from 2018: be silent if any of -S, -n, -q are mentioned.

The actual bug reverted in 1.128 was from "make print-plist-libs"
which would invoke pkg_create -n -Q and filter out the libs: but
in that case, pkg_create would not be silent, thus yielding
reading plist|-/usr/local/lib/libpython3.9.so.0.0
to filter, which obviously wouldn't work.

So, turn on silent mode for -Q as well.

if we can't open the user db, don't try to read from a closed fh

some old signing code was still around

have compute_checksum return a status, so that we don't report similar
errors several times (for instance, if a file doesn't exist, its checksum
is not going to match, obviously)

have pkg_create generate the hash for always-update

clean up and document the LRU stuff better
- create a separate path when we don't want history
- add a dummy entry to see the threshold between changed and unchanged
in verbose mode
- document better the various stuff that gets separated
- error out in case we want history (by default) and anything goes wrong

remove the gzip chunk after CONTENTS, it was only needed back when
we signed the packing-list separately. Now that we use signify to
create the signature without decompressing anything, it's completely
irrelevant.

yet another stupid shadowed variable, thx sthen@

explicitly write LRU in a comment so that it's easier to find ;)

allow disabling the @ts tweak, which may come in handy for new fw_update
that doesn't grok them.

fix typo (noticed by sthen@)

add extra support to pass REVISION, EPOCH, FLAVOR_LIST from the ports
infrastructure, so that pkg_create can catch some naming errors.

remove test that's now bogus, as fullpkgpath takes precedence.
thanks bluhm@

- grab fullpkgpath early (to decide: hard error if not provided) ?
- tweak the look-up mechanism for dependencies to NOT look at installed
packages for inter-dependencies between subpackages (and debug packages)
of the SAME pkgpath.

The rationale for looking at installed packages in the usual case
is that it's normally much faster rather than have the ports tree
spew out packing-lists.

In the case of multi-packages updates, the new version may have been built
successfully (and correctly) while the old version is still around.
However, if pkg_create looks at the installed version, it may fail to create
the package because the shared library versions will be different.

More DESC -> DESCR; spotted by espie@

tyop: DESC -> DESCR

ok landry@ robert@

give pkg_create knowledge of WRKOBJDIR, so it can error out if a symlink
points in there.

okay aja@ sthen@

nit, use "fatal" with parameters as intended

have ask_tree be more specific in its error reporting.
make sure the child has the same effective and real ids,
as perl interprets it as "setuid, refuse to debug"
go to the end of the pipe so the child doesn't exit
prematurely (didn't use to happen but now it does)

help from afresh1@ in diagnosing that one.

when asking the ports tree, we fork a child.
instead of ditching stderr entirely, keep it in a temp file, and if the
child exits with an error, we got something to display.
(note that returning and undef'd plist is enough of an error, just we
had no clue what went wrong previously)

aja@ ran into this a few weeks ago.

start re-adding the non-broken changes:
- commonalize try{} catch {} for pkg_add/delete and pkg_create, so that
debug works the same way in both.
- switch printing command name to the catch handler, so that exceptions
are simpler to handle

backout previous commits for now; some of this is shared with dpb and
results in problems. hints from pvk@ about what introduced the breakage
we were seeing, who confirms that this backout fixes things; comitting
early to unbreak package builds.

====
a bunch of changes, all related to error-handling:
- have Handle->register also create a proper END block, so that
individual packages don't have to, and explain the issue
- kill old Unlink/Copy code that migrated to State years ago
- commonalize try{} catch {} for pkg_add/delete and pkg_create, so that
debug works the same way in both.
- switch printing command name to the catch handler, so that exceptions
are simpler to handle

and a few comments for the hairy parts...

Members:
OpenBSD/AddCreateDelete.pm:1.44->1.45
OpenBSD/AddDelete.pm:1.87->1.88
OpenBSD/Error.pm:1.33->1.34
OpenBSD/PackageRepository.pm:1.164->1.165
OpenBSD/PkgCreate.pm:1.160->1.161
OpenBSD/State.pm:1.58->1.59
OpenBSD/Temp.pm:1.34->1.35
====

a bunch of changes, all related to error-handling:
- have Handle->register also create a proper END block, so that
individual packages don't have to, and explain the issue
- kill old Unlink/Copy code that migrated to State years ago
- commonalize try{} catch {} for pkg_add/delete and pkg_create, so that
debug works the same way in both.
- switch printing command name to the catch handler, so that exceptions
are simpler to handle

and a few comments for the hairy parts...

move the display of cmd into fatal, and don't say it's a fatal error, it's
already obvious

tweak the way we report bugs
- the catch/catchall distinction is no longer relevant, strip some info
by default now
- use state->{cmd} to display the command name properly

This makes sure final displays show the command name.

make pkg_create fail properly (return 1 is useless there)

OpenBSD::Temp error handling as well

actually use error, we can delay fatal problems

check ::Temp creation

oops, let cdrom normalize if it's there.
Noticed by sthen@

make PERMIT_CDROM info optional
as discussed with sthen@, code slightly adjusted

actually heed localbase when looking for groff

reorg groff runner so that failures are handled better
do the logic for manpage formatting better, so that we can't miss things
simplify filenames, fullname always has a slash

kill the solve_tags part of pkg_create for now, as it doesn't work at all

get ready to reuse verify_tag

solve_tags not yet functional, I'll reactivate the checks during creation
when it's cleaned up

well, when we read the plist from the ports tree, we need to register
the tag info similarly to registering libraries

PkgCreate has no business knowing details of define-tag

do some more basic checking of tags vs define-tag so that parameter usage
is handled more gracefully

move the tag base code into SolverBase, and use it in PkgCreate, so that
we refuse to create packages with tags without reachable definitions

split the common half of dependencies handling into SolverBase, as PkgCreate
only uses that

kill Frag/NoFrag, they only make sense in update-plist, so they've been
moved there.

Add a -Dstub option to facilitate creating/adding/deleting "stubbed" packages
with just the packing and dependency info with none of the files.

(useful for debugging tricky cases that involve large packages)

if the parser can't find a starting line in user db, it won't look
at any line, so it won't error out.
Add logic to make it error out.

port -u userlist option from update-plist, as suggested by sthen@.
main differences:
- error out on a problem instead of warning
- always parse the userlist if the option is specified. Avoid ticking time
bomb effect on bulk builds, where packages would be fine until the first
@new(user|group) usage in case the user list got corrupted.

Note it's still an option, so if you're using pkg_create outside of the
ports tree, you can do whatever you want.

adjust highly misleading comment to match reality