Merge one true awk from 2024-01-22 for the Awk Second Edition support

This brings in Unicode support, CSV support and a number of bug fixes.
They are described in _The AWK Programming Language_, Second Edition, by
Al Aho, Brian Kernighan, and Peter Weinberger (Addison-Wesley, 2024,
ISBN-13 978-0138269722, ISBN-10 0138269726).

Sponsored by: Netflix

Revert "awk: Merge upstream 2nd Edition Awk Book"

The pre-push testing I did turned out to be testing the old version with
the old testsuite (for reasons I don't understnad). There's issues with
the new version, the new test in the suite or (likely) both. Revert
until they can be chased down.

This should also fix the github CI that's gone red since this commit.

This reverts commit 3fd60a6b73ac01a72df89751f173970fae4cae73, reversing
changes made to 194df014feebd8b169b41ecd75ae73d63a792d6b.

Sponsored by: Netflix

awk: Merge upstream 2nd Edition Awk Book

Merge in the November 2nd, 2023 version of one true awk.

This brings in Unicode support, CSV support and a number of bug fixes.

Sponsored by: Netflix
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D42447

awk: Merge 20210729 from One True Awk upstream (0592de4a)

July 27, 2021:
As per IEEE Std 1003.1-2008, -F "str" is now consistent with
-v FS="str" when str is null. Thanks to Warner Losh.

July 24, 2021:
Fix readrec's definition of a record. This fixes an issue
with NetBSD's RS regular expression support that can cause
an infinite read loop. Thanks to Miguel Pineiro Jr.

Fix regular expression RS ^-anchoring. RS ^-anchoring needs to
know if it is reading the first record of a file. This change
restores a missing line that was overlooked when porting NetBSD's
RS regex functionality. Thanks to Miguel Pineiro Jr.

Fix size computation in replace_repeat() for special case
REPEAT_WITH_Q. Thanks to Todd C. Miller.

Also, included the tests from upstream, though they aren't yet connected
to the tree.

Sponsored by: Netflix

Revert "awk: Issue a warning for old hex behavior."

This reverts commit acf9cf323f8d0c844ea4a0fedeb596871794a078. It warns
about too many false positive cases.

Sponsored by: Netflix

awk: Issue a warning for old hex behavior.

Since FreeBSD has allowed "0x" hex strings to be converted to integers
for a long time, and since upstream has killed that behavior, warn about
this issue. This will allow us to deprecate this behavior for 14.0 while
giving our users of 12.x and 13.x fair warning.

Sponsored by: Netflix

awk: revert upstream's attempt to disallow hex strings

Upstream one-true-awk decided to disallow hex strings as numbers. This
is in line with awk's behavior prior to C99, and allowed by the POSIX
standard. The standard, however, allows them to be treated as numbers
because that's what the standard said in the 2001 through 2004 editions.
Since 2001, the nawk in FreeBSD has treated them as numbers, so restore
that behavior, allowed by the standard.

A number of scripts in the FreeBSD tree depend on this interpretation,
including scripts to build the kernel which had mysteriously started
failing for some people and not others. By re-allowing 0x hex numbers,
this fixes those scripts and restores POLA.

Upstream issue: https://github.com/onetrueawk/awk/issues/126
Sponsored by: Netflix
Reviewed by: kevans
MFC After: asap due to regression alrady merged to stable
Differential Revision: https://reviews.freebsd.org/D31199

one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs

Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by: Mikolaj Golub <to.my.trociny@gmail.com>
PR: 143363,143365,143368,143369,143373,143375,214782
Sponsored by: Netflix

Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git

Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).

Don't display empty error context.

Context extraction didn't handle this case and showed uninitialized memory.

Obtained from: OpenBSD lib.c 1.21
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Fix uninitialized variable

echo | awk 'BEGIN {i=$1; print i}' prints a boatload of stack
garbage. NUL terminate the memory returned from malloc to prevent it.

Obtained from: OpenBSD run.c 1.40
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

- Merged awk upstream that includes a fix for a bug exposed by kmod_syms.mk.
- Provide a build aid for those who already have a buggy awk(1) installed.

Approved by: re (kib)

Update to a 7-Aug-2011 release.

Approved by: re (kib)

Update to a 1-May-2011 release (except for the isblank change).

Update to a 26-Nov-2009 release.

Flatten out vendor tree.

Vendor import of bwk's 23-Oct-2007 release.
This includes fixes for FreeBSD PR's: bin/104795, bin/100443

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.

Revert "awk: Merge upstream 2nd Edition Awk Book"

The pre-push testing I did turned out to be testing the old version with
the old testsuite (for reasons I don't understnad). There's issues with
the new version, the new test in the suite or (likely) both. Revert
until they can be chased down.

This should also fix the github CI that's gone red since this commit.

This reverts commit 3fd60a6b73ac01a72df89751f173970fae4cae73, reversing
changes made to 194df014feebd8b169b41ecd75ae73d63a792d6b.

Sponsored by: Netflix

awk: Merge upstream 2nd Edition Awk Book

Merge in the November 2nd, 2023 version of one true awk.

This brings in Unicode support, CSV support and a number of bug fixes.

Sponsored by: Netflix
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D42447

awk: Merge 20210729 from One True Awk upstream (0592de4a)

July 27, 2021:
As per IEEE Std 1003.1-2008, -F "str" is now consistent with
-v FS="str" when str is null. Thanks to Warner Losh.

July 24, 2021:
Fix readrec's definition of a record. This fixes an issue
with NetBSD's RS regular expression support that can cause
an infinite read loop. Thanks to Miguel Pineiro Jr.

Fix regular expression RS ^-anchoring. RS ^-anchoring needs to
know if it is reading the first record of a file. This change
restores a missing line that was overlooked when porting NetBSD's
RS regex functionality. Thanks to Miguel Pineiro Jr.

Fix size computation in replace_repeat() for special case
REPEAT_WITH_Q. Thanks to Todd C. Miller.

Also, included the tests from upstream, though they aren't yet connected
to the tree.

Sponsored by: Netflix

Revert "awk: Issue a warning for old hex behavior."

This reverts commit acf9cf323f8d0c844ea4a0fedeb596871794a078. It warns
about too many false positive cases.

Sponsored by: Netflix

awk: Issue a warning for old hex behavior.

Since FreeBSD has allowed "0x" hex strings to be converted to integers
for a long time, and since upstream has killed that behavior, warn about
this issue. This will allow us to deprecate this behavior for 14.0 while
giving our users of 12.x and 13.x fair warning.

Sponsored by: Netflix

awk: revert upstream's attempt to disallow hex strings

Upstream one-true-awk decided to disallow hex strings as numbers. This
is in line with awk's behavior prior to C99, and allowed by the POSIX
standard. The standard, however, allows them to be treated as numbers
because that's what the standard said in the 2001 through 2004 editions.
Since 2001, the nawk in FreeBSD has treated them as numbers, so restore
that behavior, allowed by the standard.

A number of scripts in the FreeBSD tree depend on this interpretation,
including scripts to build the kernel which had mysteriously started
failing for some people and not others. By re-allowing 0x hex numbers,
this fixes those scripts and restores POLA.

Upstream issue: https://github.com/onetrueawk/awk/issues/126
Sponsored by: Netflix
Reviewed by: kevans
MFC After: asap due to regression alrady merged to stable
Differential Revision: https://reviews.freebsd.org/D31199

one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs

Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by: Mikolaj Golub <to.my.trociny@gmail.com>
PR: 143363,143365,143368,143369,143373,143375,214782
Sponsored by: Netflix

Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git

Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).

Don't display empty error context.

Context extraction didn't handle this case and showed uninitialized memory.

Obtained from: OpenBSD lib.c 1.21
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Fix uninitialized variable

echo | awk 'BEGIN {i=$1; print i}' prints a boatload of stack
garbage. NUL terminate the memory returned from malloc to prevent it.

Obtained from: OpenBSD run.c 1.40
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

- Merged awk upstream that includes a fix for a bug exposed by kmod_syms.mk.
- Provide a build aid for those who already have a buggy awk(1) installed.

Approved by: re (kib)

Update to a 7-Aug-2011 release.

Approved by: re (kib)

Update to a 1-May-2011 release (except for the isblank change).

Update to a 26-Nov-2009 release.

Flatten out vendor tree.

Vendor import of bwk's 23-Oct-2007 release.
This includes fixes for FreeBSD PR's: bin/104795, bin/100443

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.

awk: Merge 20210729 from One True Awk upstream (0592de4a)

July 27, 2021:
As per IEEE Std 1003.1-2008, -F "str" is now consistent with
-v FS="str" when str is null. Thanks to Warner Losh.

July 24, 2021:
Fix readrec's definition of a record. This fixes an issue
with NetBSD's RS regular expression support that can cause
an infinite read loop. Thanks to Miguel Pineiro Jr.

Fix regular expression RS ^-anchoring. RS ^-anchoring needs to
know if it is reading the first record of a file. This change
restores a missing line that was overlooked when porting NetBSD's
RS regex functionality. Thanks to Miguel Pineiro Jr.

Fix size computation in replace_repeat() for special case
REPEAT_WITH_Q. Thanks to Todd C. Miller.

Also, included the tests from upstream, though they aren't yet connected
to the tree.

Sponsored by: Netflix

Revert "awk: Issue a warning for old hex behavior."

This reverts commit acf9cf323f8d0c844ea4a0fedeb596871794a078. It warns
about too many false positive cases.

Sponsored by: Netflix

awk: Issue a warning for old hex behavior.

Since FreeBSD has allowed "0x" hex strings to be converted to integers
for a long time, and since upstream has killed that behavior, warn about
this issue. This will allow us to deprecate this behavior for 14.0 while
giving our users of 12.x and 13.x fair warning.

Sponsored by: Netflix

awk: revert upstream's attempt to disallow hex strings

Upstream one-true-awk decided to disallow hex strings as numbers. This
is in line with awk's behavior prior to C99, and allowed by the POSIX
standard. The standard, however, allows them to be treated as numbers
because that's what the standard said in the 2001 through 2004 editions.
Since 2001, the nawk in FreeBSD has treated them as numbers, so restore
that behavior, allowed by the standard.

A number of scripts in the FreeBSD tree depend on this interpretation,
including scripts to build the kernel which had mysteriously started
failing for some people and not others. By re-allowing 0x hex numbers,
this fixes those scripts and restores POLA.

Upstream issue: https://github.com/onetrueawk/awk/issues/126
Sponsored by: Netflix
Reviewed by: kevans
MFC After: asap due to regression alrady merged to stable
Differential Revision: https://reviews.freebsd.org/D31199

one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs

Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by: Mikolaj Golub <to.my.trociny@gmail.com>
PR: 143363,143365,143368,143369,143373,143375,214782
Sponsored by: Netflix

Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git

Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).

Don't display empty error context.

Context extraction didn't handle this case and showed uninitialized memory.

Obtained from: OpenBSD lib.c 1.21
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Fix uninitialized variable

echo | awk 'BEGIN {i=$1; print i}' prints a boatload of stack
garbage. NUL terminate the memory returned from malloc to prevent it.

Obtained from: OpenBSD run.c 1.40
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

- Merged awk upstream that includes a fix for a bug exposed by kmod_syms.mk.
- Provide a build aid for those who already have a buggy awk(1) installed.

Approved by: re (kib)

Update to a 7-Aug-2011 release.

Approved by: re (kib)

Update to a 1-May-2011 release (except for the isblank change).

Update to a 26-Nov-2009 release.

Flatten out vendor tree.

Vendor import of bwk's 23-Oct-2007 release.
This includes fixes for FreeBSD PR's: bin/104795, bin/100443

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.

Revert "awk: Issue a warning for old hex behavior."

This reverts commit acf9cf323f8d0c844ea4a0fedeb596871794a078. It warns
about too many false positive cases.

Sponsored by: Netflix

awk: Issue a warning for old hex behavior.

Since FreeBSD has allowed "0x" hex strings to be converted to integers
for a long time, and since upstream has killed that behavior, warn about
this issue. This will allow us to deprecate this behavior for 14.0 while
giving our users of 12.x and 13.x fair warning.

Sponsored by: Netflix

awk: revert upstream's attempt to disallow hex strings

Upstream one-true-awk decided to disallow hex strings as numbers. This
is in line with awk's behavior prior to C99, and allowed by the POSIX
standard. The standard, however, allows them to be treated as numbers
because that's what the standard said in the 2001 through 2004 editions.
Since 2001, the nawk in FreeBSD has treated them as numbers, so restore
that behavior, allowed by the standard.

A number of scripts in the FreeBSD tree depend on this interpretation,
including scripts to build the kernel which had mysteriously started
failing for some people and not others. By re-allowing 0x hex numbers,
this fixes those scripts and restores POLA.

Upstream issue: https://github.com/onetrueawk/awk/issues/126
Sponsored by: Netflix
Reviewed by: kevans
MFC After: asap due to regression alrady merged to stable
Differential Revision: https://reviews.freebsd.org/D31199

one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs

Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by: Mikolaj Golub <to.my.trociny@gmail.com>
PR: 143363,143365,143368,143369,143373,143375,214782
Sponsored by: Netflix

Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git

Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).

Don't display empty error context.

Context extraction didn't handle this case and showed uninitialized memory.

Obtained from: OpenBSD lib.c 1.21
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Fix uninitialized variable

echo | awk 'BEGIN {i=$1; print i}' prints a boatload of stack
garbage. NUL terminate the memory returned from malloc to prevent it.

Obtained from: OpenBSD run.c 1.40
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

- Merged awk upstream that includes a fix for a bug exposed by kmod_syms.mk.
- Provide a build aid for those who already have a buggy awk(1) installed.

Approved by: re (kib)

Update to a 7-Aug-2011 release.

Approved by: re (kib)

Update to a 1-May-2011 release (except for the isblank change).

Update to a 26-Nov-2009 release.

Flatten out vendor tree.

Vendor import of bwk's 23-Oct-2007 release.
This includes fixes for FreeBSD PR's: bin/104795, bin/100443

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.

Don't display empty error context.

Context extraction didn't handle this case and showed uninitialized memory.

Obtained from: OpenBSD lib.c 1.21
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Fix uninitialized variable

echo | awk 'BEGIN {i=$1; print i}' prints a boatload of stack
garbage. NUL terminate the memory returned from malloc to prevent it.

Obtained from: OpenBSD run.c 1.40
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Update to a 26-Nov-2009 release.

- Import the HEAD csup code which is the basis for the cvsmode work.

Vendor import of bwk's 23-Oct-2007 release.
This includes fixes for FreeBSD PR's: bin/104795, bin/100443

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.