Implement else branching logic including 'else if'.

Statement lists for if & else conditions are now wrapped in a new
'struct bt_cond'. Handling B_AC_TEST statements moved to stmt_eval()
to handle nested conditional statements.

From Christian Ludwig christian_ludwig at genua.de

Trailing spaces

Prevent tuples to be used as map key, associative array is what we want.

Handle variable names (things strating with $ or @) in yylex() this way
the error handling of strange variable names can be better controlled.
With and OK dv@

btrace: add support for hex and octal values.

Changes number tokenizing and parsing to support hex & octal values.
Does not address other lexer issues (e.g. $0x1) to close gaps with
bpftrace.

OK claudio@

btrace: "too long line" -> "line too long"

ok claudio@

bt(5), btrace(8): add support for binary modulo operator ('%')

Link: https://marc.info/?l=openbsd-tech&m=169695435209410&w=2

ok mpi@

Implement tuples, needed to export per-CPU scheduling data.

It is now possible to save and print immutable arrays as below:

..$t = (1, 42, "something");
..printf("%d %s\n", $t.1, $t.2);

Also add support for evaluating builtin in order to save them in variables
(required by tuples)

btrace(8): allow empty statements in probes.

Allows for probes like `BEGIN {}`, in parity with bpftrace.

Also fixes an incorrect syntax error parsing argN builtins in
subsequent probes after an empty BEGIN block.

ok mpi@

btrace(8): prohibit use of argN builtins in BEGIN/END.

The argN builtins are undefined for BEGIN and END special probes.
Similar to bpftrace, produce an error from the parser.

Adds a regress test, as well.

ok mpi@

btrace(8): fix yacc reduce conflicts and undefined symbol warnings.

Define the STR symbol, used for the str function.

Tune the grammar, simplifying the 'pat' rule to 'expr'. Resolves
the reduce conflicts related to 'pat' and 'factor' both matching a
lone CSTRING token.

ok mpi@

spelling fixes; from paul tagliamonte
any parts of his diff not taken are noted on tech

Add support for string comparison in filters.

It is now possible to filter by process name, like:

syscall:mmap:entry
/comm == "ld"/
{
...
}

Currently the parser treats C-string like any other expression member even
if arithmetic operations do no apply to strings.

Add support for multiple statements in if () blocks.

btrace(8): fix lexer to allow whitespace after filters.

Whitespace is allowed after the closing slash of a filter and before
the opening brace of an action. This makes the lexer scan ahead and
collect any whitespace and newlines into the end of filter token.

ok mpi@

Implement the probe variable. Can be used for example with
@map[probe] = count();
OK mpi@

bt(5)/btrace(8): add support for str()

Implement initial support for the str() function, which is used
primarily to truncate or NUL-terminate strings from either cli args
or args to tracepoints and syscalls.

Current implementation only supports cli args and is primarily for
compatability with bpftrace. Future work is needed once dt(4)
supports builtin args other than long values.

Adds a regress test and wires in argument-based tests again.

ok mpi@

Relax the check for accessing variable before setting them.

First we can't assume rules are written in the order they will be executed.
Secondly filters might need to check variables before they had a chance to
be populated by the right event.

Make it possible to associate multiple probes to a single rule.

The following syntax, reducing duplication, is now allowed:

END,
interval:hz:2
{
...
}

Rule descriptors are now linked to a list of probe descriptors instead of
a single one. Enabled kernel probes are now linked to btrace(8) probe
descriptors.

While here stop parsing filter and probe if debug is not enabled.

Revert a chunk committed by inadvertence in my last fix.

btrace(8): add initial support for cli arguments

This adds initial support for passing cli args to btrace(8) for use
in bt(5) scripts. Similar to bpftrace, they are referenced via $1,
$2, etc. with $# being the number of command line arguments provided.

Adds an initial regress test and a Makefile change to allow providing
arguments to regress tests in a .args file.

Currently no limit is imposed on the number of arguments, keeping
a similar approach as observed in bpftrace. References to undefined
arguments result in a new "nil" type that contextually acts as a
zero or empty string. More work can be done here to improve bpftrace
compatibility.

ok mpi@, jasper@

Check that map/hist functions are called with the right argument.

Change the parser to make clear() and zero() accept only local and
global variables as arguments.

Since the parser has no knowledge of the type of a variable abort
the execution if clear() or zero() are being called with something
other than a map or hist.

Fix assertions found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Return early if a parsing error has been found and do not perform any
sanity check as they might obviously fail.

Fix an assert found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Basic test, if (no else atm), support with a single statement.

Rewrite grammar to implement operator precedence without using %right or %prec.

Arithmetic operator should now behave as expeted and tests can now be written
without superfluous parenthesis, for example:

syscall:select:entry
/($1 == 0) || (pid == $1)/
{
}

Can now be written:

syscall:select:entry
/$1 == 0 || pid == $1/
{
}

While here improve filter debugging support.

Implement '<' and '>' operators in filters.

Based on a diff from and ok dv@

Indent and simplify the grammar.

Eliminate S/R conflicts and simplify filter grammar.

Simplify token declaration.

Simplify now that TID and PID are now only being parsed as builtin.

Remove support for in-kernel filters.

This might be added back in a future if copying events to userland becomes
a performance issue. However note that it is not always possible to filter
in-kernel if, for example. a variable has to be evaluated when a rule fires.

Extend filters to support any conditionnal test including global variables.

Stop using in-kernel filtering for the moment except for not tracing the
tracer.

Keep track of the number of filtered events.

Always initialized min value for an histogram.

typo

Support for local (scratch) variables: "$var_name".

Every rule gets its own list of (local) variables.

Extend print() to support any kind of variable.

Support first shell argument as $1 in order to use it in filters.

Remove '-p' option now that scripts can filter by pid/tid.

Extend binary operators support, required for more filter features.

Improve debugging of filters and print operator names in debug output.

Start implementing conditionals for filters.

Allows to check the existence of a variable in predicates, making it
possible to trace syscall latency, as follow:

syscall:select:entry
{
@start[pid] = nsecs;
}

syscall:select:return
/@start[pid]/
{
@usecs = hist((nsecs - @start[pid]) / 1000);
delete(@start[pid]);
}

Exclude the 'hz' token from the lexer backdoor, so interval and profile
probes can be parsed again.

ok anton@ kn@

Tracing syscalls which conflict with reserved bt(5) keywords currently
causes btrace to reject such programs. An example is exit which is both
a syscall and a builtin btrace function. Pointed out by bluhm@

To resolve the conflict, make use of a lexer backdoor. A concept
described in the original yacc paper and also found in other yacc
parsers in our tree.

ok bluhm@ mpi@

In revision 1.18 of bt_parse.y, I missed the fact that the print()
function accepts an optional argument representing the number of map
entries to print.

ok bluhm@ mpi@

The print() function only accepts a single argument.

ok mpi@

add support for '&' and '|' operators in btrace scripts

feedback from otto@
ok mpi@ kn@ semarie@

Implement linear and power-of-two histograms: hist() and lhist() keywords.

This is built on top of maps which are currently built on top of RB-trees.
Improvements are welcome! For example the use of a hashing table as pointed
by espie@.

The following one-liner produce an histogram of power-of-two values returned
by the read(2) syscall:

btrace 'syscall:read:return { @bytes = hist(retval); }'
^C
@bytes:
[0] 19 |@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1] 26 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1, 2) 1 |@ |
[2, 4) 13 |@@@@@@@@@@@@@@@@@@ |
[4, 8) 4 |@@@@@ |
[8, 16) 3 |@@@@ |
[16, 32) 1 |@ |
[32, 64) 8 |@@@@@@@@@@@ |
[64, 128) 14 |@@@@@@@@@@@@@@@@@@@ |
[128, 256) 7 |@@@@@@@@@ |
[256, 512) 37 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[512, 1K) 1 |@ |
[1K, 2K) 10 |@@@@@@@@@@@@@@ |
[2K, 4K) 11 |@@@@@@@@@@@@@@@ |
[8K, 16K) 1 |@ |

Increment line number when skipping multi-line comments.

Fix "the symbol HZ is undefined" yacc warning

OK mpi

Simplify the grammar by letting the lexer handle builtin keywords.

Also clarify various constructs by using commonly understood names like
'expr' and 'vargs'.

Implement builtin 'cpu' keyword.

This is useful to know which CPU recorded a given event.

While here document 'retval' and comment out 'ustack' until we have a
way to capture userland stacks.

Extend map to support keys composed of multiple arguments.

Keys are still strings representing the output value.

The following example is now possible to count the number of "on CPU"
events ordered by thread ID and executable name:

# btrace -e 'tracepoint:sched:on__cpu { @[tid, comm] = count() }'
^C
@[138836, idle0]: 830941
@[161307, sshd]: 716476
@[482901, softnet]: 582008
@[104443, systqmp]: 405749
@[269230, update]: 396133
@[326533, softclock]: 316926
@[61040, sshd]: 177201
@[453567, reaper]: 119676
@[446052, ksh]: 85675
@[26270, syslogd]: 66625
@[504699, sshd]: 52958
@[446052, sshd]: 32207
@[44046, tset]: 13333
@[162960, zerothread]: 101
@[313046, ntpd]: 1

Skip first line if it starts with '#!'

Add support for unnamed map.

The following is now possible but won't print anything by default:

# btrace -e 'profile:hz:99 { @[kstack] = count() }'

ok patrick@, tobhe@

Implement map functions min(), max() and sum().

For the moment map values are currently limited to integers.

Use %left to prevent S/R conflicts, arithmetic precedence is still broken.

Simplify statement rules by using an optional new line.

Implement delete() and @map[key] access.

Rename *_concat() into *_append() to match reality.

Parse '==' without error in filters.

From Benjamin Baier.

Implement builtin time() function.

Import a bug tracer, companion of dt(4), that speaks the bt(5) language.

ok kettenis@, visa@, jasper@, deraadt@

Trailing spaces

Prevent tuples to be used as map key, associative array is what we want.

Handle variable names (things strating with $ or @) in yylex() this way
the error handling of strange variable names can be better controlled.
With and OK dv@

btrace: add support for hex and octal values.

Changes number tokenizing and parsing to support hex & octal values.
Does not address other lexer issues (e.g. $0x1) to close gaps with
bpftrace.

OK claudio@

btrace: "too long line" -> "line too long"

ok claudio@

bt(5), btrace(8): add support for binary modulo operator ('%')

Link: https://marc.info/?l=openbsd-tech&m=169695435209410&w=2

ok mpi@

Implement tuples, needed to export per-CPU scheduling data.

It is now possible to save and print immutable arrays as below:

..$t = (1, 42, "something");
..printf("%d %s\n", $t.1, $t.2);

Also add support for evaluating builtin in order to save them in variables
(required by tuples)

btrace(8): allow empty statements in probes.

Allows for probes like `BEGIN {}`, in parity with bpftrace.

Also fixes an incorrect syntax error parsing argN builtins in
subsequent probes after an empty BEGIN block.

ok mpi@

btrace(8): prohibit use of argN builtins in BEGIN/END.

The argN builtins are undefined for BEGIN and END special probes.
Similar to bpftrace, produce an error from the parser.

Adds a regress test, as well.

ok mpi@

btrace(8): fix yacc reduce conflicts and undefined symbol warnings.

Define the STR symbol, used for the str function.

Tune the grammar, simplifying the 'pat' rule to 'expr'. Resolves
the reduce conflicts related to 'pat' and 'factor' both matching a
lone CSTRING token.

ok mpi@

spelling fixes; from paul tagliamonte
any parts of his diff not taken are noted on tech

Add support for string comparison in filters.

It is now possible to filter by process name, like:

syscall:mmap:entry
/comm == "ld"/
{
...
}

Currently the parser treats C-string like any other expression member even
if arithmetic operations do no apply to strings.

Add support for multiple statements in if () blocks.

btrace(8): fix lexer to allow whitespace after filters.

Whitespace is allowed after the closing slash of a filter and before
the opening brace of an action. This makes the lexer scan ahead and
collect any whitespace and newlines into the end of filter token.

ok mpi@

Implement the probe variable. Can be used for example with
@map[probe] = count();
OK mpi@

bt(5)/btrace(8): add support for str()

Implement initial support for the str() function, which is used
primarily to truncate or NUL-terminate strings from either cli args
or args to tracepoints and syscalls.

Current implementation only supports cli args and is primarily for
compatability with bpftrace. Future work is needed once dt(4)
supports builtin args other than long values.

Adds a regress test and wires in argument-based tests again.

ok mpi@

Relax the check for accessing variable before setting them.

First we can't assume rules are written in the order they will be executed.
Secondly filters might need to check variables before they had a chance to
be populated by the right event.

Make it possible to associate multiple probes to a single rule.

The following syntax, reducing duplication, is now allowed:

END,
interval:hz:2
{
...
}

Rule descriptors are now linked to a list of probe descriptors instead of
a single one. Enabled kernel probes are now linked to btrace(8) probe
descriptors.

While here stop parsing filter and probe if debug is not enabled.

Revert a chunk committed by inadvertence in my last fix.

btrace(8): add initial support for cli arguments

This adds initial support for passing cli args to btrace(8) for use
in bt(5) scripts. Similar to bpftrace, they are referenced via $1,
$2, etc. with $# being the number of command line arguments provided.

Adds an initial regress test and a Makefile change to allow providing
arguments to regress tests in a .args file.

Currently no limit is imposed on the number of arguments, keeping
a similar approach as observed in bpftrace. References to undefined
arguments result in a new "nil" type that contextually acts as a
zero or empty string. More work can be done here to improve bpftrace
compatibility.

ok mpi@, jasper@

Check that map/hist functions are called with the right argument.

Change the parser to make clear() and zero() accept only local and
global variables as arguments.

Since the parser has no knowledge of the type of a variable abort
the execution if clear() or zero() are being called with something
other than a map or hist.

Fix assertions found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Return early if a parsing error has been found and do not perform any
sanity check as they might obviously fail.

Fix an assert found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Basic test, if (no else atm), support with a single statement.

Rewrite grammar to implement operator precedence without using %right or %prec.

Arithmetic operator should now behave as expeted and tests can now be written
without superfluous parenthesis, for example:

syscall:select:entry
/($1 == 0) || (pid == $1)/
{
}

Can now be written:

syscall:select:entry
/$1 == 0 || pid == $1/
{
}

While here improve filter debugging support.

Implement '<' and '>' operators in filters.

Based on a diff from and ok dv@

Indent and simplify the grammar.

Eliminate S/R conflicts and simplify filter grammar.

Simplify token declaration.

Simplify now that TID and PID are now only being parsed as builtin.

Remove support for in-kernel filters.

This might be added back in a future if copying events to userland becomes
a performance issue. However note that it is not always possible to filter
in-kernel if, for example. a variable has to be evaluated when a rule fires.

Extend filters to support any conditionnal test including global variables.

Stop using in-kernel filtering for the moment except for not tracing the
tracer.

Keep track of the number of filtered events.

Always initialized min value for an histogram.

typo

Support for local (scratch) variables: "$var_name".

Every rule gets its own list of (local) variables.

Extend print() to support any kind of variable.

Support first shell argument as $1 in order to use it in filters.

Remove '-p' option now that scripts can filter by pid/tid.

Extend binary operators support, required for more filter features.

Improve debugging of filters and print operator names in debug output.

Start implementing conditionals for filters.

Allows to check the existence of a variable in predicates, making it
possible to trace syscall latency, as follow:

syscall:select:entry
{
@start[pid] = nsecs;
}

syscall:select:return
/@start[pid]/
{
@usecs = hist((nsecs - @start[pid]) / 1000);
delete(@start[pid]);
}

Exclude the 'hz' token from the lexer backdoor, so interval and profile
probes can be parsed again.

ok anton@ kn@

Tracing syscalls which conflict with reserved bt(5) keywords currently
causes btrace to reject such programs. An example is exit which is both
a syscall and a builtin btrace function. Pointed out by bluhm@

To resolve the conflict, make use of a lexer backdoor. A concept
described in the original yacc paper and also found in other yacc
parsers in our tree.

ok bluhm@ mpi@

In revision 1.18 of bt_parse.y, I missed the fact that the print()
function accepts an optional argument representing the number of map
entries to print.

ok bluhm@ mpi@

The print() function only accepts a single argument.

ok mpi@

add support for '&' and '|' operators in btrace scripts

feedback from otto@
ok mpi@ kn@ semarie@

Implement linear and power-of-two histograms: hist() and lhist() keywords.

This is built on top of maps which are currently built on top of RB-trees.
Improvements are welcome! For example the use of a hashing table as pointed
by espie@.

The following one-liner produce an histogram of power-of-two values returned
by the read(2) syscall:

btrace 'syscall:read:return { @bytes = hist(retval); }'
^C
@bytes:
[0] 19 |@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1] 26 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1, 2) 1 |@ |
[2, 4) 13 |@@@@@@@@@@@@@@@@@@ |
[4, 8) 4 |@@@@@ |
[8, 16) 3 |@@@@ |
[16, 32) 1 |@ |
[32, 64) 8 |@@@@@@@@@@@ |
[64, 128) 14 |@@@@@@@@@@@@@@@@@@@ |
[128, 256) 7 |@@@@@@@@@ |
[256, 512) 37 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[512, 1K) 1 |@ |
[1K, 2K) 10 |@@@@@@@@@@@@@@ |
[2K, 4K) 11 |@@@@@@@@@@@@@@@ |
[8K, 16K) 1 |@ |

Increment line number when skipping multi-line comments.

Fix "the symbol HZ is undefined" yacc warning

OK mpi

Simplify the grammar by letting the lexer handle builtin keywords.

Also clarify various constructs by using commonly understood names like
'expr' and 'vargs'.

Implement builtin 'cpu' keyword.

This is useful to know which CPU recorded a given event.

While here document 'retval' and comment out 'ustack' until we have a
way to capture userland stacks.

Extend map to support keys composed of multiple arguments.

Keys are still strings representing the output value.

The following example is now possible to count the number of "on CPU"
events ordered by thread ID and executable name:

# btrace -e 'tracepoint:sched:on__cpu { @[tid, comm] = count() }'
^C
@[138836, idle0]: 830941
@[161307, sshd]: 716476
@[482901, softnet]: 582008
@[104443, systqmp]: 405749
@[269230, update]: 396133
@[326533, softclock]: 316926
@[61040, sshd]: 177201
@[453567, reaper]: 119676
@[446052, ksh]: 85675
@[26270, syslogd]: 66625
@[504699, sshd]: 52958
@[446052, sshd]: 32207
@[44046, tset]: 13333
@[162960, zerothread]: 101
@[313046, ntpd]: 1

Skip first line if it starts with '#!'

Add support for unnamed map.

The following is now possible but won't print anything by default:

# btrace -e 'profile:hz:99 { @[kstack] = count() }'

ok patrick@, tobhe@

Implement map functions min(), max() and sum().

For the moment map values are currently limited to integers.

Use %left to prevent S/R conflicts, arithmetic precedence is still broken.

Simplify statement rules by using an optional new line.

Implement delete() and @map[key] access.

Rename *_concat() into *_append() to match reality.

Parse '==' without error in filters.

From Benjamin Baier.

Implement builtin time() function.

Import a bug tracer, companion of dt(4), that speaks the bt(5) language.

ok kettenis@, visa@, jasper@, deraadt@

Prevent tuples to be used as map key, associative array is what we want.

Handle variable names (things strating with $ or @) in yylex() this way
the error handling of strange variable names can be better controlled.
With and OK dv@

btrace: add support for hex and octal values.

Changes number tokenizing and parsing to support hex & octal values.
Does not address other lexer issues (e.g. $0x1) to close gaps with
bpftrace.

OK claudio@

btrace: "too long line" -> "line too long"

ok claudio@

bt(5), btrace(8): add support for binary modulo operator ('%')

Link: https://marc.info/?l=openbsd-tech&m=169695435209410&w=2

ok mpi@

Implement tuples, needed to export per-CPU scheduling data.

It is now possible to save and print immutable arrays as below:

..$t = (1, 42, "something");
..printf("%d %s\n", $t.1, $t.2);

Also add support for evaluating builtin in order to save them in variables
(required by tuples)

btrace(8): allow empty statements in probes.

Allows for probes like `BEGIN {}`, in parity with bpftrace.

Also fixes an incorrect syntax error parsing argN builtins in
subsequent probes after an empty BEGIN block.

ok mpi@

btrace(8): prohibit use of argN builtins in BEGIN/END.

The argN builtins are undefined for BEGIN and END special probes.
Similar to bpftrace, produce an error from the parser.

Adds a regress test, as well.

ok mpi@

btrace(8): fix yacc reduce conflicts and undefined symbol warnings.

Define the STR symbol, used for the str function.

Tune the grammar, simplifying the 'pat' rule to 'expr'. Resolves
the reduce conflicts related to 'pat' and 'factor' both matching a
lone CSTRING token.

ok mpi@

spelling fixes; from paul tagliamonte
any parts of his diff not taken are noted on tech

Add support for string comparison in filters.

It is now possible to filter by process name, like:

syscall:mmap:entry
/comm == "ld"/
{
...
}

Currently the parser treats C-string like any other expression member even
if arithmetic operations do no apply to strings.

Add support for multiple statements in if () blocks.

btrace(8): fix lexer to allow whitespace after filters.

Whitespace is allowed after the closing slash of a filter and before
the opening brace of an action. This makes the lexer scan ahead and
collect any whitespace and newlines into the end of filter token.

ok mpi@

Implement the probe variable. Can be used for example with
@map[probe] = count();
OK mpi@

bt(5)/btrace(8): add support for str()

Implement initial support for the str() function, which is used
primarily to truncate or NUL-terminate strings from either cli args
or args to tracepoints and syscalls.

Current implementation only supports cli args and is primarily for
compatability with bpftrace. Future work is needed once dt(4)
supports builtin args other than long values.

Adds a regress test and wires in argument-based tests again.

ok mpi@

Relax the check for accessing variable before setting them.

First we can't assume rules are written in the order they will be executed.
Secondly filters might need to check variables before they had a chance to
be populated by the right event.

Make it possible to associate multiple probes to a single rule.

The following syntax, reducing duplication, is now allowed:

END,
interval:hz:2
{
...
}

Rule descriptors are now linked to a list of probe descriptors instead of
a single one. Enabled kernel probes are now linked to btrace(8) probe
descriptors.

While here stop parsing filter and probe if debug is not enabled.

Revert a chunk committed by inadvertence in my last fix.

btrace(8): add initial support for cli arguments

This adds initial support for passing cli args to btrace(8) for use
in bt(5) scripts. Similar to bpftrace, they are referenced via $1,
$2, etc. with $# being the number of command line arguments provided.

Adds an initial regress test and a Makefile change to allow providing
arguments to regress tests in a .args file.

Currently no limit is imposed on the number of arguments, keeping
a similar approach as observed in bpftrace. References to undefined
arguments result in a new "nil" type that contextually acts as a
zero or empty string. More work can be done here to improve bpftrace
compatibility.

ok mpi@, jasper@

Check that map/hist functions are called with the right argument.

Change the parser to make clear() and zero() accept only local and
global variables as arguments.

Since the parser has no knowledge of the type of a variable abort
the execution if clear() or zero() are being called with something
other than a map or hist.

Fix assertions found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Return early if a parsing error has been found and do not perform any
sanity check as they might obviously fail.

Fix an assert found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Basic test, if (no else atm), support with a single statement.

Rewrite grammar to implement operator precedence without using %right or %prec.

Arithmetic operator should now behave as expeted and tests can now be written
without superfluous parenthesis, for example:

syscall:select:entry
/($1 == 0) || (pid == $1)/
{
}

Can now be written:

syscall:select:entry
/$1 == 0 || pid == $1/
{
}

While here improve filter debugging support.

Implement '<' and '>' operators in filters.

Based on a diff from and ok dv@

Indent and simplify the grammar.

Eliminate S/R conflicts and simplify filter grammar.

Simplify token declaration.

Simplify now that TID and PID are now only being parsed as builtin.

Remove support for in-kernel filters.

This might be added back in a future if copying events to userland becomes
a performance issue. However note that it is not always possible to filter
in-kernel if, for example. a variable has to be evaluated when a rule fires.

Extend filters to support any conditionnal test including global variables.

Stop using in-kernel filtering for the moment except for not tracing the
tracer.

Keep track of the number of filtered events.

Always initialized min value for an histogram.

typo

Support for local (scratch) variables: "$var_name".

Every rule gets its own list of (local) variables.

Extend print() to support any kind of variable.

Support first shell argument as $1 in order to use it in filters.

Remove '-p' option now that scripts can filter by pid/tid.

Extend binary operators support, required for more filter features.

Improve debugging of filters and print operator names in debug output.

Start implementing conditionals for filters.

Allows to check the existence of a variable in predicates, making it
possible to trace syscall latency, as follow:

syscall:select:entry
{
@start[pid] = nsecs;
}

syscall:select:return
/@start[pid]/
{
@usecs = hist((nsecs - @start[pid]) / 1000);
delete(@start[pid]);
}

Exclude the 'hz' token from the lexer backdoor, so interval and profile
probes can be parsed again.

ok anton@ kn@

Tracing syscalls which conflict with reserved bt(5) keywords currently
causes btrace to reject such programs. An example is exit which is both
a syscall and a builtin btrace function. Pointed out by bluhm@

To resolve the conflict, make use of a lexer backdoor. A concept
described in the original yacc paper and also found in other yacc
parsers in our tree.

ok bluhm@ mpi@

In revision 1.18 of bt_parse.y, I missed the fact that the print()
function accepts an optional argument representing the number of map
entries to print.

ok bluhm@ mpi@

The print() function only accepts a single argument.

ok mpi@

add support for '&' and '|' operators in btrace scripts

feedback from otto@
ok mpi@ kn@ semarie@

Implement linear and power-of-two histograms: hist() and lhist() keywords.

This is built on top of maps which are currently built on top of RB-trees.
Improvements are welcome! For example the use of a hashing table as pointed
by espie@.

The following one-liner produce an histogram of power-of-two values returned
by the read(2) syscall:

btrace 'syscall:read:return { @bytes = hist(retval); }'
^C
@bytes:
[0] 19 |@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1] 26 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1, 2) 1 |@ |
[2, 4) 13 |@@@@@@@@@@@@@@@@@@ |
[4, 8) 4 |@@@@@ |
[8, 16) 3 |@@@@ |
[16, 32) 1 |@ |
[32, 64) 8 |@@@@@@@@@@@ |
[64, 128) 14 |@@@@@@@@@@@@@@@@@@@ |
[128, 256) 7 |@@@@@@@@@ |
[256, 512) 37 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[512, 1K) 1 |@ |
[1K, 2K) 10 |@@@@@@@@@@@@@@ |
[2K, 4K) 11 |@@@@@@@@@@@@@@@ |
[8K, 16K) 1 |@ |

Increment line number when skipping multi-line comments.

Fix "the symbol HZ is undefined" yacc warning

OK mpi

Simplify the grammar by letting the lexer handle builtin keywords.

Also clarify various constructs by using commonly understood names like
'expr' and 'vargs'.

Implement builtin 'cpu' keyword.

This is useful to know which CPU recorded a given event.

While here document 'retval' and comment out 'ustack' until we have a
way to capture userland stacks.

Extend map to support keys composed of multiple arguments.

Keys are still strings representing the output value.

The following example is now possible to count the number of "on CPU"
events ordered by thread ID and executable name:

# btrace -e 'tracepoint:sched:on__cpu { @[tid, comm] = count() }'
^C
@[138836, idle0]: 830941
@[161307, sshd]: 716476
@[482901, softnet]: 582008
@[104443, systqmp]: 405749
@[269230, update]: 396133
@[326533, softclock]: 316926
@[61040, sshd]: 177201
@[453567, reaper]: 119676
@[446052, ksh]: 85675
@[26270, syslogd]: 66625
@[504699, sshd]: 52958
@[446052, sshd]: 32207
@[44046, tset]: 13333
@[162960, zerothread]: 101
@[313046, ntpd]: 1

Skip first line if it starts with '#!'

Add support for unnamed map.

The following is now possible but won't print anything by default:

# btrace -e 'profile:hz:99 { @[kstack] = count() }'

ok patrick@, tobhe@

Implement map functions min(), max() and sum().

For the moment map values are currently limited to integers.

Use %left to prevent S/R conflicts, arithmetic precedence is still broken.

Simplify statement rules by using an optional new line.

Implement delete() and @map[key] access.

Rename *_concat() into *_append() to match reality.

Parse '==' without error in filters.

From Benjamin Baier.

Implement builtin time() function.

Import a bug tracer, companion of dt(4), that speaks the bt(5) language.

ok kettenis@, visa@, jasper@, deraadt@

Handle variable names (things strating with $ or @) in yylex() this way
the error handling of strange variable names can be better controlled.
With and OK dv@

btrace: add support for hex and octal values.

Changes number tokenizing and parsing to support hex & octal values.
Does not address other lexer issues (e.g. $0x1) to close gaps with
bpftrace.

OK claudio@

btrace: "too long line" -> "line too long"

ok claudio@

bt(5), btrace(8): add support for binary modulo operator ('%')

Link: https://marc.info/?l=openbsd-tech&m=169695435209410&w=2

ok mpi@

Implement tuples, needed to export per-CPU scheduling data.

It is now possible to save and print immutable arrays as below:

..$t = (1, 42, "something");
..printf("%d %s\n", $t.1, $t.2);

Also add support for evaluating builtin in order to save them in variables
(required by tuples)

btrace(8): allow empty statements in probes.

Allows for probes like `BEGIN {}`, in parity with bpftrace.

Also fixes an incorrect syntax error parsing argN builtins in
subsequent probes after an empty BEGIN block.

ok mpi@

btrace(8): prohibit use of argN builtins in BEGIN/END.

The argN builtins are undefined for BEGIN and END special probes.
Similar to bpftrace, produce an error from the parser.

Adds a regress test, as well.

ok mpi@

btrace(8): fix yacc reduce conflicts and undefined symbol warnings.

Define the STR symbol, used for the str function.

Tune the grammar, simplifying the 'pat' rule to 'expr'. Resolves
the reduce conflicts related to 'pat' and 'factor' both matching a
lone CSTRING token.

ok mpi@

spelling fixes; from paul tagliamonte
any parts of his diff not taken are noted on tech

Add support for string comparison in filters.

It is now possible to filter by process name, like:

syscall:mmap:entry
/comm == "ld"/
{
...
}

Currently the parser treats C-string like any other expression member even
if arithmetic operations do no apply to strings.

Add support for multiple statements in if () blocks.

btrace(8): fix lexer to allow whitespace after filters.

Whitespace is allowed after the closing slash of a filter and before
the opening brace of an action. This makes the lexer scan ahead and
collect any whitespace and newlines into the end of filter token.

ok mpi@

Implement the probe variable. Can be used for example with
@map[probe] = count();
OK mpi@

bt(5)/btrace(8): add support for str()

Implement initial support for the str() function, which is used
primarily to truncate or NUL-terminate strings from either cli args
or args to tracepoints and syscalls.

Current implementation only supports cli args and is primarily for
compatability with bpftrace. Future work is needed once dt(4)
supports builtin args other than long values.

Adds a regress test and wires in argument-based tests again.

ok mpi@

Relax the check for accessing variable before setting them.

First we can't assume rules are written in the order they will be executed.
Secondly filters might need to check variables before they had a chance to
be populated by the right event.

Make it possible to associate multiple probes to a single rule.

The following syntax, reducing duplication, is now allowed:

END,
interval:hz:2
{
...
}

Rule descriptors are now linked to a list of probe descriptors instead of
a single one. Enabled kernel probes are now linked to btrace(8) probe
descriptors.

While here stop parsing filter and probe if debug is not enabled.

Revert a chunk committed by inadvertence in my last fix.

btrace(8): add initial support for cli arguments

This adds initial support for passing cli args to btrace(8) for use
in bt(5) scripts. Similar to bpftrace, they are referenced via $1,
$2, etc. with $# being the number of command line arguments provided.

Adds an initial regress test and a Makefile change to allow providing
arguments to regress tests in a .args file.

Currently no limit is imposed on the number of arguments, keeping
a similar approach as observed in bpftrace. References to undefined
arguments result in a new "nil" type that contextually acts as a
zero or empty string. More work can be done here to improve bpftrace
compatibility.

ok mpi@, jasper@

Check that map/hist functions are called with the right argument.

Change the parser to make clear() and zero() accept only local and
global variables as arguments.

Since the parser has no knowledge of the type of a variable abort
the execution if clear() or zero() are being called with something
other than a map or hist.

Fix assertions found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Return early if a parsing error has been found and do not perform any
sanity check as they might obviously fail.

Fix an assert found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Basic test, if (no else atm), support with a single statement.

Rewrite grammar to implement operator precedence without using %right or %prec.

Arithmetic operator should now behave as expeted and tests can now be written
without superfluous parenthesis, for example:

syscall:select:entry
/($1 == 0) || (pid == $1)/
{
}

Can now be written:

syscall:select:entry
/$1 == 0 || pid == $1/
{
}

While here improve filter debugging support.

Implement '<' and '>' operators in filters.

Based on a diff from and ok dv@

Indent and simplify the grammar.

Eliminate S/R conflicts and simplify filter grammar.

Simplify token declaration.

Simplify now that TID and PID are now only being parsed as builtin.

Remove support for in-kernel filters.

This might be added back in a future if copying events to userland becomes
a performance issue. However note that it is not always possible to filter
in-kernel if, for example. a variable has to be evaluated when a rule fires.

Extend filters to support any conditionnal test including global variables.

Stop using in-kernel filtering for the moment except for not tracing the
tracer.

Keep track of the number of filtered events.

Always initialized min value for an histogram.

typo

Support for local (scratch) variables: "$var_name".

Every rule gets its own list of (local) variables.

Extend print() to support any kind of variable.

Support first shell argument as $1 in order to use it in filters.

Remove '-p' option now that scripts can filter by pid/tid.

Extend binary operators support, required for more filter features.

Improve debugging of filters and print operator names in debug output.

Start implementing conditionals for filters.

Allows to check the existence of a variable in predicates, making it
possible to trace syscall latency, as follow:

syscall:select:entry
{
@start[pid] = nsecs;
}

syscall:select:return
/@start[pid]/
{
@usecs = hist((nsecs - @start[pid]) / 1000);
delete(@start[pid]);
}

Exclude the 'hz' token from the lexer backdoor, so interval and profile
probes can be parsed again.

ok anton@ kn@

Tracing syscalls which conflict with reserved bt(5) keywords currently
causes btrace to reject such programs. An example is exit which is both
a syscall and a builtin btrace function. Pointed out by bluhm@

To resolve the conflict, make use of a lexer backdoor. A concept
described in the original yacc paper and also found in other yacc
parsers in our tree.

ok bluhm@ mpi@

In revision 1.18 of bt_parse.y, I missed the fact that the print()
function accepts an optional argument representing the number of map
entries to print.

ok bluhm@ mpi@

The print() function only accepts a single argument.

ok mpi@

add support for '&' and '|' operators in btrace scripts

feedback from otto@
ok mpi@ kn@ semarie@

Implement linear and power-of-two histograms: hist() and lhist() keywords.

This is built on top of maps which are currently built on top of RB-trees.
Improvements are welcome! For example the use of a hashing table as pointed
by espie@.

The following one-liner produce an histogram of power-of-two values returned
by the read(2) syscall:

btrace 'syscall:read:return { @bytes = hist(retval); }'
^C
@bytes:
[0] 19 |@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1] 26 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1, 2) 1 |@ |
[2, 4) 13 |@@@@@@@@@@@@@@@@@@ |
[4, 8) 4 |@@@@@ |
[8, 16) 3 |@@@@ |
[16, 32) 1 |@ |
[32, 64) 8 |@@@@@@@@@@@ |
[64, 128) 14 |@@@@@@@@@@@@@@@@@@@ |
[128, 256) 7 |@@@@@@@@@ |
[256, 512) 37 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[512, 1K) 1 |@ |
[1K, 2K) 10 |@@@@@@@@@@@@@@ |
[2K, 4K) 11 |@@@@@@@@@@@@@@@ |
[8K, 16K) 1 |@ |

Increment line number when skipping multi-line comments.

Fix "the symbol HZ is undefined" yacc warning

OK mpi

Simplify the grammar by letting the lexer handle builtin keywords.

Also clarify various constructs by using commonly understood names like
'expr' and 'vargs'.

Implement builtin 'cpu' keyword.

This is useful to know which CPU recorded a given event.

While here document 'retval' and comment out 'ustack' until we have a
way to capture userland stacks.

Extend map to support keys composed of multiple arguments.

Keys are still strings representing the output value.

The following example is now possible to count the number of "on CPU"
events ordered by thread ID and executable name:

# btrace -e 'tracepoint:sched:on__cpu { @[tid, comm] = count() }'
^C
@[138836, idle0]: 830941
@[161307, sshd]: 716476
@[482901, softnet]: 582008
@[104443, systqmp]: 405749
@[269230, update]: 396133
@[326533, softclock]: 316926
@[61040, sshd]: 177201
@[453567, reaper]: 119676
@[446052, ksh]: 85675
@[26270, syslogd]: 66625
@[504699, sshd]: 52958
@[446052, sshd]: 32207
@[44046, tset]: 13333
@[162960, zerothread]: 101
@[313046, ntpd]: 1

Skip first line if it starts with '#!'

Add support for unnamed map.

The following is now possible but won't print anything by default:

# btrace -e 'profile:hz:99 { @[kstack] = count() }'

ok patrick@, tobhe@

Implement map functions min(), max() and sum().

For the moment map values are currently limited to integers.

Use %left to prevent S/R conflicts, arithmetic precedence is still broken.

Simplify statement rules by using an optional new line.

Implement delete() and @map[key] access.

Rename *_concat() into *_append() to match reality.

Parse '==' without error in filters.

From Benjamin Baier.

Implement builtin time() function.

Import a bug tracer, companion of dt(4), that speaks the bt(5) language.

ok kettenis@, visa@, jasper@, deraadt@

btrace: add support for hex and octal values.

Changes number tokenizing and parsing to support hex & octal values.
Does not address other lexer issues (e.g. $0x1) to close gaps with
bpftrace.

OK claudio@

btrace: "too long line" -> "line too long"

ok claudio@

bt(5), btrace(8): add support for binary modulo operator ('%')

Link: https://marc.info/?l=openbsd-tech&m=169695435209410&w=2

ok mpi@

Implement tuples, needed to export per-CPU scheduling data.

It is now possible to save and print immutable arrays as below:

..$t = (1, 42, "something");
..printf("%d %s\n", $t.1, $t.2);

Also add support for evaluating builtin in order to save them in variables
(required by tuples)

btrace(8): allow empty statements in probes.

Allows for probes like `BEGIN {}`, in parity with bpftrace.

Also fixes an incorrect syntax error parsing argN builtins in
subsequent probes after an empty BEGIN block.

ok mpi@

btrace(8): prohibit use of argN builtins in BEGIN/END.

The argN builtins are undefined for BEGIN and END special probes.
Similar to bpftrace, produce an error from the parser.

Adds a regress test, as well.

ok mpi@

btrace(8): fix yacc reduce conflicts and undefined symbol warnings.

Define the STR symbol, used for the str function.

Tune the grammar, simplifying the 'pat' rule to 'expr'. Resolves
the reduce conflicts related to 'pat' and 'factor' both matching a
lone CSTRING token.

ok mpi@

spelling fixes; from paul tagliamonte
any parts of his diff not taken are noted on tech

Add support for string comparison in filters.

It is now possible to filter by process name, like:

syscall:mmap:entry
/comm == "ld"/
{
...
}

Currently the parser treats C-string like any other expression member even
if arithmetic operations do no apply to strings.

Add support for multiple statements in if () blocks.

btrace(8): fix lexer to allow whitespace after filters.

Whitespace is allowed after the closing slash of a filter and before
the opening brace of an action. This makes the lexer scan ahead and
collect any whitespace and newlines into the end of filter token.

ok mpi@

Implement the probe variable. Can be used for example with
@map[probe] = count();
OK mpi@

bt(5)/btrace(8): add support for str()

Implement initial support for the str() function, which is used
primarily to truncate or NUL-terminate strings from either cli args
or args to tracepoints and syscalls.

Current implementation only supports cli args and is primarily for
compatability with bpftrace. Future work is needed once dt(4)
supports builtin args other than long values.

Adds a regress test and wires in argument-based tests again.

ok mpi@

Relax the check for accessing variable before setting them.

First we can't assume rules are written in the order they will be executed.
Secondly filters might need to check variables before they had a chance to
be populated by the right event.

Make it possible to associate multiple probes to a single rule.

The following syntax, reducing duplication, is now allowed:

END,
interval:hz:2
{
...
}

Rule descriptors are now linked to a list of probe descriptors instead of
a single one. Enabled kernel probes are now linked to btrace(8) probe
descriptors.

While here stop parsing filter and probe if debug is not enabled.

Revert a chunk committed by inadvertence in my last fix.

btrace(8): add initial support for cli arguments

This adds initial support for passing cli args to btrace(8) for use
in bt(5) scripts. Similar to bpftrace, they are referenced via $1,
$2, etc. with $# being the number of command line arguments provided.

Adds an initial regress test and a Makefile change to allow providing
arguments to regress tests in a .args file.

Currently no limit is imposed on the number of arguments, keeping
a similar approach as observed in bpftrace. References to undefined
arguments result in a new "nil" type that contextually acts as a
zero or empty string. More work can be done here to improve bpftrace
compatibility.

ok mpi@, jasper@

Check that map/hist functions are called with the right argument.

Change the parser to make clear() and zero() accept only local and
global variables as arguments.

Since the parser has no knowledge of the type of a variable abort
the execution if clear() or zero() are being called with something
other than a map or hist.

Fix assertions found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Return early if a parsing error has been found and do not perform any
sanity check as they might obviously fail.

Fix an assert found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Basic test, if (no else atm), support with a single statement.

Rewrite grammar to implement operator precedence without using %right or %prec.

Arithmetic operator should now behave as expeted and tests can now be written
without superfluous parenthesis, for example:

syscall:select:entry
/($1 == 0) || (pid == $1)/
{
}

Can now be written:

syscall:select:entry
/$1 == 0 || pid == $1/
{
}

While here improve filter debugging support.

Implement '<' and '>' operators in filters.

Based on a diff from and ok dv@

Indent and simplify the grammar.

Eliminate S/R conflicts and simplify filter grammar.

Simplify token declaration.

Simplify now that TID and PID are now only being parsed as builtin.

Remove support for in-kernel filters.

This might be added back in a future if copying events to userland becomes
a performance issue. However note that it is not always possible to filter
in-kernel if, for example. a variable has to be evaluated when a rule fires.

Extend filters to support any conditionnal test including global variables.

Stop using in-kernel filtering for the moment except for not tracing the
tracer.

Keep track of the number of filtered events.

Always initialized min value for an histogram.

typo

Support for local (scratch) variables: "$var_name".

Every rule gets its own list of (local) variables.

Extend print() to support any kind of variable.

Support first shell argument as $1 in order to use it in filters.

Remove '-p' option now that scripts can filter by pid/tid.

Extend binary operators support, required for more filter features.

Improve debugging of filters and print operator names in debug output.

Start implementing conditionals for filters.

Allows to check the existence of a variable in predicates, making it
possible to trace syscall latency, as follow:

syscall:select:entry
{
@start[pid] = nsecs;
}

syscall:select:return
/@start[pid]/
{
@usecs = hist((nsecs - @start[pid]) / 1000);
delete(@start[pid]);
}

Exclude the 'hz' token from the lexer backdoor, so interval and profile
probes can be parsed again.

ok anton@ kn@

Tracing syscalls which conflict with reserved bt(5) keywords currently
causes btrace to reject such programs. An example is exit which is both
a syscall and a builtin btrace function. Pointed out by bluhm@

To resolve the conflict, make use of a lexer backdoor. A concept
described in the original yacc paper and also found in other yacc
parsers in our tree.

ok bluhm@ mpi@

In revision 1.18 of bt_parse.y, I missed the fact that the print()
function accepts an optional argument representing the number of map
entries to print.

ok bluhm@ mpi@

The print() function only accepts a single argument.

ok mpi@

add support for '&' and '|' operators in btrace scripts

feedback from otto@
ok mpi@ kn@ semarie@

Implement linear and power-of-two histograms: hist() and lhist() keywords.

This is built on top of maps which are currently built on top of RB-trees.
Improvements are welcome! For example the use of a hashing table as pointed
by espie@.

The following one-liner produce an histogram of power-of-two values returned
by the read(2) syscall:

btrace 'syscall:read:return { @bytes = hist(retval); }'
^C
@bytes:
[0] 19 |@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1] 26 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1, 2) 1 |@ |
[2, 4) 13 |@@@@@@@@@@@@@@@@@@ |
[4, 8) 4 |@@@@@ |
[8, 16) 3 |@@@@ |
[16, 32) 1 |@ |
[32, 64) 8 |@@@@@@@@@@@ |
[64, 128) 14 |@@@@@@@@@@@@@@@@@@@ |
[128, 256) 7 |@@@@@@@@@ |
[256, 512) 37 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[512, 1K) 1 |@ |
[1K, 2K) 10 |@@@@@@@@@@@@@@ |
[2K, 4K) 11 |@@@@@@@@@@@@@@@ |
[8K, 16K) 1 |@ |

Increment line number when skipping multi-line comments.

Fix "the symbol HZ is undefined" yacc warning

OK mpi

Simplify the grammar by letting the lexer handle builtin keywords.

Also clarify various constructs by using commonly understood names like
'expr' and 'vargs'.

Implement builtin 'cpu' keyword.

This is useful to know which CPU recorded a given event.

While here document 'retval' and comment out 'ustack' until we have a
way to capture userland stacks.

Extend map to support keys composed of multiple arguments.

Keys are still strings representing the output value.

The following example is now possible to count the number of "on CPU"
events ordered by thread ID and executable name:

# btrace -e 'tracepoint:sched:on__cpu { @[tid, comm] = count() }'
^C
@[138836, idle0]: 830941
@[161307, sshd]: 716476
@[482901, softnet]: 582008
@[104443, systqmp]: 405749
@[269230, update]: 396133
@[326533, softclock]: 316926
@[61040, sshd]: 177201
@[453567, reaper]: 119676
@[446052, ksh]: 85675
@[26270, syslogd]: 66625
@[504699, sshd]: 52958
@[446052, sshd]: 32207
@[44046, tset]: 13333
@[162960, zerothread]: 101
@[313046, ntpd]: 1

Skip first line if it starts with '#!'

Add support for unnamed map.

The following is now possible but won't print anything by default:

# btrace -e 'profile:hz:99 { @[kstack] = count() }'

ok patrick@, tobhe@

Implement map functions min(), max() and sum().

For the moment map values are currently limited to integers.

Use %left to prevent S/R conflicts, arithmetic precedence is still broken.

Simplify statement rules by using an optional new line.

Implement delete() and @map[key] access.

Rename *_concat() into *_append() to match reality.

Parse '==' without error in filters.

From Benjamin Baier.

Implement builtin time() function.

Import a bug tracer, companion of dt(4), that speaks the bt(5) language.

ok kettenis@, visa@, jasper@, deraadt@

bt(5), btrace(8): add support for binary modulo operator ('%')

Link: https://marc.info/?l=openbsd-tech&m=169695435209410&w=2

ok mpi@

Implement tuples, needed to export per-CPU scheduling data.

It is now possible to save and print immutable arrays as below:

..$t = (1, 42, "something");
..printf("%d %s\n", $t.1, $t.2);

Also add support for evaluating builtin in order to save them in variables
(required by tuples)

btrace(8): allow empty statements in probes.

Allows for probes like `BEGIN {}`, in parity with bpftrace.

Also fixes an incorrect syntax error parsing argN builtins in
subsequent probes after an empty BEGIN block.

ok mpi@

btrace(8): prohibit use of argN builtins in BEGIN/END.

The argN builtins are undefined for BEGIN and END special probes.
Similar to bpftrace, produce an error from the parser.

Adds a regress test, as well.

ok mpi@

btrace(8): fix yacc reduce conflicts and undefined symbol warnings.

Define the STR symbol, used for the str function.

Tune the grammar, simplifying the 'pat' rule to 'expr'. Resolves
the reduce conflicts related to 'pat' and 'factor' both matching a
lone CSTRING token.

ok mpi@

spelling fixes; from paul tagliamonte
any parts of his diff not taken are noted on tech

Add support for string comparison in filters.

It is now possible to filter by process name, like:

syscall:mmap:entry
/comm == "ld"/
{
...
}

Currently the parser treats C-string like any other expression member even
if arithmetic operations do no apply to strings.

Add support for multiple statements in if () blocks.

btrace(8): fix lexer to allow whitespace after filters.

Whitespace is allowed after the closing slash of a filter and before
the opening brace of an action. This makes the lexer scan ahead and
collect any whitespace and newlines into the end of filter token.

ok mpi@

Implement the probe variable. Can be used for example with
@map[probe] = count();
OK mpi@

bt(5)/btrace(8): add support for str()

Implement initial support for the str() function, which is used
primarily to truncate or NUL-terminate strings from either cli args
or args to tracepoints and syscalls.

Current implementation only supports cli args and is primarily for
compatability with bpftrace. Future work is needed once dt(4)
supports builtin args other than long values.

Adds a regress test and wires in argument-based tests again.

ok mpi@

Relax the check for accessing variable before setting them.

First we can't assume rules are written in the order they will be executed.
Secondly filters might need to check variables before they had a chance to
be populated by the right event.

Make it possible to associate multiple probes to a single rule.

The following syntax, reducing duplication, is now allowed:

END,
interval:hz:2
{
...
}

Rule descriptors are now linked to a list of probe descriptors instead of
a single one. Enabled kernel probes are now linked to btrace(8) probe
descriptors.

While here stop parsing filter and probe if debug is not enabled.

Revert a chunk committed by inadvertence in my last fix.

btrace(8): add initial support for cli arguments

This adds initial support for passing cli args to btrace(8) for use
in bt(5) scripts. Similar to bpftrace, they are referenced via $1,
$2, etc. with $# being the number of command line arguments provided.

Adds an initial regress test and a Makefile change to allow providing
arguments to regress tests in a .args file.

Currently no limit is imposed on the number of arguments, keeping
a similar approach as observed in bpftrace. References to undefined
arguments result in a new "nil" type that contextually acts as a
zero or empty string. More work can be done here to improve bpftrace
compatibility.

ok mpi@, jasper@

Check that map/hist functions are called with the right argument.

Change the parser to make clear() and zero() accept only local and
global variables as arguments.

Since the parser has no knowledge of the type of a variable abort
the execution if clear() or zero() are being called with something
other than a map or hist.

Fix assertions found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Return early if a parsing error has been found and do not perform any
sanity check as they might obviously fail.

Fix an assert found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Basic test, if (no else atm), support with a single statement.

Rewrite grammar to implement operator precedence without using %right or %prec.

Arithmetic operator should now behave as expeted and tests can now be written
without superfluous parenthesis, for example:

syscall:select:entry
/($1 == 0) || (pid == $1)/
{
}

Can now be written:

syscall:select:entry
/$1 == 0 || pid == $1/
{
}

While here improve filter debugging support.

Implement '<' and '>' operators in filters.

Based on a diff from and ok dv@

Indent and simplify the grammar.

Eliminate S/R conflicts and simplify filter grammar.

Simplify token declaration.

Simplify now that TID and PID are now only being parsed as builtin.

Remove support for in-kernel filters.

This might be added back in a future if copying events to userland becomes
a performance issue. However note that it is not always possible to filter
in-kernel if, for example. a variable has to be evaluated when a rule fires.

Extend filters to support any conditionnal test including global variables.

Stop using in-kernel filtering for the moment except for not tracing the
tracer.

Keep track of the number of filtered events.

Always initialized min value for an histogram.

typo

Support for local (scratch) variables: "$var_name".

Every rule gets its own list of (local) variables.

Extend print() to support any kind of variable.

Support first shell argument as $1 in order to use it in filters.

Remove '-p' option now that scripts can filter by pid/tid.

Extend binary operators support, required for more filter features.

Improve debugging of filters and print operator names in debug output.

Start implementing conditionals for filters.

Allows to check the existence of a variable in predicates, making it
possible to trace syscall latency, as follow:

syscall:select:entry
{
@start[pid] = nsecs;
}

syscall:select:return
/@start[pid]/
{
@usecs = hist((nsecs - @start[pid]) / 1000);
delete(@start[pid]);
}

Exclude the 'hz' token from the lexer backdoor, so interval and profile
probes can be parsed again.

ok anton@ kn@

Tracing syscalls which conflict with reserved bt(5) keywords currently
causes btrace to reject such programs. An example is exit which is both
a syscall and a builtin btrace function. Pointed out by bluhm@

To resolve the conflict, make use of a lexer backdoor. A concept
described in the original yacc paper and also found in other yacc
parsers in our tree.

ok bluhm@ mpi@

In revision 1.18 of bt_parse.y, I missed the fact that the print()
function accepts an optional argument representing the number of map
entries to print.

ok bluhm@ mpi@

The print() function only accepts a single argument.

ok mpi@

add support for '&' and '|' operators in btrace scripts

feedback from otto@
ok mpi@ kn@ semarie@

Implement linear and power-of-two histograms: hist() and lhist() keywords.

This is built on top of maps which are currently built on top of RB-trees.
Improvements are welcome! For example the use of a hashing table as pointed
by espie@.

The following one-liner produce an histogram of power-of-two values returned
by the read(2) syscall:

btrace 'syscall:read:return { @bytes = hist(retval); }'
^C
@bytes:
[0] 19 |@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1] 26 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1, 2) 1 |@ |
[2, 4) 13 |@@@@@@@@@@@@@@@@@@ |
[4, 8) 4 |@@@@@ |
[8, 16) 3 |@@@@ |
[16, 32) 1 |@ |
[32, 64) 8 |@@@@@@@@@@@ |
[64, 128) 14 |@@@@@@@@@@@@@@@@@@@ |
[128, 256) 7 |@@@@@@@@@ |
[256, 512) 37 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[512, 1K) 1 |@ |
[1K, 2K) 10 |@@@@@@@@@@@@@@ |
[2K, 4K) 11 |@@@@@@@@@@@@@@@ |
[8K, 16K) 1 |@ |

Increment line number when skipping multi-line comments.

Fix "the symbol HZ is undefined" yacc warning

OK mpi

Simplify the grammar by letting the lexer handle builtin keywords.

Also clarify various constructs by using commonly understood names like
'expr' and 'vargs'.

Implement builtin 'cpu' keyword.

This is useful to know which CPU recorded a given event.

While here document 'retval' and comment out 'ustack' until we have a
way to capture userland stacks.

Extend map to support keys composed of multiple arguments.

Keys are still strings representing the output value.

The following example is now possible to count the number of "on CPU"
events ordered by thread ID and executable name:

# btrace -e 'tracepoint:sched:on__cpu { @[tid, comm] = count() }'
^C
@[138836, idle0]: 830941
@[161307, sshd]: 716476
@[482901, softnet]: 582008
@[104443, systqmp]: 405749
@[269230, update]: 396133
@[326533, softclock]: 316926
@[61040, sshd]: 177201
@[453567, reaper]: 119676
@[446052, ksh]: 85675
@[26270, syslogd]: 66625
@[504699, sshd]: 52958
@[446052, sshd]: 32207
@[44046, tset]: 13333
@[162960, zerothread]: 101
@[313046, ntpd]: 1

Skip first line if it starts with '#!'

Add support for unnamed map.

The following is now possible but won't print anything by default:

# btrace -e 'profile:hz:99 { @[kstack] = count() }'

ok patrick@, tobhe@

Implement map functions min(), max() and sum().

For the moment map values are currently limited to integers.

Use %left to prevent S/R conflicts, arithmetic precedence is still broken.

Simplify statement rules by using an optional new line.

Implement delete() and @map[key] access.

Rename *_concat() into *_append() to match reality.

Parse '==' without error in filters.

From Benjamin Baier.

Implement builtin time() function.

Import a bug tracer, companion of dt(4), that speaks the bt(5) language.

ok kettenis@, visa@, jasper@, deraadt@

Implement tuples, needed to export per-CPU scheduling data.

It is now possible to save and print immutable arrays as below:

..$t = (1, 42, "something");
..printf("%d %s\n", $t.1, $t.2);

Also add support for evaluating builtin in order to save them in variables
(required by tuples)

btrace(8): allow empty statements in probes.

Allows for probes like `BEGIN {}`, in parity with bpftrace.

Also fixes an incorrect syntax error parsing argN builtins in
subsequent probes after an empty BEGIN block.

ok mpi@

btrace(8): prohibit use of argN builtins in BEGIN/END.

The argN builtins are undefined for BEGIN and END special probes.
Similar to bpftrace, produce an error from the parser.

Adds a regress test, as well.

ok mpi@

btrace(8): fix yacc reduce conflicts and undefined symbol warnings.

Define the STR symbol, used for the str function.

Tune the grammar, simplifying the 'pat' rule to 'expr'. Resolves
the reduce conflicts related to 'pat' and 'factor' both matching a
lone CSTRING token.

ok mpi@

spelling fixes; from paul tagliamonte
any parts of his diff not taken are noted on tech

Add support for string comparison in filters.

It is now possible to filter by process name, like:

syscall:mmap:entry
/comm == "ld"/
{
...
}

Currently the parser treats C-string like any other expression member even
if arithmetic operations do no apply to strings.

Add support for multiple statements in if () blocks.

btrace(8): fix lexer to allow whitespace after filters.

Whitespace is allowed after the closing slash of a filter and before
the opening brace of an action. This makes the lexer scan ahead and
collect any whitespace and newlines into the end of filter token.

ok mpi@

Implement the probe variable. Can be used for example with
@map[probe] = count();
OK mpi@

bt(5)/btrace(8): add support for str()

Implement initial support for the str() function, which is used
primarily to truncate or NUL-terminate strings from either cli args
or args to tracepoints and syscalls.

Current implementation only supports cli args and is primarily for
compatability with bpftrace. Future work is needed once dt(4)
supports builtin args other than long values.

Adds a regress test and wires in argument-based tests again.

ok mpi@

Relax the check for accessing variable before setting them.

First we can't assume rules are written in the order they will be executed.
Secondly filters might need to check variables before they had a chance to
be populated by the right event.

Make it possible to associate multiple probes to a single rule.

The following syntax, reducing duplication, is now allowed:

END,
interval:hz:2
{
...
}

Rule descriptors are now linked to a list of probe descriptors instead of
a single one. Enabled kernel probes are now linked to btrace(8) probe
descriptors.

While here stop parsing filter and probe if debug is not enabled.

Revert a chunk committed by inadvertence in my last fix.

btrace(8): add initial support for cli arguments

This adds initial support for passing cli args to btrace(8) for use
in bt(5) scripts. Similar to bpftrace, they are referenced via $1,
$2, etc. with $# being the number of command line arguments provided.

Adds an initial regress test and a Makefile change to allow providing
arguments to regress tests in a .args file.

Currently no limit is imposed on the number of arguments, keeping
a similar approach as observed in bpftrace. References to undefined
arguments result in a new "nil" type that contextually acts as a
zero or empty string. More work can be done here to improve bpftrace
compatibility.

ok mpi@, jasper@

Check that map/hist functions are called with the right argument.

Change the parser to make clear() and zero() accept only local and
global variables as arguments.

Since the parser has no knowledge of the type of a variable abort
the execution if clear() or zero() are being called with something
other than a map or hist.

Fix assertions found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Return early if a parsing error has been found and do not perform any
sanity check as they might obviously fail.

Fix an assert found by jasper@ with AFL++ (port coming soon!).

ok jasper@

Basic test, if (no else atm), support with a single statement.

Rewrite grammar to implement operator precedence without using %right or %prec.

Arithmetic operator should now behave as expeted and tests can now be written
without superfluous parenthesis, for example:

syscall:select:entry
/($1 == 0) || (pid == $1)/
{
}

Can now be written:

syscall:select:entry
/$1 == 0 || pid == $1/
{
}

While here improve filter debugging support.

Implement '<' and '>' operators in filters.

Based on a diff from and ok dv@

Indent and simplify the grammar.

Eliminate S/R conflicts and simplify filter grammar.

Simplify token declaration.

Simplify now that TID and PID are now only being parsed as builtin.

Remove support for in-kernel filters.

This might be added back in a future if copying events to userland becomes
a performance issue. However note that it is not always possible to filter
in-kernel if, for example. a variable has to be evaluated when a rule fires.

Extend filters to support any conditionnal test including global variables.

Stop using in-kernel filtering for the moment except for not tracing the
tracer.

Keep track of the number of filtered events.

Always initialized min value for an histogram.

typo

Support for local (scratch) variables: "$var_name".

Every rule gets its own list of (local) variables.

Extend print() to support any kind of variable.

Support first shell argument as $1 in order to use it in filters.

Remove '-p' option now that scripts can filter by pid/tid.

Extend binary operators support, required for more filter features.

Improve debugging of filters and print operator names in debug output.

Start implementing conditionals for filters.

Allows to check the existence of a variable in predicates, making it
possible to trace syscall latency, as follow:

syscall:select:entry
{
@start[pid] = nsecs;
}

syscall:select:return
/@start[pid]/
{
@usecs = hist((nsecs - @start[pid]) / 1000);
delete(@start[pid]);
}

Exclude the 'hz' token from the lexer backdoor, so interval and profile
probes can be parsed again.

ok anton@ kn@

Tracing syscalls which conflict with reserved bt(5) keywords currently
causes btrace to reject such programs. An example is exit which is both
a syscall and a builtin btrace function. Pointed out by bluhm@

To resolve the conflict, make use of a lexer backdoor. A concept
described in the original yacc paper and also found in other yacc
parsers in our tree.

ok bluhm@ mpi@

In revision 1.18 of bt_parse.y, I missed the fact that the print()
function accepts an optional argument representing the number of map
entries to print.

ok bluhm@ mpi@

The print() function only accepts a single argument.

ok mpi@

add support for '&' and '|' operators in btrace scripts

feedback from otto@
ok mpi@ kn@ semarie@

Implement linear and power-of-two histograms: hist() and lhist() keywords.

This is built on top of maps which are currently built on top of RB-trees.
Improvements are welcome! For example the use of a hashing table as pointed
by espie@.

The following one-liner produce an histogram of power-of-two values returned
by the read(2) syscall:

btrace 'syscall:read:return { @bytes = hist(retval); }'
^C
@bytes:
[0] 19 |@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1] 26 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1, 2) 1 |@ |
[2, 4) 13 |@@@@@@@@@@@@@@@@@@ |
[4, 8) 4 |@@@@@ |
[8, 16) 3 |@@@@ |
[16, 32) 1 |@ |
[32, 64) 8 |@@@@@@@@@@@ |
[64, 128) 14 |@@@@@@@@@@@@@@@@@@@ |
[128, 256) 7 |@@@@@@@@@ |
[256, 512) 37 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[512, 1K) 1 |@ |
[1K, 2K) 10 |@@@@@@@@@@@@@@ |
[2K, 4K) 11 |@@@@@@@@@@@@@@@ |
[8K, 16K) 1 |@ |

Increment line number when skipping multi-line comments.

Fix "the symbol HZ is undefined" yacc warning

OK mpi

Simplify the grammar by letting the lexer handle builtin keywords.

Also clarify various constructs by using commonly understood names like
'expr' and 'vargs'.

Implement builtin 'cpu' keyword.

This is useful to know which CPU recorded a given event.

While here document 'retval' and comment out 'ustack' until we have a
way to capture userland stacks.

Extend map to support keys composed of multiple arguments.

Keys are still strings representing the output value.

The following example is now possible to count the number of "on CPU"
events ordered by thread ID and executable name:

# btrace -e 'tracepoint:sched:on__cpu { @[tid, comm] = count() }'
^C
@[138836, idle0]: 830941
@[161307, sshd]: 716476
@[482901, softnet]: 582008
@[104443, systqmp]: 405749
@[269230, update]: 396133
@[326533, softclock]: 316926
@[61040, sshd]: 177201
@[453567, reaper]: 119676
@[446052, ksh]: 85675
@[26270, syslogd]: 66625
@[504699, sshd]: 52958
@[446052, sshd]: 32207
@[44046, tset]: 13333
@[162960, zerothread]: 101
@[313046, ntpd]: 1

Skip first line if it starts with '#!'

Add support for unnamed map.

The following is now possible but won't print anything by default:

# btrace -e 'profile:hz:99 { @[kstack] = count() }'

ok patrick@, tobhe@

Implement map functions min(), max() and sum().

For the moment map values are currently limited to integers.

Use %left to prevent S/R conflicts, arithmetic precedence is still broken.

Simplify statement rules by using an optional new line.