RFC2578 section 7.1 specifies the ranges and in the case of opaque the
format to which the values need to adhere. Implement checks, so that we
don't send illegal values to the client.

OK tb@

Replace most smi_oid2string() calls with the new mib_oid2string().
smi_oid2string() is still called from trap handle context to not break
any existing scripts.

OK tb@

Clean up snmpd's header situation.

With the help of tb@ and include-what-you-use.

OK tb@

In appl_processpdu() no need to set avi_next, and only set av_next up
to varbindlen, since its only use is to print the varbindlist via
appl_pdu_log() and both are further properly initialized in
appl_request_upstream_resolve().

This fixes a cosmetic off by one for getbulk requests.

OK tb@

avi_origid must also be set when transitioning out of the
APPL_VBSTATE_MUSTFILL state, else snmpd won't like use once we reach
EOMV of our view of the world.

OK tb@

Make sure we allocate the correct size for an appl_agentcap.

OK claudio@ miod@

struct appl_varbind_internal's avi_index is used to give the index to
the original varbindlist's index. In the case of a GetBulkRequest this
must never be larger than the length of the original varbindlist.

OK tb@

Now that the last consumer of mps.c is gone, remove it and its
application_legacy.c companion.

OK tb@

Let usm_make_report() utilize appl_report(). usm_make_report utilized
mps_getstr(), which after moving the SNMPv2-SMI::snmpV2 into
application_internal returned a noSuchObject. This doesn't seem to have
broken any tools that I'm aware of, but this returns the correct result.

OK tb@

Let appl_report() collect its own metrics. This simplifies the interface
and gives us a free report-pdu log line in debug mode.

OK tb@

Don't rely on aru_pdu to rebuild the original varbindlist on error.
Now that we have avi_origid it's not needed anymore and aru_pdu needs
to go.

OK tb@

Don't use aru_pdu for determining the requesttype. It's owned by
snmp_message and getting rid of it is also needed for appl_report() to
gather its own information.

OK tb@

export SNMP-TARGET-MIB::{snmpUnavailableContexts,snmpUnknownContexts}
via application_internal.

OK tb@

Readd the sysORTable based on the new struct appl_agentcap.

OK tb@

Allow agent capabilities to be stored on a per appl_context basis.
This is needed for AgentX's {add,remove}agentcaps, and the sysORTable.

OK tb@

Introduce application_internal.c. This backend is meant to replace
application_legacy.c, mps.c, and mib.c. This commit just introduces the
backend. The existing MIBs inside mib.c will be copied over in
subsequent commits.

OK tb@

When doing a get{next,bulk} below an instance we must move the OID to
the next sibling. Not simply copying over the value of the next to use
region, since that might be the parent and we would walk backwards in
the tree.

OK tb@

searchrange.end is non-inclusive. Adjust the tests for that.

OK tb@

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

Replace most smi_oid2string() calls with the new mib_oid2string().
smi_oid2string() is still called from trap handle context to not break
any existing scripts.

OK tb@

Clean up snmpd's header situation.

With the help of tb@ and include-what-you-use.

OK tb@

In appl_processpdu() no need to set avi_next, and only set av_next up
to varbindlen, since its only use is to print the varbindlist via
appl_pdu_log() and both are further properly initialized in
appl_request_upstream_resolve().

This fixes a cosmetic off by one for getbulk requests.

OK tb@

avi_origid must also be set when transitioning out of the
APPL_VBSTATE_MUSTFILL state, else snmpd won't like use once we reach
EOMV of our view of the world.

OK tb@

Make sure we allocate the correct size for an appl_agentcap.

OK claudio@ miod@

struct appl_varbind_internal's avi_index is used to give the index to
the original varbindlist's index. In the case of a GetBulkRequest this
must never be larger than the length of the original varbindlist.

OK tb@

Now that the last consumer of mps.c is gone, remove it and its
application_legacy.c companion.

OK tb@

Let usm_make_report() utilize appl_report(). usm_make_report utilized
mps_getstr(), which after moving the SNMPv2-SMI::snmpV2 into
application_internal returned a noSuchObject. This doesn't seem to have
broken any tools that I'm aware of, but this returns the correct result.

OK tb@

Let appl_report() collect its own metrics. This simplifies the interface
and gives us a free report-pdu log line in debug mode.

OK tb@

Don't rely on aru_pdu to rebuild the original varbindlist on error.
Now that we have avi_origid it's not needed anymore and aru_pdu needs
to go.

OK tb@

Don't use aru_pdu for determining the requesttype. It's owned by
snmp_message and getting rid of it is also needed for appl_report() to
gather its own information.

OK tb@

export SNMP-TARGET-MIB::{snmpUnavailableContexts,snmpUnknownContexts}
via application_internal.

OK tb@

Readd the sysORTable based on the new struct appl_agentcap.

OK tb@

Allow agent capabilities to be stored on a per appl_context basis.
This is needed for AgentX's {add,remove}agentcaps, and the sysORTable.

OK tb@

Introduce application_internal.c. This backend is meant to replace
application_legacy.c, mps.c, and mib.c. This commit just introduces the
backend. The existing MIBs inside mib.c will be copied over in
subsequent commits.

OK tb@

When doing a get{next,bulk} below an instance we must move the OID to
the next sibling. Not simply copying over the value of the next to use
region, since that might be the parent and we would walk backwards in
the tree.

OK tb@

searchrange.end is non-inclusive. Adjust the tests for that.

OK tb@

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

Clean up snmpd's header situation.

With the help of tb@ and include-what-you-use.

OK tb@

In appl_processpdu() no need to set avi_next, and only set av_next up
to varbindlen, since its only use is to print the varbindlist via
appl_pdu_log() and both are further properly initialized in
appl_request_upstream_resolve().

This fixes a cosmetic off by one for getbulk requests.

OK tb@

avi_origid must also be set when transitioning out of the
APPL_VBSTATE_MUSTFILL state, else snmpd won't like use once we reach
EOMV of our view of the world.

OK tb@

Make sure we allocate the correct size for an appl_agentcap.

OK claudio@ miod@

struct appl_varbind_internal's avi_index is used to give the index to
the original varbindlist's index. In the case of a GetBulkRequest this
must never be larger than the length of the original varbindlist.

OK tb@

Now that the last consumer of mps.c is gone, remove it and its
application_legacy.c companion.

OK tb@

Let usm_make_report() utilize appl_report(). usm_make_report utilized
mps_getstr(), which after moving the SNMPv2-SMI::snmpV2 into
application_internal returned a noSuchObject. This doesn't seem to have
broken any tools that I'm aware of, but this returns the correct result.

OK tb@

Let appl_report() collect its own metrics. This simplifies the interface
and gives us a free report-pdu log line in debug mode.

OK tb@

Don't rely on aru_pdu to rebuild the original varbindlist on error.
Now that we have avi_origid it's not needed anymore and aru_pdu needs
to go.

OK tb@

Don't use aru_pdu for determining the requesttype. It's owned by
snmp_message and getting rid of it is also needed for appl_report() to
gather its own information.

OK tb@

export SNMP-TARGET-MIB::{snmpUnavailableContexts,snmpUnknownContexts}
via application_internal.

OK tb@

Readd the sysORTable based on the new struct appl_agentcap.

OK tb@

Allow agent capabilities to be stored on a per appl_context basis.
This is needed for AgentX's {add,remove}agentcaps, and the sysORTable.

OK tb@

Introduce application_internal.c. This backend is meant to replace
application_legacy.c, mps.c, and mib.c. This commit just introduces the
backend. The existing MIBs inside mib.c will be copied over in
subsequent commits.

OK tb@

When doing a get{next,bulk} below an instance we must move the OID to
the next sibling. Not simply copying over the value of the next to use
region, since that might be the parent and we would walk backwards in
the tree.

OK tb@

searchrange.end is non-inclusive. Adjust the tests for that.

OK tb@

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

In appl_processpdu() no need to set avi_next, and only set av_next up
to varbindlen, since its only use is to print the varbindlist via
appl_pdu_log() and both are further properly initialized in
appl_request_upstream_resolve().

This fixes a cosmetic off by one for getbulk requests.

OK tb@

avi_origid must also be set when transitioning out of the
APPL_VBSTATE_MUSTFILL state, else snmpd won't like use once we reach
EOMV of our view of the world.

OK tb@

Make sure we allocate the correct size for an appl_agentcap.

OK claudio@ miod@

struct appl_varbind_internal's avi_index is used to give the index to
the original varbindlist's index. In the case of a GetBulkRequest this
must never be larger than the length of the original varbindlist.

OK tb@

Now that the last consumer of mps.c is gone, remove it and its
application_legacy.c companion.

OK tb@

Let usm_make_report() utilize appl_report(). usm_make_report utilized
mps_getstr(), which after moving the SNMPv2-SMI::snmpV2 into
application_internal returned a noSuchObject. This doesn't seem to have
broken any tools that I'm aware of, but this returns the correct result.

OK tb@

Let appl_report() collect its own metrics. This simplifies the interface
and gives us a free report-pdu log line in debug mode.

OK tb@

Don't rely on aru_pdu to rebuild the original varbindlist on error.
Now that we have avi_origid it's not needed anymore and aru_pdu needs
to go.

OK tb@

Don't use aru_pdu for determining the requesttype. It's owned by
snmp_message and getting rid of it is also needed for appl_report() to
gather its own information.

OK tb@

export SNMP-TARGET-MIB::{snmpUnavailableContexts,snmpUnknownContexts}
via application_internal.

OK tb@

Readd the sysORTable based on the new struct appl_agentcap.

OK tb@

Allow agent capabilities to be stored on a per appl_context basis.
This is needed for AgentX's {add,remove}agentcaps, and the sysORTable.

OK tb@

Introduce application_internal.c. This backend is meant to replace
application_legacy.c, mps.c, and mib.c. This commit just introduces the
backend. The existing MIBs inside mib.c will be copied over in
subsequent commits.

OK tb@

When doing a get{next,bulk} below an instance we must move the OID to
the next sibling. Not simply copying over the value of the next to use
region, since that might be the parent and we would walk backwards in
the tree.

OK tb@

searchrange.end is non-inclusive. Adjust the tests for that.

OK tb@

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

Make sure we allocate the correct size for an appl_agentcap.

OK claudio@ miod@

struct appl_varbind_internal's avi_index is used to give the index to
the original varbindlist's index. In the case of a GetBulkRequest this
must never be larger than the length of the original varbindlist.

OK tb@

Now that the last consumer of mps.c is gone, remove it and its
application_legacy.c companion.

OK tb@

Let usm_make_report() utilize appl_report(). usm_make_report utilized
mps_getstr(), which after moving the SNMPv2-SMI::snmpV2 into
application_internal returned a noSuchObject. This doesn't seem to have
broken any tools that I'm aware of, but this returns the correct result.

OK tb@

Let appl_report() collect its own metrics. This simplifies the interface
and gives us a free report-pdu log line in debug mode.

OK tb@

Don't rely on aru_pdu to rebuild the original varbindlist on error.
Now that we have avi_origid it's not needed anymore and aru_pdu needs
to go.

OK tb@

Don't use aru_pdu for determining the requesttype. It's owned by
snmp_message and getting rid of it is also needed for appl_report() to
gather its own information.

OK tb@

export SNMP-TARGET-MIB::{snmpUnavailableContexts,snmpUnknownContexts}
via application_internal.

OK tb@

Readd the sysORTable based on the new struct appl_agentcap.

OK tb@

Allow agent capabilities to be stored on a per appl_context basis.
This is needed for AgentX's {add,remove}agentcaps, and the sysORTable.

OK tb@

Introduce application_internal.c. This backend is meant to replace
application_legacy.c, mps.c, and mib.c. This commit just introduces the
backend. The existing MIBs inside mib.c will be copied over in
subsequent commits.

OK tb@

When doing a get{next,bulk} below an instance we must move the OID to
the next sibling. Not simply copying over the value of the next to use
region, since that might be the parent and we would walk backwards in
the tree.

OK tb@

searchrange.end is non-inclusive. Adjust the tests for that.

OK tb@

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

struct appl_varbind_internal's avi_index is used to give the index to
the original varbindlist's index. In the case of a GetBulkRequest this
must never be larger than the length of the original varbindlist.

OK tb@

Now that the last consumer of mps.c is gone, remove it and its
application_legacy.c companion.

OK tb@

Let usm_make_report() utilize appl_report(). usm_make_report utilized
mps_getstr(), which after moving the SNMPv2-SMI::snmpV2 into
application_internal returned a noSuchObject. This doesn't seem to have
broken any tools that I'm aware of, but this returns the correct result.

OK tb@

Let appl_report() collect its own metrics. This simplifies the interface
and gives us a free report-pdu log line in debug mode.

OK tb@

Don't rely on aru_pdu to rebuild the original varbindlist on error.
Now that we have avi_origid it's not needed anymore and aru_pdu needs
to go.

OK tb@

Don't use aru_pdu for determining the requesttype. It's owned by
snmp_message and getting rid of it is also needed for appl_report() to
gather its own information.

OK tb@

export SNMP-TARGET-MIB::{snmpUnavailableContexts,snmpUnknownContexts}
via application_internal.

OK tb@

Readd the sysORTable based on the new struct appl_agentcap.

OK tb@

Allow agent capabilities to be stored on a per appl_context basis.
This is needed for AgentX's {add,remove}agentcaps, and the sysORTable.

OK tb@

Introduce application_internal.c. This backend is meant to replace
application_legacy.c, mps.c, and mib.c. This commit just introduces the
backend. The existing MIBs inside mib.c will be copied over in
subsequent commits.

OK tb@

When doing a get{next,bulk} below an instance we must move the OID to
the next sibling. Not simply copying over the value of the next to use
region, since that might be the parent and we would walk backwards in
the tree.

OK tb@

searchrange.end is non-inclusive. Adjust the tests for that.

OK tb@

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

Let usm_make_report() utilize appl_report(). usm_make_report utilized
mps_getstr(), which after moving the SNMPv2-SMI::snmpV2 into
application_internal returned a noSuchObject. This doesn't seem to have
broken any tools that I'm aware of, but this returns the correct result.

OK tb@

Let appl_report() collect its own metrics. This simplifies the interface
and gives us a free report-pdu log line in debug mode.

OK tb@

Don't rely on aru_pdu to rebuild the original varbindlist on error.
Now that we have avi_origid it's not needed anymore and aru_pdu needs
to go.

OK tb@

Don't use aru_pdu for determining the requesttype. It's owned by
snmp_message and getting rid of it is also needed for appl_report() to
gather its own information.

OK tb@

export SNMP-TARGET-MIB::{snmpUnavailableContexts,snmpUnknownContexts}
via application_internal.

OK tb@

Readd the sysORTable based on the new struct appl_agentcap.

OK tb@

Allow agent capabilities to be stored on a per appl_context basis.
This is needed for AgentX's {add,remove}agentcaps, and the sysORTable.

OK tb@

Introduce application_internal.c. This backend is meant to replace
application_legacy.c, mps.c, and mib.c. This commit just introduces the
backend. The existing MIBs inside mib.c will be copied over in
subsequent commits.

OK tb@

When doing a get{next,bulk} below an instance we must move the OID to
the next sibling. Not simply copying over the value of the next to use
region, since that might be the parent and we would walk backwards in
the tree.

OK tb@

searchrange.end is non-inclusive. Adjust the tests for that.

OK tb@

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

Readd the sysORTable based on the new struct appl_agentcap.

OK tb@

Allow agent capabilities to be stored on a per appl_context basis.
This is needed for AgentX's {add,remove}agentcaps, and the sysORTable.

OK tb@

Introduce application_internal.c. This backend is meant to replace
application_legacy.c, mps.c, and mib.c. This commit just introduces the
backend. The existing MIBs inside mib.c will be copied over in
subsequent commits.

OK tb@

When doing a get{next,bulk} below an instance we must move the OID to
the next sibling. Not simply copying over the value of the next to use
region, since that might be the parent and we would walk backwards in
the tree.

OK tb@

searchrange.end is non-inclusive. Adjust the tests for that.

OK tb@

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

Introduce application_internal.c. This backend is meant to replace
application_legacy.c, mps.c, and mib.c. This commit just introduces the
backend. The existing MIBs inside mib.c will be copied over in
subsequent commits.

OK tb@

When doing a get{next,bulk} below an instance we must move the OID to
the next sibling. Not simply copying over the value of the next to use
region, since that might be the parent and we would walk backwards in
the tree.

OK tb@

searchrange.end is non-inclusive. Adjust the tests for that.

OK tb@

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

When doing a get{next,bulk} below an instance we must move the OID to
the next sibling. Not simply copying over the value of the next to use
region, since that might be the parent and we would walk backwards in
the tree.

OK tb@

searchrange.end is non-inclusive. Adjust the tests for that.

OK tb@

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

If an invalid varbindlist was returned by a backend we would call
appl_request_upstream_resolve() twice where the first call would already
return a reply and free the upstream request, leading to a use after
free.
Make appl_request_downstream_free() call appl_request_upstream_resolve()
unconditionally and remove the call from appl_response().

OK tb@

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@

Allow overlapping regions (if subtree claim flag is set) when the backends
are identical.

OK tb@

Make sure oidbuf is initialized when we hit the overlap case.

OK tb@

When there are two overlapping regions handled by the same backend we might
traverse back in the tree; Make sure this doesn't happen.

OK tb@

When a backend disappears while handling a request, make sure that the
outstanding requests are handled by the next backend, instead of leaking
memory.

OK tb@

Let snmpd check a response package against the requested searchrange end.

If the returned OID is beyond the searchrange end we have two cases:
- If the backend supports searchranges (agentx) we generate a GENERR and
close the connection.
- If the backend doesn't support searchranges (legacy and maybe a future
snmp proxy) we simply fix-up the result.

OK tb@

(Re)add support for agentx in snmpd
Current omissions in protocol support are notifications,
index (de)allocation, and agent capabilities.

Help testing sthen@
Feedback/tweaks/OK jmatthew@

Introduce a blocklist backend and keyword.

This allows the admin to specify a full region of the OID tree to be
blocked and simply returns NOSUCHOBJECT/ENDOFMIBVIEW.

This deprecates filter-pf-addresses in favour of:
blocklist pfTblAddrTable

OK tb@

Mostly rewrite appl_request_upstream_reply.
The old code had a potential off by one underflow, which is unlikely to be
hit with the current builtin backend, and didn't show the returned
varbindlist correct.

OK sthen@

Initialize oidbuf and regionbuf when registering a region in appl_region.
This prevents us from spewing garbage on error.

regionbuf case pointed out by tb@

OK deraadt@ tb@

requestid is a 32 bit integer, make sure that we request that with
ober_scanf_elements.

Found the hard way on octeon and patch tested by sebastian <at> rostwald
<dot> de
OK tb

Add missing NULL check.

OK benno@

Add the new application layer. Changes include:
- Asynchronous design, which should allow us to cleanly implement agentx
support.
- Cluster requests when sending them to backends
- Return a better error code in a lot of cases.
- Allow bulkget to return row by row instead of column by column (as per
RFC3416)
- Better SNMPv1 mapping as per RFC3584
- Allow registration of overlapping regions.
- Stricter OID comparison.
- We loose write support. Previous write support didn't guarantee
atomicity, wasn't persistent across restarts and didn't implement
anything useful. This can be added later if it's missed.
- This is quite a bit slower, but this should clear up once the current
mps.c and mib.c code gets pushed out. Other tricks could help speed
things up, but I don't want to resort to extra tricks if it's not needed.
- More detailed debugging output.

This commit is stand-alone and gets hooked in with the following commit.

"Looks good at first glance" benno@
minor issues pointed out by and OK jmatthew@
Performance loss aceptable to sthen@
tested as part of larger diff by sthen@ and Joel Carnat

FC3416 section 4.2.1 (and others) tells us that if an error occurs the
varbindlist in the response must be identical to the original request.

OK tb@

Certain error codes are only intended for certain request-types. Add an
appl_error_valid() function to test for this.

OK tb@

If a backend registers as an instance it must never return OIDs below
their registration. Add a test for this in appl_varbind_valid().

OK and minor nit tb@

When returning an endOfMibView we must always set it on the requested OID.

OK tb@

Fix appl_unregister() when called with range_subid set to !0.

OK tb@

Fix appl_register() when called with range_subid set to !0.

OK tb@

Always check if the context is available inside appl_agentx_recv().
Not every PDU goes through application.c.

OK tb@

Make ar[du]_{nonrepeaters,maxrepetitions} uint16_t instead of int16_t.
snmp uses signed 32 bits integers and agentx uses unsigned 16 bits
integers. I somehow ended up somewhere in between.

OK sthen@, kn@

varbind was designed to allow both a ber NULL and a NULL pointer for
value. The ber NULL case is there for when it was received via a PDU.
The NULL pointer case can happen if application.c runs into a timeout
or when a backend runs into problems.

The NULL pointer case however was overlooked in appl_varbind_valid and
results in an "missing value" error, (needlessly) terminating the
connection to the backend.

Found the hard way by Mischa Peters while stress testing agentx support
for vmd.

OK tb@, sthen@

Rewrite the searchrange end calculation routine.
The old one had a bug which allowed it to move backwards on overlapping
regions and also didn't always returned the optimal end position.

OK tb@

Remove now unused search variable. Missed in previous.

OK tb@

Remove the downwards check for overlapping regions when the subtree flag is
set.

There's a bit of inverted logic in there and this feature will probably get
in the way of the blocklist feature (and maybe others)

OK tb@