network: Update device statistics (mostly) in the stack.

We bypass device logic in datalink_send_routed_data() in the case
of RTF_LOCAL, so if we don't update the stats there, they'll never
get updated. Furthermore, there's places packets can be dropped
inside the device reader thread. So, we might as well consolidate
the stats-updating logic and get it out of drivers.

(The only remaining case where drivers need to update stats is when
they drop a packet in receive(), as the stack can't tell when an error
from receive() is due to a dropped packet or not.)

Fixes a potential leak on packet drops in the device reader thread,
and fixes loopback statistics for TCP/UDP/etc.

loopback_frame: apply an ethernet header for packet capture

* the packet capture expects frames, so loopback_frame should actually apply framing and deframing
for loopback packets.
* datalink: set the address for host routes
* device_interfaces: call the deframing if any in device_enqueue_buffer()
tested with ping 127.0.0.1 and ping6 ::1, while removing the local flag on both routes.

Change-Id: I2085735bdac3bb85908189a2e1acb2540818c7bd
Reviewed-on: https://review.haiku-os.org/c/haiku/+/6451
Reviewed-by: Fredrik Holmqvist <fredrik.holmqvist@gmail.com>
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

network/stack: Return immediately from device_reader_thread after calling device_removed.

Otherwise we can and will use-after-free the device structures.
It seems this code has been broken this way since it was first added
in cc8eceb0af9ece5bcea51b106fa73e83b6b750a8 (2010).

network/stack: Update and add comments in down_device_interface.

The function names in the comment were not been updated in some time,
so synchronize those with how things presently look.

Also add another possible codepath, from device_removed. This one can
and is invoked from the device_reader_thread, in the case where
we get B_DEVICE_NOT_FOUND from receive_data. That is now taken care of
by having wait_for_thread return EDEADLK, we need only note it here.

Change-Id: I18320c2831dce99a34b07de862a0a5c8b4e4e98d
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5311
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

network/stack: Fix a race condition in device interface teardown.

Inside the device_consumer thread, the sequence of operations was this:
1. Call fifo_dequeue_buffer()
2. If it returned B_INTERRUPTED, exit the thread. Otherwise, process
the buffer it returned as normal.
3. Loop.

Thus, if the FIFO was destroyed not during a call to
fifo_dequeue_buffer, the next loop, fifo_dequeue_buffer would
be called on a destroyed FIFO. It would then try to lock the mutex
in the FIFO which had been destroyed, causing an assertion failure
and thus a panic.

Now, we check the ref_count on every loop, and set it to 0 before
calling uninit_fifo(). Thus, even if we are in the middle of a loop
inside the FIFO thread, the loop iteration condition will fail
and thus the thread will exit, avoiding the race.

Probably this was not an issue before because the timing required
to hit this is incredibly unlikely. With the new ipro1000 driver
(or kallisti5's WIP TUN/TAP driver), the timing makes this much
more likely.

Should fix #15024.

device_interfaces: Fix memory leak on success.

The module list was closed properly at the end of the function body
(failure case) but not inside the loop on the success case.

CID 605738.

network stack: Fixed double mutex lock.

* device_removed() no longer uses get_device_interface() to avoid
the double locking.
* Introduced net_device_interface::busy member to avoid locking in
device_removed() for too long.
* This should fix #7252.

NetStack: Fix locking
Change the monitor_lock to be a recursive_lock, fixing #7039.
Notify_device_monitor acquires the monitor lock, then the
monitor unregisters itself, calling unregister_device_monitor, which
acquires the monitor lock again.

Use close_module_list() to free the list.
There are a pair of Coverity CIDs for these two.

Initialization of net_device_interface::monitor_count corrected

Fixes #8839

Signed-off-by: Siarzhuk Zharski <zharik@gmx.li>

Fixed up network stack and drivers for x86_64.

* Various compilation fixes.
* Fixes to the FreeBSD compatibility layer (from comparing the x86-
specific bits with the equivalent amd64 sources in FreeBSD).
* Compile all the Ethernet drivers except for sis900 and wb840, these
require a bit more work to fix (will file a ticket soon). Tested
ipro1000 and rtl81xx, no issues.

Since sdl_e_type is now stored in network byte order, device_consumer_thread()
needed a corresponding adjustment to convert it back to host order before
working with it. Fixes #6972.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39812 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Set the new net_buffer::index field on retrieval of a buffer.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38394 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Spotted a few reference counting bugs that were probably responsible for
#6446, although I could not reproduce the exact problem.
* net_datalink::is_local_[link_]address() now releases a previous reference if
the _interfaceAddress arguments does not point to NULL.
* When a buffer is received from a device, it's interface_address should be NULL
already.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38118 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Changed the _SIZEOF_ADDR_IFREQ() macro such that it can be used how we are
using it - hopefully, that is actually correct (it's adopted from FreeBSD).
* Fixed bug that cut off the sockaddr_dl reported by the system, so that DHCP
wouldn't work anymore (and ifconfig wouldn't show the actual MAC address
anymore).
* Changed the listing code to actually pad to ifreq size, and leave the length
of the sockaddr untouched.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37941 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Make sure the SIOCGIFCONF never returns an address with an address length
smaller than sizeof(sockaddr). This fixes a compatibility issue with other
platforms - portable software often assumes that the amount of bytes to
add to an ifreq structure is the larger amount between sizeof(ifreq), and
basically what the _SIZEOF_ADDR_IFREQ() macro returns, instead of always
relying on that macro.
* Renamed UserBuffer::Copy() to Push, ConsumedAmount() to BytesConsumed(),
added Pad() method.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37927 a95241bf-73f2-0310-859d-f6bbb57e9c96

Report the actual error when read() or write() a network device failed.
This enable to actually detect and handle device removal (USB...).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37897 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Enabled removing the interface on device removal again.
* Made the return type of remove_interface() void, as it cannot fail.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37884 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced a datalink layer (2) independent way of specifying the packet
type one wants to receive. Changed ipv6_datagram to use that (but note that
it currently does not compile).
* Header cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37868 a95241bf-73f2-0310-859d-f6bbb57e9c96

* put_device_interface() now also accepts a NULL pointer.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37852 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a dedicated lock for the device monitors. This fixes a locking issue in
interface_protocol_send_data() which accessed the monitors unlocked.
* Changed SIOCCPACKETCAP to check if the device name matches the one used with
SIOCSPACKETCAP.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37845 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced ENODEV with B_DEVICE_NOT_FOUND.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37803 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Reworked the complete stack to allow more than one address per network
interface - this caused quite a number of changes.
* Network interfaces, and its addresses are now reference counted (not yet
complete, though, InterfaceAddresses need to hold references to their
interface as well).
* There are two known regressions of this commit that I will fix later:
- you cannot remove interfaces anymore
- IPv4 multicast was broken anyway, but now it's disabled, too.
* Moved a device_interfaces.cpp|h out of interfaces.cpp.
* The datalink layer chain is now instantiated per domain per interface,
not just per interface anymore.
* When a buffer reaches the network layer, it has no known interface yet, ie.
the ipv4|6|whatever modules need to set this manually.
* Added more debug output, and some new debugger commands, the control option
is now printed in clear text.
* Added hash_address() function to the address modules. Added "const" to
set_to_defaults() where needed.
* Fixed net_buffer's restore header functions offset use as reported by Atis.
* Improved buffer dump output, use the domain module to print the address if
available.
* Moved net_buffer::type into the union, as it's not needed by the upper layers
anymore.
* Moved IPv6 specific code from {add|remove}_default_route() to where it
belongs, but disabled it for the time being.
* Completely discarded useless ipv4_datagram module.
* Added ping6 to the build.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37794 a95241bf-73f2-0310-859d-f6bbb57e9c96

NetStack: Fix locking
Change the monitor_lock to be a recursive_lock, fixing #7039.
Notify_device_monitor acquires the monitor lock, then the
monitor unregisters itself, calling unregister_device_monitor, which
acquires the monitor lock again.

Use close_module_list() to free the list.
There are a pair of Coverity CIDs for these two.

Initialization of net_device_interface::monitor_count corrected

Fixes #8839

Signed-off-by: Siarzhuk Zharski <zharik@gmx.li>

Fixed up network stack and drivers for x86_64.

* Various compilation fixes.
* Fixes to the FreeBSD compatibility layer (from comparing the x86-
specific bits with the equivalent amd64 sources in FreeBSD).
* Compile all the Ethernet drivers except for sis900 and wb840, these
require a bit more work to fix (will file a ticket soon). Tested
ipro1000 and rtl81xx, no issues.

Since sdl_e_type is now stored in network byte order, device_consumer_thread()
needed a corresponding adjustment to convert it back to host order before
working with it. Fixes #6972.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39812 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Set the new net_buffer::index field on retrieval of a buffer.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38394 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Spotted a few reference counting bugs that were probably responsible for
#6446, although I could not reproduce the exact problem.
* net_datalink::is_local_[link_]address() now releases a previous reference if
the _interfaceAddress arguments does not point to NULL.
* When a buffer is received from a device, it's interface_address should be NULL
already.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38118 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Changed the _SIZEOF_ADDR_IFREQ() macro such that it can be used how we are
using it - hopefully, that is actually correct (it's adopted from FreeBSD).
* Fixed bug that cut off the sockaddr_dl reported by the system, so that DHCP
wouldn't work anymore (and ifconfig wouldn't show the actual MAC address
anymore).
* Changed the listing code to actually pad to ifreq size, and leave the length
of the sockaddr untouched.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37941 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Make sure the SIOCGIFCONF never returns an address with an address length
smaller than sizeof(sockaddr). This fixes a compatibility issue with other
platforms - portable software often assumes that the amount of bytes to
add to an ifreq structure is the larger amount between sizeof(ifreq), and
basically what the _SIZEOF_ADDR_IFREQ() macro returns, instead of always
relying on that macro.
* Renamed UserBuffer::Copy() to Push, ConsumedAmount() to BytesConsumed(),
added Pad() method.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37927 a95241bf-73f2-0310-859d-f6bbb57e9c96

Report the actual error when read() or write() a network device failed.
This enable to actually detect and handle device removal (USB...).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37897 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Enabled removing the interface on device removal again.
* Made the return type of remove_interface() void, as it cannot fail.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37884 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Introduced a datalink layer (2) independent way of specifying the packet
type one wants to receive. Changed ipv6_datagram to use that (but note that
it currently does not compile).
* Header cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37868 a95241bf-73f2-0310-859d-f6bbb57e9c96

* put_device_interface() now also accepts a NULL pointer.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37852 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a dedicated lock for the device monitors. This fixes a locking issue in
interface_protocol_send_data() which accessed the monitors unlocked.
* Changed SIOCCPACKETCAP to check if the device name matches the one used with
SIOCSPACKETCAP.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37845 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Replaced ENODEV with B_DEVICE_NOT_FOUND.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37803 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Reworked the complete stack to allow more than one address per network
interface - this caused quite a number of changes.
* Network interfaces, and its addresses are now reference counted (not yet
complete, though, InterfaceAddresses need to hold references to their
interface as well).
* There are two known regressions of this commit that I will fix later:
- you cannot remove interfaces anymore
- IPv4 multicast was broken anyway, but now it's disabled, too.
* Moved a device_interfaces.cpp|h out of interfaces.cpp.
* The datalink layer chain is now instantiated per domain per interface,
not just per interface anymore.
* When a buffer reaches the network layer, it has no known interface yet, ie.
the ipv4|6|whatever modules need to set this manually.
* Added more debug output, and some new debugger commands, the control option
is now printed in clear text.
* Added hash_address() function to the address modules. Added "const" to
set_to_defaults() where needed.
* Fixed net_buffer's restore header functions offset use as reported by Atis.
* Improved buffer dump output, use the domain module to print the address if
available.
* Moved net_buffer::type into the union, as it's not needed by the upper layers
anymore.
* Moved IPv6 specific code from {add|remove}_default_route() to where it
belongs, but disabled it for the time being.
* Completely discarded useless ipv4_datagram module.
* Added ping6 to the build.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37794 a95241bf-73f2-0310-859d-f6bbb57e9c96