MFC r318353:
Avoid use of contiguous memory allocations in busdma when possible.

This patch improves the boundary checks in busdma to allow more cases
using the regular page based kernel memory allocator. Especially in
the case of having a non-zero boundary in the parent DMA tag. For
example AMD64 based platforms set the PCI DMA tag boundary to
PCI_DMA_BOUNDARY, 4GB, which before this patch caused contiguous
memory allocations to be preferred when allocating more than PAGE_SIZE
bytes. Even if the required alignment was less than PAGE_SIZE bytes.

This patch also fixes the nsegments check for using kmem_alloc_attr()
when the maximum segment size is less than PAGE_SIZE bytes.

Updated some comments describing the code in question.

Differential Revision: https://reviews.freebsd.org/D10645
Reviewed by: kib, jhb, gallatin, scottl
Sponsored by: Mellanox Technologies

MFC r282120:
The add_bounce_page() function can be called when loading physical
pages which pass a NULL virtual address. If the BUS_DMA_KEEP_PG_OFFSET
flag is set, use the physical address to compute the page offset
instead. The physical address should always be valid when adding
bounce pages and should contain the same page offset like the virtual
address.

Submitted by: Svatopluk Kraus <onwahe@gmail.com>
Reviewed by: jhb@

MFC r278031: Remove a stale comment.

MFC r273599:

Fix a bug where DMA maps created with bus_dmamap_create() won't increment
the map count and without being able to keep track of the current map
allocation, bus_dma_tag_destroy() will fail to proceed and will return
EBUSY even after all the maps have been correctly destroyed with
bus_dmamap_destroy().

Found while testing the detach method of a NIC.

MFC r274538, r274545, r274596, r274602, r274603, r274604, r274605, r274839:

When doing busdma sync ops for BUSDMA_COHERENT memory, there is no need
for cache maintenance operations, but ensure that all prior writes have
reached memory when doing a PREWRITE sync.

Do not do a cache invalidate on a PREREAD sync that is also a PREWRITE sync.

Do the cache invalidate sequence from the outermost to innermost, required
for correct operation.

Correct the sequence of busdma sync ops involved with PRE/POSTREAD syncs.

When doing a PREREAD sync of an mbuf-type dma buffer, do a writeback of
the first cacheline if the buffer start address is not on a cacheline
boundary.

MFC r263710, r273377, r273378, r273423 and r273455:

- De-vnet hash sizes and hash masks.
- Fix multiple issues related to arguments passed to SYSCTL macros.

Sponsored by: Mellanox Technologies

MFC r269206, r269207, r269208, r269209, r269210, r269211, r269212, r269213,
r269214, r269215, r269216, r269217, r269221:

busdma-v6 improvements, primarily:
- Allocate the temporary segments array per-map rather than per-tag.
- Avoid needlessly bouncing IO for mbufs and buffers allocated by
bus_dmamem_alloc() (in both situations we known they're allocated
on cacheline boundaries and don't need bouncing).
- Various minor reformatting and cleanups.

MFC r261414, r261415, r261417, r261418, r261419

Don't call device_set_ivars() for the mmchs

Change the way pcpu and curthread are stored per-core

Invalidate cachelines for bounce pages on PREREAD too, there may still be
stale entries from a previous transfer.

Only use the CPU ID register if SMP is defined. Some non-MPCore armv6 cpu,
such as the one found in the RPi, don't have it, and just hang when we try
to access it.

MFC r257228:
Add bus_dmamap_load_ma() function to load map with the array of
vm_pages.

MFC r256638:

Add cases for the combinations of busdma sync op flags that we handle
correctly by doing nothing, then add a panic for the default case, because
that implies that some driver asked for a sync (probably incorrectly) and
nothing was done.

MFC r256637:

When calculating the number of bounce pages needed, round the maxsize
up to a multiple of PAGE_SIZE, and add one page because there can always
be one more boundary crossing than the number of pages in the transfer.

MFC r282120:
The add_bounce_page() function can be called when loading physical
pages which pass a NULL virtual address. If the BUS_DMA_KEEP_PG_OFFSET
flag is set, use the physical address to compute the page offset
instead. The physical address should always be valid when adding
bounce pages and should contain the same page offset like the virtual
address.

Submitted by: Svatopluk Kraus <onwahe@gmail.com>
Reviewed by: jhb@

MFC r278031: Remove a stale comment.

MFC r273599:

Fix a bug where DMA maps created with bus_dmamap_create() won't increment
the map count and without being able to keep track of the current map
allocation, bus_dma_tag_destroy() will fail to proceed and will return
EBUSY even after all the maps have been correctly destroyed with
bus_dmamap_destroy().

Found while testing the detach method of a NIC.

MFC r274538, r274545, r274596, r274602, r274603, r274604, r274605, r274839:

When doing busdma sync ops for BUSDMA_COHERENT memory, there is no need
for cache maintenance operations, but ensure that all prior writes have
reached memory when doing a PREWRITE sync.

Do not do a cache invalidate on a PREREAD sync that is also a PREWRITE sync.

Do the cache invalidate sequence from the outermost to innermost, required
for correct operation.

Correct the sequence of busdma sync ops involved with PRE/POSTREAD syncs.

When doing a PREREAD sync of an mbuf-type dma buffer, do a writeback of
the first cacheline if the buffer start address is not on a cacheline
boundary.

MFC r263710, r273377, r273378, r273423 and r273455:

- De-vnet hash sizes and hash masks.
- Fix multiple issues related to arguments passed to SYSCTL macros.

Sponsored by: Mellanox Technologies

MFC r269206, r269207, r269208, r269209, r269210, r269211, r269212, r269213,
r269214, r269215, r269216, r269217, r269221:

busdma-v6 improvements, primarily:
- Allocate the temporary segments array per-map rather than per-tag.
- Avoid needlessly bouncing IO for mbufs and buffers allocated by
bus_dmamem_alloc() (in both situations we known they're allocated
on cacheline boundaries and don't need bouncing).
- Various minor reformatting and cleanups.

MFC r261414, r261415, r261417, r261418, r261419

Don't call device_set_ivars() for the mmchs

Change the way pcpu and curthread are stored per-core

Invalidate cachelines for bounce pages on PREREAD too, there may still be
stale entries from a previous transfer.

Only use the CPU ID register if SMP is defined. Some non-MPCore armv6 cpu,
such as the one found in the RPi, don't have it, and just hang when we try
to access it.

MFC r257228:
Add bus_dmamap_load_ma() function to load map with the array of
vm_pages.

MFC r256638:

Add cases for the combinations of busdma sync op flags that we handle
correctly by doing nothing, then add a panic for the default case, because
that implies that some driver asked for a sync (probably incorrectly) and
nothing was done.

MFC r256637:

When calculating the number of bounce pages needed, round the maxsize
up to a multiple of PAGE_SIZE, and add one page because there can always
be one more boundary crossing than the number of pages in the transfer.