Fix fallout from previous: ci_ipending is in the low bytes of ci_istate,
so we should not left-shift by 8 the ci_ipending value we want to add there.
Should fix KASSERT in Xen's idle_block() reported by Frank Kardel in private
mail.

NetBSD/x86: Raise the number of interrupt sources per CPU from 32 to 56.

There has been no objection for three years.
https://mail-index.netbsd.org/port-amd64/2019/09/22/msg003012.html
Implemented by nonaka@n.o, updated by me.

Release mutexes earlier in xenevt_free(), to call *destroy and kmem_free
without holding spin locks. Fixes an occasional deadlock seen when the qemu
backing a HVM domU exits.

xenevt_setipending(): also increase xenevt_ih->ih_pending, so that
the handler will be called when the IPL is lowered.

Fix all remaining typos, mainly in comments but also in few definitions and log messages, reported by me in PR kern/54889.
Also fixed some additional typos in comments, found on review of same files or typos.

branches: 1.63.4;
Trailing whitespace

Fix spelling of preferred in comments. While here improve grammar in
one of the comments.

Work in progress on dom0 PVH support. kernel boots and xl info works,
but we can't start a domU yet.

branches: 1.60.2;
Go back using cpu_info_primary, all events are bound to vCPU 0 by default.
Register the event handler on cpu_info_primary.
While there. update the event counter on interrupts.
Finally this should be MP-safe.

The event channel is not necesserely bound to the primary CPU.
Use the CPU returned in the interrupt handler instead.

Make DOM0OPS build for PVH/PVHVM too

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

branches: 1.56.2;
switch from malloc() to kmem_zalloc()

just pass true as known_mpsafe for interrupt establish, drop the
(level != IPL_VM); NFCI, this uses IPL_HIGH so was mpsafe already

Make this files compilable w/o options DIAGNOSTIC

branches: 1.53.4;
Excise XEN specific code out of x86/x86/intr.c into xen/x86/xen_intr.c

While at it, separate the source function tracking so that the interrupt
paths are truly independant.

Use weak symbol exporting to provision for future PVHVM co-existence
of both files, but with independant paths. Introduce assembler code
such that in a unified scenario, native interrupts get first priority
in spllower(), followed by XEN event callbacks. IPL management and
semantics are unchanged - native handlers and xen callbacks are
expected to maintain their ipl related semantics.

In summary, after this commit, native and XEN now have completely
unrelated interrupt handling mechanisms, including
intr_establish_xname() and assembler stubs and intr handler
management.

Happy Christmas!

Bifurcate the interrupt establish functions between XEN and non-XEN

Thus intr_establish_xname() becomes xen_intr_establish_xname() etc.

One consequence of this is that dom0 devices expect the native
function calls to be available and we thus provide weak aliasing for
dom0 builds to succeed. XEN and non-XEN devices are distinguished by
the PIC they are established on. XEN interrupts are exclusively
established on xen_pic, while dom0 interrupts are established on
natively available PICs.

This allows us an orthogonal path to xen device management (eg:
xenstore events) in XENPVHVM, without having to worry about unifying
the vector entry paths, etc., which is quite challenging.

When using the intr_establish_xname() interface to register
XEN events, follow established x86/intr.c conventions - set
the 'legacy' irq value to -1, to indicate that the pic, pin
combination (&xen_pic, port) is used for registration.

The known_mpsafe parameter is used by intr.c:intr_establish_xname()
especially in the non -D MULTIPROCESSOR case. We used it incorrectly.

Fix this.

Do not re-expose the innards of evtchn.c, now that we have a way
to register interrupts via intr.c:intr_establish_xname()

evtchn.c is going to get refactored soon, so use the latter method.

branches: 1.48.2; 1.48.4;
add fo_name so we can identify the fileops in a simple way.

branches: 1.47.2;
Unify the xen and native x86/ interrupt setup functions and
spl traversal data structures.

This is towards PVHVM.

remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

KNF. Remove extra spaces. No functional change.

include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.

Improve wording.

Fix off by one error, pointed out by Wei Liu in port-xen/49919

branches: 1.41.2; 1.41.4;
Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.

branches: 1.40.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.

branches: 1.39.4; 1.39.8; 1.39.10; 1.39.12; 1.39.18;
hypervisor_unmask_event(): don't check/update evtchn_pending_sel for the
current CPU, but for any CPU which may accept this event.
xen/xenevt.c: more use of atomic ops and locks where appropriate, and some
other SMP fixes. Handle all events on the primary CPU (may be revisited
later). Set/clear ci_evtmask[] for watched events.

This should fix the problems on dom0 kernels reported by jym@

branches: 1.38.2;
Make event/interrupt handling MP aware

branches: 1.37.2;
- Replace uses of simple_lock and ltsleep with mutex and condvar.
- Improve some parts of the code to be more MP-friendly.

Tested by jakllsch@.

branches: 1.36.4; 1.36.6;
If a multithreaded app closes an fd while another thread is blocked in
read/write/accept, then the expectation is that the blocked thread will
exit and the close complete.
Since only one fd is affected, but many fd can refer to the same file,
the close code can only request the fs code unblock with ERESTART.
Fixed for pipes and sockets, ERESTART will only be generated after such
a close - so there should be no change for other programs.
Also rename fo_abort() to fo_restart() (this used to be fo_drain()).
Fixes PR/26567

Rename fo_drain() to fo_abort(), 'drain' is used to mean 'wait for output
do drain' in many places, whereas fo_drain() was called in order to force
blocking read()/write() etc calls to return to userspace so that a close()
call from a different thread can complete.
In the sockets code comment out the broken code in the inner function,
it was being called from compat code.

Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !

remove Xen2 support.
ok bouyer@

Add fileops::fo_drain(), to be called from fd_close() when there is more
than one active reference to a file descriptor. It should dislodge threads
sleeping while holding a reference to the descriptor. Implemented only for
sockets but should be extended to pipes, fifos, etc.

Fixes the case of a multithreaded process doing something like the
following, which would have hung until the process got a signal.

thr0 accept(fd, ...)
thr1 close(fd)

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.30.2;
kill MALLOC and FREE macros.

branches: 1.29.2; 1.29.4;
Compile NetBSD/amd64 kernels with "-Wextra". Patches contributed by
Juan RP in PR port-amd64/39266.

branches: 1.28.4;
Initialize ih_ilevel for the special xenevt_processevt() handler.
Should fix KASSERT panic reported by Sarton O'Brien and Kazushi Marukawa
on port-xen@

branches: 1.27.2; 1.27.4;
Use interrupt biglock wrapper as in x86/x86/intr.c
This change is based on http://mail-index.netbsd.org/port-amd64/2004/02/22/0000.html
OK bouyer

branches: 1.26.2;
- use POSIX integer types
- ansify functions

Make it compile.

Fix my previous confuse of the code: use selinit/seldestroy in correct
places, and fix missed selnotify call.

Welcome to 4.99.55:

- Add a lot of missing selinit() and seldestroy() calls.

- Merge selwakeup() and selnotify() calls into a single selnotify().

- Add an additional 'events' argument to selnotify() call. It will
indicate which event (POLL_IN, POLL_OUT, etc) happen. If unknown,
zero may be used.

Note: please pass appropriate value of 'events' where possible.
Proposed on: <tech-kern>

branches: 1.22.2; 1.22.6;
The event bitmasks provided by the hypervisor are unsigned long (so 64bits
on amd64). Make sure to use the right type to store and manipulate them.
This fixes amd64, where basically any event channel > 31 was not working
(and you get there after starting/stopping a domU a few times). Things
would occasionally unwedge though the spllower() callbacks.

Fix xenevt to not call softint_schedule() above IPL_HIGH:
Register a ipl callback for IPL_HIGH.
if the current ipl level is too high, just record the event in a bitmap,
and record IPL_HIGH as pending. The callback will process the pending events.

Add missing __KERNEL_RCSID()

Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.

Allocating up to 8k on stack isn't a good idea ...

branches: 1.17.6;
Interrupt handling changes, in discussion since February:

- Reduce available SPL levels for hardware devices to none, vm, sched, high.
- Acquire kernel_lock only for interrupts at IPL_VM.
- Implement threaded soft interrupts.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

branches: 1.15.2;
Merge the ppcoea-renovation branch to HEAD.

This branch was a major cleanup and rototill of many of the various OEA
cpu based PPC ports that focused on sharing as much code as possible
between the various ports to eliminate near-identical copies of files in
every tree. Additionally there is a new PIC system that unifies the
interface to interrupt code for all different OEA ppc arches. The work
for this branch was done by a variety of people, too long to list here.

TODO:
bebox still needs work to complete the transition to -renovation.
ofppc still needs a bunch of work, which I will be looking at.
ev64260 still needs to be renovated
amigappc was not attempted.

NOTES:
pmppc was removed as an arch, and moved to a evbppc target.

branches: 1.14.8; 1.14.10; 1.14.14;
Add needed locking in xenevt_fpoll; this fixes at least a lost-wakeup bug.
OK'ed by bouyer@.

branches: 1.13.4; 1.13.6; 1.13.12;
TRUE -> true, FALSE -> false

Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.

branches: 1.11.4; 1.11.6; 1.11.8;
Hypercalls returns negative error codes, so return -error in ioctl
function.

branches: 1.10.4;
Adapt to change in args for falloc(). Reported by Hisashi T Fujinaka
on current-users.

integrate kauth.

Add dom0 operation support for Xen3. Probably buggy, but it's enouth to have
xend and xenstored starting; xm info and xm list works.

branches: 1.7.4; 1.7.6; 1.7.8; 1.7.10; 1.7.12;
Make xenevt.c compile after lwp changes; not tested yet.

merge ktrace-lwp.

branches: 1.5.2; 1.5.4;
Avoid a race between do_hypervisor_event() and stipending() that could
cause an event to be both handled and marked as pending, or being
marked as pending twice (triggering the diagnostic check
evtch_maskcount[port] == 0 in hypervisor_set_ipending):
mask and clear event by word of 32bit in do_hypervisor_event() or stipending(),
instead of by indiviual bits in do_event() or xenevt_event().
In addition this is marginally more efficient.

Get rid of the event to pseudo-irq mapping. We are limited to 32 pseudo-irq,
including soft interrupt, and this is way too low in some use (lots of domains,
or domains with lots of xennet, or even hardware with lots of devices at
different interrupts).
Based on idea from YAMAMOTO Takashi, keep one list of handler per-event and
one per-IPL (so the same handler is now in 2 lists). In the common case were
an event is received at low IPL, we can call the handlers quickly (there
is usually only one handler per event, unless the event is mapped to a
physical interrupt and this interrupt is shared by different devices).
Deffered events and software interrupts are handled by a bitmask (as before)
with one bit per IPL. When one IPL has an event pending all handlers for
this IPL will be called.
With this change, it is now possible to have all the 1024 events active.

While here, handle debug event in a special way: the handler is always called,
regardless of the current IPL. Make the handler print usefull informations
about events and IPL states.
Also remove code not used on Xen in files inherited from the x86 port.

fix a bug which corrupts runqueue.
when dealing with events, which are handed to xenevt pseudo device,
don't call wakeup(9)/selnotify(9) at too high IPL. PR/29792.

branches: 1.2.2; 1.2.4; 1.2.6;
Merge the bouyer-xen2 branch. This add supports for the Xen 2.0 virtual
machine kernel (both privileged and non-privileged domains), and remove support
for the old xen 1.2.

branches: 1.1.2;
file xenevt.c was initially added on branch bouyer-xen2.

NetBSD/x86: Raise the number of interrupt sources per CPU from 32 to 56.

There has been no objection for three years.
https://mail-index.netbsd.org/port-amd64/2019/09/22/msg003012.html
Implemented by nonaka@n.o, updated by me.

Release mutexes earlier in xenevt_free(), to call *destroy and kmem_free
without holding spin locks. Fixes an occasional deadlock seen when the qemu
backing a HVM domU exits.

xenevt_setipending(): also increase xenevt_ih->ih_pending, so that
the handler will be called when the IPL is lowered.

Fix all remaining typos, mainly in comments but also in few definitions and log messages, reported by me in PR kern/54889.
Also fixed some additional typos in comments, found on review of same files or typos.

branches: 1.63.4;
Trailing whitespace

Fix spelling of preferred in comments. While here improve grammar in
one of the comments.

Work in progress on dom0 PVH support. kernel boots and xl info works,
but we can't start a domU yet.

branches: 1.60.2;
Go back using cpu_info_primary, all events are bound to vCPU 0 by default.
Register the event handler on cpu_info_primary.
While there. update the event counter on interrupts.
Finally this should be MP-safe.

The event channel is not necesserely bound to the primary CPU.
Use the CPU returned in the interrupt handler instead.

Make DOM0OPS build for PVH/PVHVM too

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

branches: 1.56.2;
switch from malloc() to kmem_zalloc()

just pass true as known_mpsafe for interrupt establish, drop the
(level != IPL_VM); NFCI, this uses IPL_HIGH so was mpsafe already

Make this files compilable w/o options DIAGNOSTIC

branches: 1.53.4;
Excise XEN specific code out of x86/x86/intr.c into xen/x86/xen_intr.c

While at it, separate the source function tracking so that the interrupt
paths are truly independant.

Use weak symbol exporting to provision for future PVHVM co-existence
of both files, but with independant paths. Introduce assembler code
such that in a unified scenario, native interrupts get first priority
in spllower(), followed by XEN event callbacks. IPL management and
semantics are unchanged - native handlers and xen callbacks are
expected to maintain their ipl related semantics.

In summary, after this commit, native and XEN now have completely
unrelated interrupt handling mechanisms, including
intr_establish_xname() and assembler stubs and intr handler
management.

Happy Christmas!

Bifurcate the interrupt establish functions between XEN and non-XEN

Thus intr_establish_xname() becomes xen_intr_establish_xname() etc.

One consequence of this is that dom0 devices expect the native
function calls to be available and we thus provide weak aliasing for
dom0 builds to succeed. XEN and non-XEN devices are distinguished by
the PIC they are established on. XEN interrupts are exclusively
established on xen_pic, while dom0 interrupts are established on
natively available PICs.

This allows us an orthogonal path to xen device management (eg:
xenstore events) in XENPVHVM, without having to worry about unifying
the vector entry paths, etc., which is quite challenging.

When using the intr_establish_xname() interface to register
XEN events, follow established x86/intr.c conventions - set
the 'legacy' irq value to -1, to indicate that the pic, pin
combination (&xen_pic, port) is used for registration.

The known_mpsafe parameter is used by intr.c:intr_establish_xname()
especially in the non -D MULTIPROCESSOR case. We used it incorrectly.

Fix this.

Do not re-expose the innards of evtchn.c, now that we have a way
to register interrupts via intr.c:intr_establish_xname()

evtchn.c is going to get refactored soon, so use the latter method.

branches: 1.48.2; 1.48.4;
add fo_name so we can identify the fileops in a simple way.

branches: 1.47.2;
Unify the xen and native x86/ interrupt setup functions and
spl traversal data structures.

This is towards PVHVM.

remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

KNF. Remove extra spaces. No functional change.

include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.

Improve wording.

Fix off by one error, pointed out by Wei Liu in port-xen/49919

branches: 1.41.2; 1.41.4;
Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.

branches: 1.40.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.

branches: 1.39.4; 1.39.8; 1.39.10; 1.39.12; 1.39.18;
hypervisor_unmask_event(): don't check/update evtchn_pending_sel for the
current CPU, but for any CPU which may accept this event.
xen/xenevt.c: more use of atomic ops and locks where appropriate, and some
other SMP fixes. Handle all events on the primary CPU (may be revisited
later). Set/clear ci_evtmask[] for watched events.

This should fix the problems on dom0 kernels reported by jym@

branches: 1.38.2;
Make event/interrupt handling MP aware

branches: 1.37.2;
- Replace uses of simple_lock and ltsleep with mutex and condvar.
- Improve some parts of the code to be more MP-friendly.

Tested by jakllsch@.

branches: 1.36.4; 1.36.6;
If a multithreaded app closes an fd while another thread is blocked in
read/write/accept, then the expectation is that the blocked thread will
exit and the close complete.
Since only one fd is affected, but many fd can refer to the same file,
the close code can only request the fs code unblock with ERESTART.
Fixed for pipes and sockets, ERESTART will only be generated after such
a close - so there should be no change for other programs.
Also rename fo_abort() to fo_restart() (this used to be fo_drain()).
Fixes PR/26567

Rename fo_drain() to fo_abort(), 'drain' is used to mean 'wait for output
do drain' in many places, whereas fo_drain() was called in order to force
blocking read()/write() etc calls to return to userspace so that a close()
call from a different thread can complete.
In the sockets code comment out the broken code in the inner function,
it was being called from compat code.

Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !

remove Xen2 support.
ok bouyer@

Add fileops::fo_drain(), to be called from fd_close() when there is more
than one active reference to a file descriptor. It should dislodge threads
sleeping while holding a reference to the descriptor. Implemented only for
sockets but should be extended to pipes, fifos, etc.

Fixes the case of a multithreaded process doing something like the
following, which would have hung until the process got a signal.

thr0 accept(fd, ...)
thr1 close(fd)

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.30.2;
kill MALLOC and FREE macros.

branches: 1.29.2; 1.29.4;
Compile NetBSD/amd64 kernels with "-Wextra". Patches contributed by
Juan RP in PR port-amd64/39266.

branches: 1.28.4;
Initialize ih_ilevel for the special xenevt_processevt() handler.
Should fix KASSERT panic reported by Sarton O'Brien and Kazushi Marukawa
on port-xen@

branches: 1.27.2; 1.27.4;
Use interrupt biglock wrapper as in x86/x86/intr.c
This change is based on http://mail-index.netbsd.org/port-amd64/2004/02/22/0000.html
OK bouyer

branches: 1.26.2;
- use POSIX integer types
- ansify functions

Make it compile.

Fix my previous confuse of the code: use selinit/seldestroy in correct
places, and fix missed selnotify call.

Welcome to 4.99.55:

- Add a lot of missing selinit() and seldestroy() calls.

- Merge selwakeup() and selnotify() calls into a single selnotify().

- Add an additional 'events' argument to selnotify() call. It will
indicate which event (POLL_IN, POLL_OUT, etc) happen. If unknown,
zero may be used.

Note: please pass appropriate value of 'events' where possible.
Proposed on: <tech-kern>

branches: 1.22.2; 1.22.6;
The event bitmasks provided by the hypervisor are unsigned long (so 64bits
on amd64). Make sure to use the right type to store and manipulate them.
This fixes amd64, where basically any event channel > 31 was not working
(and you get there after starting/stopping a domU a few times). Things
would occasionally unwedge though the spllower() callbacks.

Fix xenevt to not call softint_schedule() above IPL_HIGH:
Register a ipl callback for IPL_HIGH.
if the current ipl level is too high, just record the event in a bitmap,
and record IPL_HIGH as pending. The callback will process the pending events.

Add missing __KERNEL_RCSID()

Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.

Allocating up to 8k on stack isn't a good idea ...

branches: 1.17.6;
Interrupt handling changes, in discussion since February:

- Reduce available SPL levels for hardware devices to none, vm, sched, high.
- Acquire kernel_lock only for interrupts at IPL_VM.
- Implement threaded soft interrupts.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

branches: 1.15.2;
Merge the ppcoea-renovation branch to HEAD.

This branch was a major cleanup and rototill of many of the various OEA
cpu based PPC ports that focused on sharing as much code as possible
between the various ports to eliminate near-identical copies of files in
every tree. Additionally there is a new PIC system that unifies the
interface to interrupt code for all different OEA ppc arches. The work
for this branch was done by a variety of people, too long to list here.

TODO:
bebox still needs work to complete the transition to -renovation.
ofppc still needs a bunch of work, which I will be looking at.
ev64260 still needs to be renovated
amigappc was not attempted.

NOTES:
pmppc was removed as an arch, and moved to a evbppc target.

branches: 1.14.8; 1.14.10; 1.14.14;
Add needed locking in xenevt_fpoll; this fixes at least a lost-wakeup bug.
OK'ed by bouyer@.

branches: 1.13.4; 1.13.6; 1.13.12;
TRUE -> true, FALSE -> false

Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.

branches: 1.11.4; 1.11.6; 1.11.8;
Hypercalls returns negative error codes, so return -error in ioctl
function.

branches: 1.10.4;
Adapt to change in args for falloc(). Reported by Hisashi T Fujinaka
on current-users.

integrate kauth.

Add dom0 operation support for Xen3. Probably buggy, but it's enouth to have
xend and xenstored starting; xm info and xm list works.

branches: 1.7.4; 1.7.6; 1.7.8; 1.7.10; 1.7.12;
Make xenevt.c compile after lwp changes; not tested yet.

merge ktrace-lwp.

branches: 1.5.2; 1.5.4;
Avoid a race between do_hypervisor_event() and stipending() that could
cause an event to be both handled and marked as pending, or being
marked as pending twice (triggering the diagnostic check
evtch_maskcount[port] == 0 in hypervisor_set_ipending):
mask and clear event by word of 32bit in do_hypervisor_event() or stipending(),
instead of by indiviual bits in do_event() or xenevt_event().
In addition this is marginally more efficient.

Get rid of the event to pseudo-irq mapping. We are limited to 32 pseudo-irq,
including soft interrupt, and this is way too low in some use (lots of domains,
or domains with lots of xennet, or even hardware with lots of devices at
different interrupts).
Based on idea from YAMAMOTO Takashi, keep one list of handler per-event and
one per-IPL (so the same handler is now in 2 lists). In the common case were
an event is received at low IPL, we can call the handlers quickly (there
is usually only one handler per event, unless the event is mapped to a
physical interrupt and this interrupt is shared by different devices).
Deffered events and software interrupts are handled by a bitmask (as before)
with one bit per IPL. When one IPL has an event pending all handlers for
this IPL will be called.
With this change, it is now possible to have all the 1024 events active.

While here, handle debug event in a special way: the handler is always called,
regardless of the current IPL. Make the handler print usefull informations
about events and IPL states.
Also remove code not used on Xen in files inherited from the x86 port.

fix a bug which corrupts runqueue.
when dealing with events, which are handed to xenevt pseudo device,
don't call wakeup(9)/selnotify(9) at too high IPL. PR/29792.

branches: 1.2.2; 1.2.4; 1.2.6;
Merge the bouyer-xen2 branch. This add supports for the Xen 2.0 virtual
machine kernel (both privileged and non-privileged domains), and remove support
for the old xen 1.2.

branches: 1.1.2;
file xenevt.c was initially added on branch bouyer-xen2.

Release mutexes earlier in xenevt_free(), to call *destroy and kmem_free
without holding spin locks. Fixes an occasional deadlock seen when the qemu
backing a HVM domU exits.

xenevt_setipending(): also increase xenevt_ih->ih_pending, so that
the handler will be called when the IPL is lowered.

Fix all remaining typos, mainly in comments but also in few definitions and log messages, reported by me in PR kern/54889.
Also fixed some additional typos in comments, found on review of same files or typos.

branches: 1.63.4;
Trailing whitespace

Fix spelling of preferred in comments. While here improve grammar in
one of the comments.

Work in progress on dom0 PVH support. kernel boots and xl info works,
but we can't start a domU yet.

branches: 1.60.2;
Go back using cpu_info_primary, all events are bound to vCPU 0 by default.
Register the event handler on cpu_info_primary.
While there. update the event counter on interrupts.
Finally this should be MP-safe.

The event channel is not necesserely bound to the primary CPU.
Use the CPU returned in the interrupt handler instead.

Make DOM0OPS build for PVH/PVHVM too

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

branches: 1.56.2;
switch from malloc() to kmem_zalloc()

just pass true as known_mpsafe for interrupt establish, drop the
(level != IPL_VM); NFCI, this uses IPL_HIGH so was mpsafe already

Make this files compilable w/o options DIAGNOSTIC

branches: 1.53.4;
Excise XEN specific code out of x86/x86/intr.c into xen/x86/xen_intr.c

While at it, separate the source function tracking so that the interrupt
paths are truly independant.

Use weak symbol exporting to provision for future PVHVM co-existence
of both files, but with independant paths. Introduce assembler code
such that in a unified scenario, native interrupts get first priority
in spllower(), followed by XEN event callbacks. IPL management and
semantics are unchanged - native handlers and xen callbacks are
expected to maintain their ipl related semantics.

In summary, after this commit, native and XEN now have completely
unrelated interrupt handling mechanisms, including
intr_establish_xname() and assembler stubs and intr handler
management.

Happy Christmas!

Bifurcate the interrupt establish functions between XEN and non-XEN

Thus intr_establish_xname() becomes xen_intr_establish_xname() etc.

One consequence of this is that dom0 devices expect the native
function calls to be available and we thus provide weak aliasing for
dom0 builds to succeed. XEN and non-XEN devices are distinguished by
the PIC they are established on. XEN interrupts are exclusively
established on xen_pic, while dom0 interrupts are established on
natively available PICs.

This allows us an orthogonal path to xen device management (eg:
xenstore events) in XENPVHVM, without having to worry about unifying
the vector entry paths, etc., which is quite challenging.

When using the intr_establish_xname() interface to register
XEN events, follow established x86/intr.c conventions - set
the 'legacy' irq value to -1, to indicate that the pic, pin
combination (&xen_pic, port) is used for registration.

The known_mpsafe parameter is used by intr.c:intr_establish_xname()
especially in the non -D MULTIPROCESSOR case. We used it incorrectly.

Fix this.

Do not re-expose the innards of evtchn.c, now that we have a way
to register interrupts via intr.c:intr_establish_xname()

evtchn.c is going to get refactored soon, so use the latter method.

branches: 1.48.2; 1.48.4;
add fo_name so we can identify the fileops in a simple way.

branches: 1.47.2;
Unify the xen and native x86/ interrupt setup functions and
spl traversal data structures.

This is towards PVHVM.

remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

KNF. Remove extra spaces. No functional change.

include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.

Improve wording.

Fix off by one error, pointed out by Wei Liu in port-xen/49919

branches: 1.41.2; 1.41.4;
Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.

branches: 1.40.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.

branches: 1.39.4; 1.39.8; 1.39.10; 1.39.12; 1.39.18;
hypervisor_unmask_event(): don't check/update evtchn_pending_sel for the
current CPU, but for any CPU which may accept this event.
xen/xenevt.c: more use of atomic ops and locks where appropriate, and some
other SMP fixes. Handle all events on the primary CPU (may be revisited
later). Set/clear ci_evtmask[] for watched events.

This should fix the problems on dom0 kernels reported by jym@

branches: 1.38.2;
Make event/interrupt handling MP aware

branches: 1.37.2;
- Replace uses of simple_lock and ltsleep with mutex and condvar.
- Improve some parts of the code to be more MP-friendly.

Tested by jakllsch@.

branches: 1.36.4; 1.36.6;
If a multithreaded app closes an fd while another thread is blocked in
read/write/accept, then the expectation is that the blocked thread will
exit and the close complete.
Since only one fd is affected, but many fd can refer to the same file,
the close code can only request the fs code unblock with ERESTART.
Fixed for pipes and sockets, ERESTART will only be generated after such
a close - so there should be no change for other programs.
Also rename fo_abort() to fo_restart() (this used to be fo_drain()).
Fixes PR/26567

Rename fo_drain() to fo_abort(), 'drain' is used to mean 'wait for output
do drain' in many places, whereas fo_drain() was called in order to force
blocking read()/write() etc calls to return to userspace so that a close()
call from a different thread can complete.
In the sockets code comment out the broken code in the inner function,
it was being called from compat code.

Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !

remove Xen2 support.
ok bouyer@

Add fileops::fo_drain(), to be called from fd_close() when there is more
than one active reference to a file descriptor. It should dislodge threads
sleeping while holding a reference to the descriptor. Implemented only for
sockets but should be extended to pipes, fifos, etc.

Fixes the case of a multithreaded process doing something like the
following, which would have hung until the process got a signal.

thr0 accept(fd, ...)
thr1 close(fd)

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.30.2;
kill MALLOC and FREE macros.

branches: 1.29.2; 1.29.4;
Compile NetBSD/amd64 kernels with "-Wextra". Patches contributed by
Juan RP in PR port-amd64/39266.

branches: 1.28.4;
Initialize ih_ilevel for the special xenevt_processevt() handler.
Should fix KASSERT panic reported by Sarton O'Brien and Kazushi Marukawa
on port-xen@

branches: 1.27.2; 1.27.4;
Use interrupt biglock wrapper as in x86/x86/intr.c
This change is based on http://mail-index.netbsd.org/port-amd64/2004/02/22/0000.html
OK bouyer

branches: 1.26.2;
- use POSIX integer types
- ansify functions

Make it compile.

Fix my previous confuse of the code: use selinit/seldestroy in correct
places, and fix missed selnotify call.

Welcome to 4.99.55:

- Add a lot of missing selinit() and seldestroy() calls.

- Merge selwakeup() and selnotify() calls into a single selnotify().

- Add an additional 'events' argument to selnotify() call. It will
indicate which event (POLL_IN, POLL_OUT, etc) happen. If unknown,
zero may be used.

Note: please pass appropriate value of 'events' where possible.
Proposed on: <tech-kern>

branches: 1.22.2; 1.22.6;
The event bitmasks provided by the hypervisor are unsigned long (so 64bits
on amd64). Make sure to use the right type to store and manipulate them.
This fixes amd64, where basically any event channel > 31 was not working
(and you get there after starting/stopping a domU a few times). Things
would occasionally unwedge though the spllower() callbacks.

Fix xenevt to not call softint_schedule() above IPL_HIGH:
Register a ipl callback for IPL_HIGH.
if the current ipl level is too high, just record the event in a bitmap,
and record IPL_HIGH as pending. The callback will process the pending events.

Add missing __KERNEL_RCSID()

Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.

Allocating up to 8k on stack isn't a good idea ...

branches: 1.17.6;
Interrupt handling changes, in discussion since February:

- Reduce available SPL levels for hardware devices to none, vm, sched, high.
- Acquire kernel_lock only for interrupts at IPL_VM.
- Implement threaded soft interrupts.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

branches: 1.15.2;
Merge the ppcoea-renovation branch to HEAD.

This branch was a major cleanup and rototill of many of the various OEA
cpu based PPC ports that focused on sharing as much code as possible
between the various ports to eliminate near-identical copies of files in
every tree. Additionally there is a new PIC system that unifies the
interface to interrupt code for all different OEA ppc arches. The work
for this branch was done by a variety of people, too long to list here.

TODO:
bebox still needs work to complete the transition to -renovation.
ofppc still needs a bunch of work, which I will be looking at.
ev64260 still needs to be renovated
amigappc was not attempted.

NOTES:
pmppc was removed as an arch, and moved to a evbppc target.

branches: 1.14.8; 1.14.10; 1.14.14;
Add needed locking in xenevt_fpoll; this fixes at least a lost-wakeup bug.
OK'ed by bouyer@.

branches: 1.13.4; 1.13.6; 1.13.12;
TRUE -> true, FALSE -> false

Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.

branches: 1.11.4; 1.11.6; 1.11.8;
Hypercalls returns negative error codes, so return -error in ioctl
function.

branches: 1.10.4;
Adapt to change in args for falloc(). Reported by Hisashi T Fujinaka
on current-users.

integrate kauth.

Add dom0 operation support for Xen3. Probably buggy, but it's enouth to have
xend and xenstored starting; xm info and xm list works.

branches: 1.7.4; 1.7.6; 1.7.8; 1.7.10; 1.7.12;
Make xenevt.c compile after lwp changes; not tested yet.

merge ktrace-lwp.

branches: 1.5.2; 1.5.4;
Avoid a race between do_hypervisor_event() and stipending() that could
cause an event to be both handled and marked as pending, or being
marked as pending twice (triggering the diagnostic check
evtch_maskcount[port] == 0 in hypervisor_set_ipending):
mask and clear event by word of 32bit in do_hypervisor_event() or stipending(),
instead of by indiviual bits in do_event() or xenevt_event().
In addition this is marginally more efficient.

Get rid of the event to pseudo-irq mapping. We are limited to 32 pseudo-irq,
including soft interrupt, and this is way too low in some use (lots of domains,
or domains with lots of xennet, or even hardware with lots of devices at
different interrupts).
Based on idea from YAMAMOTO Takashi, keep one list of handler per-event and
one per-IPL (so the same handler is now in 2 lists). In the common case were
an event is received at low IPL, we can call the handlers quickly (there
is usually only one handler per event, unless the event is mapped to a
physical interrupt and this interrupt is shared by different devices).
Deffered events and software interrupts are handled by a bitmask (as before)
with one bit per IPL. When one IPL has an event pending all handlers for
this IPL will be called.
With this change, it is now possible to have all the 1024 events active.

While here, handle debug event in a special way: the handler is always called,
regardless of the current IPL. Make the handler print usefull informations
about events and IPL states.
Also remove code not used on Xen in files inherited from the x86 port.

fix a bug which corrupts runqueue.
when dealing with events, which are handed to xenevt pseudo device,
don't call wakeup(9)/selnotify(9) at too high IPL. PR/29792.

branches: 1.2.2; 1.2.4; 1.2.6;
Merge the bouyer-xen2 branch. This add supports for the Xen 2.0 virtual
machine kernel (both privileged and non-privileged domains), and remove support
for the old xen 1.2.

branches: 1.1.2;
file xenevt.c was initially added on branch bouyer-xen2.

xenevt_setipending(): also increase xenevt_ih->ih_pending, so that
the handler will be called when the IPL is lowered.

Fix all remaining typos, mainly in comments but also in few definitions and log messages, reported by me in PR kern/54889.
Also fixed some additional typos in comments, found on review of same files or typos.

branches: 1.63.4;
Trailing whitespace

Fix spelling of preferred in comments. While here improve grammar in
one of the comments.

Work in progress on dom0 PVH support. kernel boots and xl info works,
but we can't start a domU yet.

branches: 1.60.2;
Go back using cpu_info_primary, all events are bound to vCPU 0 by default.
Register the event handler on cpu_info_primary.
While there. update the event counter on interrupts.
Finally this should be MP-safe.

The event channel is not necesserely bound to the primary CPU.
Use the CPU returned in the interrupt handler instead.

Make DOM0OPS build for PVH/PVHVM too

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

branches: 1.56.2;
switch from malloc() to kmem_zalloc()

just pass true as known_mpsafe for interrupt establish, drop the
(level != IPL_VM); NFCI, this uses IPL_HIGH so was mpsafe already

Make this files compilable w/o options DIAGNOSTIC

branches: 1.53.4;
Excise XEN specific code out of x86/x86/intr.c into xen/x86/xen_intr.c

While at it, separate the source function tracking so that the interrupt
paths are truly independant.

Use weak symbol exporting to provision for future PVHVM co-existence
of both files, but with independant paths. Introduce assembler code
such that in a unified scenario, native interrupts get first priority
in spllower(), followed by XEN event callbacks. IPL management and
semantics are unchanged - native handlers and xen callbacks are
expected to maintain their ipl related semantics.

In summary, after this commit, native and XEN now have completely
unrelated interrupt handling mechanisms, including
intr_establish_xname() and assembler stubs and intr handler
management.

Happy Christmas!

Bifurcate the interrupt establish functions between XEN and non-XEN

Thus intr_establish_xname() becomes xen_intr_establish_xname() etc.

One consequence of this is that dom0 devices expect the native
function calls to be available and we thus provide weak aliasing for
dom0 builds to succeed. XEN and non-XEN devices are distinguished by
the PIC they are established on. XEN interrupts are exclusively
established on xen_pic, while dom0 interrupts are established on
natively available PICs.

This allows us an orthogonal path to xen device management (eg:
xenstore events) in XENPVHVM, without having to worry about unifying
the vector entry paths, etc., which is quite challenging.

When using the intr_establish_xname() interface to register
XEN events, follow established x86/intr.c conventions - set
the 'legacy' irq value to -1, to indicate that the pic, pin
combination (&xen_pic, port) is used for registration.

The known_mpsafe parameter is used by intr.c:intr_establish_xname()
especially in the non -D MULTIPROCESSOR case. We used it incorrectly.

Fix this.

Do not re-expose the innards of evtchn.c, now that we have a way
to register interrupts via intr.c:intr_establish_xname()

evtchn.c is going to get refactored soon, so use the latter method.

branches: 1.48.2; 1.48.4;
add fo_name so we can identify the fileops in a simple way.

branches: 1.47.2;
Unify the xen and native x86/ interrupt setup functions and
spl traversal data structures.

This is towards PVHVM.

remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

KNF. Remove extra spaces. No functional change.

include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.

Improve wording.

Fix off by one error, pointed out by Wei Liu in port-xen/49919

branches: 1.41.2; 1.41.4;
Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.

branches: 1.40.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.

branches: 1.39.4; 1.39.8; 1.39.10; 1.39.12; 1.39.18;
hypervisor_unmask_event(): don't check/update evtchn_pending_sel for the
current CPU, but for any CPU which may accept this event.
xen/xenevt.c: more use of atomic ops and locks where appropriate, and some
other SMP fixes. Handle all events on the primary CPU (may be revisited
later). Set/clear ci_evtmask[] for watched events.

This should fix the problems on dom0 kernels reported by jym@

branches: 1.38.2;
Make event/interrupt handling MP aware

branches: 1.37.2;
- Replace uses of simple_lock and ltsleep with mutex and condvar.
- Improve some parts of the code to be more MP-friendly.

Tested by jakllsch@.

branches: 1.36.4; 1.36.6;
If a multithreaded app closes an fd while another thread is blocked in
read/write/accept, then the expectation is that the blocked thread will
exit and the close complete.
Since only one fd is affected, but many fd can refer to the same file,
the close code can only request the fs code unblock with ERESTART.
Fixed for pipes and sockets, ERESTART will only be generated after such
a close - so there should be no change for other programs.
Also rename fo_abort() to fo_restart() (this used to be fo_drain()).
Fixes PR/26567

Rename fo_drain() to fo_abort(), 'drain' is used to mean 'wait for output
do drain' in many places, whereas fo_drain() was called in order to force
blocking read()/write() etc calls to return to userspace so that a close()
call from a different thread can complete.
In the sockets code comment out the broken code in the inner function,
it was being called from compat code.

Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !

remove Xen2 support.
ok bouyer@

Add fileops::fo_drain(), to be called from fd_close() when there is more
than one active reference to a file descriptor. It should dislodge threads
sleeping while holding a reference to the descriptor. Implemented only for
sockets but should be extended to pipes, fifos, etc.

Fixes the case of a multithreaded process doing something like the
following, which would have hung until the process got a signal.

thr0 accept(fd, ...)
thr1 close(fd)

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.30.2;
kill MALLOC and FREE macros.

branches: 1.29.2; 1.29.4;
Compile NetBSD/amd64 kernels with "-Wextra". Patches contributed by
Juan RP in PR port-amd64/39266.

branches: 1.28.4;
Initialize ih_ilevel for the special xenevt_processevt() handler.
Should fix KASSERT panic reported by Sarton O'Brien and Kazushi Marukawa
on port-xen@

branches: 1.27.2; 1.27.4;
Use interrupt biglock wrapper as in x86/x86/intr.c
This change is based on http://mail-index.netbsd.org/port-amd64/2004/02/22/0000.html
OK bouyer

branches: 1.26.2;
- use POSIX integer types
- ansify functions

Make it compile.

Fix my previous confuse of the code: use selinit/seldestroy in correct
places, and fix missed selnotify call.

Welcome to 4.99.55:

- Add a lot of missing selinit() and seldestroy() calls.

- Merge selwakeup() and selnotify() calls into a single selnotify().

- Add an additional 'events' argument to selnotify() call. It will
indicate which event (POLL_IN, POLL_OUT, etc) happen. If unknown,
zero may be used.

Note: please pass appropriate value of 'events' where possible.
Proposed on: <tech-kern>

branches: 1.22.2; 1.22.6;
The event bitmasks provided by the hypervisor are unsigned long (so 64bits
on amd64). Make sure to use the right type to store and manipulate them.
This fixes amd64, where basically any event channel > 31 was not working
(and you get there after starting/stopping a domU a few times). Things
would occasionally unwedge though the spllower() callbacks.

Fix xenevt to not call softint_schedule() above IPL_HIGH:
Register a ipl callback for IPL_HIGH.
if the current ipl level is too high, just record the event in a bitmap,
and record IPL_HIGH as pending. The callback will process the pending events.

Add missing __KERNEL_RCSID()

Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.

Allocating up to 8k on stack isn't a good idea ...

branches: 1.17.6;
Interrupt handling changes, in discussion since February:

- Reduce available SPL levels for hardware devices to none, vm, sched, high.
- Acquire kernel_lock only for interrupts at IPL_VM.
- Implement threaded soft interrupts.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

branches: 1.15.2;
Merge the ppcoea-renovation branch to HEAD.

This branch was a major cleanup and rototill of many of the various OEA
cpu based PPC ports that focused on sharing as much code as possible
between the various ports to eliminate near-identical copies of files in
every tree. Additionally there is a new PIC system that unifies the
interface to interrupt code for all different OEA ppc arches. The work
for this branch was done by a variety of people, too long to list here.

TODO:
bebox still needs work to complete the transition to -renovation.
ofppc still needs a bunch of work, which I will be looking at.
ev64260 still needs to be renovated
amigappc was not attempted.

NOTES:
pmppc was removed as an arch, and moved to a evbppc target.

branches: 1.14.8; 1.14.10; 1.14.14;
Add needed locking in xenevt_fpoll; this fixes at least a lost-wakeup bug.
OK'ed by bouyer@.

branches: 1.13.4; 1.13.6; 1.13.12;
TRUE -> true, FALSE -> false

Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.

branches: 1.11.4; 1.11.6; 1.11.8;
Hypercalls returns negative error codes, so return -error in ioctl
function.

branches: 1.10.4;
Adapt to change in args for falloc(). Reported by Hisashi T Fujinaka
on current-users.

integrate kauth.

Add dom0 operation support for Xen3. Probably buggy, but it's enouth to have
xend and xenstored starting; xm info and xm list works.

branches: 1.7.4; 1.7.6; 1.7.8; 1.7.10; 1.7.12;
Make xenevt.c compile after lwp changes; not tested yet.

merge ktrace-lwp.

branches: 1.5.2; 1.5.4;
Avoid a race between do_hypervisor_event() and stipending() that could
cause an event to be both handled and marked as pending, or being
marked as pending twice (triggering the diagnostic check
evtch_maskcount[port] == 0 in hypervisor_set_ipending):
mask and clear event by word of 32bit in do_hypervisor_event() or stipending(),
instead of by indiviual bits in do_event() or xenevt_event().
In addition this is marginally more efficient.

Get rid of the event to pseudo-irq mapping. We are limited to 32 pseudo-irq,
including soft interrupt, and this is way too low in some use (lots of domains,
or domains with lots of xennet, or even hardware with lots of devices at
different interrupts).
Based on idea from YAMAMOTO Takashi, keep one list of handler per-event and
one per-IPL (so the same handler is now in 2 lists). In the common case were
an event is received at low IPL, we can call the handlers quickly (there
is usually only one handler per event, unless the event is mapped to a
physical interrupt and this interrupt is shared by different devices).
Deffered events and software interrupts are handled by a bitmask (as before)
with one bit per IPL. When one IPL has an event pending all handlers for
this IPL will be called.
With this change, it is now possible to have all the 1024 events active.

While here, handle debug event in a special way: the handler is always called,
regardless of the current IPL. Make the handler print usefull informations
about events and IPL states.
Also remove code not used on Xen in files inherited from the x86 port.

fix a bug which corrupts runqueue.
when dealing with events, which are handed to xenevt pseudo device,
don't call wakeup(9)/selnotify(9) at too high IPL. PR/29792.

branches: 1.2.2; 1.2.4; 1.2.6;
Merge the bouyer-xen2 branch. This add supports for the Xen 2.0 virtual
machine kernel (both privileged and non-privileged domains), and remove support
for the old xen 1.2.

branches: 1.1.2;
file xenevt.c was initially added on branch bouyer-xen2.

Fix all remaining typos, mainly in comments but also in few definitions and log messages, reported by me in PR kern/54889.
Also fixed some additional typos in comments, found on review of same files or typos.

Trailing whitespace

Fix spelling of preferred in comments. While here improve grammar in
one of the comments.

Work in progress on dom0 PVH support. kernel boots and xl info works,
but we can't start a domU yet.

branches: 1.60.2;
Go back using cpu_info_primary, all events are bound to vCPU 0 by default.
Register the event handler on cpu_info_primary.
While there. update the event counter on interrupts.
Finally this should be MP-safe.

The event channel is not necesserely bound to the primary CPU.
Use the CPU returned in the interrupt handler instead.

Make DOM0OPS build for PVH/PVHVM too

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

branches: 1.56.2;
switch from malloc() to kmem_zalloc()

just pass true as known_mpsafe for interrupt establish, drop the
(level != IPL_VM); NFCI, this uses IPL_HIGH so was mpsafe already

Make this files compilable w/o options DIAGNOSTIC

branches: 1.53.4;
Excise XEN specific code out of x86/x86/intr.c into xen/x86/xen_intr.c

While at it, separate the source function tracking so that the interrupt
paths are truly independant.

Use weak symbol exporting to provision for future PVHVM co-existence
of both files, but with independant paths. Introduce assembler code
such that in a unified scenario, native interrupts get first priority
in spllower(), followed by XEN event callbacks. IPL management and
semantics are unchanged - native handlers and xen callbacks are
expected to maintain their ipl related semantics.

In summary, after this commit, native and XEN now have completely
unrelated interrupt handling mechanisms, including
intr_establish_xname() and assembler stubs and intr handler
management.

Happy Christmas!

Bifurcate the interrupt establish functions between XEN and non-XEN

Thus intr_establish_xname() becomes xen_intr_establish_xname() etc.

One consequence of this is that dom0 devices expect the native
function calls to be available and we thus provide weak aliasing for
dom0 builds to succeed. XEN and non-XEN devices are distinguished by
the PIC they are established on. XEN interrupts are exclusively
established on xen_pic, while dom0 interrupts are established on
natively available PICs.

This allows us an orthogonal path to xen device management (eg:
xenstore events) in XENPVHVM, without having to worry about unifying
the vector entry paths, etc., which is quite challenging.

When using the intr_establish_xname() interface to register
XEN events, follow established x86/intr.c conventions - set
the 'legacy' irq value to -1, to indicate that the pic, pin
combination (&xen_pic, port) is used for registration.

The known_mpsafe parameter is used by intr.c:intr_establish_xname()
especially in the non -D MULTIPROCESSOR case. We used it incorrectly.

Fix this.

Do not re-expose the innards of evtchn.c, now that we have a way
to register interrupts via intr.c:intr_establish_xname()

evtchn.c is going to get refactored soon, so use the latter method.

branches: 1.48.2; 1.48.4;
add fo_name so we can identify the fileops in a simple way.

branches: 1.47.2;
Unify the xen and native x86/ interrupt setup functions and
spl traversal data structures.

This is towards PVHVM.

remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

KNF. Remove extra spaces. No functional change.

include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.

Improve wording.

Fix off by one error, pointed out by Wei Liu in port-xen/49919

branches: 1.41.2; 1.41.4;
Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.

branches: 1.40.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.

branches: 1.39.4; 1.39.8; 1.39.10; 1.39.12; 1.39.18;
hypervisor_unmask_event(): don't check/update evtchn_pending_sel for the
current CPU, but for any CPU which may accept this event.
xen/xenevt.c: more use of atomic ops and locks where appropriate, and some
other SMP fixes. Handle all events on the primary CPU (may be revisited
later). Set/clear ci_evtmask[] for watched events.

This should fix the problems on dom0 kernels reported by jym@

branches: 1.38.2;
Make event/interrupt handling MP aware

branches: 1.37.2;
- Replace uses of simple_lock and ltsleep with mutex and condvar.
- Improve some parts of the code to be more MP-friendly.

Tested by jakllsch@.

branches: 1.36.4; 1.36.6;
If a multithreaded app closes an fd while another thread is blocked in
read/write/accept, then the expectation is that the blocked thread will
exit and the close complete.
Since only one fd is affected, but many fd can refer to the same file,
the close code can only request the fs code unblock with ERESTART.
Fixed for pipes and sockets, ERESTART will only be generated after such
a close - so there should be no change for other programs.
Also rename fo_abort() to fo_restart() (this used to be fo_drain()).
Fixes PR/26567

Rename fo_drain() to fo_abort(), 'drain' is used to mean 'wait for output
do drain' in many places, whereas fo_drain() was called in order to force
blocking read()/write() etc calls to return to userspace so that a close()
call from a different thread can complete.
In the sockets code comment out the broken code in the inner function,
it was being called from compat code.

Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !

remove Xen2 support.
ok bouyer@

Add fileops::fo_drain(), to be called from fd_close() when there is more
than one active reference to a file descriptor. It should dislodge threads
sleeping while holding a reference to the descriptor. Implemented only for
sockets but should be extended to pipes, fifos, etc.

Fixes the case of a multithreaded process doing something like the
following, which would have hung until the process got a signal.

thr0 accept(fd, ...)
thr1 close(fd)

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.30.2;
kill MALLOC and FREE macros.

branches: 1.29.2; 1.29.4;
Compile NetBSD/amd64 kernels with "-Wextra". Patches contributed by
Juan RP in PR port-amd64/39266.

branches: 1.28.4;
Initialize ih_ilevel for the special xenevt_processevt() handler.
Should fix KASSERT panic reported by Sarton O'Brien and Kazushi Marukawa
on port-xen@

branches: 1.27.2; 1.27.4;
Use interrupt biglock wrapper as in x86/x86/intr.c
This change is based on http://mail-index.netbsd.org/port-amd64/2004/02/22/0000.html
OK bouyer

branches: 1.26.2;
- use POSIX integer types
- ansify functions

Make it compile.

Fix my previous confuse of the code: use selinit/seldestroy in correct
places, and fix missed selnotify call.

Welcome to 4.99.55:

- Add a lot of missing selinit() and seldestroy() calls.

- Merge selwakeup() and selnotify() calls into a single selnotify().

- Add an additional 'events' argument to selnotify() call. It will
indicate which event (POLL_IN, POLL_OUT, etc) happen. If unknown,
zero may be used.

Note: please pass appropriate value of 'events' where possible.
Proposed on: <tech-kern>

branches: 1.22.2; 1.22.6;
The event bitmasks provided by the hypervisor are unsigned long (so 64bits
on amd64). Make sure to use the right type to store and manipulate them.
This fixes amd64, where basically any event channel > 31 was not working
(and you get there after starting/stopping a domU a few times). Things
would occasionally unwedge though the spllower() callbacks.

Fix xenevt to not call softint_schedule() above IPL_HIGH:
Register a ipl callback for IPL_HIGH.
if the current ipl level is too high, just record the event in a bitmap,
and record IPL_HIGH as pending. The callback will process the pending events.

Add missing __KERNEL_RCSID()

Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.

Allocating up to 8k on stack isn't a good idea ...

branches: 1.17.6;
Interrupt handling changes, in discussion since February:

- Reduce available SPL levels for hardware devices to none, vm, sched, high.
- Acquire kernel_lock only for interrupts at IPL_VM.
- Implement threaded soft interrupts.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

branches: 1.15.2;
Merge the ppcoea-renovation branch to HEAD.

This branch was a major cleanup and rototill of many of the various OEA
cpu based PPC ports that focused on sharing as much code as possible
between the various ports to eliminate near-identical copies of files in
every tree. Additionally there is a new PIC system that unifies the
interface to interrupt code for all different OEA ppc arches. The work
for this branch was done by a variety of people, too long to list here.

TODO:
bebox still needs work to complete the transition to -renovation.
ofppc still needs a bunch of work, which I will be looking at.
ev64260 still needs to be renovated
amigappc was not attempted.

NOTES:
pmppc was removed as an arch, and moved to a evbppc target.

branches: 1.14.8; 1.14.10; 1.14.14;
Add needed locking in xenevt_fpoll; this fixes at least a lost-wakeup bug.
OK'ed by bouyer@.

branches: 1.13.4; 1.13.6; 1.13.12;
TRUE -> true, FALSE -> false

Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.

branches: 1.11.4; 1.11.6; 1.11.8;
Hypercalls returns negative error codes, so return -error in ioctl
function.

branches: 1.10.4;
Adapt to change in args for falloc(). Reported by Hisashi T Fujinaka
on current-users.

integrate kauth.

Add dom0 operation support for Xen3. Probably buggy, but it's enouth to have
xend and xenstored starting; xm info and xm list works.

branches: 1.7.4; 1.7.6; 1.7.8; 1.7.10; 1.7.12;
Make xenevt.c compile after lwp changes; not tested yet.

merge ktrace-lwp.

branches: 1.5.2; 1.5.4;
Avoid a race between do_hypervisor_event() and stipending() that could
cause an event to be both handled and marked as pending, or being
marked as pending twice (triggering the diagnostic check
evtch_maskcount[port] == 0 in hypervisor_set_ipending):
mask and clear event by word of 32bit in do_hypervisor_event() or stipending(),
instead of by indiviual bits in do_event() or xenevt_event().
In addition this is marginally more efficient.

Get rid of the event to pseudo-irq mapping. We are limited to 32 pseudo-irq,
including soft interrupt, and this is way too low in some use (lots of domains,
or domains with lots of xennet, or even hardware with lots of devices at
different interrupts).
Based on idea from YAMAMOTO Takashi, keep one list of handler per-event and
one per-IPL (so the same handler is now in 2 lists). In the common case were
an event is received at low IPL, we can call the handlers quickly (there
is usually only one handler per event, unless the event is mapped to a
physical interrupt and this interrupt is shared by different devices).
Deffered events and software interrupts are handled by a bitmask (as before)
with one bit per IPL. When one IPL has an event pending all handlers for
this IPL will be called.
With this change, it is now possible to have all the 1024 events active.

While here, handle debug event in a special way: the handler is always called,
regardless of the current IPL. Make the handler print usefull informations
about events and IPL states.
Also remove code not used on Xen in files inherited from the x86 port.

fix a bug which corrupts runqueue.
when dealing with events, which are handed to xenevt pseudo device,
don't call wakeup(9)/selnotify(9) at too high IPL. PR/29792.

branches: 1.2.2; 1.2.4; 1.2.6;
Merge the bouyer-xen2 branch. This add supports for the Xen 2.0 virtual
machine kernel (both privileged and non-privileged domains), and remove support
for the old xen 1.2.

branches: 1.1.2;
file xenevt.c was initially added on branch bouyer-xen2.

Trailing whitespace

Fix spelling of preferred in comments. While here improve grammar in
one of the comments.

Work in progress on dom0 PVH support. kernel boots and xl info works,
but we can't start a domU yet.

branches: 1.60.2;
Go back using cpu_info_primary, all events are bound to vCPU 0 by default.
Register the event handler on cpu_info_primary.
While there. update the event counter on interrupts.
Finally this should be MP-safe.

The event channel is not necesserely bound to the primary CPU.
Use the CPU returned in the interrupt handler instead.

Make DOM0OPS build for PVH/PVHVM too

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

branches: 1.56.2;
switch from malloc() to kmem_zalloc()

just pass true as known_mpsafe for interrupt establish, drop the
(level != IPL_VM); NFCI, this uses IPL_HIGH so was mpsafe already

Make this files compilable w/o options DIAGNOSTIC

branches: 1.53.4;
Excise XEN specific code out of x86/x86/intr.c into xen/x86/xen_intr.c

While at it, separate the source function tracking so that the interrupt
paths are truly independant.

Use weak symbol exporting to provision for future PVHVM co-existence
of both files, but with independant paths. Introduce assembler code
such that in a unified scenario, native interrupts get first priority
in spllower(), followed by XEN event callbacks. IPL management and
semantics are unchanged - native handlers and xen callbacks are
expected to maintain their ipl related semantics.

In summary, after this commit, native and XEN now have completely
unrelated interrupt handling mechanisms, including
intr_establish_xname() and assembler stubs and intr handler
management.

Happy Christmas!

Bifurcate the interrupt establish functions between XEN and non-XEN

Thus intr_establish_xname() becomes xen_intr_establish_xname() etc.

One consequence of this is that dom0 devices expect the native
function calls to be available and we thus provide weak aliasing for
dom0 builds to succeed. XEN and non-XEN devices are distinguished by
the PIC they are established on. XEN interrupts are exclusively
established on xen_pic, while dom0 interrupts are established on
natively available PICs.

This allows us an orthogonal path to xen device management (eg:
xenstore events) in XENPVHVM, without having to worry about unifying
the vector entry paths, etc., which is quite challenging.

When using the intr_establish_xname() interface to register
XEN events, follow established x86/intr.c conventions - set
the 'legacy' irq value to -1, to indicate that the pic, pin
combination (&xen_pic, port) is used for registration.

The known_mpsafe parameter is used by intr.c:intr_establish_xname()
especially in the non -D MULTIPROCESSOR case. We used it incorrectly.

Fix this.

Do not re-expose the innards of evtchn.c, now that we have a way
to register interrupts via intr.c:intr_establish_xname()

evtchn.c is going to get refactored soon, so use the latter method.

branches: 1.48.2; 1.48.4;
add fo_name so we can identify the fileops in a simple way.

branches: 1.47.2;
Unify the xen and native x86/ interrupt setup functions and
spl traversal data structures.

This is towards PVHVM.

remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

KNF. Remove extra spaces. No functional change.

include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.

Improve wording.

Fix off by one error, pointed out by Wei Liu in port-xen/49919

branches: 1.41.2; 1.41.4;
Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.

branches: 1.40.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.

branches: 1.39.4; 1.39.8; 1.39.10; 1.39.12; 1.39.18;
hypervisor_unmask_event(): don't check/update evtchn_pending_sel for the
current CPU, but for any CPU which may accept this event.
xen/xenevt.c: more use of atomic ops and locks where appropriate, and some
other SMP fixes. Handle all events on the primary CPU (may be revisited
later). Set/clear ci_evtmask[] for watched events.

This should fix the problems on dom0 kernels reported by jym@

branches: 1.38.2;
Make event/interrupt handling MP aware

branches: 1.37.2;
- Replace uses of simple_lock and ltsleep with mutex and condvar.
- Improve some parts of the code to be more MP-friendly.

Tested by jakllsch@.

branches: 1.36.4; 1.36.6;
If a multithreaded app closes an fd while another thread is blocked in
read/write/accept, then the expectation is that the blocked thread will
exit and the close complete.
Since only one fd is affected, but many fd can refer to the same file,
the close code can only request the fs code unblock with ERESTART.
Fixed for pipes and sockets, ERESTART will only be generated after such
a close - so there should be no change for other programs.
Also rename fo_abort() to fo_restart() (this used to be fo_drain()).
Fixes PR/26567

Rename fo_drain() to fo_abort(), 'drain' is used to mean 'wait for output
do drain' in many places, whereas fo_drain() was called in order to force
blocking read()/write() etc calls to return to userspace so that a close()
call from a different thread can complete.
In the sockets code comment out the broken code in the inner function,
it was being called from compat code.

Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !

remove Xen2 support.
ok bouyer@

Add fileops::fo_drain(), to be called from fd_close() when there is more
than one active reference to a file descriptor. It should dislodge threads
sleeping while holding a reference to the descriptor. Implemented only for
sockets but should be extended to pipes, fifos, etc.

Fixes the case of a multithreaded process doing something like the
following, which would have hung until the process got a signal.

thr0 accept(fd, ...)
thr1 close(fd)

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.30.2;
kill MALLOC and FREE macros.

branches: 1.29.2; 1.29.4;
Compile NetBSD/amd64 kernels with "-Wextra". Patches contributed by
Juan RP in PR port-amd64/39266.

branches: 1.28.4;
Initialize ih_ilevel for the special xenevt_processevt() handler.
Should fix KASSERT panic reported by Sarton O'Brien and Kazushi Marukawa
on port-xen@

branches: 1.27.2; 1.27.4;
Use interrupt biglock wrapper as in x86/x86/intr.c
This change is based on http://mail-index.netbsd.org/port-amd64/2004/02/22/0000.html
OK bouyer

branches: 1.26.2;
- use POSIX integer types
- ansify functions

Make it compile.

Fix my previous confuse of the code: use selinit/seldestroy in correct
places, and fix missed selnotify call.

Welcome to 4.99.55:

- Add a lot of missing selinit() and seldestroy() calls.

- Merge selwakeup() and selnotify() calls into a single selnotify().

- Add an additional 'events' argument to selnotify() call. It will
indicate which event (POLL_IN, POLL_OUT, etc) happen. If unknown,
zero may be used.

Note: please pass appropriate value of 'events' where possible.
Proposed on: <tech-kern>

branches: 1.22.2; 1.22.6;
The event bitmasks provided by the hypervisor are unsigned long (so 64bits
on amd64). Make sure to use the right type to store and manipulate them.
This fixes amd64, where basically any event channel > 31 was not working
(and you get there after starting/stopping a domU a few times). Things
would occasionally unwedge though the spllower() callbacks.

Fix xenevt to not call softint_schedule() above IPL_HIGH:
Register a ipl callback for IPL_HIGH.
if the current ipl level is too high, just record the event in a bitmap,
and record IPL_HIGH as pending. The callback will process the pending events.

Add missing __KERNEL_RCSID()