clockintr: remove clockintr_init(), clockintr_flags

All the state initialization once done in clockintr_init() has been
moved to other parts of the kernel. It's a dead function. Remove it.

Likewise, the clockintr_flags variable no longer sports any meaningful
flags. Remove it. This frees up the CL_* flag namespace, which might
be useful to the clockintr frontend if we ever need to add behavior
flags to any of those functions.

clockintr: replace CL_RNDSTAT with global variable statclock_is_randomized

In order to separate the statclock from the clock interrupt subsystem
we need to move all statclock state out into the broader kernel.

Start by replacing the CL_RNDSTAT flag with a new global variable,
"statclock_is_randomized", in kern_clock.c. Update all clockintr_init()
callers to set the boolean instead of passing the flag.

Thread: https://marc.info/?l=openbsd-tech&m=169428749720476&w=2

all platforms: separate cpu_initclocks() from cpu_startclock()

To give the primary CPU an opportunity to perform clock interrupt
preparation in a machine-independent manner we need to separate the
"initialization" parts of cpu_initclocks() from the "start the clock
interrupt" parts. Currently, cpu_initclocks() does everything all at
once, so there is no space for this MI setup.

Many platforms have more-or-less already done this separation by
implementing a separate routine named "cpu_startclock()". This patch
promotes cpu_startclock() from de facto standard to mandatory API.

- Prototype cpu_startclock() in sys/systm.h alongside cpu_initclocks().
The separation of responsibility between the two routines is a bit
fuzzy but the basic guidelines are as follows:

+ cpu_initclocks() must initialize hz, stathz, and profhz, and call
clockintr_init().

+ cpu_startclock() must call clockintr_cpu_init() and start the clock
interrupt cycle on the calling CPU.

These guidelines will shift in the future, but that's the way things
stand as of *this* commit.

- In initclocks(): first call cpu_initclocks(), then do MI setup, and
last call cpu_startclock().

- On platforms where cpu_startclock() already exists: don't call
cpu_startclock() from cpu_initclocks() anymore.

- On platforms where cpu_startclock() doesn't yet exist: implement it.
Usually this is as simple as dividing cpu_initclocks() in two.

Tested on amd64 (i8254, lapic), arm64, i386 (i8254, lapic), macppc,
mips64/octeon, and sparc64. Tested on arm/armv7 (agtimer(4)) by
phessler@ and jmatthew@. Tested on m88k/luna88k by aoyama@. Tested
on powerpc64 by gkoehler@ and mlarkin@. Tested on riscv64 by
jmatthew@.

Thread: https://marc.info/?l=openbsd-tech&m=169195251322149&w=2

agtimer(4/arm64): call CPU_BUSY_CYCLE() during spin-loop

For consistency with other delay(9) implementations, agtimer(4/arm64)
ought to call CPU_BUSY_CYCLE() as it spins.

kettenis@ notes that we could reduce the power consumed in
agtimer_delay() by enabling CNTKCTL_EL1.EVNTEN and configuring
ENTKCTL_EL1.EVNTI.

kettenis@ also notes that Armv8.7 adds FEAT_WFxT, which will, when the
feature appears in real hardware, make it even easier to save power in
agtimer_delay().

With input from drahn@ and kettenis@.

Thread: https://marc.info/?l=openbsd-tech&m=169146193022516&w=2

ok kettenis@

agtimer(4/arm64): agtimer_delay: compute cycle count with 64-bit arithmetic

Converting from microseconds to timer cycles is much simpler with
64-bit arithmetic.

Thread: https://marc.info/?l=openbsd-tech&m=169146193022516&w=2

ok drahn@ kettenis@

statclock: move profil(2), GPROF code to profclock(), gmonclock()

This patch isolates profil(2) and GPROF from statclock(). Currently,
statclock() implements both profil(2) and GPROF through a complex
mechanism involving both platform code (setstatclockrate) and the
scheduler (pscnt, psdiv, and psratio). We have a machine-independent
interface to the clock interrupt hardware now, so we no longer need to
do it this way.

- Move profil(2)-specific code from statclock() to a new clock
interrupt callback, profclock(), in subr_prof.c. Each
schedstate_percpu has its own profclock handle. The profclock is
enabled/disabled for a given CPU when it is needed by the running
thread during mi_switch() and sched_exit().

- Move GPROF-specific code from statclock() to a new clock interrupt
callback, gmonclock(), in subr_prof.c. Where available, each cpu_info
has its own gmonclock handle . The gmonclock is enabled/disabled for
a given CPU via sysctl(2) in prof_state_toggle().

- Both profclock() and gmonclock() have a fixed period, profclock_period,
that is initialized during initclocks().

- Export clockintr_advance(), clockintr_cancel(), clockintr_establish(),
and clockintr_stagger() via <sys/clockintr.h>. They have external
callers now.

- Delete pscnt, psdiv, psratio. From schedstate_percpu, also delete
spc_pscnt and spc_psdiv. The statclock frequency is not dynamic
anymore so these variables are now useless.

- Delete code/state related to the dynamic statclock frequency from
kern_clockintr.c. The statclock frequency can still be pseudo-random,
so move the contents of clockintr_statvar_init() into clockintr_init().

With input from miod@, deraadt@, and claudio@. Early revisions
cleaned up by claudio. Early revisions tested by claudio@. Tested by
cheloha@ on amd64, arm64, macppc, octeon, and sparc64 (sun4v).
Compile- and boot- tested on i386 by mlarkin@. riscv64 compilation
bugs found by mlarkin@. Tested on riscv64 by jca@. Tested on
powerpc64 by gkoehler@.

timecounting: remove incomplete PPS support

The timecounting code has had stubs for pulse-per-second (PPS) polling
since it was imported in 2004. At this point it seems unlikely that
anyone is going to finish adding PPS support, so let's remove the stubs:

- Delete the dead tc_poll_pps() call from tc_windup().
- Remove all tc_poll_pps symbols from the kernel.

Link: https://marc.info/?l=openbsd-tech&m=167519035723210&w=2

ok miod@

Allwinner hardware sucks! The ARM generic timer on the A64 has a bug
where the bottom 9 bits of the counter register can't be trusted if any of
the higher bits are rolling over. This is an unpublished errata so the
details aren't known. Adopt the same workaround that Linux has.

This will disable the userland timecounter support on hardware affected
by the hardware. We will need a similar workaround in libc to restore
that functionality.

tested by semarie@
ok cheloha@

arm64: switch to clockintr(9)

Switch arm64 to the clockintr(9) subsystem.

- Remove the custom per-CPU clock interrupt schedule from agtimer(4).
- Remove the custom randomized statclock() pieces from agtimer(4).
- Add agtimer_rearm(), agtimer_trigger(), and wire up agtimer_intrclock.

There is one wart:

- The AArch64 spec says that a value written to CNTV_TVAL_EL0 is
"treated as a signed 32-bit integer" [1]. kettenis@ doesn't know
what to make of this. I'm capping the value at INT32_MAX for
now. It's possible I am misreading this, though.

Tested by kettenis@ on his Apple M1 mini. Tested by me on my
Raspberry Pi 4B.

Link: https://marc.info/?l=openbsd-tech&m=166776342503304&w=2

[1] "Arm Architecture Reference Manual for A-profile architecture"
issue I.a, section D17.11.27 ("CNTV_TVAL_EL0").

ok kettenis@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

spelling

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

clockintr: replace CL_RNDSTAT with global variable statclock_is_randomized

In order to separate the statclock from the clock interrupt subsystem
we need to move all statclock state out into the broader kernel.

Start by replacing the CL_RNDSTAT flag with a new global variable,
"statclock_is_randomized", in kern_clock.c. Update all clockintr_init()
callers to set the boolean instead of passing the flag.

Thread: https://marc.info/?l=openbsd-tech&m=169428749720476&w=2

all platforms: separate cpu_initclocks() from cpu_startclock()

To give the primary CPU an opportunity to perform clock interrupt
preparation in a machine-independent manner we need to separate the
"initialization" parts of cpu_initclocks() from the "start the clock
interrupt" parts. Currently, cpu_initclocks() does everything all at
once, so there is no space for this MI setup.

Many platforms have more-or-less already done this separation by
implementing a separate routine named "cpu_startclock()". This patch
promotes cpu_startclock() from de facto standard to mandatory API.

- Prototype cpu_startclock() in sys/systm.h alongside cpu_initclocks().
The separation of responsibility between the two routines is a bit
fuzzy but the basic guidelines are as follows:

+ cpu_initclocks() must initialize hz, stathz, and profhz, and call
clockintr_init().

+ cpu_startclock() must call clockintr_cpu_init() and start the clock
interrupt cycle on the calling CPU.

These guidelines will shift in the future, but that's the way things
stand as of *this* commit.

- In initclocks(): first call cpu_initclocks(), then do MI setup, and
last call cpu_startclock().

- On platforms where cpu_startclock() already exists: don't call
cpu_startclock() from cpu_initclocks() anymore.

- On platforms where cpu_startclock() doesn't yet exist: implement it.
Usually this is as simple as dividing cpu_initclocks() in two.

Tested on amd64 (i8254, lapic), arm64, i386 (i8254, lapic), macppc,
mips64/octeon, and sparc64. Tested on arm/armv7 (agtimer(4)) by
phessler@ and jmatthew@. Tested on m88k/luna88k by aoyama@. Tested
on powerpc64 by gkoehler@ and mlarkin@. Tested on riscv64 by
jmatthew@.

Thread: https://marc.info/?l=openbsd-tech&m=169195251322149&w=2

agtimer(4/arm64): call CPU_BUSY_CYCLE() during spin-loop

For consistency with other delay(9) implementations, agtimer(4/arm64)
ought to call CPU_BUSY_CYCLE() as it spins.

kettenis@ notes that we could reduce the power consumed in
agtimer_delay() by enabling CNTKCTL_EL1.EVNTEN and configuring
ENTKCTL_EL1.EVNTI.

kettenis@ also notes that Armv8.7 adds FEAT_WFxT, which will, when the
feature appears in real hardware, make it even easier to save power in
agtimer_delay().

With input from drahn@ and kettenis@.

Thread: https://marc.info/?l=openbsd-tech&m=169146193022516&w=2

ok kettenis@

agtimer(4/arm64): agtimer_delay: compute cycle count with 64-bit arithmetic

Converting from microseconds to timer cycles is much simpler with
64-bit arithmetic.

Thread: https://marc.info/?l=openbsd-tech&m=169146193022516&w=2

ok drahn@ kettenis@

statclock: move profil(2), GPROF code to profclock(), gmonclock()

This patch isolates profil(2) and GPROF from statclock(). Currently,
statclock() implements both profil(2) and GPROF through a complex
mechanism involving both platform code (setstatclockrate) and the
scheduler (pscnt, psdiv, and psratio). We have a machine-independent
interface to the clock interrupt hardware now, so we no longer need to
do it this way.

- Move profil(2)-specific code from statclock() to a new clock
interrupt callback, profclock(), in subr_prof.c. Each
schedstate_percpu has its own profclock handle. The profclock is
enabled/disabled for a given CPU when it is needed by the running
thread during mi_switch() and sched_exit().

- Move GPROF-specific code from statclock() to a new clock interrupt
callback, gmonclock(), in subr_prof.c. Where available, each cpu_info
has its own gmonclock handle . The gmonclock is enabled/disabled for
a given CPU via sysctl(2) in prof_state_toggle().

- Both profclock() and gmonclock() have a fixed period, profclock_period,
that is initialized during initclocks().

- Export clockintr_advance(), clockintr_cancel(), clockintr_establish(),
and clockintr_stagger() via <sys/clockintr.h>. They have external
callers now.

- Delete pscnt, psdiv, psratio. From schedstate_percpu, also delete
spc_pscnt and spc_psdiv. The statclock frequency is not dynamic
anymore so these variables are now useless.

- Delete code/state related to the dynamic statclock frequency from
kern_clockintr.c. The statclock frequency can still be pseudo-random,
so move the contents of clockintr_statvar_init() into clockintr_init().

With input from miod@, deraadt@, and claudio@. Early revisions
cleaned up by claudio. Early revisions tested by claudio@. Tested by
cheloha@ on amd64, arm64, macppc, octeon, and sparc64 (sun4v).
Compile- and boot- tested on i386 by mlarkin@. riscv64 compilation
bugs found by mlarkin@. Tested on riscv64 by jca@. Tested on
powerpc64 by gkoehler@.

timecounting: remove incomplete PPS support

The timecounting code has had stubs for pulse-per-second (PPS) polling
since it was imported in 2004. At this point it seems unlikely that
anyone is going to finish adding PPS support, so let's remove the stubs:

- Delete the dead tc_poll_pps() call from tc_windup().
- Remove all tc_poll_pps symbols from the kernel.

Link: https://marc.info/?l=openbsd-tech&m=167519035723210&w=2

ok miod@

Allwinner hardware sucks! The ARM generic timer on the A64 has a bug
where the bottom 9 bits of the counter register can't be trusted if any of
the higher bits are rolling over. This is an unpublished errata so the
details aren't known. Adopt the same workaround that Linux has.

This will disable the userland timecounter support on hardware affected
by the hardware. We will need a similar workaround in libc to restore
that functionality.

tested by semarie@
ok cheloha@

arm64: switch to clockintr(9)

Switch arm64 to the clockintr(9) subsystem.

- Remove the custom per-CPU clock interrupt schedule from agtimer(4).
- Remove the custom randomized statclock() pieces from agtimer(4).
- Add agtimer_rearm(), agtimer_trigger(), and wire up agtimer_intrclock.

There is one wart:

- The AArch64 spec says that a value written to CNTV_TVAL_EL0 is
"treated as a signed 32-bit integer" [1]. kettenis@ doesn't know
what to make of this. I'm capping the value at INT32_MAX for
now. It's possible I am misreading this, though.

Tested by kettenis@ on his Apple M1 mini. Tested by me on my
Raspberry Pi 4B.

Link: https://marc.info/?l=openbsd-tech&m=166776342503304&w=2

[1] "Arm Architecture Reference Manual for A-profile architecture"
issue I.a, section D17.11.27 ("CNTV_TVAL_EL0").

ok kettenis@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

spelling

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

all platforms: separate cpu_initclocks() from cpu_startclock()

To give the primary CPU an opportunity to perform clock interrupt
preparation in a machine-independent manner we need to separate the
"initialization" parts of cpu_initclocks() from the "start the clock
interrupt" parts. Currently, cpu_initclocks() does everything all at
once, so there is no space for this MI setup.

Many platforms have more-or-less already done this separation by
implementing a separate routine named "cpu_startclock()". This patch
promotes cpu_startclock() from de facto standard to mandatory API.

- Prototype cpu_startclock() in sys/systm.h alongside cpu_initclocks().
The separation of responsibility between the two routines is a bit
fuzzy but the basic guidelines are as follows:

+ cpu_initclocks() must initialize hz, stathz, and profhz, and call
clockintr_init().

+ cpu_startclock() must call clockintr_cpu_init() and start the clock
interrupt cycle on the calling CPU.

These guidelines will shift in the future, but that's the way things
stand as of *this* commit.

- In initclocks(): first call cpu_initclocks(), then do MI setup, and
last call cpu_startclock().

- On platforms where cpu_startclock() already exists: don't call
cpu_startclock() from cpu_initclocks() anymore.

- On platforms where cpu_startclock() doesn't yet exist: implement it.
Usually this is as simple as dividing cpu_initclocks() in two.

Tested on amd64 (i8254, lapic), arm64, i386 (i8254, lapic), macppc,
mips64/octeon, and sparc64. Tested on arm/armv7 (agtimer(4)) by
phessler@ and jmatthew@. Tested on m88k/luna88k by aoyama@. Tested
on powerpc64 by gkoehler@ and mlarkin@. Tested on riscv64 by
jmatthew@.

Thread: https://marc.info/?l=openbsd-tech&m=169195251322149&w=2

agtimer(4/arm64): call CPU_BUSY_CYCLE() during spin-loop

For consistency with other delay(9) implementations, agtimer(4/arm64)
ought to call CPU_BUSY_CYCLE() as it spins.

kettenis@ notes that we could reduce the power consumed in
agtimer_delay() by enabling CNTKCTL_EL1.EVNTEN and configuring
ENTKCTL_EL1.EVNTI.

kettenis@ also notes that Armv8.7 adds FEAT_WFxT, which will, when the
feature appears in real hardware, make it even easier to save power in
agtimer_delay().

With input from drahn@ and kettenis@.

Thread: https://marc.info/?l=openbsd-tech&m=169146193022516&w=2

ok kettenis@

agtimer(4/arm64): agtimer_delay: compute cycle count with 64-bit arithmetic

Converting from microseconds to timer cycles is much simpler with
64-bit arithmetic.

Thread: https://marc.info/?l=openbsd-tech&m=169146193022516&w=2

ok drahn@ kettenis@

statclock: move profil(2), GPROF code to profclock(), gmonclock()

This patch isolates profil(2) and GPROF from statclock(). Currently,
statclock() implements both profil(2) and GPROF through a complex
mechanism involving both platform code (setstatclockrate) and the
scheduler (pscnt, psdiv, and psratio). We have a machine-independent
interface to the clock interrupt hardware now, so we no longer need to
do it this way.

- Move profil(2)-specific code from statclock() to a new clock
interrupt callback, profclock(), in subr_prof.c. Each
schedstate_percpu has its own profclock handle. The profclock is
enabled/disabled for a given CPU when it is needed by the running
thread during mi_switch() and sched_exit().

- Move GPROF-specific code from statclock() to a new clock interrupt
callback, gmonclock(), in subr_prof.c. Where available, each cpu_info
has its own gmonclock handle . The gmonclock is enabled/disabled for
a given CPU via sysctl(2) in prof_state_toggle().

- Both profclock() and gmonclock() have a fixed period, profclock_period,
that is initialized during initclocks().

- Export clockintr_advance(), clockintr_cancel(), clockintr_establish(),
and clockintr_stagger() via <sys/clockintr.h>. They have external
callers now.

- Delete pscnt, psdiv, psratio. From schedstate_percpu, also delete
spc_pscnt and spc_psdiv. The statclock frequency is not dynamic
anymore so these variables are now useless.

- Delete code/state related to the dynamic statclock frequency from
kern_clockintr.c. The statclock frequency can still be pseudo-random,
so move the contents of clockintr_statvar_init() into clockintr_init().

With input from miod@, deraadt@, and claudio@. Early revisions
cleaned up by claudio. Early revisions tested by claudio@. Tested by
cheloha@ on amd64, arm64, macppc, octeon, and sparc64 (sun4v).
Compile- and boot- tested on i386 by mlarkin@. riscv64 compilation
bugs found by mlarkin@. Tested on riscv64 by jca@. Tested on
powerpc64 by gkoehler@.

timecounting: remove incomplete PPS support

The timecounting code has had stubs for pulse-per-second (PPS) polling
since it was imported in 2004. At this point it seems unlikely that
anyone is going to finish adding PPS support, so let's remove the stubs:

- Delete the dead tc_poll_pps() call from tc_windup().
- Remove all tc_poll_pps symbols from the kernel.

Link: https://marc.info/?l=openbsd-tech&m=167519035723210&w=2

ok miod@

Allwinner hardware sucks! The ARM generic timer on the A64 has a bug
where the bottom 9 bits of the counter register can't be trusted if any of
the higher bits are rolling over. This is an unpublished errata so the
details aren't known. Adopt the same workaround that Linux has.

This will disable the userland timecounter support on hardware affected
by the hardware. We will need a similar workaround in libc to restore
that functionality.

tested by semarie@
ok cheloha@

arm64: switch to clockintr(9)

Switch arm64 to the clockintr(9) subsystem.

- Remove the custom per-CPU clock interrupt schedule from agtimer(4).
- Remove the custom randomized statclock() pieces from agtimer(4).
- Add agtimer_rearm(), agtimer_trigger(), and wire up agtimer_intrclock.

There is one wart:

- The AArch64 spec says that a value written to CNTV_TVAL_EL0 is
"treated as a signed 32-bit integer" [1]. kettenis@ doesn't know
what to make of this. I'm capping the value at INT32_MAX for
now. It's possible I am misreading this, though.

Tested by kettenis@ on his Apple M1 mini. Tested by me on my
Raspberry Pi 4B.

Link: https://marc.info/?l=openbsd-tech&m=166776342503304&w=2

[1] "Arm Architecture Reference Manual for A-profile architecture"
issue I.a, section D17.11.27 ("CNTV_TVAL_EL0").

ok kettenis@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

spelling

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

agtimer(4/arm64): call CPU_BUSY_CYCLE() during spin-loop

For consistency with other delay(9) implementations, agtimer(4/arm64)
ought to call CPU_BUSY_CYCLE() as it spins.

kettenis@ notes that we could reduce the power consumed in
agtimer_delay() by enabling CNTKCTL_EL1.EVNTEN and configuring
ENTKCTL_EL1.EVNTI.

kettenis@ also notes that Armv8.7 adds FEAT_WFxT, which will, when the
feature appears in real hardware, make it even easier to save power in
agtimer_delay().

With input from drahn@ and kettenis@.

Thread: https://marc.info/?l=openbsd-tech&m=169146193022516&w=2

ok kettenis@

agtimer(4/arm64): agtimer_delay: compute cycle count with 64-bit arithmetic

Converting from microseconds to timer cycles is much simpler with
64-bit arithmetic.

Thread: https://marc.info/?l=openbsd-tech&m=169146193022516&w=2

ok drahn@ kettenis@

statclock: move profil(2), GPROF code to profclock(), gmonclock()

This patch isolates profil(2) and GPROF from statclock(). Currently,
statclock() implements both profil(2) and GPROF through a complex
mechanism involving both platform code (setstatclockrate) and the
scheduler (pscnt, psdiv, and psratio). We have a machine-independent
interface to the clock interrupt hardware now, so we no longer need to
do it this way.

- Move profil(2)-specific code from statclock() to a new clock
interrupt callback, profclock(), in subr_prof.c. Each
schedstate_percpu has its own profclock handle. The profclock is
enabled/disabled for a given CPU when it is needed by the running
thread during mi_switch() and sched_exit().

- Move GPROF-specific code from statclock() to a new clock interrupt
callback, gmonclock(), in subr_prof.c. Where available, each cpu_info
has its own gmonclock handle . The gmonclock is enabled/disabled for
a given CPU via sysctl(2) in prof_state_toggle().

- Both profclock() and gmonclock() have a fixed period, profclock_period,
that is initialized during initclocks().

- Export clockintr_advance(), clockintr_cancel(), clockintr_establish(),
and clockintr_stagger() via <sys/clockintr.h>. They have external
callers now.

- Delete pscnt, psdiv, psratio. From schedstate_percpu, also delete
spc_pscnt and spc_psdiv. The statclock frequency is not dynamic
anymore so these variables are now useless.

- Delete code/state related to the dynamic statclock frequency from
kern_clockintr.c. The statclock frequency can still be pseudo-random,
so move the contents of clockintr_statvar_init() into clockintr_init().

With input from miod@, deraadt@, and claudio@. Early revisions
cleaned up by claudio. Early revisions tested by claudio@. Tested by
cheloha@ on amd64, arm64, macppc, octeon, and sparc64 (sun4v).
Compile- and boot- tested on i386 by mlarkin@. riscv64 compilation
bugs found by mlarkin@. Tested on riscv64 by jca@. Tested on
powerpc64 by gkoehler@.

timecounting: remove incomplete PPS support

The timecounting code has had stubs for pulse-per-second (PPS) polling
since it was imported in 2004. At this point it seems unlikely that
anyone is going to finish adding PPS support, so let's remove the stubs:

- Delete the dead tc_poll_pps() call from tc_windup().
- Remove all tc_poll_pps symbols from the kernel.

Link: https://marc.info/?l=openbsd-tech&m=167519035723210&w=2

ok miod@

Allwinner hardware sucks! The ARM generic timer on the A64 has a bug
where the bottom 9 bits of the counter register can't be trusted if any of
the higher bits are rolling over. This is an unpublished errata so the
details aren't known. Adopt the same workaround that Linux has.

This will disable the userland timecounter support on hardware affected
by the hardware. We will need a similar workaround in libc to restore
that functionality.

tested by semarie@
ok cheloha@

arm64: switch to clockintr(9)

Switch arm64 to the clockintr(9) subsystem.

- Remove the custom per-CPU clock interrupt schedule from agtimer(4).
- Remove the custom randomized statclock() pieces from agtimer(4).
- Add agtimer_rearm(), agtimer_trigger(), and wire up agtimer_intrclock.

There is one wart:

- The AArch64 spec says that a value written to CNTV_TVAL_EL0 is
"treated as a signed 32-bit integer" [1]. kettenis@ doesn't know
what to make of this. I'm capping the value at INT32_MAX for
now. It's possible I am misreading this, though.

Tested by kettenis@ on his Apple M1 mini. Tested by me on my
Raspberry Pi 4B.

Link: https://marc.info/?l=openbsd-tech&m=166776342503304&w=2

[1] "Arm Architecture Reference Manual for A-profile architecture"
issue I.a, section D17.11.27 ("CNTV_TVAL_EL0").

ok kettenis@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

spelling

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

statclock: move profil(2), GPROF code to profclock(), gmonclock()

This patch isolates profil(2) and GPROF from statclock(). Currently,
statclock() implements both profil(2) and GPROF through a complex
mechanism involving both platform code (setstatclockrate) and the
scheduler (pscnt, psdiv, and psratio). We have a machine-independent
interface to the clock interrupt hardware now, so we no longer need to
do it this way.

- Move profil(2)-specific code from statclock() to a new clock
interrupt callback, profclock(), in subr_prof.c. Each
schedstate_percpu has its own profclock handle. The profclock is
enabled/disabled for a given CPU when it is needed by the running
thread during mi_switch() and sched_exit().

- Move GPROF-specific code from statclock() to a new clock interrupt
callback, gmonclock(), in subr_prof.c. Where available, each cpu_info
has its own gmonclock handle . The gmonclock is enabled/disabled for
a given CPU via sysctl(2) in prof_state_toggle().

- Both profclock() and gmonclock() have a fixed period, profclock_period,
that is initialized during initclocks().

- Export clockintr_advance(), clockintr_cancel(), clockintr_establish(),
and clockintr_stagger() via <sys/clockintr.h>. They have external
callers now.

- Delete pscnt, psdiv, psratio. From schedstate_percpu, also delete
spc_pscnt and spc_psdiv. The statclock frequency is not dynamic
anymore so these variables are now useless.

- Delete code/state related to the dynamic statclock frequency from
kern_clockintr.c. The statclock frequency can still be pseudo-random,
so move the contents of clockintr_statvar_init() into clockintr_init().

With input from miod@, deraadt@, and claudio@. Early revisions
cleaned up by claudio. Early revisions tested by claudio@. Tested by
cheloha@ on amd64, arm64, macppc, octeon, and sparc64 (sun4v).
Compile- and boot- tested on i386 by mlarkin@. riscv64 compilation
bugs found by mlarkin@. Tested on riscv64 by jca@. Tested on
powerpc64 by gkoehler@.

timecounting: remove incomplete PPS support

The timecounting code has had stubs for pulse-per-second (PPS) polling
since it was imported in 2004. At this point it seems unlikely that
anyone is going to finish adding PPS support, so let's remove the stubs:

- Delete the dead tc_poll_pps() call from tc_windup().
- Remove all tc_poll_pps symbols from the kernel.

Link: https://marc.info/?l=openbsd-tech&m=167519035723210&w=2

ok miod@

Allwinner hardware sucks! The ARM generic timer on the A64 has a bug
where the bottom 9 bits of the counter register can't be trusted if any of
the higher bits are rolling over. This is an unpublished errata so the
details aren't known. Adopt the same workaround that Linux has.

This will disable the userland timecounter support on hardware affected
by the hardware. We will need a similar workaround in libc to restore
that functionality.

tested by semarie@
ok cheloha@

arm64: switch to clockintr(9)

Switch arm64 to the clockintr(9) subsystem.

- Remove the custom per-CPU clock interrupt schedule from agtimer(4).
- Remove the custom randomized statclock() pieces from agtimer(4).
- Add agtimer_rearm(), agtimer_trigger(), and wire up agtimer_intrclock.

There is one wart:

- The AArch64 spec says that a value written to CNTV_TVAL_EL0 is
"treated as a signed 32-bit integer" [1]. kettenis@ doesn't know
what to make of this. I'm capping the value at INT32_MAX for
now. It's possible I am misreading this, though.

Tested by kettenis@ on his Apple M1 mini. Tested by me on my
Raspberry Pi 4B.

Link: https://marc.info/?l=openbsd-tech&m=166776342503304&w=2

[1] "Arm Architecture Reference Manual for A-profile architecture"
issue I.a, section D17.11.27 ("CNTV_TVAL_EL0").

ok kettenis@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

spelling

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

timecounting: remove incomplete PPS support

The timecounting code has had stubs for pulse-per-second (PPS) polling
since it was imported in 2004. At this point it seems unlikely that
anyone is going to finish adding PPS support, so let's remove the stubs:

- Delete the dead tc_poll_pps() call from tc_windup().
- Remove all tc_poll_pps symbols from the kernel.

Link: https://marc.info/?l=openbsd-tech&m=167519035723210&w=2

ok miod@

Allwinner hardware sucks! The ARM generic timer on the A64 has a bug
where the bottom 9 bits of the counter register can't be trusted if any of
the higher bits are rolling over. This is an unpublished errata so the
details aren't known. Adopt the same workaround that Linux has.

This will disable the userland timecounter support on hardware affected
by the hardware. We will need a similar workaround in libc to restore
that functionality.

tested by semarie@
ok cheloha@

arm64: switch to clockintr(9)

Switch arm64 to the clockintr(9) subsystem.

- Remove the custom per-CPU clock interrupt schedule from agtimer(4).
- Remove the custom randomized statclock() pieces from agtimer(4).
- Add agtimer_rearm(), agtimer_trigger(), and wire up agtimer_intrclock.

There is one wart:

- The AArch64 spec says that a value written to CNTV_TVAL_EL0 is
"treated as a signed 32-bit integer" [1]. kettenis@ doesn't know
what to make of this. I'm capping the value at INT32_MAX for
now. It's possible I am misreading this, though.

Tested by kettenis@ on his Apple M1 mini. Tested by me on my
Raspberry Pi 4B.

Link: https://marc.info/?l=openbsd-tech&m=166776342503304&w=2

[1] "Arm Architecture Reference Manual for A-profile architecture"
issue I.a, section D17.11.27 ("CNTV_TVAL_EL0").

ok kettenis@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

spelling

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Allwinner hardware sucks! The ARM generic timer on the A64 has a bug
where the bottom 9 bits of the counter register can't be trusted if any of
the higher bits are rolling over. This is an unpublished errata so the
details aren't known. Adopt the same workaround that Linux has.

This will disable the userland timecounter support on hardware affected
by the hardware. We will need a similar workaround in libc to restore
that functionality.

tested by semarie@
ok cheloha@

arm64: switch to clockintr(9)

Switch arm64 to the clockintr(9) subsystem.

- Remove the custom per-CPU clock interrupt schedule from agtimer(4).
- Remove the custom randomized statclock() pieces from agtimer(4).
- Add agtimer_rearm(), agtimer_trigger(), and wire up agtimer_intrclock.

There is one wart:

- The AArch64 spec says that a value written to CNTV_TVAL_EL0 is
"treated as a signed 32-bit integer" [1]. kettenis@ doesn't know
what to make of this. I'm capping the value at INT32_MAX for
now. It's possible I am misreading this, though.

Tested by kettenis@ on his Apple M1 mini. Tested by me on my
Raspberry Pi 4B.

Link: https://marc.info/?l=openbsd-tech&m=166776342503304&w=2

[1] "Arm Architecture Reference Manual for A-profile architecture"
issue I.a, section D17.11.27 ("CNTV_TVAL_EL0").

ok kettenis@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

spelling

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

arm64: switch to clockintr(9)

Switch arm64 to the clockintr(9) subsystem.

- Remove the custom per-CPU clock interrupt schedule from agtimer(4).
- Remove the custom randomized statclock() pieces from agtimer(4).
- Add agtimer_rearm(), agtimer_trigger(), and wire up agtimer_intrclock.

There is one wart:

- The AArch64 spec says that a value written to CNTV_TVAL_EL0 is
"treated as a signed 32-bit integer" [1]. kettenis@ doesn't know
what to make of this. I'm capping the value at INT32_MAX for
now. It's possible I am misreading this, though.

Tested by kettenis@ on his Apple M1 mini. Tested by me on my
Raspberry Pi 4B.

Link: https://marc.info/?l=openbsd-tech&m=166776342503304&w=2

[1] "Arm Architecture Reference Manual for A-profile architecture"
issue I.a, section D17.11.27 ("CNTV_TVAL_EL0").

ok kettenis@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

spelling

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

spelling

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

spelling

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

timecounting: use C99-style initialization for all timecounter structs

The timecounter struct is large and I think it may change in the
future. Changing it later will be easier if we use C99-style
initialization for all timecounter structs. It also makes reading the
code a bit easier.

For reasons I cannot explain, switching to C99-style initialization
sometimes changes the hash of the resulting object file, even though
the resulting struct should be the same. So there is a binary change
here, but only sometimes. No behavior should change in either case.

I can't compile-test this everywhere but I have been staring at the
diff for days now and I'm relatively confident this will not break
compilation. Fingers crossed.

ok gnezdo@

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

s/KHz/kHz/ and reduce dmesg spam a bit

ok tb@, deraadt@

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Userland timecounter implementation for arm64.

ok naddy@

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Some whitespace fixes for the inline assembly.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add support for timeconting in userland.

This diff exposes parts of clock_gettime(2) and gettimeofday(2) to
userland via libc eliberating processes from the need for a context
switch everytime they want to count the passage of time.

If a timecounter clock can be exposed to userland than it needs to set
its tc_user member to a non-zero value. Tested with one or multiple
counters per architecture.

The timing data is shared through a pointer found in the new ELF
auxiliary vector AUX_openbsd_timekeep containing timehands information
that is frequently updated by the kernel.

Timing differences between the last kernel update and the current time
are adjusted in userland by the tc_get_timecount() function inside the
MD usertc.c file.

This permits a much more responsive environment, quite visible in
browsers, office programs and gaming (apparently one is are able to fly
in Minecraft now).

Tested by robert@, sthen@, naddy@, kmos@, phessler@, and many others!

OK from at least kettenis@, cheloha@, naddy@, sthen@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add workaround for Cortex-A73 errata 858921. Pointed out by drahn@ who
also came up with the initial implementation.

ok drahn@, jsg@

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Use MAXCPUS as the number of elements for the array of per-cpu data.
Fixes machines with more than 8 cores.

ok jsg@, patrick@

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add MULTIPROCESSOR support to the interrupt controller drivers. This makes
the secondary CPUs receive clock interrupts. Based on diffs from drahn@.

ok patrick@

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Switch arm64 generic timer to use virtual timer instead of physical
timer. virtual timer will always be present where physical timer may
be disabled by hypervisor. Other OSes use virtual timer. ok patrick@

Make sure that the timer control registers are written before any
further code is executed. Additionally, make sure the counter is
read only after all previous code has executed.

Bump variables related to arithmetic operations to 64-bit. Especially
the bump of usec and csec is of concern since a timer frequency of around
187MHz overflows in a 32-bit only calculation, as seen on the AMD Seattle
SoC. Since we are running a 64-bit architecture, doing 64-bit arithmetic
operations doesn't hurt us as much as on the 32-bit ARMv7 port.

Initialize the generic timer early so that its delay function can be
used early, similar to the armv7 implementation.

The default frequency we chose for the generic timer does not always ring
true. Instead, unless overwritten by the device tree, we should ask the
generic timer for its frequency. This fixes time on my AMD Seattle and
should improve time management on QEMU as well.

Also attach to "arm,armv7-timer".

ok patrick@

Pass value as input instead of output register, otherwise we write
garbage into the control register. While there remove positional
argument leftover from the 32-bit version.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.