xtensa: add load/store exception handler

Memory attached to instruction bus of the xtensa CPU is only accessible
for a limited subset of opcodes. Other opcodes generate an exception
with the load/store error cause code. This property complicates use of
such systems. Provide a handler that recognizes and transparently fixes
such exceptions. The following opcodes are recognized when used outside
of FLIX bundles: l32i, l32i.n, l16ui, l16si, l8ui.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: move early_trap_init from kasan_early_init to init_arch

There may be other users for the early traps besides KASAN. Move call to
the early_trap_init from kasan_early_init. Protect init_exc_table
initializer with ifdef to make sure it builds on noMMU configurations.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: support coprocessors on SMP

Current coprocessor support on xtensa only works correctly on
uniprocessor configurations. Make it work on SMP too and keep it lazy.

Make coprocessor_owner array per-CPU and move it to struct exc_table for
easy access from the fast_coprocessor exception handler. Allow task to
have live coprocessors only on single CPU, record this CPU number in the
struct thread_info::cp_owner_cpu. Change struct thread_info::cpenable
meaning to be 'coprocessors live on cp_owner_cpu'.
Introduce C-level coprocessor exception handler that flushes and
releases live coprocessors of the task taking 'coprocessor disabled'
exception and call it from the fast_coprocessor handler when the task
has live coprocessors on other CPU.
Make coprocessor_flush_all and coprocessor_release_all work correctly
when called from any CPU by sending IPI to the cp_owner_cpu. Add
function coprocessor_flush_release_all to do flush followed by release
atomically. Add function local_coprocessors_flush_release_all to flush
and release all coprocessors on the local CPU and use it to flush
coprocessor contexts from the CPU that goes offline.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: clean up excsave1 initialization

Use xtensa_set_sr instead of inline assembly.
Rename local variable exc_table in early_trap_init to avoid conflict
with per-CPU variable of the same name.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: clean up exception handler prototypes

Exception handlers are currently passed as void pointers because they
may have one or two parameters. Only two handlers uses the second
parameter and it is available in the struct pt_regs anyway. Make all
handlers have only one parameter, introduce xtensa_exception_handler
type for handlers and use it in trap_set_handler.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: clean up function declarations in traps.c

Drop 'extern' from all function declarations and move those that need to
be visible from traps.c to traps.h. Add 'asmlinkage' to declarations of
fucntions defined in assembly. Add 'static' to declarations and
definitions only used locally. Add argument names in declarations.
Drop unused second argument from do_multihit and do_page_fault.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: only build windowed register support code when needed

There's no need in window overflow/underflow/alloca exception handlers
or window spill code when neither kernel nor userspace support windowed
registers. Don't build or link it.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: drop unused field from the struct exc_table

exc_table::syscall_save and corresponding macro EXC_TABLE_SYSCALL_SAVE
have never been used by the xtensa code. Drop them.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: implement early_trap_init

Paging on xtensa architecture requires functioning exception handling
because hardware cannot transparently access page tables that are not
currently mapped by TLB. Exception handling is set up late in the
initialization process, but working paging is needed for KASAN.

Provide early_trap_init that sets up minimal exception handling
sufficient for KASAN to work.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: clean up exception handling structure

Instead of using flat array of longs use normal C structure and generate
EXC_TABLE_* constants in the asm-offsets.c

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: support hardware breakpoints/watchpoints

Use perf framework to manage hardware instruction and data breakpoints.
Add two new ptrace calls: PTRACE_GETHBPREGS and PTRACE_SETHBPREGS to
query and set instruction and data breakpoints.
Address bit 0 choose instruction (0) or data (1) break register, bits
31..1 are the register number.
Both calls transfer two 32-bit words: address (0) and control (1).
Instruction breakpoint contorl word is 0 to clear breakpoint, 1 to set.
Data breakpoint control word bit 31 is 'trigger on store', bit 30 is
'trigger on load, bits 29..0 are length. Length 0 is used to clear a
breakpoint. To set a breakpoint length must be a power of 2 in the range
1..64 and the address must be length-aligned.

Introduce new thread_info flag: TIF_DB_DISABLED. Set it if debug
exception is raised by the kernel code accessing watched userspace
address and disable corresponding data breakpoint. On exit to userspace
check that flag and, if set, restore all data breakpoints.

Handle debug exceptions raised with PS.EXCM set. This may happen when
window overflow/underflow handler or fast exception handler hits data
breakpoint, in which case save and disable all data breakpoints,
single-step faulting instruction and restore data breakpoints.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: use context structure for debug exceptions

With implementation of data breakpoints debug exceptions raised when
PS.EXCM is set need to be handled, e.g. window overflow code can write
to watched userspace address. Currently debug exception handler uses
EXCSAVE and DEPC SRs to save temporary registers, but DEPC may not be
available when PS.EXCM is set and more space will be needed to save
additional state.
Reorganize debug context: create per-CPU structure debug_table instance
and store its address in the EXCSAVE<debug level> instead of
debug_exception function address. Expand this structure when more save
space is needed.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: fix kernel register spilling

call12 can't be safely used as the first call in the inline function,
because the compiler does not extend the stack frame of the bounding
function accordingly, which may result in corruption of local variables.

If a call needs to be done, do call8 first followed by call12.

For pure assembly code in _switch_to increase stack frame size of the
bounding function.

Cc: stable@vger.kernel.org
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: introduce spill_registers_kernel macro

Most in-kernel users want registers spilled on the kernel stack and
don't require PS.EXCM to be set. That means that they don't need fixup
routine and could reuse regular window overflow mechanism for that,
which makes spill routine very simple.

Cc: stable@vger.kernel.org
Suggested-by: Chris Zankel <chris@zankel.net>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

xtensa: add SMP support

This is largely based on SMP code from the xtensa-2.6.29-smp tree by
Piet Delaney, Marc Gauthier, Joe Taylor, Christian Zankel (and possibly
other Tensilica folks).

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Chris Zankel <chris@zankel.net>

xtensa: clean up stpill_registers

- remove unused asm parameters;
- fix EXCM bit setting in the PS SR during _spill_registers call.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Chris Zankel <chris@zankel.net>

xtensa: move spill_registers to traps.h

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Chris Zankel <chris@zankel.net>

xtensa: add trap_set_handler function

trap_set_handler sets new C-handler in the exception table and returns
previous handler.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Chris Zankel <chris@zankel.net>