math: use fnstsw consistently instead of fstsw in x87 asm

fnstsw does not wait for pending unmasked x87 floating-point exceptions
and it is the same as fstsw when all exceptions are masked which is the
only environment libc supports.

assembly optimizations for fmod/remainder functions