2.6.10-rc3, i386: fpu handling on sigreturn

[Bodo Stroesser]

Maybe, there is a problem in i386 fpu/signal handling:

On i386, if a signal handler is started, the kernel saves the fpu-state
of the interrupted routine in the sigcontext on the stack. Calling
unlazy_fpu() and setting current->used_math=0, the kernel supplies the
signal-handler with a cleared virtual fpu.
On sigreturn(), the old fpu-state of the interrupted routine is
restored.

If a process never used the fpu, it virtually has a cleared fpu.
If such a process is interrupted by a signal handler, no fpu-context is
saved and sigcontext->fpstate is set to NULL.

Assume, that the signal handler uses the fpu. Then, AFAICS, on sigreturn
current->used_math will be 1. Since sigcontext->fpstate still is NULL,
restore_sigcontext() doesn't call restore_i387(). Thus, no
clear_fpu() is done, current->used_math is not reset.

Now, the interrupted processes fpu no longer is cleared!

I don't know, if this could cause trouble, since I'm not an expert for
i386-fpu. But it seems to be not clean.

Best regards
Bodo


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/