Alpha FP Fixes

Daryll Strauss (daryll@d2.com)
Tue, 10 Jun 1997 12:24:48 -0700 (PDT)


We have some serious numerical applications that were running fine under
Dec Unix, but would fail with floating point exceptions under Linux. I
tracked the problem down to two different areas.

The first problem is that the Dec Unix boxes default to trapping
floating point exceptions and doing the IEEE math, where the default
under Linux is not do that. I changes set_program_attributes to reset
those flags for the current process. That's patch number one.

The second problem is that there is an instruct CVTQL which actually
causes a floating point exception even though it isn't a floating point
operation. The code to emulate this instruction was writen, but was
never being called in alpha_fp_emul_imprecise. That's patch number two.

Once these were installed all our applications started behaving much
better. These are patches against 2.1.42.

- |Daryll

*** linux/arch/alpha/kernel/osf_sys.c.dist Tue Apr 22 22:38:40 1997
--- linux/arch/alpha/kernel/osf_sys.c Sun Jun 8 10:32:53 1997
***************
*** 75,80 ****
--- 75,81 ----
mm = current->mm;
mm->end_code = bss_start + bss_len;
mm->brk = bss_start + bss_len;
+ current->tss.flags &= ~IEEE_TRAP_ENABLE_MASK;
printk("set_program_attributes(%lx %lx %lx %lx)\n",
text_start, text_len, bss_start, bss_len);
unlock_kernel();
*** linux/arch/alpha/math-emu/fp-emul.c.dist Tue Jun 10 12:00:03 1997
--- linux/arch/alpha/math-emu/fp-emul.c Tue Jun 10 11:53:41 1997
***************
*** 16,21 ****
--- 16,22 ----
#define OPC_FLTV 0x14
#define OPC_FLTI 0x15
#define OPC_FLTL 0x16
+ #define OPC_CVTQL 0x17

#define OPC_MISC 0x18

***************
*** 369,379 ****
case OPC_FLTV:
case OPC_FLTI:
case OPC_FLTL:
write_mask &= ~(1UL << (rc + 32));
break;
}
if (!write_mask) {
! if ((opcode == OPC_FLTI || opcode == OPC_FLTL)
&& alpha_fp_emul(trigger_pc))
{
/* re-execute insns in trap-shadow: */
--- 370,382 ----
case OPC_FLTV:
case OPC_FLTI:
case OPC_FLTL:
+ case OPC_CVTQL:
write_mask &= ~(1UL << (rc + 32));
break;
}
if (!write_mask) {
! if ((opcode == OPC_FLTI || opcode == OPC_FLTL ||
! opcode == OPC_CVTQL)
&& alpha_fp_emul(trigger_pc))
{
/* re-execute insns in trap-shadow: */