Skip to content

cpu: raise the FP exceptions IRIX's diagnostics check for - #406

Draft
jamesbraid wants to merge 2 commits into
techomancer:mainfrom
jamesbraid:ide-fpu-fix
Draft

cpu: raise the FP exceptions IRIX's diagnostics check for#406
jamesbraid wants to merge 2 commits into
techomancer:mainfrom
jamesbraid:ide-fpu-fix

Conversation

@jamesbraid

@jamesbraid jamesbraid commented Nov 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

IRIX programs and SGI's diagnostics that check FCSR or NaNs got the wrong answer in iris, because it raised the wrong FP exceptions and used modern IEEE NaNs. The Octane PROM's ide FPU test failed 2 of its 14 checks: 0/0 raised divide-by-zero instead of Invalid, and an overflowing add left FCSR empty. Now all 14 pass, under the interpreter and jitv2.

  • ADD, SUB, MUL and DIV raise Invalid, and Overflow plus Inexact. Results at the edge of the range go through one cold path both engines share. Normal results pay one compare, which replaces an existing one.
  • NaNs use the original MIPS encoding: a quiet NaN has the top fraction bit clear. iris's own R4400 hardware results in cpu-tests already showed this. Invalid results return the MIPS default NaN.
  • SQRT, RECIP, RSQRT, the MADD family and CVT.S.D return the same default NaN and raise Invalid for cases like inf×0 and sqrt(−1). This costs madd.s 60→66 host instructions under the JIT.
  • Fixes a JIT bug on main: with FCSR.FS set, compiled code trapped on denormals instead of flushing them to zero.

The FPU code is shared, so the R4400 and R5000 change too.

Testing

  • ide fpu: 14/14 under jitv2 and the interpreter. It was 12/14.
  • cargo test --release --lib --features jitv2: 2892 passed. Each fix has a test that fails without it.

@jamesbraid
cpu: classify FP NaNs with the MIPS encoding
6073413 
The R4400, R5000 and R10000 use the original MIPS NaN encoding: a NaN
with the fraction MSB set is signalling, one with it clear is quiet. The
default NaN is 0x7FBFFFFF for that reason. iris used the IEEE 754-2008
convention, the reverse, in the interpreter and in compiled code.

An R4400 shows it. The cpu-tests case fpu/qnan_operand feeds 0x7FC00000
to ADD and gets Invalid and the default NaN, not Unimplemented, and the
signalling-predicate compares agree.

Swap the sense of the fraction-MSB test in is_snan and is_qnan and in
their compiled twins, including the compare path. With the bit set the
fraction is nonzero by construction, so the interpreter's signalling
test shrinks to one mask and compare, and an ordinary operand skips the
subnormal and quiet-NaN checks after one exponent test. Compiled code
swaps a comparison and costs the same.

The tests cover the default NaN as an operand (Unimplemented, no
Invalid), a signalling NaN operand (Invalid, trapped and untrapped), and
a compiled-versus-interpreter check over ADD, MUL, DIV, SQRT and the
quiet and signalling compare predicates.
@jamesbraid
jamesbraid force-pushed the ide-fpu-fix branch 2 times, most recently from ad267e6 to c968e9c Compare November 2, 2026 15:47
@jamesbraid
jamesbraid marked this pull request as draft November 2, 2026 16:29
@jamesbraid
jamesbraid force-pushed the ide-fpu-fix branch 2 times, most recently from 4474a7f to 4bce574 Compare November 2, 2026 18:54
@jamesbraid
cpu: report Invalid and Overflow as the hardware does
776e029 
The Octane PROM's ide FPU test checks results the host FPU gets wrong
for MIPS: infinity minus infinity, zero times infinity, 0/0 and
infinity/infinity are Invalid Operations that deliver the default NaN
(0x7FBFFFFF), 0/0 is not a divide by zero, and MAX+MAX is Overflow with
Inexact. iris returned the host's NaN with no flags. 12 of the 14
subtests passed.

Classify those results on every CPU model, in both engines, at no cost
to an ordinary result.

ADD, SUB, MUL and DIV test the result's magnitude once. That test
replaces the old denormal-result test, and the top of the range, which
is where Invalid and Overflow live, goes to a cold classifier shared by
both engines (a hook for compiled code). The operand flags those ops
used to compute up front are gone: an ordinary result proves there were
none, and the classifier finds a divide by zero itself.

SQRT, RECIP, RSQRT, the multiply-add family and CVT.S.D swap the host's
NaN, a signalling NaN under the MIPS encoding, for the default NaN and
add V. They do it the same way round: one compare of the result's
magnitude, and only an infinity or a NaN goes on to classify the
operands. A finite result cannot come from a NaN operand, a zero
divisor or a negative square root, so the signalling-NaN tests on up to
three operands no longer run for ordinary arithmetic. CVT.S.D checks its
source for a signalling NaN only after the conversion has already left
the exact path, since a NaN is never equal to itself.

Host instructions per FP op, perf stat instructions:u, ordinary
operands, ship profile, against the commit before this series (slope
over a 120-op region, three alternating rounds, counts do not move
between rounds). R10000 numbers; R5000 and R4400 differ by a constant:
  compiled add.d, mul.d, div.d, sqrt.s, cvt.s.d   no change
  compiled madd/msub/nmadd/nmsub .s and .d        56-63 -> 25-31
  compiled recip.s, rsqrt.s                       43 -> 25, 51 -> 27
  interpreter add.d, mul.d, div.d                 197-200 -> 171
  interpreter madd/msub/nmadd/nmsub               168-181 -> 153-159
  interpreter recip.s, rsqrt.s                    145 -> 136, 153 -> 137
  interpreter sqrt.s, cvt.s.d                     no change
Every measured op, in both engines, on the R4400, R5000 and R10000, is
within 0.1 instruction of the old count or cheaper.

The tests that check results run on the ordinary passthrough models in
both engines, and compiled code is checked against the interpreter for
each rule. On an emulated Octane the ide fpu test passes 14 of 14 with
the compiled engine and with the interpreter, and an IRIX 6.5.30 boot
reaches login and powers off with halt -p.
@jamesbraid
jamesbraid force-pushed the ide-fpu-fix branch 2 times, most recently from 710711f to 776e029 Compare November 2, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jamesbraid