Skip to content

Commit 13923a9

Browse files
committed
Fix mantissa overflow bug
1 parent 4d54720 commit 13923a9

1 file changed

Lines changed: 7 additions & 3 deletions

File tree

src/backend/mode_gpu.js

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -315,12 +315,16 @@
315315
' test += mantissa_part3 * exp2(exponent-7.0);',
316316
' test += mantissa_part2 * exp2(exponent-15.0);',
317317
' test += mantissa_part1 * exp2(exponent-23.0);',
318-
' float error = floor(log2(test) - log2F);',
319-
' if (abs(error) > 0.0) {',
320-
' exponent -= error;',
318+
' float error = log2(test) - log2F;',
319+
' if (abs(error) >= 1.0) {',
320+
' exponent -= floor(error);',
321321
' mantissa_part1 = integerMod(F * exp2(23.0-exponent), 256.0);',
322322
' mantissa_part2 = integerMod(F * exp2(15.0-exponent), 256.0);',
323323
' mantissa_part3 = integerMod(F * exp2(7.0-exponent), 128.0);',
324+
' } else if (abs(error) > 0.0) {',
325+
' mantissa_part1 = 0.0;',
326+
' mantissa_part2 = 0.0;',
327+
' mantissa_part3 = 0.0;',
324328
' }',
325329
' exponent += 127.0;',
326330
' vec4 rgba;',

0 commit comments

Comments
 (0)