BigDecimal/float difference between MRI and JRuby #5904

Activity

1. mohits commented on May 8, 2021

   

   mohits

   on May 8, 2021

   More actions

   I think that this issue should be closed since this is probably implementation dependent, and should not be something that a programmer should rely on.
2. headius commented on May 10, 2021

   

   headius

   on May 10, 2021

   Member

   More actions

   Looking into this a little big (apparently for the first time)...

   I believe this is an artifact of how we compare a BigDecimal with a Float. In JRuby, we promote the Float into a BigDecimal by using the java.math.BigDecimal constructor that takes a java.math.MathContext for precision. Specifically here in the code, where we use a precision of 15 for floats but unlimited for integer values:

   jruby/core/src/main/java/org/jruby/ext/bigdecimal/RubyBigDecimal.java

   Lines 495 to 509 in 57f9668

   	private static RubyBigDecimal getVpValue(ThreadContext context, IRubyObject value, boolean must) {
   	switch (((RubyBasicObject) value).getNativeClassIndex()) {
   	case BIGDECIMAL:
   	return (RubyBigDecimal) value;
   	case FIXNUM:
   	return newInstance(context.runtime, context.runtime.getClass("BigDecimal"), (RubyFixnum) value, MathContext.UNLIMITED);
   	case BIGNUM:
   	return newInstance(context.runtime, context.runtime.getClass("BigDecimal"), (RubyBignum) value, MathContext.UNLIMITED);
   	case FLOAT:
   	return newInstance(context.runtime, context.runtime.getClass("BigDecimal"), (RubyFloat) value, new MathContext(RubyFloat.DIG));
   	case RATIONAL:
   	return newInstance(context, (RubyRational) value, new MathContext(RubyFloat.DIG));
   	}
   	return cannotBeCoerced(context, value, must);
   	}

   Duplicating this code from Ruby we see the same result:

   $ jruby -e "p java.math.BigDecimal.new('0.803e1').compareTo(java.math.BigDecimal.new(8.03, java.math.MathContext.new(15)))"
   0
   

   Leaving the precision off gets us a result that matches CRuby:

   $ rvm ruby-3.0 do ruby -rbigdecimal -e "p BigDecimal('0.803e1') <=> 8.03"
   1
   
   $ jruby -e "p java.math.BigDecimal.new('0.803e1').compareTo(java.math.BigDecimal.new(8.03))"
   1
   

   As near as I can tell this change came in 73764ea as part of improving the coercion process for Float values. I am not sure where the 15 precision came from, but if we revisit CRuby's equivalent to getVpValue we may be able to match their behavior more closely.
3. headius commented on May 10, 2021

   

   headius

   on May 10, 2021

   Member

   More actions

   The CRuby GetVpValue code basically converts the Float value into a string, then normalizes that string into an Integer and an exponent. The logic bottoms out around here:

   https://github.com/ruby/ruby/blob/4e2e1d60936a3dcbc0e51b1c9c925ef41c1780f6/ext/bigdecimal/bigdecimal.c#L2960

   I believe this means it will end up using infinite precision based on the normalized Integer form of the Float which would explain the behavioral difference from our 15 precision logic.

   As of Java 8, the java.math.BigDecimal logic does not convert the incoming double value into a String. Instead it works with the raw bits directly and pulls out each element of the floating-point value to use in constructing the BigDecimal instance. The value is then rounded to the requested precision, which results in the exact BigDecimal 8.03 actually match the inaccurate floating point 8.029999999999999-something.

   So basically, by providing the precision and allowing it to round off, we are choosing one inaccurate representation over another, and end up matching.

   @ahorek I believe this was your commit. I know it has been a while, but do you remember why you chose to pass in a MathContext with a precision here?

   The fix for this would be to figure out what precision and rounding behavior matches CRuby's float-to-string-to-integer path.
4. ahorek commented on May 11, 2021

   

   ahorek

   on May 11, 2021

   Contributor

   More actions

   actually 73764ea didn't cause the regression.

   why you chose to pass in a MathContext with a precision here?

   the idea behind this is that more accurate precision than the maximum float precision doesn't matter for a comparsion with another float.

   the flaw is we're not comparing the real float representation which is inacurrate.

   BigDecimal('.803e1') == 8.03
   
   [BigDecimal('.803e1'), BigDecimal('.803e1')]
   => true # current behaviour
   
   [BigDecimal(8.02999973297119140625), BigDecimal('.803e1')]
   => false # desired behaviour
   

   so the value should be normalized before converting to a bigdecimal. The precision isn't a real problem.

   also the way how CRuby do it seems to be complex and probably slow. The best way would be to use the fast path which works for most cases and do an additional validation step if there's a match for corner cases like this.

   btw: I don't see a real use case why should anyone depend on precise comparsions between floats and bigdecimals.
   But yes, it's a bug - BigDecimal('8.03') and Float(8.03) aren't the same numbers even if they look very simmilar.
5. headius commented on May 13, 2021

   

   headius

   on May 13, 2021

   Member

   More actions

   the flaw is we're not comparing the real float representation which is inacurrate.

   You are correct. I did not mean to imply that you broke anything. It is largely a bug in us following CRuby logic for the comparison but not for the "GetVpValue" logic that converts float to BigDecimal via a string.

   I think we will need to diverge on the implementation of RubyBigDecimal.cmp in order to do more accurate comparisons. Truth be told, we could probably drop most of the ported logic from CRuby if we can get the JDK BigDecimal to do the equivalent work.

   At this point I am not sure what the fix should be, or indeed whether a fix is necessary. These values are logically equal but physically unequal, and every resource I have looked at recommends against comparing floating-point values and BigDecimal values without explicitly opting into the rounding. In this case we are rounding to 15 digits which causes them to appear equal, while 16 would cause them to be unequal:

   $ jruby -rbigdecimal -e "p BigDecimal(8.03, 15)"
   0.803e1
   
   $ jruby -rbigdecimal -e "p BigDecimal(8.03, 16)"
   0.8029999999999999e1
   

   Even if we were to skip this coercion to BigDecimal at a Ruby level, we would still need to coerce to a JDK BigDecimal in order to do the comparison, at which point we would need to choose some level of rounding.
6. kares

   linked a pull request that will close this issue[compat] match BigDecimal float comparison (#5904) #9816on Oct 17, 2026
7. headius

   added 2 commits that reference this issue on Oct 19, 2026

   [compat] match BigDecimal float comparison (#5904)

   ...

   ac3fee9

   Merge pull request #9816 from kares/fix_bigdec-float-convert

   ...

   c1b58a9
8. headius

   closed this as completedin #9816on Oct 19, 2026
9. headius commented on Oct 19, 2026

   

   headius

   on Oct 19, 2026

   Member

   More actions

   I believe there are still issues outstanding here as noted in #9816, so reopening.

   👍React with 👍1Reacted by Karol Bucek
10. headius

    reopened this on Oct 19, 2026