Decimal precision seems limited

The decimal data type in Appian seems very limited, e.g. todecimal(100000.99) returns 100001

Any ideas?

https://docs.appian.com/suite/help/18.2/fnc_conversion_todecimal.html

  Discussion posts and replies are publicly visible

Parents Reply Children
  • 0
    Certified Lead Developer
    in reply to Mike Schmitt
    I'm pretty sure the result that fixed() has returned is actually correct, as strange as it seems! Remember that rounding from the 5th decimal place bumps the next digit to the left up by one and sets the current digit to 0. This cascades to the left until a digit is able to be incremented by one without affecting the next digit along; you wouldn't expect 44.99 rounded to 0 decimal places to be 44 just because you've got to the right number of decimal places, right? :)

    That's not a very good explanation... but plug 4447.989999999998 into any rounding calculator on the web and you'll find it comes back with 4447.9900.

    The text() result is a good example of the way that text truncates rather than rounds!
  • 0
    Certified Lead Developer
    in reply to PhilB
    Ah, yep that's fair, sorry.