KB-1426 Unexpected results on decimal calculations and rounding

Symptom

Designers will sometimes receive unexpected rounding/results for calculations involving decimals. See some examples below:

Observed: 92000 - (7000.27 + 73000.27 + 2000.33 + 9999.13) = -1.455192e-011
Expected: 92000 - (7000.27 + 73000.27 + 2000.33 + 9999.13) = 0
Observed: round(123/240*100,1) = 15.2
Expected: round(123/240*100,1) = 15.3 since the result is actually 15.25
Observed: fixed(8192.80*100,0,true()) = 819279
Expected: fixed(8192.80*100,0,true()) = 819280
Observed: text(1.9999999999999, "0.00") = 2.00
Expected: text(1.9999999999999, "0.00") = 1.99
Observed: a!textField(value:12.236319) = 12.23632
Expected: a!textField(value:12.236319) = 12.236319

Cause

This is not a bug with Appian, but it is rather a limitation of the IEEE 754 standard. The Appian Data Type 'Number (Decimal)' is stored as double precision floating-point decimal, which is a 64-bit IEEE 754 floating point. There is no such thing as perfect precision with double floating point numbers since base 10 numbers are being represented as base 2 numbers. 

Workaround

Depending on the calculation, there could be various workarounds to attempt.

If you are seeing unexpected rounding with the round() function, try rounding with more digits of precision or try using the roundup() or rounddown() functions. For the above example,

round(123/240*100,6) = 15.25
roundup(123/240*100,1) = 15.3

If a decimal calculation using the fixed() function is providing unexpected results, try using round() in conjunction with fixed(). For the above example,

fixed(round(8192.80*100),0,true()) = 819280

If the text() function seems to incorrectly round a number, try using less digits after the decimal point. For the above example,

text(1.9999, "0.00") = 1.99

If the Text Component is rounding a decimal rule input, try wrapping the fixed() function to indicate the desired number of digits after the decimal to be maintained. For the above example,

a!textField(value: fixed(12.236319,6) = 12.236319

There is a Decimal (Floating Point) Component that can be used.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: December 2017

Related
Recommended