You are currently reviewing an older revision of this page.

KB-1426 Unexpected results on decimal calculations

Symptom

Designers will sometimes receive unexpected results for calculations involving decimals. See some examples below:

Observed: 92000 - (7000.27 + 73000.27 + 2000.33 + 9999.13) = -1.455192e-011
Expected: 92000 - (7000.27 + 73000.27 + 2000.33 + 9999.13) = 0
Observed: round(123/240*100,1) = 15.2
Expected: round(123/240*100,1) = 15.3 since the result is actually 15.25
Observed: fixed(8192.80*100,0,true()) = 819279
Expected: fixed(8192.80*100,0,true()) = 819280

Cause

This is not a bug with Appian, but it is rather a limitation of the IEEE 754 standard. The Appian Data Type 'Number (Decimal)' is stored as double precision floating-point decimal, which is a 64-bit IEEE 754 floating point. There is no such thing as perfect precision with double floating point numbers since base 10 numbers are being represented as base 2 numbers. 

Workaround

Depending on the calculation, there could be various workarounds to attempt.

If you are seeing unexpected rounding, try rounding with more digits of precision or try using the roundup() or rounddown() functions. For the above example,

round(123/240*100,6) = 15.25
roundup(123/240*100,1) = 15.3

If the fixed() function is used with a decimal calculation and the results are unexpected, try using round() in conjunction with fixed(). For the above example,

fixed(round(8192.80*100),0,true()) = 819280

There is a Decimal (Floating Point) Component that can be used as well.

Affected Versions

This article applies to all versions of Appian.

Last Reviewed: November 2017