PROBLEMS MAPPING DECIMAL

RE: PROBLEMS MAPPING DECIMAL

sandrap845 — Mon, 02 Dec 2019 13:06:47 GMT

is that it is the mapping in the node directly, take the process variable and in the node add it to the corresponding value.

RE: PROBLEMS MAPPING DECIMAL

sandrap845 — Mon, 02 Dec 2019 13:05:49 GMT

Thank you very much David !!, very interesting information

RE: PROBLEMS MAPPING DECIMAL

davel001150 — Fri, 29 Nov 2019 13:43:07 GMT

It's because of floating point math.

0.1 is only an approximation, because of the way Decimal is stored (as a Double precision floating point number in Java). In binary, in that format, 1 / 10 is an infinitely repeating number, something like 0.00011001100110011. Nearly every number you see in decimal format is only an approximation of what can be represented in binary. It's like, how many halves, plus how many fourths, plus how many eighths, plus how many sixteenths, plus how many thirty-secondths, etc. Unless it's exactly a sum of negative powers of two, it can't be represented fully accurately. The further away from 1 or -1 you go, the wider the gaps between the numbers you can approximate. All software everywhere has this problem. Try it in Java: 0.1 + 0.1 + 0.1 == 0.3 returns false.

What can you do? Nothing, really. You can try multiplying it by 100, 982,065, moving it to it's own PV, then dividing that PV by 100 and putting the answer into the other CDT and see if you don't get exactly the same behavior. You could create a custom plugin to implement C#'s Decimal type, which stores the number as a 28 or 29 digit integer and then stores the location of the decimal point. Or you could create a custom plugin to utilize the Java arbitrary precision mathematics library.

Or as the other posters have said, you can use the fixed function for display purposes. So it's off in the back-end; you just don't let the end users know about it. For the database, just store the digits you want using number(), for instance number(12,4) is up to 8 digits before the decimal and up to 4 digits after the decimal. It will probably still be off deep within, but should refrain from telling you about it.

RE: PROBLEMS MAPPING DECIMAL

Harshit Bumb (Appyzie) — Fri, 29 Nov 2019 12:43:52 GMT

Can you please paste the code snippet which you are using to cast/map the value to another CDT?

RE: PROBLEMS MAPPING DECIMAL

sandrap845 — Thu, 28 Nov 2019 10:35:22 GMT

Ok, I'll try it, but do you know why this problem may be occurring?

Greetings and thank you very much!

RE: PROBLEMS MAPPING DECIMAL

Sandeep — Thu, 28 Nov 2019 10:31:02 GMT

Hi, i think it will work. before saving data, you just use fixed() function for that specific attribute and then save data in DB.

RE: PROBLEMS MAPPING DECIMAL

sandrap845 — Thu, 28 Nov 2019 09:48:53 GMT

Hi,
If I make a fixed, I would return a string but I have to keep that data in a decimal in a database.
Wouldn't this give problems?

Greetings and thank you

RE: PROBLEMS MAPPING DECIMAL

Sandeep — Thu, 28 Nov 2019 09:42:37 GMT

Hi,

Try using fixed() function for it.