It's not actually "returning infiniti", it's just showing that output because your first number is well past the integer overflow of just over 2 billion. Just like if you only tried to output the first number by itself.
If you force the first number to evaluate as a decimal instead, you get a more helpful answer: (i figured wrapping the first number in "todecimal()" would do this too, but for some reason it isn't. Appian is a little wonky about data typing though - and I might just be doing something wrong, too.)