Autoplay Calculates expressions wrong

[rayh34]

Example: in Autoplay 8003203833 * 12 = 1549165484
But in reality 8003203833 * 12 = 96038445996

Seems there is a cap in the size it can process. I was playing with NIC Addresses when I came across this.

[Derek]

Re: Autoplay Calculates expressions wrong

I believe this was mentioned before when someone made a calculator. [img]/ubbthreads/images/icons/smile.gif[/img]

[Lorne]

Re: Autoplay Calculates expressions wrong

It isn't calculating the expression wrong, exactly. [img]/ubbthreads/images/icons/smile.gif[/img] It's just limited to signed integers, which forces all the values to fit inside 32 bits, with 1 bit used for a sign flag (+/-). The largest value it can handle is plus or minus 2 the the 31st power, i.e. –2,147,483,648 to 2,147,483,647.

Looking at the code, I see a workaround, however. (Kind of odd that I didn't realize this before.) Real numbers are handled as doubles internally, which gives you 1.7E +/- 308. (Technically I think it gives you 15 digits of precision, but in any case it's a really big number.)

So, if you add a decimal place to one of your numbers, you can perform calculations on much larger values. I just tested it, and yep:

8003203833 * 12.0 = 96038445996.0

You can use a "Variable - Format Number" action to remove the decimal place from the result. (Just set the decimal places to 0 and set it to Truncate.)

[rayh34]

Re: Autoplay Calculates expressions wrong

Thanks Lorne,
I found that out while playing with Truncate & Rounding.