LoginLogin
Nintendo shutting down 3DS + Wii U online services, see our post

Modulo Operator Overflow

Root / Programming Questions / [.]

amihartCreated:
I've run into this issue with the mod operator throwing an overflow error with larger numbers when using the mod operator. This code throws the "overflow" error:
? 274313e6 MOD 65536e0
Overflow
Yet, I can still calculate the modulo using this:
? 274313e6-65536e0*FLOOR(274313e6/65536e0)
13376
Internally, why does one work and the other doesn't? The output of the mod operator is inherently smaller than the value after the mod, in this case 65536e0, so the result will be of size x<65536e0 so I don't know why it'd overflow.

Trying to make a pseudo-random number generator? This is probably part of the weirdness that Square fingers found over here: https://smilebasicsource.com/forum?ftid=104 Basically, MOD is an integer command; underneath, it uses the integer divide hardware of a processor. Since SB is probably trying to make things easier on us, it tells us that the number is "too big" to use modulus on rather than telling us the type in inconsistent (although if you look at the above thread, you'll see that this too is inconsistent, as you can apparently OR float values with a wonky undocumented conversion). FLOOR, subtraction, division, and multiplication in the second one are all float operations, so there's no overflow.

Trying to make a pseudo-random number generator? This is probably part of the weirdness that Square fingers found over here: https://smilebasicsource.com/forum?ftid=104 Basically, MOD is an integer command; underneath, it uses the integer divide hardware of a processor. Since SB is probably trying to make things easier on us, it tells us that the number is "too big" to use modulus on rather than telling us the type in inconsistent (although if you look at the above thread, you'll see that this too is inconsistent, as you can apparently OR float values with a wonky undocumented conversion). FLOOR, subtraction, division, and multiplication in the second one are all float operations, so there's no overflow.
Actually I was trying to decompress an unsigned 16-bit integer array that I compressed as a signed 32-bit integer array. That is strange. I just assumed mod would work on floating point since the other operators do. Is there a floating-point equivalent for the MOD operator?

In short, no. In long, C's modf or whatever it's called in the standard library is insanely easy to port as a function.

If you're packing 16 bit numbers into integers, you should do something like:
INT%=HIGH%<<16 OR LOW%
HIGH%=INT>>16
LOW%=INT AND &HFFFF