LoginLogin
Nintendo shutting down 3DS + Wii U online services, see our post

Bizarre modulo (MOD()) problems

Root / Programming Questions / [.]

SP_SourCreated:
I'm creating a program that formats floats into uniform strings (i.e. makes sure 5, 88.8, 100.25, and 3.7 can all be displayed in ###.## format) and partially do this by performing modulo operations to check if the numbers in question have anything in the tenths or hundredths places. Here is the specific code I'm using:
DEF DISP(I)
 I$=STR$(I)
 IF ((100*I) MOD 100)==0 THEN I$=I$+".0"
 IF ((100*I) MOD 10)==0 THEN I$=I$+"0"
 '(more code not shown)
RETURN I$
Basically: if you want to check if 6.76 has any decimals you see if 676 can evenly be divided by 100, then if if can be evenly divided by 10. (There are also checks for whether there is anything in the whole 10s or 100s places, but there haven't been any problems with these so far.) DISP(5.55) returns "005.55", 3.5 returns "003.50", 103 returns "103.00", etc. However I ran into a strange problem. Just to calibrate the function, I tried running it on a variety of numbers which happened to include 67.1. Instead of giving me "067.10" like it should, instead I got back "067.1" I tried to figure out where it was going wrong and this happened.
?((67.1*100) MOD 10)
9
OK
?(67.1*100)
6710
OK
?(6710 MOD 10)
0
OK
Yeah, uh, what? So for some reason when I do those things at the same time it suddenly makes 9 instead of 0? I even tried setting a variable =I*100 then using the variable instead of the direct calculation (i.e. Y=I*100 [...] Y MOD 10) but that didn't do anything either. Through trial and error, I discovered that just a weirdly specific range of numbers has this problem: any number between 64 and 81 inclusive that ends in .1 is affected. So (X*100) MOD 10, where 64.1<=X<=81.1 and X ends in .1, always returns 9 instead of 0. I did some relatively light searching\browsing of the forums and the problem may lie in how apparently MOD is an integer function, which is why just using X MOD 1 to check for decimals seems to not work either. But if that's the case, why does doing the multiplication and the modulo in the same line make it mess up? And why only that specific range of numbers, and why just .1? I realize that there are probably other ways of checking for decimals, such as using FLOOR and then checking for equivalence, but this is really bothering me. Sorry for the super-long post, but does anyone know what's going on here?

67.1 can't be represented exactly as a 64 bit floating point number; it's actually stored as 67.099999999999994. Multiplying that by 100 gives 6709.9999999999991, which is rounded down to 6709 when it's converted to an integer by MOD.

not related to float precision issues but don't you just want
I$ = FORMAT$("%03.2F",I)
Edit: float format doesn't respect places before the radix point, but it does solve one side
FORMAT$("%06S",FORMAT$("%3.2F",I))

not related to float precision issues but don't you just want
I$ = FORMAT$("%03.2F",I)
Edit: float format doesn't respect places before the radix point, but it does solve one side
FORMAT$("%06S",FORMAT$("%3.2F",I))
The first number is the total length of the inserted value:
I$ = FORMAT$("%06.2F",I)
FORMAT$ guide This should do what you want.

Thanks so much for the input guys. I did end up creating a working system but I had no idea the FORMAT() command existed. Works even better! That's interesting about floats, too. (All I know about them is I've gotten from old pannenkoek videos, so it was news to me!)