Effectively, &H literal values have 'type' integer.Thank you SquareFingers, I guess I should have realized that would represent the integer type. But how would I test that I really can get double precision? If I do something like this:
VAR T# = 0.333333333333333 PRINT STR$(T#)I get "0.333333", which is much fewer digits of precision than I'd expect. Shouldn't I get closer to 16 or did I do something wrong in that example? EDIT: I also tried using FORMAT$(), to make sure it was printing the appropriate type and got the same result
A#=1E15 B#=A#+1 ?B#==A#It will show 0, or FALSE. If you try with 1E16, the system will tell you they are equal. EDIT: Also try
A#=1/3
?FORMAT$("%1.20F",A#)I think this will show the kind of thing you're looking for.
STR$ has limited precision and is also evil.While STR$ does have a limited precision, it does all of the proper truncating for you when returning the string, while with a format string you have to do all of that yourself. For example, STR$(.1) returns "0.1", but FORMAT$("%F",.1) returns "0.100000". STR$ basically exists for convenience.
EDIT: Also try
A#=1/3
?FORMAT$("%1.20F",A#)I think this will show the kind of thing you're looking for.I don't mean to be going around being Mr. Corrector, but your format code isn't entirely correct. The leading number (the one directly after the %) is the total length of that formatted value, not the number of integer digits. So %24.22F would produce the same result. Just wanted to clarify that.your format code isn't entirely correct.Thanks for the better information. I'll try and remember it.
Ah, yep, that worked. Thank you. One last question, (and sorry for hijacking this thread), if I want to do bitmasking for 64 bit types is that possible? I've been using &H as a mask to isolate individual bytes, but if that is limited to 32 bit types, then how would I mask anything beyond 32 bit? It appears &B is also integer type as it can't go beyond 32bits either. EDIT: Ah, slackerSnail already mentioned about the &B type, I'm too slow to post :PWhen you use bitwise operators, the values are implicitly turned into integers if possible. Any fractional portion is thrown out and the operations only work on the integer portion, treated as if it were a SB integer. Numbers outside of integer range get trimmed down to fit. For example, POW(2,32) AND POW(2,32) just returns 231-1.