LoginLogin
Nintendo shutting down 3DS + Wii U online services, see our post

Float-to-string and back

Root / General / [.]

SquareFingersCreated:
SquareFingers#1
I found I wanted to convert a floating-point value to a string, and back, without any loss of precision, if possible. This is what I came up with. 
OPTION STRICT

DEF NTOS$(N#)
 VAR S$=STR$(N#)
 VAR S0$="+",S1$="+",E%=0,NN#,N1%,N2%
 IF S$[0]=="-" THEN
  S0$="-"
  S$=RIGHT$(S$,LEN(S$)-1)
  N#=-N#
 ENDIF
 IF N#==0 THEN
  RETURN S0$+"0"
 ELSEIF INSTR("in",S$[0])>=0 THEN
  RETURN S0$+S$[0]
 ENDIF
 IF N#<1
  S1$="-"
  NN#=N#*256
  WHILE NN#<2
   INC E%,8
   N#=NN#
   NN#=N#*256
  WEND
  NN#=N#*2
  WHILE NN#<2
   INC E%
   N#=NN#
   NN#=N#*2
  WEND
  DEC N#
  N#=N#*POW(2,28)
  N1%=FLOOR(N#)
  DEC N#,N1%
  N2%=N#*POW(2,24)
  RETURN S0$+S1$+FORMAT$("%03X%07X%06X",E%,N1%,N2%)
 ELSE
  NN#=N#/256
  WHILE NN#>=1
   INC E%,8
   N#=NN#
   NN#=N#/256
  WEND
  NN#=N#/2
  WHILE NN#>=1
   INC E%
   N#=NN#
   NN#=N#/2
  WEND
  DEC N#
  N#=N#*POW(2,28)
  N1%=FLOOR(N#)
  DEC N#,N1%
  N2%=N#*POW(2,24)
  RETURN S0$+S1$+FORMAT$("%03X%07X%06X",E%,N1%,N2%)
 ENDIF
END

DEF NYBBLE#(N$)
 RETURN ASC(N$)-48-7*(ASC(N$)>=58)
END

DEF STON#(S$)
 VAR S0$=S$[0],RESULT#=0,I%,E%
 S$=RIGHT$(S$,LEN(S$)-1)
 IF S%=="0" GOTO @ZERO
 ON INSTR("in",S$[0]) GOTO @INF,@NAN
@NUM
 RESULT#=1
 FOR I%=4 TO 16
  RESULT#=RESULT#*16+NYBBLE#(S$[I%])
 NEXT I%
 E%=NYBBLE#(S$[1])*256+NYBBLE#(S$[2])*16+NYBBLE#(S$[3])
 IF S$[0]=="-" THEN E%=-E%
 RESULT#=RESULT#/POW(2,52)
 RESULT#=RESULT#*POW(2,E%)
 GOTO @END
@INF
 RESULT#=1E200*1E200
 GOTO @END
@NAN
 RESULT#=1E200*1E200
 DEC RESULT#,RESULT#
 GOTO @END
@ZERO
@END
 IF S0$=="-" THEN RESULT#=-RESULT#
 RETURN RESULT#
END
Some edge cases I have tested with: 
VAR INF#=1E200*1E200 ' infinity
VAR INF_N#=-INF# ' negative infinity
VAR NAN#=INF#-INF# ' not-a-number
VAR NAN_N#=-NAN# ' negative not-a-number
VAR E#=POW(2,-1022) ' smallest positive number that can be represented
VAR O#=1+POW(2,-52) ' smallest number greater than 1 that can be represented
VAR E1#=E#*O# ' smallest number greater than E# that can be represented
VAR T#=2-POW(2,-52) ' greatest number less than 2 that can be represented
VAR M2#=POW(2,1023) ' greatest power of 2 that can be represented
VAR M#=M2#*T# ' greatest number that can be represented
VAR Z#=0 ' zero
VAR Z_N#=-Z# ' negative zero
It's not high-quality code. I may come back and make a neater version. If anyone discovers a value that turns out different after being passed through NTOS$ then STON#, please let me know. NAN does not 'equal' itself, for any other floating point value V# my hope is that STON#(NTOS$(V#))==V# - let me know if you find a counterexample.
haloopdy#2
Dang, that's a lot of code
SquareFingers#3
Well, there's a lot of code in SmileBasic, specifically to hide details of the internal representation of floats. Makes sense that it would take a lot of code to undo that. - But, in fairness, yes, it is longer than it could be. Length of code was a lower priority for me than other characteristics the code should have.
snail_#4
That's an impressive amount of code, yes. Though, I was of the impression that a double only had about 17 fractional digits, and that something like FORMAT$("%.17F",NUM#) was enough to represent it properly in a string. Of course that won't handle inf and nan, but CLASSIFY can catch that for you. If you could explain why this isn't enough I'd like to know, doubles have a lot going on.
SquareFingers#5
With E# as above, FORMAT$("%.17F",E#) gives 0.00000000000000000, the same result as if you give it 0, but E#==0 is false.
snail_#6
With E# as above, FORMAT$("%.17F",E#) gives 0.00000000000000000, the same result as if you give it 0, but E#==0 is false.
Oh, I see, yeah. That's an issue. Would E# be a denormal number? I could certainly see that as being a potential issue with the formatter.
SquareFingers#7
With E# as above, FORMAT$("%.17F",E#) gives 0.00000000000000000, the same result as if you give it 0, but E#==0 is false.
Oh, I see, yeah. That's an issue. Would E# be a denormal number? I could certainly see that as being a potential issue with the formatter.
No, E# is not a denormal number... I tried generating a denormal number with (E#*T#)-E#, but it turns out the result of this calculation is ==0. E# has the same string representation as 0 because it is less than 0.000000000000000005. Perhaps FORMAT$("%.324F",E#) would work, but that's a long string. EDIT: FORMAT$("%.324F",VALUE#) does work, but VAL does not reverse the transformation - for E#, the result is 0.