Dynamic multidimensional arrays

I've been trying to think of the right way to represent data for an MML editor I'm working on. I need to be able to encapsulate each note as a position, duration, and note value. Furthermore each channel can have an arbitrary number of notes, and there are 16 different channels. Now, in order to organize actual songs, the 16 channels, with their arbitrary number of notes would define a pattern, and user could create and order any number of unique patterns. If you are familiar with a standard music tracker, you've probably seen this type of structure. So in theory that all makes sense, but I'm finding actually representing that kind of data dynamically as troublesome. Representing this as a multidimensional array might look like this: Song[PATTERN#][CHANNEL#][NOTE#][3] PATTERN# and NOTE# would need to be redim'able. As far as I'm aware, that's not possible in SB. Is there a different way to represent this data that allows for the flexibility of a multidimensional array that can be resized? Thanks!

One way is to use a string, and some characters to indicate "end of element/row" for various levels of 'row', e.g.

32 43 88/54 23:77 22 84/55/72/198 43:55/62#98

This has four 'end of element/row' characters (including space), so is four-dimensional. The top-level split is at the #, and there is one of them, so the 'dimension' at that level is two. The first of the two is subdivided by two :, so had a 'dimension' of three, the second of the two has a 'dimension' of 1.

Thanks, SquareFingers, I think I understand the gist of this. I'm a little confused about the dimensional representation, but let me stew on it a bit. Using this method, I'd tokenize the string to retrieve the info I needed, edit the data some, and rebuild the string again. So I'm concerned with how this solution would scale, but I suppose I could just remove a dimension, and then just have my top dimension be an array of strings. So each pattern could be a string. That might help keep it maintainable. VAR SONG$[1] SONG$[0] = "32 43 88/54 23:77 22 84/55/72/198 43:55/62" ' Pattern 0's data PUSH SONG$, "" ' Adding another pattern SONG$[1] = "40 10 12/69 1:40 14 91/36/10/220 12:40/60" ' Pattern 1's data Where each index in the string array is a pattern. Let me know if I've misunderstood you. Thanks for you help SquareFingers.

Just to chime in with an FYI: The problem with multidimensional arrays is that they are actually flat in memory. To explain: DIM A[4,4] is equivalent to B[16] except for how SmileBasic lets you access it. So hypothetically if you could push to A[3] , SB would have to push everything after B[12] over 1. Now anything at A[3,4] is at A[4,0]. Thats not good... EDIT: You could use a program that manages PRGINS for you to create new arrays for each pattern... I wonder if someones made one of those...

PUSH only works on one-dimensional arrays (and strings).

kldck_hul: Thanks for the clarification. PRGINS is an interesting idea for dynamically instantiating arrays. Would that work though? Can you use PRGINS to write to another slot, and then reload that program and access the data without blowing away data in the currently existing patterns. Is it possible to reference a variable by name, in string form? Like:

VAR("Pattern"+STR$(pnum))

. I think that would be necessary. Now I've got to think about this a bit. SquareFingers: Is a string considered a 1d array? Now I feel foolish. EDIT: fixed code, readable

SquareFingers: Is a string considered a 1d array? Now I feel foolish.

Nothing to feel foolish about. A string is very much like a 1D array. The first character of a string variable can be retrieved with S$[0], and the PUSH, POP, SHIFT and UNSHIFT commands all work on strings. In other ways, not so much. If you say S$="01234567", then S$[5] will contain the value "5". Assigning to any other index, in a regular array, would not affect S$[5]. But, if you were to execute S$[0]="", then S$[5] becomes "6". If you were to execute S$[4]="XYZ", then S$[5] becomes "Y". EDIT:

Is it possible to reference a variable by name, in string form?

If you had two particular characters that are not permitted in variable names or values, say ^ and &, you could make a string like "^VarName1&77^VarName2&90^". You can use INSTR with the variable name (surrounded by ^ and &) to find in the string where the corresponding value is, and everything up to the next ^ is the value. EDIT 2:

Can you use PRGINS to write to another slot ... and access the data

Variable values in one program slot are not accessible from another slot, except in so far as SHARED DEF is accessible from another slot, and within a DEF block, all variable names other than the parameters belong to the same slot as the DEF. EDIT: and by SHARED, of course, I mean COMMON - thanks, slackerSnail.

It's alright, my embarrassment is always overshadowed by the joy of learning new things :) Thanks for such a detailed explanation, SquareFingers.

Variable values in one program slot are not accessible from another slot, except in so far as SHARED DEF is accessible from another slot, and within a DEF block, all variable names other than the parameters belong to the same slot as the DEF.

There is indeed a VAR() function that can return the value of a variable given it's name in a string, but it's not documented. If you include a slot number in the variable name, you can get it's value like so: VAR("1:VARNAME"). How this managed to remain undocumented is baffling to me. Also, don't you mean COMMON DEF?

There is indeed a VAL() function that can return the value of a variable given it's name in a string, but it's not documented.

Ahh, this was what I meant when in my example, but I used VAR instead of VAL (>_<) Something like this instead:

pnum = 0
Pattern0 = "Some string"
a = VAL("Pattern"+STR$(pnum))

So variable "a" would contain the contents of variable "Pattern0", "Some string". Thanks slackerSnail, this was what I meant. EDIT: Arggg, did it again, VAL

Ahh, I meant VAR! Not VAL, that was a typo. Please take note, I'll edit my post.

slackerSnail, I think you were right the first time! Unless I'm just super confused The documentation for VAL says: Arguments: A character string representing a number (e.g., "123"), or a string variable So VAL returns a numerical representation of a string, or if the string contains a variable name, the contents of the variable.

slackerSnail, I think you were right the first time! Unless I'm just super confused The documentation for VAL says: Arguments: A character string representing a number (e.g., "123"), or a string variable So VAL returns a numerical representation of a string, or if the string contains a variable name, the contents of the variable.

No, VAL only exists to convert a string of numbers into a numerical value. VAR obtains a variable's value given it's name.

Ah, you are right, I found the post concerning this behavior. The VAR command/function And it looks like VAR() goes both ways, can set and get values. That's really useful.

kldck_hul: Like:

VAR("Pattern"+STR$(pnum))

. I think that would be necessary. Now I've got to think about this a bit.

Here are a couple of functions which I use to read and write arrays to prog slots. I created them because I wanted to save any number of arrays at runtime, and didn't want the SAVE command's dialog prompt to appear. Firstly a little helper function, ARR2STR:

COMMON DEF ARR2STR(A)
VAR O$="" 'OUTPUT STRING
VAR I%=0 'COUNTER
'USING PUSH() IS MUCH FASTER THAN APPENDING TO STRING
FOR I%=0 TO LEN(A)-1
 PUSH O$,STR$(A[I%])+","
NEXT
'STRING TRAILING COMMA AND RETURN:
RETURN LEFT$(T$,LEN(T$)-1)
END

And here are the two functions:

'COPY ARRAY A TO PRG SLOT P%
'AS DATA LINES WITH LABEL L$
COMMON DEF ARR2PRG A,P%,L$
DIM S$ 'STRING USED TO SEARCH PRG
IF P%>=0 AND P%<4 THEN 
 USE P%
 IF CHKLABEL(STR$(P%)+":@"+L$) THEN 
  'L$ EXISTS, FIND WHERE TO OVERWRITE:
  PRGEDIT P%,1
  REPEAT
   S$=PRGGET()
  UNTIL "@"+L$+CHR$(10)==S$ OR ""==S$
  'NOW HAVE THE INSERTION POINT.
  'EMPTY STRING CONDITION SHOULD NEVER MATCH
 ELSE
  PRGEDIT P%,-1 'START INSERTION AT EOF
 ENDIF
 PRGSET "@"+L$
 PRGSET "DATA "+STR$(LEN(A))+" 'ARRAY LENGTH"
 PRGSET "DATA "+ARR2STR(A)
ELSE
 ?"ARR2PRG(ARRAY,PRG%,LABEL$)"
 ?"* PRG% MUST BE 0,1,2, OR 3"
ENDIF
END

COMMON DEF PRG2ARR(P%,L$)
DIM L%=0 'LENGTH
DIM T 'TEMP VARIABLE
DIM O[0] 'OUTPUT ARRAY
IF P%>=0 AND P<4 THEN 
 IF P%>0 THEN
  USE P%
 ENDIF
 L$=STR$(P%)+":@"+L$
 IF CHKLABEL(L$) THEN
  RESTORE L$
  READ L%
  WHILE L%>0
   READ T
   PUSH O,T
   DEC L%
  WEND
  RETURN O
 ELSE
  ?"ERROR: '"+L$+"' NOT FOUND"
 ENDIF
ELSE
 ?"PRG2ARR(PRG%,LABEL$) RETURNS ARRAY"
 ?"* PRG% MUST BE 0,1,2, OR 3"
ENDIF
END

Hope this helps. Bear in mind these functions only work with number types. You'll need to tweak them for strings. EDIT: just to follow up: these functions only work with 1D arrays. Also, large arrays (eg. the output of GSAVE()) will probably use too much memory and fail. I have a version of ARR2STR which uses COPY to copy the input to a temporary array first, which has the side-effect of flattening it if it's a multi-dimensional array. The main problem is that there's no way to determine the width of a multidimensional array, so you need to store that extra bit of info when saving and restoring. It just ends up being a pain, and easier to use multiple 1D arrays than to mess with 2D or higher...

I've been trying to think of the right way to represent data for an MML editor I'm working on. I need to be able to encapsulate each note as a position, duration, and note value. Furthermore each channel can have an arbitrary number of notes, and there are 16 different channels. Now, in order to organize actual songs, the 16 channels, with their arbitrary number of notes would define a pattern, and user could create and order any number of unique patterns. If you are familiar with a standard music tracker, you've probably seen this type of structure. So in theory that all makes sense, but I'm finding actually representing that kind of data dynamically as troublesome. Representing this as a multidimensional array might look like this: Song[PATTERN#][CHANNEL#][NOTE#][3] PATTERN# and NOTE# would need to be redim'able. As far as I'm aware, that's not possible in SB. Is there a different way to represent this data that allows for the flexibility of a multidimensional array that can be resized? Thanks!

Sorry to double post, but this is separate to my last post and related to your OP. Recently I've been playing with using the RGB() and RGBREAD functions to pack and unpack four bytes of data. I found that using RGB(a,b,c,d) is a lot faster than using four bitshift operations to get the same result. Similarly RGBREAD is a lot faster in reverse than using bitshifts. (I've been using this method to try and make a fast 3D engine and storing 3D and 4D Vectors in this structure, and combining them into arrays of length 3 or 4 to encapsulate matrices.) The problem is that you can only store four values of 0-255, so it's not suitable for every purpose. It could be suited to yours though, let's say your notes have these properties: value, position, duration all map to the range 0-255. Channel is already defined as being 0-15. So you could try: RGB(chan%,pos%,dur%,val%) to encode a note val% of duration dur% at position pos% belonging to channel chan%. This will be a number between &H00000000 and &H10FFFFFF. To get those values back out just use RGBREAD variable OUT chan%,pos%,dur%,val%. You can then store these 16-bit numbers in a pattern[id,notes] array, and your song array should then just be a list of IDs in your pattern array. I think that'll work, not 100% sure though...

ezman, man, you're reading my mind. I was just thinking about stuffing data under a label in a prog slot, and yesterday I was considering consolidating all my note information in a single variable and then just masking out the relevant bits for duration, length, and value. But I never thought about using the RGB built ins to do the heavy lifting, that's ingenious. I'm definitely going to try this. Plus it's economic for memory. slackerSnail, My exploration into VAR was confusing as it behaves strangely:

VAR("TMP")=2

(this errors on the VAR line)

VAR("TMP")=2
PRINT TMP

(but printing the variable makes it not error...?)

OPTION STRICT
VAR("TMP")=2
PRINT TMP

(this errors unless I first DIM the variable) And doing something like this to construct, as needed, variables is out of the question:

VAR I=2
VAR("TMP_"+STR$(I))=10
PRINT VAR("TMP_"+STR$(I)) ' this doesn't work
PRINT VAR("TMP_2")              ' this also doesn't work
PRINT TMP_2                           ' this works, but defeats the purpose

(and of course with OPTION STRICT, none of the above is applicable because I'd need to DIM first which means, I'd needed to explicitly state the name of the variable, which is what I was trying to get around) Probably the above is a no brainer, but I found it interesting. Anyway, there's a lot of great info in this thread, I feel like I can take a crack at it now. Thanks everyone!

VAR("TMP")=2

(this errors on the VAR line)

VAR("TMP")=2
PRINT TMP

(but printing the variable makes it not error...?)

OPTION STRICT
VAR("TMP")=2
PRINT TMP

(this errors unless I first DIM the variable)

If there is no OPTION STRICT, the system will examine the whole program, looking for variable names. When it finds one, it will add it to the list of recognized variables. It does all this before even executing the first line. In the first piece of code, TMP does not get added to the list, in the second, it does.

Wow, never in my life would I have expected RGB() and RGBREAD to be faster than a straight binary op. You'd imagine the call overhead would offset that quite a lot, since binary is pretty much the fastest thing a CPU can do. That value packing concept is brilliant, by the way. I never would've thought of that. Now, if only we knew how to pack 64 bits into a single real number...

SquareFingers, this makes sense now. I guess it would be weird if variables could be ambiguous. So, this sounds a bit like a preprocessor , but SmileBASIC isn't compiled is it? Or maybe it's just super fast, but I always imagined it just ran through a VM or something. Do you know how that's handled?