_ArrayAdd + MultiDimensional Array..??

[cypher175]

what am I suppose to use for adding data into multiple columns/rows for an Array..?? since _ArrayAdd only works for 1 dimensional arrays..??

[Malkey]

There is an _Array2D_Add () function in the Array2D.au3 UDF file found here.

[jchd]

That will need a ReDim somewhere, likely to add only one more entry at a time. If you're going to add up to the million rows, then that will sum up to as much ReDim. You should probably allocate new entries by chunks of 64, 256 or 1024 (I just love 2).

BTW, my answer to the other post still holds and further begs a question: will you search in this array? How?

[cypher175]

ok got the redim part down..

no i will not be searching this array, i just need to be able to sort column 0 descendingly and be able to export the array to like a spreadsheet file or something every so often..

[czardas]

This seems less straight forward than I thought it would be. I haven't looked at the UDF, but I have thought about a possible solution. It's a bit long winded though. Create the new entries in a second array. Then when you are satisfied with the new entries, create a third array large enough to hold all the entries. Voila!

For Example:

#include <Array.au3>

Local $a1[2][2], $a2[2][2]

$a1[0][0] = 1
$a1[0][1] = "a"
$a1[1][0] = 2
$a1[1][1] = "b"

$a2[0][0] = 3
$a2[0][1] = "c"
$a2[1][0] = 4
$a2[1][1] = "d"

Local $a3[Ubound($a1) + UBound($a2)][2]

For $i = 0 To UBound($a1) -1
    $a3[$i][0] = $a1[$i][0]
    $a3[$i][1] = $a1[$i][1]
Next

$j = 0
For $i = Ubound($a1) To Ubound($a3) -1
    $a3[$i][0] = $a2[$j][0]
    $a3[$i][1] = $a2[$j][1]
    $j += 1
Next
;_ArrayDisplay($a3)

$a1 = $a3
_ArrayDisplay($a1)

If you use this method, make sure you make the second array large enough to hold all the new entries. Count the number of entries as you add them, to determine the size of the third array.

Edited April 13, 2010 by czardas

[jchd]

Yes of course that's possible!

#include <Array.au3>

Local $a[5][3]
$a[0][0] = "abc"
$a[4][2] = "def"
_ArrayDisplay($a)

ReDim $a[6][3]
$a[5][1] = "ghi"
_ArrayDisplay($a)

But it doesn't make much sense to me: what happens at a power outage, can you recreate the data easily?

@czardas: PLEASE!

Edited April 13, 2010 by jchd

[czardas]

Yes of course that's possible!

#include <Array.au3>

Local $a[5][3]
$a[0][0] = "abc"
$a[4][2] = "def"
_ArrayDisplay($a)

ReDim $a[6][3]
$a[5][1] = "ghi"
_ArrayDisplay($a)

But it doesn't make much sense to me: what happens at a power outage, can you recreate the data easily?

@czardas: PLEASE!

Sorry JCHD, I'm not sure what power outages have to do with it. If there's a more reliable way to do this then okay. Perhaps I missed something? Chances are you are going to lose data at a power outage anyway? I have not used Redim before, so I'll check it out. I think your method is better. Thanks for the tip. Edited April 13, 2010 by czardas

[jchd]

Accumulating millions of entries is likely to take some time, in particular when the OP talks about dumping the array to a speadsheet "every so often". That evokes some persistence to me and I naturally ask about data re-creation in case of power outage or system crash à la BSOD. Memory "storage" isn't really storage...(*)

I continue to believe that an SQLite DB would do it more nicely, but that also depend on the rate of inserts and/or updates and the expected size of the cells.

About your solution: it's a terrible idea. Where did you get it?

* Not everybody can get it: I mean "now".(**)

** For those who still don't get it, memory storage has been persistent at some time in IT history and ... well ... I've been there!

Edited April 13, 2010 by jchd

[czardas]

Accumulating millions of entries is likely to take some time, in particular when the OP talks about dumping the array to a speadsheet "every so often". That evokes some persistence to me and I naturally ask about data re-creation in case of power outage or system crash à la BSOD. Memory "storage" isn't really storage...

I continue to believe that an SQLite DB would do it more nicely, but that also depend on the rate of inserts and/or updates and the expected size of the cells.

About your solution: it's a terrible idea. Where did you get it?

My solution is a terrible idea! Come on. Cap the number of new entries and update as you go. The solution I presented has some advantages. The Redim method may have some disadvantages. Considering I didn't know about Redim my solution is damn good. Where did I get it? Product of a fertile imagination.

Besides: the OP never mentioned anything about millions of entries.

Edited April 13, 2010 by czardas

[jchd]

My solution is a terrible idea! Come on. Cap the number of new entries and update as you go. The solution I presented has some advantages.

Allocate a 1Kb fixed-size file on a 2Tb harddisk. When the file is populated, allocated another 1Kb fixed-size file on another 2Tb harddisk. When the second file is fully populated, allocate a fixed-size 2Kb file onto a third harddisk where you can merge your two 1K files.

I admit I'm a bit making it up, but I'm sorry but it was not a good idea at all.

Besides: the OP never mentioned anything about millions of entries.

Sure!

[czardas]

Okay, it's about 5.30 in the morning and the forum is about to be backed up in one hit. Lets hope there isn't a power outage. Maybe it's time for me to sleep. I'll look at the other post tomorrow. Good night.

[czardas]

Ok I got a couple of hours shut eye, and now have I looked at the other thread. To add to JCHD's earlier comments, an excel spread sheet holds a maximum of 2^16 rows. That's no where near a million. SQLite sounds good, I think I'll give it a try.

One final thought on this: I am not at all convinced that the method I came up with is so bad. You either keep updating the main array, one entry at a time, or you do it in chunks. The methods are simply different, not better or worse. One of the advantages to updating in chunks is that you do not need to hold all the information in RAM while you add new entries. You only need to do that when you update the main data dump, be it a spread sheet or whatever. And with Redim you only need to use two arrays (or if you're clever only one array) instead of three. I didn't know that yesterday.

Edited April 13, 2010 by czardas

[jchd]

ReDim reallocates the array. It doesn't have to copy the data anywhere. Only one copy ever exists. No copy needed.

Sorry to point that out: your approach is bad in that it needs twice as much array writes and twice as much memory compared to ReDim.

And, as I already noted, ReDim should be made in chunks, not one entry at a time.

SQLite offers large storage (up to 500Gb per database), instant sorting using indexing, proven reliability, ACID transactions, live backups, superfast queries, ...

Excel 2007 allows for 2^14 columns and 2^20 rows.

[czardas]

Well this is an enlightening thread. Thanks to your explanaitions JCHD. I have learned something here.

[jchd]

No problem, you're welcome!