Proper way of incrementing an array's size by 1

[jchd]

Correct, and doubling can very quickly hit the limit of 16 *1024*1024 array entries.

[Bowmore]

An array with 524288 elements (all empty) would take 3.4 MB of RAM

​That's is not correct AutoIt does not allocate storage for array elements until they actually contain data.

[TheDcoder]

@Bowmore Try it:

Local $aArray[524288]

MsgBox(0, "Script Paused", "Open Task Manager & Have A Look At Its RAM Usage")

TD

[BrewManNH]

An array with 524288 elements (all empty) would take 3.4 MB of RAM

​The point of doubling the array to add more elements is because you're putting something into the array, they would never be empty until you doubled the array. The first half should always have something in it, otherwise there's no point in expanding the array.

[water]

TD,
until you do not have a rough estimate about how many elements the array will hold at maximum, it is useless to discuss if factor 1.5 or 2 or increasing the size by a fixed number best fits your needs.
Until you do not know how important performance is for the problem you try to solve it is useless to ...

First goes the planning, then the coding!

[mikell]

.. I need to expand an array only ~10 times

... and as the title tells about  "incrementing an array's size by 1", this should be enough :

Redim $array[UBound($array)+10]

[TheDcoder]

@mikell I said ~10 not exactly 10

[AspirinJunkie]

Correct, and doubling can very quickly hit the limit of 16 *1024*1024 array entries.

​Why? If it hits the limit you filled at least 8.388.608 elements. If you don't use the index 0 as the counter you can even fill the maximum number of elements inside such a array because 16.777.216 is a power of 2. If you increase by 1.5x the maximum value of elements is limited to 11.958.656.

So you only hit the limit if you put a corresponding number of elements. The limit isn't reached faster - you only need lesser ReDims until the limit is reached.

btw.: i made a little UDF which builds on the concept of doubling/halfing the array size if the limit is reached. I putted it as an attachment.

DynArray.zip

[jchd]

Well, I never intended to start a flame war about this.

But at this rate, why not allocate at once the maximum size head first and finally ReDim it to truncate to the actual used size?

If you don't use the index 0 as the counter you can even fill the maximum number of elements inside such a array because 16.777.216 is a power of 2.

Only if you started at a power of 2.

If you increase by 1.5x the maximum value of elements is limited to 11.958.656.

Why so? Min() can do that.

Anyway, things aren't that simple when the array has more than one dimension.

[AspirinJunkie]

Well, I never intended to start a flame war about this.

​I also not. It was meant as clarification that the total array size limit  doesn't depend on which increasing factor is chosen.

Why so? Min() can do that.

​Sure - that's why i not understood why you said that doubling will reach the array size faster than with a factor of 1.5x. With both factors you can hold up to 16.777.216 values inside the array. Only the number of redims and the size of the unused space differs - not more. The maximum count of array elements doesn't really influence the choice of the increasing factor. That clarification was my intention here.

Maybe i misunderstood you - maybe you misunderstood me. I never wanted to start a flame war - sorry if it feels like it for you.

[czardas]

I think 2x is too much and 1.5 times not enough. I reckon 1.61803398875 times to be about perfect. JOKE!

After all is said and done, doubling hits the limit on smaller input. Underestimating an array's size by a factor of 2 can happen, but it's generally an exception. Something like that happens when you need to catalogue a large, and unknown, number of items. In this case it makes sense to pump up the size in larger leaps. Doubling should not be used as a rule of thumb and nobody said it should. I wanted to make this a little clearer.

[jchd]

The flame war remark was a joke, nothing more.

why you said that doubling will reach the array size faster than with a factor of 1.5x

Because in average use you may want to keep anticipated memory allocation relatively small as a conservative measure and only in rare instances you need to reach a very large number of entries. Expansion factor in many libraries are more in the range 1.2 - 1.5 just to balance number of reallocations and size growth rate. Of course both the initial size and the "optimal" growth factor heavily depend on how well you can estimate the order of magnitude of the final number of entries.

[jchd]

The flame war remark was a joke, nothing more.

why you said that doubling will reach the array size faster than with a factor of 1.5x

Because in average use you may want to keep anticipated memory allocation relatively small as a conservative measure and only in rare instances you need to reach a very large number of entries. Expansion factor in many libraries are more in the range 1.2 - 1.5 just to balance number of reallocations and size growth rate. Of course both the initial size and the "optimal" growth factor heavily depend on how well you can estimate the order of magnitude of the final number of entries.

[czardas]

jchd, the forum is doubling your input.

Edited June 1, 2015 by czardas

[jchd]

I saw that but there is no "delete post" feature. It wasn't to make my point more significant!

There are issues while you have an answer awaiting send and when another post shows up. Not only the post may appear twice but you're unable to go back to where you came from: the history loops on the page.

[czardas]

Oh but it does illustrate your point quite nicely.

[AspirinJunkie]

It wasn't to make my point more significant!​

Because in average use you may want to keep anticipated memory allocation relatively small as a conservative measure and only in rare instances you need to reach a very large number of entries.

Exactly. Now you should understand my question why the maximum number of array elements for a AutoIt array should influence the choise of the growth factor.

I think 2x is too much and 1.5 times not enough.

​I didn't say that 2x or any other factor is the best choice or better than another factor. My point was to scrutinise the argumentation with the maximum array size because it has no real influence. The choice rather depends - as you both already said - on the expectable number of to adding elements and the balancing of avoiding as much redims as possible vs. wasted unused memory amount.

We are in complete agreement - maybe my bad english is the reason for the misunderstandings here - sorry for that.

[czardas]

Now you should understand my question why the maximum number of array elements for a AutoIt array should influence the choise of the growth factor.

Using a smaller increment allows you to work with larger arrays without going beyond the limit imposed by AutoIt. It's nothing tricky or complicated.

[AspirinJunkie]

As jchd already mentioned: The factor doesn't influence the maximum number of elements you can store in the array.

Realise the size increasing this way:

ReDim $Array[(UBound($Array) * $factor >= 16777216 ? 16777216 : UBound($Array) * $factor)]

As you can see you will reach the maximum number of 16.777.216 elements always with every growth factor > 1.0.

[TheDcoder]

@czardas Congratulations on your 7,800th post

The best thing to do when you need to ReDim an array is:

 

TD