Beta Array.au3 code - testing and comments welcomed

[Melba23]

Deadline of 1800 UTC 28 Jan 14 for bug reports, comments and suggestions

Thread closed - final Beta code here

I have rewritten the Array.au3 include file to support 2D arrays where possible. I would be grateful if people would test it rigourously as I never seem to find all the things that can go wrong.

Note: You need the Beta 3.3.11.3 to run this code - it is Beta itself after all.

Current public functions and any changes made:

expandcollapse popup

_ArrayAdd:  Now supports 2D arrays.  Data for addition to 2D arrays can be a single value, a delimited string or another 1D array.  The delimiter and the column in which the insertion is to start can be user-defined.

_ArrayBinarySearch:  Now supports 2D arrays searching in columns.

_ArrayColDelete: New function to delete a column from an array.  Optional to convert to 1D if only 1 column left.

_ArrayColInsert:  New function to insert a column into an array.  Converts 1D to 2D automatically.

_ArrayCombinations: Unchanged

_ArrayConcatenate: Will concatenate 2D arrays with the same number of columns.

_ArrayDelete: Now supports 2D arrays.  Will also delete multiple rows if passed either a range string detailing the rows ("3:7-9:15-25;56") or a 1D array of all the rows to be deleted with a count in the [0] element.

_ArrayDisplay:  Unchanged

_ArrayExtract:  New function to return a section of the source array as defined by the user.

_ArrayFindAll: Unchanged

_ArrayInsert: Now supports 2D arrays.  Data for insertion into 2D arrays can be a single value, a delimited string or another 1D array.  The delimiter and the column in which the insertion is to start can be user-defined.

_ArrayMax: Now supports 2D arrays

_ArrayMaxIndex: Now supports 2D arrays

_ArrayMin: Now supports 2D arrays

_ArrayMinIndex: Now supports 2D arrays

_ArrayPermute: Unchanged

_ArrayPop: Unchanged

_ArrayPush: Unchanged

_ArrayReverse: Unchanged

_ArraySearch: Now supports searching rows as well as columns

_ArraySort: Unchanged

_ArraySwap: Removed from library. SCRIPT BREAKING!!!!!

_ArrayToClip:  Now a simple wrapper for _ArrayToClip - so default delimiter is now "|" and not @CR. SCRIPT BREAKING!!!!!

_ArrayToString: Now supports 2D arrays - user-defined delimiters between items and rows.  SCRIPT BREAKING!!!!!

_ArrayTranspose: New and improved jchd code.  Automatically changes from 1D to 2D and vice versa if needed

_ArrayTrim: Now supports 2D arrays

_ArrayUnique: Column number now 0-based not 1-based - this is for consistency with all other array functions. SCRIPT BREAKING!!!!!

Here is a zip with the new include (Array_New.au3) and examples for each of the modified/new functions showing the new behaviour and syntax:

If you run into any problems then do let me know - please provide both the array and the code you used so I can debug properly. And please also comment if you feel that any of the changes I have made are not what should have been done - although I would ask that you provide good reasons for your opposition.

Thanks in advance to those who test.

M23

Edited January 25, 2014 by Melba23
Code removed

[water]

Looks perfect

I've tested all functions and get the expected results.

_ArrayMaxIndex confused me a bit until I realized that StringSplit added a count at the top of array $avArray

[Iczer]

Is all functions are first class?

i hoped someday heavy _Array_..() functions move to internal...

but anyway - shouldn't this functionality be part of ReDim() function ?

_ArrayColDelete: New function to delete a column from an array. 
_ArrayColInsert:  New function to insert a column into an array.

also - why remove and get script breaking change? Is it interfere with something?

_ArraySwap: Removed from library. SCRIPT BREAKING!!!!!

Edited January 22, 2014 by Iczer

[BrewManNH]

but anyway - shouldn't this functionality be part of ReDim() function ?

_ArrayColDelete: New function to delete a column from an array. 
_ArrayColInsert:  New function to insert a column into an array.

No, you can't use just ReDim to add/remove columns because it will delete the contents of the array when you do that. These functions preserve the contents and just add/remove a column to an existing array. Edited January 22, 2014 by BrewManNH

[Melba23]

Iczer,

 

Is all functions are first class?

i hoped someday heavy _Array_..() functions move to internal...

I only have a red banner - not a green one.....

 

shouldn't this functionality be part of ReDim()

What BrewManNH said.

 

why remove and get script breaking change?

Have you looked at the current function? Firstly, it has nothing to do with arrays and so should not be in this library; next, it is so simple that it should never have been an #include function in the first place; and finally, its awfulness offended even my hobbyist coder sensibility.

All,

Without wishing to pick on Iczer, that was exactly the sort of post I was hoping not to see - a pure moan rather than sensible comment or reasoned argument about the new/modified functions. I hope my attitude to comments is sufficiently well-known that people realise that I will not take offence if you point out even the stupidest mistakes (that I hope I have not made). All I want is to get a better Array include for the next release - so by all means post, but please make the comments relevant.

M23

[Iczer]

No, you can't use just ReDim to add/remove columns because it will delete the contents of the array when you do that. These functions preserve the contents and just add/remove a column to an existing array.

You think i would ask if i do not know current functionality of ReDim() ? it seems my english is not that good - i mean : shouldn't this functionality be MADE part of ReDim() function ? Very logical extension i may say. Or it not possible from some point of view?

 

Iczer,

 

I only have a red banner - not a green one.....

 

 

   

Have you looked at the current function? Firstly, it has nothing to do with arrays and so should not be in this library; next, it is so simple that it should never have been an #include function in the first place; and finally, its awfulness offended even my hobbyist coder sensibility.

 

but it was added for some reason?  i'm just happen to hate script breaking changes piling up , dont mind me

[asdf8]

I long ago already not use function _ArrayAdd because it is very slow. I propose to add two more functions in the UDF and little change _ArrayAdd function, to compensate for this deficiency:

expandcollapse popup

#Region
Global Const $UBOUND_DIMENSIONS = 0
Global Const $UBOUND_ROWS = 1
Global Const $UBOUND_COLUMNS = 2
Global Const $STR_ENTIRESPLIT = 1
Global Const $STR_NOCOUNT = 2
#EndRegion

Global $ARRAY_FAST_UPDATE[1][2]

Func _ArrayBeginUpdate(ByRef $avArray)
    If Not IsArray($avArray) Or UBound($avArray, $UBOUND_DIMENSIONS) > 2 Then Return SetError(1)
    For $i = 1 To $ARRAY_FAST_UPDATE[0][0]
        If $ARRAY_FAST_UPDATE[$i][0] == $avArray Then
            Return 1
        EndIf
    Next
    $ARRAY_FAST_UPDATE[0][0] += 1
    ReDim $ARRAY_FAST_UPDATE[$ARRAY_FAST_UPDATE[0][0] + 1][2]
    $ARRAY_FAST_UPDATE[$ARRAY_FAST_UPDATE[0][0]][0] = $avArray
    $ARRAY_FAST_UPDATE[$ARRAY_FAST_UPDATE[0][0]][1] = UBound($avArray)
    Return 1
EndFunc

Func _ArrayEndUpdate(ByRef $avArray)
    If Not IsArray($avArray) Then Return SetError(1)
    Local $iRet
    For $i = 1 To $ARRAY_FAST_UPDATE[0][0]
        If $ARRAY_FAST_UPDATE[$i][0] == $avArray Then
            Switch UBound($avArray, $UBOUND_DIMENSIONS)
                Case 1
                    ReDim $avArray[$ARRAY_FAST_UPDATE[$i][1]]
                Case 2
                    ReDim $avArray[$ARRAY_FAST_UPDATE[$i][1]][UBound($avArray, $UBOUND_COLUMNS)]
            EndSwitch
            $iRet = 1
        ElseIf $iRet Then
            $ARRAY_FAST_UPDATE[$i - 1][0] = $ARRAY_FAST_UPDATE[$i][0]
            $ARRAY_FAST_UPDATE[$i - 1][1] = $ARRAY_FAST_UPDATE[$i][1]
        EndIf
    Next
    If $iRet Then
        $ARRAY_FAST_UPDATE[0][0] -= 1
        ReDim $ARRAY_FAST_UPDATE[$ARRAY_FAST_UPDATE[0][0] + 1][2]
        Return $iRet
    Else
        Return SetError(2)
    EndIf
EndFunc

Func _ArrayAdd(ByRef $avArray, $vValue, $iStart = 0, $sDelim = "|")
    
    If $iStart = Default Then $iStart = 0
    If $sDelim = Default Then $sDelim = "|"
    If Not IsArray($avArray) Then Return SetError(1, 0, -1)
    Local $iDim_1 = UBound($avArray, $UBOUND_ROWS)
    Local $iFastProc
    For $j = 1 To $ARRAY_FAST_UPDATE[0][0]
        If $ARRAY_FAST_UPDATE[$j][0] == $avArray Then
            $iFastProc = $j
            ExitLoop
        EndIf
    Next
    Switch UBound($avArray, $UBOUND_DIMENSIONS)
        Case 1
            If $iFastProc Then
                $iDim_1 = $ARRAY_FAST_UPDATE[$iFastProc][1]
                $ARRAY_FAST_UPDATE[$iFastProc][1] += 1
                If UBound($avArray) < $ARRAY_FAST_UPDATE[$iFastProc][1] Then
                    ReDim $avArray[2 * $ARRAY_FAST_UPDATE[$iFastProc][1]]
                EndIf
            Else
                ReDim $avArray[$iDim_1 + 1]
            EndIf
            $avArray[$iDim_1] = $vValue
            Return $iDim_1
        Case 2
            Local $iDim_2 = UBound($avArray, $UBOUND_COLUMNS)
            If $iStart < 0 Or $iStart > $iDim_2 - 1 Then Return SetError(3, 0, -1)
            If $iFastProc Then
                $iDim_1 = $ARRAY_FAST_UPDATE[$iFastProc][1]
                $ARRAY_FAST_UPDATE[$iFastProc][1] += 1
                If UBound($avArray) < $ARRAY_FAST_UPDATE[$iFastProc][1] Then
                    ReDim $avArray[2 * $ARRAY_FAST_UPDATE[$iFastProc][1]][$iDim_2]
                EndIf
            Else
                ReDim $avArray[$iDim_1 + 1][$iDim_2]
            EndIf
            If IsArray($vValue) Then
                If UBound($vValue, $UBOUND_DIMENSIONS) <> 1 Then Return SetError(4, 0, -1)
            Else
                $vValue = StringSplit($vValue, $sDelim, $STR_NOCOUNT + $STR_ENTIRESPLIT)
            EndIf
            Local $iDim_Val = UBound($vValue, $UBOUND_ROWS)
            For $j = 0 To $iDim_2 - 1
                If $j < $iStart Then
                    $avArray[$iDim_1][$j] = ""
                ElseIf $j - $iStart > $iDim_Val - 1 Then
                    $avArray[$iDim_1][$j] = ""
                Else
                    $avArray[$iDim_1][$j] = $vValue[$j - $iStart]
                EndIf
            Next
        Case Else
            Return SetError(2, 0, -1)
    EndSwitch
    
    Return UBound($avArray, $UBOUND_ROWS)
EndFunc



#Region example
Dim $aTmp[0]
$timer = TimerInit()

_ArrayBeginUpdate($aTmp)
For $i = 1 To 2000
    _ArrayAdd($aTmp, $i)
Next
_ArrayEndUpdate($aTmp)

ConsoleWrite('->' & TimerDiff($timer) & @CRLF)
#EndRegion example

Similarly, we can accelerate other functions containing ReDim.

[Mat]

What's the difference between that and _ArrayConcatenate and StringSplit?

[asdf8]

What's the difference between that and _ArrayConcatenate and StringSplit?

 

Using StringSplit constitutes acceptance of the loss of data types, and must ensure that the separator is not found in the data.

What is different _ArrayAdd from _ArrayConcatenate can view Help.

[Melba23]

asdf8,

While I appeciate the ingenuity of your solution, there seem to be even faster ways to append multiple values to an array - just use _ArrayConcatenate (as Mat suggested) or use ReDim and then copy over the values:

#Region example
; Create an array to hold the elements to be appended
Global $aFill[2000]
For $i = 0 To 1999
    $aFill[$i] = $i
Next

Global $aTmp[0]
$timer = TimerInit()
For $i = 1 To 2000
    _ArrayAdd($aTmp, $aFill[$i - 1]) ; The function in the Beta library <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Next
ConsoleWrite("UDF: " & TimerDiff($timer) & @CRLF)

Global $aTmp[0]
$timer = TimerInit()
_ArrayBeginUpdate($aTmp)
For $i = 1 To 2000
    _ArrayAdd_asdf8($aTmp, $aFill[$i -1]) ; I renamed the function you posted above <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Next
_ArrayEndUpdate($aTmp)
ConsoleWrite("asdf8: " & TimerDiff($timer) & @CRLF)

Global $aTmp[0]
$timer = TimerInit()
_ArrayConcatenate($aTmp, $aFill)
ConsoleWrite("Concat: " & TimerDiff($timer) & @CRLF)

Global $aTmp[0]
$timer = TimerInit()
ReDim $aTmp[2000]
For $i = 1 To 2000
    $aTmp[$i - 1] = $aFill[$i - 1]
Next
ConsoleWrite("ReDim: " & TimerDiff($timer) & @CRLF)
#EndRegion example

I get 1600 - 85 - 8 - 7 ms as average timings.

On a wider note, I did rewrite _ArrayDelete to accept multiple deletions in one pass which dramatically improved the speed when that was required. I was considering how I might do the same for ArrayInsert but decided not to do so as it all got too complex. I will revisit the matter to see if I can at least do it for _Array Add where the problem is considerably simpler - and perhaps think again about _ArrayInsert too.

M23

[asdf8]

Melba23,

Examples 3 and 4 are not correct: not taken into account the duration of creating $aFill and most importantly - is considered to be known in advance the size of the array $aFill.

[Melba23]

Hi,

Spurred on by asdf8 I have rewritten _ArrayAdd to carry out multiple insertions to prevent too namy ReDim calls - and here is the result. I have added a $iMulti flag which works like this:

<snip>

Now to see if I can get _ArrayInsert to play nicely with multiple inserts as well.

M23

Edited January 25, 2014 by Melba23
Code removed

[jaberwacky]

nothing

Edited January 23, 2014 by jaberwacky

[BrewManNH]

I wrote an arraybinarysearch that works with 2D arrays, it's basically just the normal _ABS with the option to search on a specific "column" of the array. Much like how ArraySearch works, but binary.

[Mat]

I've used a similar thing a few times BrewManNH, usually when I don't want to use a dictionary for whatever reason, such as when I have 4 or 5 different values associated with a key.

[Melba23]

Hi,

I cannot find a suitable logic for multiple inserts - how do you deal with the required indices given that their initial location will change as the other items are inserted above them. An example of what I mean:

Original  - Insert 1,3 
            This way?                         Or this way?

0           0                                 0
1           New A ; Item 1                    New A
2           1                                 1
3           New B ; item 3 in new array       2
4           2                                 New B ; above item 3 in old array
            3                                 3
            4                                 4

As I am certain that there will never be agreement on which is the correct way to do it, I have decided to concentrate on other things and _ArrayInsert will remain a one-shot function. _ArrayAdd & _ArrayDelete both now have multiple options, so as MeatLoaf used to sing 2 out of 3 ain't bad.

M23

[jaberwacky]

@BrewMan, That's cool and seems more logical than the version I posted.

Edited January 23, 2014 by jaberwacky

[AdamUL]

It's nice to see Array.au3 updated to work with 2D arrays, and this is very nice work.  I did notice something, some of the 2D support added, as well as 2D support already in some function, seems to only be for working with columns, and not rows.  For example, _ArraySearch is not included in the update to full 2D support.  So you are only able to search columns, and not rows.  I use this >Array2D UDF quite a bit, and _Array2D_Search has the option for searching rows, as well as columns.  Searching rows is very helpful, like when row 0 are column headers that could change position as you add and remove columns.  Also, I noticed that _ArraySwap was removed, which I do not see a problem with.  In the Array2D UDF, there is a nice function called _Array2D_SwapByIndex, which allows you to swap entire rows, or columns, this may be more useful to add.  There may be other functions in the Array2D UDF that could be useful to add to Array.au3.  

 

Thanks again for working on this update.

 

 

 

Adam

Edited January 23, 2014 by AdamUL

[JLogan3o13]

Couldn't tell from your post if you were asking why _ArraySwap was removed, or just suggesting an alternative. Just to reiterate what has been stated already, the original _ArraySwap was literally 3 lines of (badly written) code, which could easily be reproduced, and didn't really belong in Array.au3 to begin with. Thus the removal with this update.

Edited January 23, 2014 by JLogan3o13

[Zedna]

Just for the reference:

Here is my _ArrayToString2D(), _ArrayToHtml2D()

?do=embed' frameborder='0' data-embedContent>