ArrayMultiColSort -New Release 06 April 2019

Melba23 · October 16, 2013

[New Release] - 06 April 2019

Added: Error-checking for sensible column numbers in the $aSortData array, with an additional error status.

------------------------------------------------------------------------------------------------------------------------

While answering a recent question about sorting a ListView on several columns, I developed this function to sort a 2D array on several columns and I though I might give it a wider audience.

Here is the function:

#include-once

;#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w- 4 -w 5 -w 6 -w- 7

; #INCLUDES# =========================================================================================================
#include <Array.au3>
; ===============================================================================================================================

; #INDEX# =======================================================================================================================
; Title .........: ArrayMultiColSort
; AutoIt Version : v3.3.8.1 or higher
; Language ......: English
; Description ...: Sorts 2D arrays on several columns
; Note ..........:
; Author(s) .....: Melba23
; Remarks .......:
; ===============================================================================================================================

; #CURRENT# =====================================================================================================================
; _ArrayMultiColSort : Sort 2D arrays on several columns
; ===============================================================================================================================

; #INTERNAL_USE_ONLY#=================================================================================================
; __AMCS_SortChunk : Sorts array section
; ===============================================================================================================================

; #FUNCTION# ====================================================================================================================
; Name...........: _ArrayMultiColSort
; Description ...: Sort 2D arrays on several columns
; Syntax.........: _ArrayMultiColSort(ByRef $aArray, $aSortData[, $iStart = 0[, $iEnd = 0]])
; Parameters ....: $aArray    - The 2D array to be sorted
;                  $aSortData - 2D array holding details of the sort format
;                               Format: [Column to be sorted, Sort order]
;                                   Sort order can be either numeric (0/1 = ascending/descending) or a ordered string of items
;                                   Any elements not matched in string are left unsorted after all sorted elements
;                  $iStart    - Element of array at which sort starts (default = 0)
;                  $iEnd      - Element of array at which sort endd (default = 0 - converted to end of array)
; Requirement(s).: v3.3.8.1 or higher
; Return values .: Success: No error
;                  Failure: @error set as follows
;                            @error = 1 with @extended set as follows (all refer to $sIn_Date):
;                                1 = Array to be sorted not 2D
;                                2 = Sort data array not 2D
;                                3 = More data rows in $aSortData than columns in $aArray
;                                4 = Start beyond end of array
;                                5 = Start beyond End
;                            @error = 2 with @extended set as follows:
;                                1 = Invalid string parameter in $aSortData
;                                2 = Invalid sort direction parameter in $aSortData
;                                3 = Invalid column index in $aSortData
; Author ........: Melba23
; Remarks .......: Columns can be sorted in any order
; Example .......; Yes
; ===============================================================================================================================
Func _ArrayMultiColSort(ByRef $aArray, $aSortData, $iStart = 0, $iEnd = 0)

    ; Errorchecking
    ; 2D array to be sorted
    If UBound($aArray, 2) = 0 Then
        Return SetError(1, 1, "")
    EndIf
    ; 2D sort data
    If UBound($aSortData, 2) <> 2 Then
        Return SetError(1, 2, "")
    EndIf
    If UBound($aSortData) > UBound($aArray) Then
        Return SetError(1, 3)
    EndIf
    For $i = 0 To UBound($aSortData) - 1
        If $aSortData[$i][0] < 0 Or $aSortData[$i][0] > UBound($aArray, 2) -1 Then
            Return SetError(2, 3, "")
        EndIf
    Next
    ; Start element
    If $iStart < 0 Then
        $iStart = 0
    EndIf
    If $iStart >= UBound($aArray) - 1 Then
        Return SetError(1, 4, "")
    EndIf
    ; End element
    If $iEnd <= 0 Or $iEnd >= UBound($aArray) - 1 Then
        $iEnd = UBound($aArray) - 1
    EndIf
    ; Sanity check
    If $iEnd <= $iStart Then
        Return SetError(1, 5, "")
    EndIf

    Local $iCurrCol, $iChunk_Start, $iMatchCol

    ; Sort first column
    __AMCS_SortChunk($aArray, $aSortData, 0, $aSortData[0][0], $iStart, $iEnd)
    If @error Then
        Return SetError(2, @extended, "")
    EndIf
    ; Now sort within other columns
    For $iSortData_Row = 1 To UBound($aSortData) - 1
        ; Determine column to sort
        $iCurrCol = $aSortData[$iSortData_Row][0]
        ; Create arrays to hold data from previous columns
        Local $aBaseValue[$iSortData_Row]
        ; Set base values
        For $i = 0 To $iSortData_Row - 1
            $aBaseValue[$i] = $aArray[$iStart][$aSortData[$i][0]]
        Next
        ; Set start of this chunk
        $iChunk_Start = $iStart
        ; Now work down through array
        For $iRow = $iStart + 1 To $iEnd
            ; Match each column
            For $k = 0 To $iSortData_Row - 1
                $iMatchCol = $aSortData[$k][0]
                ; See if value in each has changed
                If $aArray[$iRow][$iMatchCol] <> $aBaseValue[$k] Then
                    ; If so and row has advanced
                    If $iChunk_Start < $iRow - 1 Then
                        ; Sort this chunk
                        __AMCS_SortChunk($aArray, $aSortData, $iSortData_Row, $iCurrCol, $iChunk_Start, $iRow - 1)
                        If @error Then
                            Return SetError(2, @extended, "")
                        EndIf
                    EndIf
                    ; Set new base value
                    $aBaseValue[$k] = $aArray[$iRow][$iMatchCol]
                    ; Set new chunk start
                    $iChunk_Start = $iRow
                EndIf
            Next
        Next
        ; Sort final section
        If $iChunk_Start < $iRow - 1 Then
            __AMCS_SortChunk($aArray, $aSortData, $iSortData_Row, $iCurrCol, $iChunk_Start, $iRow - 1)
            If @error Then
                Return SetError(2, @extended, "")
            EndIf
        EndIf
    Next

EndFunc   ;==>_ArrayMultiColSort

; #INTERNAL_USE_ONLY# ===========================================================================================================
; Name...........: __AMCS_SortChunk
; Description ...: Sorts array section
; Author ........: Melba23
; Remarks .......:
; ===============================================================================================================================
Func __AMCS_SortChunk(ByRef $aArray, $aSortData, $iRow, $iColumn, $iChunkStart, $iChunkEnd)

    Local $aSortOrder

    ; Set default sort direction
    Local $iSortDirn = 1
    ; Need to prefix elements?
    If IsString($aSortData[$iRow][1]) Then
        ; Split elements
        $aSortOrder = StringSplit($aSortData[$iRow][1], ",")
        If @error Then
            Return SetError(1, 1, "")
        EndIf
        ; Add prefix to each element
        For $i = $iChunkStart To $iChunkEnd
            For $j = 1 To $aSortOrder[0]
                If $aArray[$i][$iColumn] = $aSortOrder[$j] Then
                    $aArray[$i][$iColumn] = StringFormat("%02i-", $j) & $aArray[$i][$iColumn]
                    ExitLoop
                EndIf
            Next
            ; Deal with anything that does not match
            If $j > $aSortOrder[0] Then
                $aArray[$i][$iColumn] = StringFormat("%02i-", $j) & $aArray[$i][$iColumn]
            EndIf
        Next
    Else
        Switch $aSortData[$iRow][1]
            Case 0, 1
                ; Set required sort direction if no list
                If $aSortData[$iRow][1] Then
                    $iSortDirn = -1
                Else
                    $iSortDirn = 1
                EndIf
            Case Else
                Return SetError(1, 2, "")
        EndSwitch
    EndIf

    ; Sort the chunk
    Local $iSubMax = UBound($aArray, 2) - 1
    __ArrayQuickSort2D($aArray, $iSortDirn, $iChunkStart, $iChunkEnd, $iColumn, $iSubMax)

    ; Remove any prefixes
    If IsString($aSortData[$iRow][1]) Then
        For $i = $iChunkStart To $iChunkEnd
            $aArray[$i][$iColumn] = StringTrimLeft($aArray[$i][$iColumn], 3)
        Next
    EndIf

EndFunc   ;==>__AMCS_SortChunk

And here is an example to show it working:

#include "ArrayMultiColSort.au3"
#include <String.au3> ; Only used to fill array

; Create and display array
Global $aArray[100][4]
For $i = 0 To 99
    $aArray[$i][0] = _StringRepeat(Chr(Random(65, 68, 1)), 5)
    $aArray[$i][1] = _StringRepeat(Chr(Random(74, 77, 1)), 5)
    $aArray[$i][2] = _StringRepeat(Chr(Random(80, 83, 1)), 5)
    $aArray[$i][3] = _StringRepeat(Chr(Random(87, 90, 1)), 5)
Next
_ArrayDisplay($aArray, "Unsorted")

; Copy arrays for separate examples below
$aArray_1 = $aArray
$aArray_2 = $aArray

; This sorts columns in ascending order - probably the most common requirement
; Sort requirement:
;  Col 0 = Decending
;  Col 1 = Ascending
;  Col 2 = Required order of elements (note not alphabetic PQRS nor reverse SRQP)
;  Col 3 = Ascending
Global $aSortData[][] = [ _
        [0, 1], _
        [1, 0], _
        [2, "SSSSS,QQQQQ,PPPPP,RRRRR"], _
        [3, 0]]

; Sort and display array
_ArrayMultiColSort($aArray_1, $aSortData)
; Display any errors encountered
If @error Then ConsoleWrite("Oops: " & @error & " - " & @extended & @CRLF)
_ArrayDisplay($aArray_1, "Sorted in order 0-1-2-3")

; But the UDF can sort columns in any order
; Sort requirement:
;  Col 2 = Decending
;  Col 0 = Ascending
Global $aSortData[][] = [ _
        [2, 1], _
        [0, 0]]

; Sort and display array
_ArrayMultiColSort($aArray_2, $aSortData)
; Display any errors encountered
If @error Then ConsoleWrite("Oops: " & @error & " - " & @extended & @CRLF)
_ArrayDisplay($aArray_2, "Sorted in order 2-0")

And here are both in zip form: ArrayMultiColSort.zip

As usual all comments welcome.

M23

Edited April 5, 2019 by Melba23

BrewManNH · October 16, 2013

Not really sure what column 2 is supposed to be sorting by. The explanation is a bit vague and the sort doesn't seem to fit what I think it's saying.

BTW, wouldn't this be faster using an SQLite DB in memory?

Melba23 · October 16, 2013

BrewManNH,

Sorry if the explanation was lacking. If you use a string, elements in that column will be sorted in the order set in the string. Im this case if you were to use just "ascending/descending" for column 2 then the strings would be ordered "P,Q,R,S / S,R,Q,P" - by using the string they are in fact ordered "S,Q,P,R". The idea is that if you use known strings as elements (e.g. "Male,Female,Unknown") you can order them as you wish and not necessarily in alpha/reverse alpha order. Clearer now?

And it could well be quicker using SQLite. But I wanted to do it in AutoIt and I will leave the other solution to jchd.

M23

c.haslam · June 15, 2015

IMHO This is worthy of being included in Array.au3.

I have found one small error: if I code:

Local $sortSpecsAr[][] = [[$kNatGp,0],[$kIncl,0],[$kExcl,0],[$kUdf,0]]  ; all ascending
    if _ArrayMultiColSort($resultsAr,$sortSpecsAr)='' Then
        MsgBox(0,' _ArrayMultiColSort','@error = '&@error)
    EndIf

I see @error=0

Melba23 · June 15, 2015

c.haslam,

If all goes well then @‌error should indeed be 0, but I have no way of telling if that is the case - what value for @‌error were you expecting?

M23

c.haslam · June 15, 2015

Sorry for an incomplete post.

I was expecting success. You wrote:

; Return values .: Success: The sorted array
;                  Failure: An empty string with @error set as follows
;                            @error = 1 with @extended set as follows (all refer to $sIn_Date):
;                                1 = Array to be sorted not 2D
;                                2 = Sort data array not 2D
;                                3 = More data rows in $aSortData than columns in $aArray
;                                4 = Start beyond end of array
;                                5 = Start beyond End
;                            @error = 2 with @extended set as follows:
;                                1 = Invalid string parameter in $aSortData
;                                2 = Invalid sort direction parameter in $aSortData

So I expected the sorted array to be returned.

jchd · June 15, 2015

The array is to be passed ByRef so it doesn't need to be "returned", strictly speaking.

c.haslam · June 16, 2015

True. But the documentation and code should match.

jchd · June 16, 2015

Trust the code: contrary to the dox, it's a poor liar.

Melba23 · June 16, 2015

c.haslam.

As I stated above, if the @‌error returned is 0 then the function ran successfully - what you should do is test for that rather then the return value. I will look at perhaps changing the code and documentation a bit so that they match, but do remember that a foolish consistency is the hobgoblin of little minds.

M23

Melba23 · September 15, 2015

[New Release] - 15 Sep 15

Changed: UDF can now sort columns in any order.

New files and zip in first post.

M23

c.haslam · September 15, 2015

Thank you for the update

Bizzaro · November 10, 2015

I'm wondering if this UDF can help me.

I have a 2D array and I'm ordering the items by one of the columns, lowest to high number. However some of the numbers are the same. I would like to then sort the rows by another column, so if several of the entries into the array rows in the column I'm sorting are "2", I'd then like to sort these rows by another column.

Is this what your UDF does? Or is there is a very simple solution that I can use to action my requirements?

Melba23 · November 10, 2015

Bizzaro,

Is this what your UDF does?

Try it and see - you should find that it does exactly what you want.

M23

mLipok · November 10, 2015

Just for note - added here:
https://www.autoitscript.com/wiki/User_Defined_Functions#Misc

Edited November 10, 2015 by mLipok

Melba23 · November 10, 2015

Bizzaro.

Her is a short example doing what you require:

#include <Array.au3>
#include <ArrayMultiColSort.au3>

Global $aOrg[20][2]

For $i = 0 To 19
    $aOrg[$i][0] = Random(1, 9, 1)
    $aOrg[$i][1] = Random(1, 9, 1)
Next

_ArrayDisplay($aOrg, "Original", Default, 8)

Global $aSortData[][] = [[0, 0], [1, 0]]

_ArrayMultiColSort($aOrg, $aSortData)

_ArrayDisplay($aOrg, "Multi Sorted", Default, 8)

M23

Bizzaro · November 10, 2015

Melba, thank you for your help. Your UDF is fantastic for my needs and i'll be using it a lot. It should be included automatically IMO as it is invaluable.

Trolleule · July 6, 2016

Nice work, helped me right now. Stay on !

Funtime60 · June 8, 2018

I love this, but I've hit a snag in my latest usage of it.

In ascending sorting I have 9 being set above 10,11,12,etc...

I think I understand the issue as I have seen it in other things.

I was wondering if there were any solution?

Thank you for your time.

EDIT:

I have realized that I can correct the problem externally in NP++, however this isn't viable for the full sized files as they are thousands of entries long so I'd have to add 0's to the front of every number in the sorting columns with fewer digits than the larger number. It would be nice if this could be done internally with the results matching the original format.

I have no idea how hard this would be as I have no idea how to do it on my own in an infinite manner let alone integrate it into a function I don't understand.

Again, thank you for your time.

Edit 2:

I have tested and have found the issue I think. Please forgive my poor terminology, but I think this sorts like you would alphabetize words. Compare the first digits ignoring any digits after that. A 9 is greater than 10 because It only seems to look at 9 and 1 and 9 is greater than 1 so the 0 in 10 is ignored. To solve this you can pad the sorting columns with preceding zeros 09 will be less than 10 because 0 is less than 1. As such I have write a poorly designed thing to automatically pad a column, though I am still making it procedural so any columns can be padded and the padding maybe left behind in the output. I can't say for sure on this last point as I have yet to complete a test of a final project.

On another note while I did write my bit to compensate for negatives, turning -9 into -09 and not 0-9, the original UDF does not seem to support negative numbers properly as far as I can tell.

All of this is based purely off of my minimal observations and testing with extremely limited understanding of the original UDF's inner workings. I would like apologize for any mistakes I have made in the process.

Thank you for taking the time to read this.

Edited June 9, 2018 by Funtime60
I had more Ideas

Funtime60 · June 9, 2018

#include <ArrayMultiColSort.au3>

#Region Init
;Source data to be sorted, any 2D array.
Local $testfilearray1[3][3] = [["123", "321", "-1"], ["123", "-31", "13"], ["123", "-321", "-12"]]
;Must come anywhere after line above, but before loops below.
Local $testsize = UBound($testfilearray1) - 1
;Must predetermine the sorting format.
Global $aSortData[][] = [[1, 0], [2, 0]]
#EndRegion Init


#Region Add Pad
;Sorry, I'd explain, but even after only a few hours, I've already forgotten whay most of this does.
Local $col1max = _ArrayMax($testfilearray1, 1, -1, -1, 1)
Local $col2max = _ArrayMax($testfilearray1, 1, -1, -1, 2)
Local $col1maxdig = StringLen($col1max)
Local $col2maxdig = StringLen($col2max)
Local $sorttotalcount = UBound($aSortData) - 1
For $sortcount = 0 To $sorttotalcount Step 1
    Local $colnum = $aSortData[$sortcount][0]
    For $testfilerownum2 = 0 To $testsize Step 1
        Local $neg1 = StringSplit($testfilearray1[$testfilerownum2][$colnum], "")
        If $neg1[1] = "-" Then
            Local $neg1testfile = StringSplit($testfilearray1[$testfilerownum2][$colnum], "-")
            Local $testcol1dig = StringLen($neg1testfile[2])
        Else
            Local $testcol1dig = StringLen($testfilearray1[$testfilerownum2][$colnum])
        EndIf
        While $testcol1dig < $col1maxdig
            Local $neg1 = StringSplit($testfilearray1[$testfilerownum2][$colnum], "")
            If $neg1[1] = "-" Then
                Local $neg1testfile = StringSplit($testfilearray1[$testfilerownum2][$colnum], "-")
                $testfilearray1[$testfilerownum2][$colnum] = "-0" & $neg1testfile[2]
                Local $testcol1dig = StringLen($neg1testfile[2])
            Else
                $testfilearray1[$testfilerownum2][$colnum] = "0" & $testfilearray1[$testfilerownum2][$colnum]
                Local $testcol1dig = StringLen($testfilearray1[$testfilerownum2][$colnum])
            EndIf
            Local $neg1 = StringSplit($testfilearray1[$testfilerownum2][$colnum], "")
            If $neg1[1] = "-" Then
                Local $neg1testfile = StringSplit($testfilearray1[$testfilerownum2][$colnum], "-")
                Local $testcol1dig = StringLen($neg1testfile[2])
            Else
                Local $testcol1dig = StringLen($testfilearray1[$testfilerownum2][$colnum])
            EndIf
        WEnd
    Next
Next
_ArrayDisplay($testfilearray1, "$testfilearray1")
#EndRegion Add Pad

;Now take the array and plug it into _ArrayMultiColSort($testfilearray1, $aSortData)

This is my procedural padding thing, it's technically not a UDF.

If this shouldn't be here, just ask. I'll try to remove it in a reasonable amount of time.

Thanks for your time.

EDIT:

On a side note, why does the syntax highlighter not work on my post?

EDIT 2:

Please read the next page.

Edited June 9, 2018 by Funtime60
Apologizing for my noobyness at posting code.

ArrayMultiColSort -New Release 06 April 2019

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Melba23

Melba23

Melba23

Posted Images

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members