Improving _ArrayDisplay speed

[pixelsearch]

I was reviewing some comments in this thread and maybe (Jpm) we didn't understand each other concerning this issue :

On the left of the pic is the actual ArrayDisplay (used by everybody) and its buffer of 4096 characters (259 char visibles in regular LV), as found in  __ArrayDisplay_GetItemText(), so far so good.

On the right of the pic is the Virtual Listview UDF (ArrayDisplayInternals2.au3) and its buffer of 50 characters ("wchar[50]") which will constantly truncate captions longer than 50 characters (not hide, truncate). This will result in many Forums complains concerning these truncated captions.

Here is the example I used to display the preceding pic (code placed in ArrayDisplay2-new.au3)

Example_perso()

Func Example_perso()
    Local $sRange = ""
    Local $iFlags = ""
    Local $sHeader = ""
    Local $bDebug = True

    Local $sPath = "C:\This is a 1st subfolder"
    Local $sCommand = " /c dir /a-d /b /od /s "
    Local $iPid = Run(@ComSpec & $sCommand, $sPath, @SW_HIDE, $STDOUT_CHILD)
    ProcessWaitClose($iPid)

    Local $sList = StdoutRead($iPid)
    $sList = StringTrimRight($sList, 2) ; delete last @CRLF
    Local $aArray = StringSplit($sList, @CRLF, BitOr($STR_ENTIRESPLIT, $STR_NOCOUNT))

    _DebugArrayDisplay($aArray, "Sorted by date", $sRange, $iFlags, Default, $sHeader , Default, Default)
    $g_hArrayDisplay_Share($aArray, "Sorted by date", $sRange, $iFlags, Default, $sHeader, Default, Default, $bDebug)

EndFunc   ;==>Example_perso

What would be the problem if you change ("wchar[50]") to ("wchar[260]") or ("wchar[4096]") in the Virtual LV version, without modifying anything else ?

Local Static $tText = DllStructCreate("wchar[4096]"), $pText = DllStructGetPtr($tText)

 

Edited April 29, 2021 by pixelsearch

[jpm]

Thanks, I try follow the release colwidth behavior

Cheers

ArrayDisplayInternals2.7z

[pixelsearch]

Bravo Jpm, now long captions aren't truncated anymore when using Virtual Listviews
One last thing that requires your confirmation :

Example_perso2()

Func Example_perso2()
    Local $iRows = 1, $iFlags = 0 * $ARRAYDISPLAY_NOROW + $ARRAYDISPLAY_COLALIGNRIGHT + 0 * $ARRAYDISPLAY_TRANSPOSE
    Local $sRange = "1:"
    Local $sHeader = "String|Integers|Floats|Dates|Times|R/C"
    Local $bDebug = True

    ; Generate array
    Local $hTimer = TimerInit()
    Local $aArray = FAS_Random2DArrayAu3($iRows, "sifdtr", "abcdefghijklmnopqrstuvwxyz")
    ConsoleWrite("Generating array = " & TimerDiff($hTimer)& " $iRows = " & $iRows & " $iCols = " & UBound($aArray, $UBOUND_COLUMNS) & @CRLF)

    _DebugArrayDisplay($aArray, Default, $sRange, $iFlags, Default, $sHeader , Default, Default)
    $g_hArrayDisplay_Share($aArray, Default, $sRange, $iFlags, Default, $sHeader, Default, Default, $bDebug)
EndFunc   ;==>Example_perso2

The preceding code will show :
* a blank LV with actual ArrayDisplayInternals.au3 (this seems normal)
* one row with ArrayDisplayInternals2.au3

Is this exactly the way you want it to be ?

Edited April 29, 2021 by pixelsearch

[jpm]

THanks I got the idea

but I am suprise you  get it working as $asHeader is not  global variable so should not be accessible inside the notify handler

Cheers

[pixelsearch]

Just now, jpm said:

I am suprise you  get it working as $asHeader is not  global variable

(Jpm's last comment is related to my post at the end of preceding page)
You're right Jpm, I probably changed $asHeader to a global variable during the test and forgot to mention it.

[jpm]

here is the last update ...

I fix also the Copy to clipboard

ArrayDisplayInternals2.7z

[pixelsearch]

Hi Jpm
I think it's finished now, you really did a great job, especially it was time consuming.
Concerning the last question I asked in the post above, there's probably nothing that can be done and 1 row will always appear in that special case, we'll adapt to that.

Thanks to LarsJ and his great initial examples, they enlightened us concerning Virtual ListViews

[pixelsearch]

Hi Jpm

I'm not convinced by your answer in this post and the change done in the last version :

"1 is wrong as an extra entry is needed when displaying the row number, it should be :"
Dim $_g_ArrayDisplay_aIndexes[$_g_ArrayDisplay_nCols + $_g_ArrayDisplay_iDisplayRow + 1]

Displaying the row number (or not) doesn't have an impact on the number of rows in this array, because :

* The "row number" is mandatory in the array $_g_ArrayDisplay_aIndexes[], you always add its index as [0] (no matter the column will be displayed or not +++) . This is because index [0] is required during the __ArrayDisplay_NotifyHandler() function to display the initial rows.

* Column numbers : here is what we found in the script

$_g_ArrayDisplay_nCols = ($_g_ArrayDisplay_iDims = 2) ? UBound($_g_ArrayDisplay_aArray, $UBOUND_COLUMNS) : 1

I like the fact that you "forced" $_g_ArrayDisplay_nCols = 1 in case of a 1D array, it makes things easier . Now $_g_ArrayDisplay_nCols reflect exactly how many columns exist in the array (no matter the array is 1D or 2D)

* So in the end (and if I'm not mistaken) what we simply need is this :

Dim $_g_ArrayDisplay_aIndexes[1 + $_g_ArrayDisplay_nCols]

Here is a test based on a 2D array (LarsJ's 6 cols), showing that no script error occurs when using the precedent line, no matter the row number is displayed or not. A debug line placed at the very end of Func __ArrayDisplay_SortIndexes() shows exactly which index was just added and its position in the array :

_DebugArrayDisplay($_g_ArrayDisplay_aIndexes, "$_g_ArrayDisplay_aIndexes")

* In the preceding pic, if the Array had been 1D, then the display would have been like this (tested, no matter the row number was displayed or not) :

Row|Col 0
# 0|{Array}
# 1|{Array}

* Some comments on the 2D pic above :
- I choosed to sort on rightmost column (#6 in the pic) to show that no error occurs.
- It may not be the place to discuss it (again) but you choosed to sort on column #1 in all cases, when I would have avoided it totally.
- Noone tells us that user wants a sort on col #1 or on any other column, so it's just "wasting time" to prepare index(es) that won't be used at all. Imho no column should be sorted so ArrayDisplay will immediately display the virtual LV.
- Of course col #0 (the "row column already sorted") is a very different case as it IS lightning fast to prepare (even on huge arrays) and also because index [0] is immediately required by NotifyHandler

Anyway, the final decision will be yours but I needed to share my concern. It's getting better now
Thanks for all your great work in this function. I hope you'll make it public in the beta download section when you think the time has come.

[pixelsearch]

Hello @jpm
I tweaked recently my version of ArrayDisplay (beta 3.3.15.4) for 2 reasons and I would like to explain here why I did it.

1) First reason : have all numeric columns sorted correctly.
Actually it's not the case. You'll notice the bad sort result after running the following script (with beta 3.3.15.4 or with the official release 3.3.14.5)

#include <Array.au3>
Local $aArray[12] = [0.5, 1.25, 2.75, 1.5, 0.25, 2.50, 2, 1, -1, 0.75, 1.55, 2.255]
ConsoleWrite("VarGetType($aArray[0]) = " & VarGetType($aArray[0]) & @crlf) ; Double
Local $iFlags = (0 * $ARRAYDISPLAY_NOROW) + (0 * $ARRAYDISPLAY_COLALIGNRIGHT) + (0 * $ARRAYDISPLAY_TRANSPOSE)
Local $sHeader = "Float"
_ArrayDisplay($aArray, @AutoItVersion, Default, $iFlags, Default, $sHeader)

2) Second reason : when we sort a column numerically, the time for preparing the index (in beta 3.3.15.4) can be reduced from 40-50% with the appropriate code, which means the index can be prepared nearly twice faster !

"Appropriate" code means that we have to indicate to ArrayDisplay that we want some columns to be sorted numerically (without  adding a new parameter to ArrayDisplay) and that's not an easy task because it requires some "compromise" : any way you'll do it, some users would like it to be done differently.

For the record, please note that with the official release 3.3.14.5, some interesting code is found in __ArrayDisplay_Share() ... but this code is never excuted because of this line :

__ArrayDisplay_RegisterSortCallBack($idListView, 2, True, "__ArrayDisplay_SortCallBack")

Here are the lines that, unfortunately, are never executed because the 2nd parameter of the preceding line is always 2 ("Use Windows API StrCmpLogical") and never 0 or 1 :

If $__g_aArrayDisplay_SortInfo[8] = 1 Then
        ; force Treat as Number if possible
        If (StringIsFloat($sVal1) Or StringIsInt($sVal1)) Then $sVal1 = Number($sVal1)
        If (StringIsFloat($sVal2) Or StringIsInt($sVal2)) Then $sVal2 = Number($sVal2)
    EndIf

    Local $nResult
    If $__g_aArrayDisplay_SortInfo[8] < 2 Then
        ; Treat as String or Number
        $nResult = 0 ; No change of item1 and item2 positions
        If $sVal1 < $sVal2 Then
            $nResult = -1 ; Put item2 before item1
        ElseIf $sVal1 > $sVal2 Then
            $nResult = 1 ; Put item2 behind item1
        EndIf
    Else

More explanations in this thread, where paulpmeier seemed satisfied with the solution offered to him (solution not acceptable in beta 3.3.15.4 where sort indexes are stored in arrays and it's out of question to update them constantly)

Ok, back to beta 3.3.15.4
There are a few options to force a columnn to be sorted numerically. This is the compromise I made : add a * at the end of a header to force a numerical sorting, for example in the 1st listing above :

; Local $sHeader = "Float"
Local $sHeader = "Float*" ; add * at the end of a header to force a numeric sort

Here are a couple of examples based on the asterix way :

#include <Array.au3>
Local $iRows = 12, $aArray[$iRows][3]
Local $aFloat  = StringSplit("0.5, 1.25, 2.75, 1.5, 0.25, 2.50, 2, 1, -1, 0.75, 1.55, 2.255", "," , 2) ; $STR_NOCOUNT = 2
Local $aString = StringSplit("Paul, Denise, Richard, Louis, Amanda, Keith, Helen, John, Philip, Zoe, Steve, Eva",  "," , 2)
Local $aHexa   = StringSplit("0x0B, 0x020, 0x0C, 0x03, 0x01, 0x0A, 0x0FF, 0x030, 0x04, 0x0E, 0x0D, 0x02", "," , 2)
If Ubound($aFloat) <> $iRows Or Ubound($aString) <> $iRows Or Ubound($aHexa) <> $iRows Then
    MsgBox(0, "Error", "Bad number of elements in example arrays")
    Exit
EndIf
For $i = 0 To $iRows - 1
    $aArray[$i][0] = StringStripWS($aFloat[$i],  1 + 2) ; $STR_STRIPLEADING = 1 , $STR_STRIPTRAILING = 2
    $aArray[$i][1] = StringStripWS($aString[$i], 1 + 2)
    $aArray[$i][2] = StringStripWS($aHexa[$i],   1 + 2)
Next
Local $iFlags = (0 * $ARRAYDISPLAY_NOROW) + (0 * $ARRAYDISPLAY_COLALIGNRIGHT) + (0 * $ARRAYDISPLAY_TRANSPOSE)
; Local $sHeader = "Float|String|Hexa"
Local $sHeader = "Float*|String|Hexa*" ; add * at the end of a header to force a numeric sort
_ArrayDisplay($aArray, @AutoItVersion, Default, $iFlags, Default, $sHeader)

Bad sort in 2 columns

Good sort when asterix in code (the asterix disappears from headers, below)

Here is some code where 5 columns should be sorted numerically (even the last "R/C" column which doesn't contain numbers only, as its values increment from "0/5" to "9999/5")

#include <Array.au3>
#include "RandomArray.au3" ; LarsJ
Local $iRows = 10000
Local $aArray = FAS_Random2DArrayAu3($iRows, "sifdtr", "abcdefghijklmnopqrstuvwxyz")
Local $iFlags = (0 * $ARRAYDISPLAY_NOROW) + (0 * $ARRAYDISPLAY_COLALIGNRIGHT) + (0 * $ARRAYDISPLAY_TRANSPOSE)
; Local $sHeader = "String|Integers|Floats|Dates|Times|R/C"
Local $sHeader = "String|Integers*|Floats*|Dates*|Times*|R/C*" ; add * at the end of a header to force a numeric sort
_ArrayDisplay($aArray, @AutoItVersion, Default, $iFlags, Default, $sHeader)

Here are the parts of code I added to reach this result (you know where they are placed) :

expandcollapse popup

...
Global $_g_ArrayDisplay_aNumericSort
...

; Split custom header on separator
Dim $_g_ArrayDisplay_aNumericSort[$_g_ArrayDisplay_nCols]
...

; Create custom header with available items
...
If StringRight($_g_ArrayDisplay_asHeader[$iIndex], 1) = "*" Then
    $_g_ArrayDisplay_asHeader[$iIndex] = StringTrimRight($_g_ArrayDisplay_asHeader[$iIndex], 1) ; remove "*" from right
    $_g_ArrayDisplay_aNumericSort[$iIndex - $_g_ArrayDisplay_iSubItem_Start] = 1 ; 1 (numeric sort) or empty (natural sort)
EndIf
...

Func __ArrayDisplay_SortArrayStruct(Const ByRef $aArray, $iCol)
    ...
    ; Local $lo, $hi, $mi, $r
    Local $lo, $hi, $mi, $r, $nVal1, $nVal2

    If $_g_ArrayDisplay_aNumericSort[$iCol] Then ; Numeric sort
        If $iDims = 1 Then
            $nVal1 = Number($aArray[$i])
            $nVal2 = Number($aArray[DllStructGetData($tIndex, 1, $mi + 1)])
        Else
            $nVal1 = Number($aArray[$i][$iCol])
            $nVal2 = Number($aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])
        EndIf
        $r = $nVal1 < $nVal2 ? -1 : $nVal1 > $nVal2 ? 1 : 0
    Else ; Natural sort
        If $iDims = 1 Then
            $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)])[0]
        Else
            $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i][$iCol], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])[0]
        EndIf
    EndIf
...

Now maybe the asterix * is not the perfect character to use (maybe a couple of characters could be used) but this can be changed, in the same way you scripted this line :

Global $g_sArrayDisplay_RowPrefix = "#"

Jpm, if you think this could be interesting for the AutoIt community, or if you would like to exchange ideas about it, here or in PM, please do not hesitate. The only (easy) part remaining to solve should be to make disappear the asterix when Transpose is on (as headers will be found in the "Row" colum).

Finally, I got 2 other points I wanted to share with you :

* The local variable $k = 0 found 4 times in Func __ArrayDisplay_SortArrayStruct()
It's not needed anymore as it got always the same value as the existing $i variable. This variable $k was introduced in some sorting tests that were abandoned.

* What should be done with these 2 big arrays just before leaving __ArrayDisplay_Share ?

Global $_g_ArrayDisplay_aIndex
Global $_g_ArrayDisplay_aIndexes

I choose to erase them (assigning both to 0) so it frees several megs of memory in case of big arrays and several indexes (tested) before returning to the calling script.

Thanks for reading and good luck

[jpm]

Thanks for the improvement I will commit it

just a remark I don't know where you found the $k = 0 in the beta release. there is only one time

Cheers

[pixelsearch]

Hi Jpm,
Glad you liked the improvement

4 hours ago, jpm said:

just a remark I don't know where you found the $k = 0 in the beta release. there is only one time

I meant it's found 4 times within the function __ArrayDisplay_GetSortColStruct. Here is what I did to get rid of $k :

Local $lo, $hi, $mi, $r, $k = 0
Local $lo, $hi, $mi, $r, $k = 0
...

$k += 1
...

DllCall($hDll, "none", "RtlMoveMemory", "struct*", $pIndex + ($mi + 1) * 4, "struct*", $pIndex + $mi * 4, "ulong_ptr", ($k - $mi) * 4)
DllCall($hDll, "none", "RtlMoveMemory", "struct*", $pIndex + ($mi + 1) * 4, "struct*", $pIndex + $mi * 4, "ulong_ptr", ($i - $mi) * 4)
...

DllStructSetData($tIndex, 1, $k, $mi + 1 + ($lo = $mi + 1))
DllStructSetData($tIndex, 1, $i, $mi + 1 + ($lo = $mi + 1))

Which leads to the final function that includes the numeric sort :

expandcollapse popup

Func __ArrayDisplay_SortArrayStruct(Const ByRef $aArray, $iCol)

    Local $iDims = UBound($aArray, $UBOUND_DIMENSIONS)
    Local $tIndex = DllStructCreate("uint[" & $_g_ArrayDisplay_nRows & "]")
    Local $pIndex = DllStructGetPtr($tIndex)
    Static $hDll = DllOpen("kernel32.dll")
    Static $hDllComp = DllOpen("shlwapi.dll")

    Local $lo, $hi, $mi, $r, $nVal1, $nVal2

    ; Sorting by one column
    For $i = 1 To $_g_ArrayDisplay_nRows - 1
        $lo = 0
        $hi = $i - 1
        Do
            $mi = Int(($lo + $hi) / 2)
            If $_g_ArrayDisplay_aNumericSort[$iCol] Then ; Numeric sort
                If $iDims = 1 Then
                    $nVal1 = Number($aArray[$i])
                    $nVal2 = Number($aArray[DllStructGetData($tIndex, 1, $mi + 1)])
                Else
                    $nVal1 = Number($aArray[$i][$iCol])
                    $nVal2 = Number($aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])
                EndIf
                $r = $nVal1 < $nVal2 ? -1 : $nVal1 > $nVal2 ? 1 : 0
            Else ; Natural sort
                If $iDims = 1 Then
                    $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)])[0]
                Else
                    $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i][$iCol], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])[0]
                EndIf
            EndIf
            Switch $r
                Case -1
                    $hi = $mi - 1
                Case 1
                    $lo = $mi + 1
                Case 0
                    ExitLoop
            EndSwitch
        Until $lo > $hi
        DllCall($hDll, "none", "RtlMoveMemory", "struct*", $pIndex + ($mi + 1) * 4, "struct*", $pIndex + $mi * 4, "ulong_ptr", ($i - $mi) * 4)
        DllStructSetData($tIndex, 1, $i, $mi + 1 + ($lo = $mi + 1))
    Next

    Return $tIndex
EndFunc   ;==>__ArrayDisplay_SortArrayStruct

 

12 hours ago, pixelsearch said:

The only (easy) part remaining to solve should be to make disappear the asterix when Transpose is on (as headers will be found in the "Row" colum).

On second thought, there's nothing to code in this case because the user shouldn't add any * to its headers. The user knows that his columns will become rows (Transpose On) so * are now irrelevant in header(s) . It's even better to let them appear in this case, maybe some "stars" will wake him up

[jpm]

transpose is ok  without *

Thanks for all

[pixelsearch]

"Unstable sort" vs "Stable sort", is it really important and useful ?
There are very few comments about it on our Forum. I found a trac ticket #3620 dated 4 years ago, some stackoverflow comments, Wiki & other web pages etc...

The pics above explain it :
* Left pic : the array is unsorted  (location : plant 1 after plant 2)
But let's notice how the products are nicely presented in their own column (a, b, c, d, e) for each plant.

* Middle pic : we click on the location column's header to sort it, plant 1 comes first as expected... but now the product column is a total mess because the rows order hasn't been preserved when the same key was found during the sort (the keys here are plant #1 and plant #2). This is an unstable sort.

* Right pic : with the appropriate code, we click on location column's header and get a stable sort. The rows aren't scrambled any more for each plant and the product column looks great (as it was in the pic on the left)

AutoIt sorts are natively unstable (quicksort in _ArraySort, or the creation of the indexes for the virtual listview in _ArrayDisplay) . I found it challenging to write some code for _ArrayDisplay to get a stable sort when a column needs sorting. Also it allowed me to compare AutoIt results with C++ results (as C++ got natively a stable sort algorithm in its STL, added to its non-stable sort algorithm)

Comparative tests in C++ and AutoIt, on arrays of 10.000 rows and 6 columns (which got duplicate keys in their sorted column) gave good results (no differences in the output).

Maybe the following code to achieve a stable sort in ArrayDisplay could be useful to someone one day, who knows ?

expandcollapse popup

Func __ArrayDisplay_SortArrayStruct(Const ByRef $aArray, $iCol, $bStableSort = False)

    Local $iDims = UBound($aArray, $UBOUND_DIMENSIONS)
    Local $tIndex = DllStructCreate("uint[" & $_g_ArrayDisplay_nRows & "]")
    Local $pIndex = DllStructGetPtr($tIndex)
    Static $hDll = DllOpen("kernel32.dll")
    Static $hDllComp = DllOpen("shlwapi.dll")

    Local $lo, $hi, $mi, $r, $nVal1, $nVal2

    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
    If $bStableSort Then Local $aDupe[$_g_ArrayDisplay_nRows], $iDupe, $bMove
    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

    ; Sorting by one column
    For $i = 1 To $_g_ArrayDisplay_nRows - 1
        $lo = 0
        $hi = $i - 1
        Do
            $mi = Int(($lo + $hi) / 2)

            If $_g_ArrayDisplay_aNumericSort[$iCol] Then ; Numeric sort
                If $iDims = 1 Then
                    $nVal1 = Number($aArray[$i])
                    $nVal2 = Number($aArray[DllStructGetData($tIndex, 1, $mi + 1)])
                Else
                    $nVal1 = Number($aArray[$i][$iCol])
                    $nVal2 = Number($aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])
                EndIf
                $r = $nVal1 < $nVal2 ? -1 : $nVal1 > $nVal2 ? 1 : 0
            Else ; Natural sort
                If $iDims = 1 Then
                    $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)])[0]
                Else
                    $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i][$iCol], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])[0]
                EndIf
            EndIf

            Switch $r
                Case -1
                    $hi = $mi - 1
                Case 1
                    $lo = $mi + 1
                Case 0
                    ExitLoop
            EndSwitch
        Until $lo > $hi

        ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
        If $r = 0 And $bStableSort Then
            If $i = $mi + 1 Then
                DllStructSetData($tIndex, 1, $i, $i + 1)
                $aDupe[$i] = $i
                $aDupe[DllStructGetData($tIndex, 1, $i)] = $i
            Else
                $iDupe = $aDupe[DllStructGetData($tIndex, 1, $mi + 1)]
                If $iDupe = 0 Then
                    $aDupe[$i] = $i
                    $aDupe[DllStructGetData($tIndex, 1, $mi + 1)] = $i
                    DllCall($hDll, "none", "RtlMoveMemory", "struct*", $pIndex + ($mi + 1) * 4, "struct*", $pIndex + $mi * 4, "ulong_ptr", ($i - $mi) * 4)
                    DllStructSetData($tIndex, 1, $i, $mi + 2)
                Else
                    $bMove = False
                    For $j = $mi + 2 To $i
                        If $aDupe[DllStructGetData($tIndex, 1, $j)] <> $iDupe Then
                            $bMove = True
                            ExitLoop; For $j...
                        EndIf
                    Next
                    If Not $bMove Then
                        DllStructSetData($tIndex, 1, $i, $i + 1)
                    Else
                        DllCall($hDll, "none", "RtlMoveMemory", "struct*", $pIndex + $j * 4, "struct*", $pIndex + ($j - 1) * 4, "ulong_ptr", ($i - $j + 1) * 4)
                        DllStructSetData($tIndex, 1, $i, $j)
                    EndIf
                    $aDupe[$i] = $iDupe
                EndIf
            EndIf
            ContinueLoop ; For $i...
        EndIf
        ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

        DllCall($hDll, "none", "RtlMoveMemory", "struct*", $pIndex + ($mi + 1) * 4, "struct*", $pIndex + $mi * 4, "ulong_ptr", ($i - $mi) * 4)
        DllStructSetData($tIndex, 1, $i, $mi + 1 + ($lo = $mi + 1))
    Next

    Return $tIndex
EndFunc   ;==>__ArrayDisplay_SortArrayStruct

If anyone finds a bug, please report here, thank you 

Edit: March 4, 2022 :

The code above is ok to get a stable sort.
But I'm revising my opinion concerning the speed process. If the array is big and the column to sort got plenty of duplicate keys, then the time to prepare the "stable-sort index" will be longer than preparing it with the native unstable sort code.
So one should always have this in mind and use the stable sort only when it's really needed (on small arrays of a few thousands elements for example)

Edit: March 5, 2022 :

Working on a solution for a faster stable sort, to avoid the nasty loop For $j = $mi + 2 To $i (which takes time if any duplicate key is found many times in the array)
Results look promising, When ready, I'll post an alternate way in a new post below.

Edited March 5, 2022 by pixelsearch
Comments on speed

[pixelsearch]

Stable sort : 2nd version.
The code below runs much faster than the code in the preceding post. Anyway, I'll leave both versions on the Forum so it will show the evolution of the code.

All comments are left in the code in case someone is interested.
If you guys think there is some improvement to do (or find a bug), don't hesitate to share your thoughts.
Thank you 

expandcollapse popup

Func __ArrayDisplay_SortArrayStruct(Const ByRef $aArray, $iCol, $bStableSort = False)

    Local $iDims = UBound($aArray, $UBOUND_DIMENSIONS)
    Local $tIndex = DllStructCreate("uint[" & $_g_ArrayDisplay_nRows & "]")
    Local $pIndex = DllStructGetPtr($tIndex)
    Static $hDll = DllOpen("kernel32.dll")
    Static $hDllComp = DllOpen("shlwapi.dll")

    Local $lo, $hi, $mi, $r, $nVal1, $nVal2
    If $bStableSort Then Local $aDupe[$_g_ArrayDisplay_nRows][3], $iDupe, $iNdx, $iRank, $iNb_Elem, $iDiff, $j

    ; Sorting by one column
    For $i = 1 To $_g_ArrayDisplay_nRows - 1
        $lo = 0
        $hi = $i - 1
        Do
            $mi = Int(($lo + $hi) / 2)

            If $_g_ArrayDisplay_aNumericSort[$iCol] Then ; Numeric sort
                If $iDims = 1 Then
                    $nVal1 = Number($aArray[$i])
                    $nVal2 = Number($aArray[DllStructGetData($tIndex, 1, $mi + 1)])
                Else
                    $nVal1 = Number($aArray[$i][$iCol])
                    $nVal2 = Number($aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])
                EndIf
                $r = $nVal1 < $nVal2 ? -1 : $nVal1 > $nVal2 ? 1 : 0
            Else ; Natural sort
                If $iDims = 1 Then
                    $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)])[0]
                Else
                    $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i][$iCol], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])[0]
                EndIf
            EndIf

            Switch $r
                Case -1
                    $hi = $mi - 1
                Case 1
                    $lo = $mi + 1
                Case 0
                    ExitLoop
            EndSwitch
        Until $lo > $hi

        If $r = 0 And $bStableSort Then
            $iNdx = DllStructGetData($tIndex, 1, $mi + 1) ; low element ( < $i) of a dupe serie. low doesn't mean lowest.
            $iDupe = $aDupe[$iNdx][0] ; it's empty (a new dupe serie is starting now) OR it's filled ( > 0 , a dupe serie already exists)
            If $iDupe = 0 Then ; new dupe serie (2 elements found same)
                $aDupe[$iNdx][0] = $i ; "key" for this dupe serie (NEVER = 0 as based on $i which is always > 0 in loop For $i = 1 To...)
                $aDupe[$iNdx][1] = 1 ; rank for this element in its dupe serie (1 because $i element will always got a higher rank, see below)

                $aDupe[$i][0] = $i ; "key" for this dupe serie. This is a "crucial" element (its row = its [$i][0] content, see below)
                $aDupe[$i][1] = 2  ; rank for this element in its dupe serie (always 2 for the crucial element, as there is a lower element)
                $aDupe[$i][2] = 2  ; number of elements in this dupe serie (2 of course here), always placed in Col 2 of crucial element.

                DllCall($hDll, "none", "RtlMoveMemory", "struct*", $pIndex + ($mi + 1) * 4, "struct*", $pIndex + $mi * 4, "ulong_ptr", ($i - $mi) * 4)
                DllStructSetData($tIndex, 1, $i, $mi + 2)

            Else ; 3+ elements of same dupe serie
                $iRank = $aDupe[$iNdx][1] ; 1+ (always lower or equal than $iNb_Elem below)
                $iNb_Elem = $aDupe[$iDupe][2] ; 2+ (number of dupes already existing in this serie, will be incremented below)
                $iDiff = $iNb_Elem - $iRank ; 0+ (how many elements of this dupe serie are already well placed at the right of $mi + 1 ?)
                $j = $mi + 1 + $iDiff ; 2+ ; to avoid repeating the same calculation several times below.

                DllCall($hDll, "none", "RtlMoveMemory", "struct*", $pIndex + ($j + 1) * 4, "struct*", $pIndex + $j * 4, "ulong_ptr", ($i - $j) * 4)
                DllStructSetData($tIndex, 1, $i, $j + 1)

                $aDupe[$i][0] = $iDupe ; same "key" for this dupe serie (+++ now at least 3 elements in it +++)
                $aDupe[$i][1] = $iNb_Elem + 1 ; 3+ (rank for this element in its dupe serie)
                $aDupe[$iDupe][2] = $iNb_Elem + 1 ; 3+ (number of elements in this dupe serie) updated in Col 2 of the crucial element.
            EndIf                                 ; note how Col 2 is used only by crucial elements. It's empty for all other elements.

        Else
            DllCall($hDll, "none", "RtlMoveMemory", "struct*", $pIndex + ($mi + 1) * 4, "struct*", $pIndex + $mi * 4, "ulong_ptr", ($i - $mi) * 4)
            DllStructSetData($tIndex, 1, $i, $mi + 1 + ($lo = $mi + 1))
        EndIf
    Next

    Return $tIndex
EndFunc   ;==>__ArrayDisplay_SortArrayStruct

Edit: Mar 19, 2022 :
* AutoIt Stable sort code adapted to C++ in this link

Edited March 19, 2022 by pixelsearch
comment added

[pixelsearch]

@jpm just added 2 lines in ArrayDisplayInternals.au3 to use this extended listview style :

LVS_EX_HEADERDRAGDROP : Enables drag-and-drop reordering of columns in a list-view control

According to the variables names in the function, the 2 lines are :

Local Const $_ARRAYCONSTANT_LVS_EX_HEADERDRAGDROP = 0x00000010
...
GUICtrlSendMsg($idListView, $_ARRAYCONSTANT_LVM_SETEXTENDEDLISTVIEWSTYLE, $_ARRAYCONSTANT_LVS_EX_HEADERDRAGDROP, $_ARRAYCONSTANT_LVS_EX_HEADERDRAGDROP)

It seems to work fine, even on a virtual listview.
I needed this in a script today, maybe it could be useful to some other users too, you'll decide