Is _Arraysort() a wimp?

[UEZ]

@KaFu:

This is missing:

DllClose($h_DLL_user32)
    DllClose($h_DLL_msvcrt)
    DllClose($h_DLL_Kernel32)

Br,

UEZ

[KaFu]

Is yours based upon an older version than the one at the top of this thread: ?

Or is it a refined/improved version?

Using it as a simple UDF without the #region data seems the easiest to implement...

Yes, it's based on that example. I've made the dll handles global (tiny, tiny speedup) and added reserved memory for the binary strings to execute, otherwise you might run into DEP issues on Vista+ Systems.

This is missing:

Can't hurt to explicitly close the dlls , added the lines.

[Spiff59]

I haven't a clue as to how the precompiler/interpreter handles the #region/#endregion directives.

Can I simply paste the #region block above the function and save it as a stand-alone UDF in my Include folder?

Or does the #region text need to be physically within the base module?

If not, is there any sort of requirement that the new "#Include <ArraySortCLib.au3>" statement be the first include in the source (putting the #region code at the top)?

Edited February 19, 2013 by Spiff59

[BrewManNH]

#region/#endregion are always optional, the script doesn't see it when you run it.

[guinness]

It's just like comment tags, nothing special but for readability purposes.

[Jos]

It's just like comment tags, nothing special but for readability purposes.

Readability indeed by being able to FOLD the whole #region-#endregion section in SciTE, which makes it easier to work on larger scripts.

[guinness]

Readability indeed by being able to FOLD the whole #region-#endregion section in SciTE, which makes it easier to work on larger scripts.

Accompanied with SciTE Jump too.

[Spiff59]

I see... So in order to keep KaFu's enhancement's, that only load the DLL's once, but have a single-file standalone UDF with no external requirements, one would need to merge the #region code into the function and apply some sort of test (or passed parameter) to ensure that that code was only executed on the first call?

[KaFu]

I used the region tag for pure readability, to show which part of the global code is necessary for the function to work. This should do it:

ArraySortClib.au3

expandcollapse popup

#include-once
#include <memory.au3>
Global $h_DLL_user32 = DllOpen("user32.dll")
Global $h_DLL_msvcrt = DllOpen("msvcrt.dll")
Global $h_DLL_Kernel32 = DllOpen("kernel32.dll")
Global $pEnumProc_ArraySortClib_Mem1 = _MemVirtualAlloc(0, 64, $MEM_COMMIT, $PAGE_EXECUTE_READWRITE) ; needed for _ArraySortClib()
Global $pEnumProc_ArraySortClib_Mem2 = _MemVirtualAlloc(0, 64, $MEM_COMMIT, $PAGE_EXECUTE_READWRITE) ; needed for _ArraySortClib()
Global $pEnumProc_ArraySortClib_Mem3 = _MemVirtualAlloc(0, 36, $MEM_COMMIT, $PAGE_EXECUTE_READWRITE) ; needed for _ArraySortClib()
OnAutoItExitRegister("ArraySortClib_Mem_Release")
Func ArraySortClib_Mem_Release()
    _MemVirtualFree($pEnumProc_ArraySortClib_Mem1, 0, $MEM_RELEASE) ; release mem reserved for _ArraySortClib()
    _MemVirtualFree($pEnumProc_ArraySortClib_Mem2, 0, $MEM_RELEASE) ; release mem reserved for _ArraySortClib()
    _MemVirtualFree($pEnumProc_ArraySortClib_Mem3, 0, $MEM_RELEASE) ; release mem reserved for _ArraySortClib()
    DllClose($h_DLL_user32)
    DllClose($h_DLL_msvcrt)
    DllClose($h_DLL_Kernel32)
EndFunc   ;==>ArraySortClib_Mem_Release


; http://www.autoitscript.com/forum/index.php?showtopic=63525&view=findpost&p=474072
;===============================================================================
; Function Name: _ArraySortClib() v4
; Description:     Sort 1D/2D array using qsort() from C runtime library
; Syntax:
; Parameter(s):  $Array - the array to be sorted, ByRef
;               $iMode - sort mode, can be one of the following:
;                   0 = numerical, using double precision float compare
;                   1 = string sort, case insensitive (default)
;                   2 = string sort, case sensitive
;                   3 = word sort, case insensitive - compatible with AutoIt's native compare
;               $fDescend - sort direction. True = descending, False = ascending (default)
;               $iStart - index of starting element (default 0 = $array[0])
;               $iEnd - index of ending element (default 0 = Ubound($array)-1)
;               $iColumn - index of column to sort by (default 0 = first column)
;               $iStrMax - max string length of each array element to compare (default 4095 chars)
; Requirement(s): msvcrt.dll (shipped with Windows since Win98 at least), 32-bit version of AutoIt
; Return Value(s): Success = Returns 1
;               Failure = Returns 0 and sets error:
;                   @error 1 = invalid array
;                   @error 2 = invalid param
;                   @error 3 = dll error
;                   @error 64 = 64-bit AutoIt unsupported
; Author(s): Siao
; Modification(s): KaFu, added three global _MemVirtualAlloc() calls to top of script to prevent DEP Errors:
;
; Global $pEnumProc_ArraySortClib_Mem1 = _MemVirtualAlloc(0, 64, $MEM_COMMIT, $PAGE_EXECUTE_READWRITE) ; needed for _ArraySortClib()
; Global $pEnumProc_ArraySortClib_Mem2 = _MemVirtualAlloc(0, 64, $MEM_COMMIT, $PAGE_EXECUTE_READWRITE) ; needed for _ArraySortClib()
; Global $pEnumProc_ArraySortClib_Mem3 = _MemVirtualAlloc(0, 36, $MEM_COMMIT, $PAGE_EXECUTE_READWRITE) ; needed for _ArraySortClib()
;
;===============================================================================

Func _ArraySortClib(ByRef $array, $iMode = 1, $fDescend = False, $iStart = 0, $iEnd = 0, $iColumn = 0, $iStrMax = 4095)
    If @AutoItX64 Then Return SetError(64, 0, 0)
    Local $iArrayDims = UBound($array, 0)
    If @error Or $iArrayDims > 2 Then Return SetError(1, 0, 0)
    Local $iArraySize = UBound($array, 1), $iColumnMax = UBound($array, 2)
    If $iArraySize < 2 Then Return SetError(1, 0, 0)
    If $iEnd < 1 Or $iEnd > $iArraySize - 1 Then $iEnd = $iArraySize - 1
    If ($iEnd - $iStart < 1) Then
        ; ConsoleWrite("- _ArraySortClib ERROR: #1 - " & ($iEnd - $iStart < 1) & @tab & ($iEnd - $iStart) & @crlf)
        Return SetError(2, 0, 0)
    EndIf
    If $iArrayDims = 2 And ($iColumnMax - $iColumn < 0) Then
        ; ConsoleWrite("- _ArraySortClib ERROR: #2 - " & $iArrayDims & @tab & ($iColumnMax - $iColumn < 0) & @tab & ($iColumnMax - $iColumn) & @crlf)
        Return SetError(2, 0, 0)
    EndIf
    If $iStrMax < 1 Then
        ; ConsoleWrite("- _ArraySortClib ERROR: #3 - " & $iStrMax & @crlf)
        Return SetError(2, 0, 0)
    EndIf
    Local $i, $j, $iCount = $iEnd - $iStart + 1, $fNumeric, $aRet, $sZero = ChrW(0), $sStrCmp, $sBufType = 'byte[', $tSource, $tIndex, $tFloatCmp

    Local $tEnumProc = DllStructCreate('byte[64]', $pEnumProc_ArraySortClib_Mem1)
    Local $tCmpWrap = DllStructCreate('byte[64]', $pEnumProc_ArraySortClib_Mem2)

    If $h_DLL_msvcrt = -1 Then Return SetError(3, 0, 0)

    ;; initialize compare proc
    Switch $iMode
        Case 0
            $fNumeric = True
            $tFloatCmp = DllStructCreate('byte[36]', $pEnumProc_ArraySortClib_Mem3)
            DllStructSetData($tFloatCmp, 1, '0x8B4C24048B542408DD01DC1ADFE0F6C440750D80E441740433C048C333C040C333C0C3')
            DllStructSetData($tCmpWrap, 1, '0xBA' & Hex(Binary(DllStructGetPtr($tFloatCmp)), 8) & '8B4424088B4C2404FF30FF31FFD283C408C3')
            DllStructSetData($tEnumProc, 1, '0x8B7424048B7C24088B4C240C8B442410893789470483C60883C708404975F1C21000')
        Case 1, 2
            $sStrCmp = "_strcmpi" ;case insensitive
            If $iMode = 2 Then $sStrCmp = "strcmp" ;case sensitive
            $aRet = DllCall($h_DLL_Kernel32, 'ptr', 'GetModuleHandle', 'str', 'msvcrt.dll')
            $aRet = DllCall($h_DLL_Kernel32, 'ptr', 'GetProcAddress', 'ptr', $aRet[0], 'str', $sStrCmp)
            ;If $aRet[0] = 0 Then Return SetError(3, 0, 0 * DllClose($h_DLL_msvcrt))
            If $aRet[0] = 0 Then Return SetError(3, 0, 1)
            DllStructSetData($tCmpWrap, 1, '0xBA' & Hex(Binary($aRet[0]), 8) & '8B4424088B4C2404FF30FF31FFD283C408C3')
            DllStructSetData($tEnumProc, 1, '0x8B7424048B7C24088B4C240C8B542410893789570483C7088A064684C075F9424975EDC21000')
        Case 3
            $sBufType = 'wchar['
            $aRet = DllCall($h_DLL_Kernel32, 'ptr', 'GetModuleHandle', 'str', 'kernel32.dll')
            $aRet = DllCall($h_DLL_Kernel32, 'ptr', 'GetProcAddress', 'ptr', $aRet[0], 'str', 'CompareStringW')
            ;If $aRet[0] = 0 Then Return SetError(3, 0, 0 * DllClose($h_DLL_msvcrt))
            If $aRet[0] = 0 Then Return SetError(3, 0, 1)
            DllStructSetData($tCmpWrap, 1, '0xBA' & Hex(Binary($aRet[0]), 8) & '8B4424088B4C24046AFFFF306AFFFF3168000000006800040000FFD283E802C3')
            DllStructSetData($tEnumProc, 1, '0x8B7424048B7C24088B4C240C8B542410893789570483C7080FB70683C60285C075F6424975EAC21000')
        Case Else

            Return SetError(2, 0, 0)
    EndSwitch
    ;; write data to memory
    If $fNumeric Then
        $tSource = DllStructCreate('double[' & $iCount & ']')
        If $iArrayDims = 1 Then
            For $i = 1 To $iCount
                DllStructSetData($tSource, 1, $array[$iStart + $i - 1], $i)
            Next
        Else
            For $i = 1 To $iCount
                DllStructSetData($tSource, 1, $array[$iStart + $i - 1][$iColumn], $i)
            Next
        EndIf
    Else
        Local $sMem = ""
        If $iArrayDims = 1 Then
            For $i = $iStart To $iEnd
                $sMem &= StringLeft($array[$i], $iStrMax) & $sZero
            Next
        Else
            For $i = $iStart To $iEnd
                $sMem &= StringLeft($array[$i][$iColumn], $iStrMax) & $sZero
            Next
        EndIf
        $tSource = DllStructCreate($sBufType & StringLen($sMem) + 1 & ']')
        DllStructSetData($tSource, 1, $sMem)
        $sMem = ""
    EndIf
    ;; index data
    $tIndex = DllStructCreate('int[' & $iCount * 2 & ']')
    DllCall($h_DLL_user32, 'uint', 'CallWindowProc', 'ptr', DllStructGetPtr($tEnumProc), 'ptr', DllStructGetPtr($tSource), 'ptr', DllStructGetPtr($tIndex), 'int', $iCount, 'int', $iStart)
    ;; sort
    DllCall($h_DLL_msvcrt, 'none:cdecl', 'qsort', 'ptr', DllStructGetPtr($tIndex), 'int', $iCount, 'int', 8, 'ptr', DllStructGetPtr($tCmpWrap))
    ;DllClose($h_DLL_msvcrt)
    ;; rearrange the array by sorted index
    Local $aTmp = $array, $iRef
    If $iArrayDims = 1 Then ; 1D
        If $fDescend Then
            For $i = 0 To $iCount - 1
                $iRef = DllStructGetData($tIndex, 1, $i * 2 + 2)
                $array[$iEnd - $i] = $aTmp[$iRef]
            Next
        Else ; ascending
            For $i = $iStart To $iEnd
                $iRef = DllStructGetData($tIndex, 1, ($i - $iStart) * 2 + 2)
                $array[$i] = $aTmp[$iRef]
            Next
        EndIf
    Else ; 2D
        If $fDescend Then
            For $i = 0 To $iCount - 1
                $iRef = DllStructGetData($tIndex, 1, $i * 2 + 2)
                For $j = 0 To $iColumnMax - 1
                    $array[$iEnd - $i][$j] = $aTmp[$iRef][$j]
                Next
            Next
        Else ; ascending
            For $i = $iStart To $iEnd
                $iRef = DllStructGetData($tIndex, 1, ($i - $iStart) * 2 + 2)
                For $j = 0 To $iColumnMax - 1
                    $array[$i][$j] = $aTmp[$iRef][$j]
                Next
            Next
        EndIf
    EndIf
    Return 1
EndFunc   ;==>_ArraySortClib

ArraySortClib_Example.au3

#include <ArraySortClib.au3>

#include <array.au3> ; not needed for _ArraySortClib() itself, only for example
Global $aFileList[60001][2] = [[60000, ""]]

$timer = TimerInit()
For $i = 1 To 30000
    $aFileList[$i][0] = "FileType_A" & $i
    $aFileList[$i][1] = StringRight("00000" & $i, 6)
Next
For $i = 1 To 30000
    $aFileList[$i + 30000][0] = "FileType_B" & $i
    $aFileList[$i + 30000][1] = StringRight("00000" & $i, 6)
Next
ConsoleWrite(TimerDiff($timer) & @CRLF)

$timer = TimerInit()
_ArrayDisplay($aFileList)
ConsoleWrite(TimerDiff($timer) & @CRLF)

; _ArraySort($aFileList, 0, 1, 0, 1)
$timer = TimerInit()
_ArraySortClib($aFileList, 1, 0, 0, 0, 1)
ConsoleWrite(TimerDiff($timer) & @CRLF)

$timer = TimerInit()
_ArrayDisplay($aFileList)
ConsoleWrite(TimerDiff($timer) & @CRLF)

[Spiff59]

Thank you, KaFu.

I've added that to my short list of custom UDF's.

Being x64-incompatible I guess it can't be a replacement for the flawed _ArraySort().

It might make a nice final entry to Siao's 2008 thread, although you'd be guilty of necromancy.

(A shame he hasn't logged in for a few years, he was another smart cookie)

[guinness]

If it's worth reviving then I see no reason not to.