Bug with _ArrayDisplay in 3.3.16.0?

[HurleyShanabarger]

I tested a script, that I found over here and the _ArrayDisplay at the end was not working.

I tested with 3.14.5.0 and there everything is fine.

#include <Array.au3>

Dim $aTest_Null[1][2] = [[Null, Null]]
_ArrayDisplay($aTest_Null) ; this is working
ReDim $aTest_Null[2][2]
_ArrayDisplay($aTest_Null)  ; this is never show; seems to get stuck in "__ArrayDisplay_SortArrayStruct"

The script above narrows the problem down.  It seems if an array contains a Keyword the interall call of __ArrayDisplay_SortArrayStruct (@Line 772 of ArrayDisplayInternals.au3) never exits the Do...Until loop.

Edited March 8, 2022 by HurleyShanabarger

[ad777]

@HurleyShanabarger

it get stuck becuz Array :

$aTest_Null = NULL = 0

go around to:Function

__ArrayDisplay_SortArrayStruct

just Add below Switch $r:

Case -2
    ExitLoop 2

 

[Jos]

The point is that it shouldn't hang in the first place so need to be fixed in _ArrayDisplay when that is wrong.

[pixelsearch]

It seems that the Null keyword makes StrCmpLogicalW crash (the function is supposed to "compare two Unicode strings" says MSDN). For example, AutoIt (no matter the version) crashes with this script :

Local $aArray[2] = [Null, 123]
DllCall("shlwapi.dll", 'int', 'StrCmpLogicalW', 'wstr', $aArray[0], 'wstr', $aArray[1])

Exit code: 3221225477, which is "C0000005" (0xC0000005 STATUS_ACCESS_VIOLATION)

In the new version of ArrayDisplay 3.3.16.0, data comes directly from the array, so Null is passed to the function & crash.

$r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)])[0]

In the precedent version of ArrayDisplay 3.3.14.5, Null wasn't passed to the function because of these lines :

$sVal1 = __ArrayDisplay_GetItemText(...)
$sVal2 = __ArrayDisplay_GetItemText(...)
$nResult = DllCall('shlwapi.dll', 'int', 'StrCmpLogicalW', 'wstr', $sVal1, 'wstr', $sVal2)[0]

I just tested the 3 other AutoIt keywords used as datatypes : True, False & Default, they don't make the function crash. Let's wait @jpm or anyone wishing to comment, who may have complementary informations.

Edit 1: for what it's worth, the following code doesn't crash (when we click the column header to sort) :

Local $aArray[2] = [Null, 123]

_ArrayDisplay($aArray, "Title", Default, Default, Default, _
    "Numerics*") ; * at end of any header means numeric sort for the concerned column.

Edit 2: an idea to solve it.
As only the Natural sort is concerned, instead of passing $aArray[$i] to the function, can't we pass String($aArray[$i])  or even  "" & $aArray[$i]  ?

That would solve the NULL issue (tested right now) and it doesn't seem to delay a lot the sorting result, but the result itself should be checked carefully to make sure it brings back the same sorting order than the original code (in case $aArray[$i] contains digits or not etc...)

Edit 3: I made tests like JP likes them : rebooting the computer between each test !
Tests on 100.000 rows, 6 cols, Srandom(6) to generate the same array, Natural sort on String column (column 0)

1) First test with original code version 3.3.16.0, i.e. params of the function are $aArray[$i][$iCol]  and  $aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol]

2) Second test with String($aArray[$i][$iCol])  and  String(...)

3) Third test with "" & $aArray[$i][$iCol]  and  "" & $aArray[DllStruct...]

Fastest is of course 1)
2) and 3) run in nearly same speed time, but 5% slowest than 1)

Comparing the results after each sort (with DebugArrayDisplay, button copy data, save in 3 different text files) => exactly the same result no matter the test (lucky us !)

String elements where composed of letters, digits & some other characters (see below) :

$aArray = FAS_Random2DArrayAu3($iRows, "sifdtr", "abcdefghijkl0123456789.-+*/_,;")

When it was all finished, for fun, I added the following lines in the code, just to confirm the crash (test 1) or no crash (tests 2 & 3) ...but this time it was only with 1000 rows.

$aArray[0][0] = Null
$aArray[1][0] = Null
$aArray[100][0] = Null
$aArray[500][0] = Null
$aArray[999][0] = Null

Hope it helps JP to decide, because leaving "as-is" will crash as soon as a NULL element is found when a non-numeric sort is required, while changing the code won't crash but it should be 5% slower.

Edit 4: the horrible slowest way to patch it ?

If $aArray[$i][$iCol] = NULL Or $aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol] = NULL Then

    $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', String($aArray[$i][$iCol]), 'wstr', String($aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol]))[0]

Else

    $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i][$iCol], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])[0]

EndIf

It's not because the number of rows = 100.000 that StrCmpLogicalW is called 100.000 times. 
Here are the number of times that StrCmpLogicalW is called :

528 comparaisons for 100 rows, 8571 for 1000 rows, 118.956 for 10.000 rows, 1.522.498 for 100.000 rows.
Who wants to add an If statement evaluated 1.522.498 times ?

Thanks to HurleyShanabarger who discovered this issue.

Edited March 8, 2022 by pixelsearch
Edit 4

[HurleyShanabarger]

Thank you for you effort on this one. Guess we wait for some feedback and create a ticket afterwards?

Edit: The solution from @ad777 also looks nice, as it handles the unexpected return value from StrCmpLogicalW.

Edited March 8, 2022 by HurleyShanabarger

[pixelsearch]

1 hour ago, HurleyShanabarger said:

Guess we wait for some feedback

Exactly. Jpm will certainly read all this soon (his profile stipulates he "Last visited" 13 hours ago, that was before your 1st post). After he reads all this, he'll certainly fix it, let's wait and see, especially he's involved in the Trac system and in ArrayDisplay, he's everywhere 

I just saw Nine's script (the one you indicated in your 1st post) and yes, his column 7 got 2 NULL entries on my computer (rows 0 and 1) which made the new ArrayDisplay crash when I tried to sort the column. Forcing these 2 entries from NULL to Default (the other keyword) doesn't make ArrayDisplay crash. Let's hope NULL will be the only issue we'll encounter, fingers crossed. Also, when I patched the function with String() as described above, then Nine's original script didn't crash anymore, even with its 2 NULL's.

ad777's solution didn't make it for me (I tried it) because AutoIt doesn't loop as it apparently does for you, it exits immediately ("AutoIt encountered an issue and must close") with an exit code = 3221225477

When I surround the DllCall with 2 ConsoleWrite, then the 2nd ConsoleWrite never shows for me because AutoIt quits during DllCall :

ConsoleWrite("1" & @lf) ; this one is displayed

$r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i][$iCol], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])[0]

ConsoleWrite("2" & @lf) ; this one isn't displayed

I'm not aware of this return value of -2 from StrCmpLogicalW, I always thought it was -1, 1 or 0 as stipulated on MSDN's page.

Edit: in your 1st script, you wrote :

Dim $aTest_Null[1][2] = [[Null, Null]]
_ArrayDisplay($aTest_Null) ; this is working

It's normal that it works because if you look at the function __ArrayDisplay_SortArrayStruct, you'll notice this loop :

For $i = 1 To $_g_ArrayDisplay_nRows - 1
    ...

As you got only 1 row in the array, then the loop isn't executed at all (For $i = 1 To 0), that's why you don't have a crash when 1 row, because there's nothing to sort.

And now it's time to go to bed, it was a long day 

Edited March 8, 2022 by pixelsearch

[pixelsearch]

Until it's officially fixed, this is what I did to patch ArrayDisplayInternals.au3 version 3.3.16.0, avoiding the crash when a Null element is encountered.

Lines 796 to 802 were :

Else ; Natural sort
    If $iDims = 1 Then
        $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)])[0]
    Else
        $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', $aArray[$i][$iCol], 'wstr', $aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol])[0]
    EndIf
EndIf

Patched to :

Else ; Natural sort (4 String() has been added to the following, or AutoIt crashes when a NULL element is found in the array)
    If $iDims = 1 Then
        $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', String($aArray[$i]), 'wstr', String($aArray[DllStructGetData($tIndex, 1, $mi + 1)]))[0]
    Else
        $r = DllCall($hDllComp, 'int', 'StrCmpLogicalW', 'wstr', String($aArray[$i][$iCol]), 'wstr', String($aArray[DllStructGetData($tIndex, 1, $mi + 1)][$iCol]))[0]
    EndIf
EndIf

If it's not urgent for you, please wait for the new official release, maybe the fix will be different.
If it's urgent...
 

Edited March 9, 2022 by pixelsearch

[pixelsearch]

On 3/8/2022 at 1:01 PM, ad777 said:

go around to:Function

__ArrayDisplay_SortArrayStruct

just Add below Switch $r:

Case -2
    ExitLoop 2

 

Func __ArrayDisplay_GetSortColStruct(Const ByRef $aArray, $iCol)
    ...
    Switch $r
        Case -1
            $hi = $mi - 1
        Case 1
            $lo = $mi + 1
        Case -2         ; <============== added line
            ExitLoop 2  ; <============== added line
        Case 0
            ExitLoop
    EndSwitch
    ...
EndFunc

We spent a long time yesterday with jpm studying this miraculous solution but it can't work.
It seems to work on OP's 2 rows script... because there is nothing to show in his 2 rows array, but have you tried it on a simple example like this ?

#include <Array.au3>

Local $aTest_Null[8] = ["w", "c", Null, "z", "a", "y", "x", "b"]
_ArrayDisplay($aTest_Null, "Test " & @AutoItVersion)

As soon as a Null element is found, "ExitLoop 2" breaks out of the Switch $r loop and the For $i = ... loop, to directly reach the instruction Return $tIndex. What about all the elements of the array that haven't been sorted, as shown in the pic below ?

The half-baked $tIndex returned won't sort anything good, as shown on the right pic, when you click Col0 header to sort it.

Content of the incomplete $tIndex :

$tIndex = 1  0  0  0  0  0  0  0

 

[HurleyShanabarger]

Thank you both, for looking into this.