Garbage in Virtual ListView

[RAMzor]

Hello, 
I'm trying to implement Virtual ListView in my project. I have been based on example from @LarsJ.
The script loads data from csv file instead of array as in original example and perform some parsing.
Sometimes, when running a script, it adds garbage to the end of the correct data.
The data is loaded correctly into the global array (double-clicking on a row with garbage displays it correct in the console), but is not displayed correctly.
What am I missing or doing wrong?

expandcollapse popup

#include <GUIConstants.au3>
#include <WindowsConstants.au3>
#include <GuiImageList.au3>
#include <GuiListView.au3>
#include <GuiStatusBar.au3>
#include <FileConstants.au3>
#include <GuiListBox.au3>
#include <SendMessage.au3>
#include <EditConstants.au3>


Opt("MustDeclareVars", 1)

Global $hWnd, $idLV, $hLV, $aItems[1][1]
Global $gWinXMin = 1130, $gWinYMin = 615
Global Const $TopMost = 262144 ; Set MsgBox top-most attribute


Example()


Func Example()
    Local $Ver = "Virtual LV"

    ; Create GUI
    $hWnd = GUICreate("iTS - Test Sequencer   [Ver " & $Ver & "]", $gWinXMin, $gWinYMin, -1, -1, BitOR($WS_SIZEBOX, $WS_SYSMENU, $WS_MAXIMIZEBOX, $WS_MINIMIZEBOX))

    ; Create ListView                                                                        Reduces flicker          Checkboxes        Tooltip               Icons
    Local  $iLvStyle = BitOR($WS_EX_CLIENTEDGE, $LVS_EX_FULLROWSELECT, $LVS_EX_GRIDLINES, $LVS_EX_DOUBLEBUFFER, $LVS_EX_CHECKBOXES, $LVS_EX_INFOTIP);, $LVS_EX_SUBITEMIMAGES)
    $idLV = GUICtrlCreateListView("", 8, 50, 915, 477, $LVS_OWNERDATA, $iLvStyle)
    $hLV = GUICtrlGetHandle($idLV)   ;                      Virtual listview


    ; --- ListView Add columns ---
      _GUICtrlListView_AddColumn($hLV, "# ID", 45)         ; 0
      _GUICtrlListView_AddColumn($hLV, "Group", 120)       ; 1
      _GUICtrlListView_AddColumn($hLV, "Name", 335)        ; 2
      _GUICtrlListView_AddColumn($hLV, "Min", 50)          ; 3
      _GUICtrlListView_AddColumn($hLV, "Max", 50)          ; 4
      _GUICtrlListView_AddColumn($hLV, "Result", 50)       ; 5
      _GUICtrlListView_AddColumn($hLV, "Status", 50)       ; 6
      _GUICtrlListView_AddColumn($hLV, "Description", 179) ; 7

    ; Checkboxes
    _GUICtrlListView_SetCallBackMask($hLV, 32)   ; 32 - The application stores the image list index of the current state image

    ; Load icon images
;~  Local $hImage = _GUIImageList_Create()
;~  _GUIImageList_Add($hImage, _GUICtrlListView_CreateSolidBitMap($hLV, 0xFF0000, 16, 16))     ; Index = 0
;~  _GUIImageList_Add($hImage, _GUICtrlListView_CreateSolidBitMap($hLV, 0x00FF00, 16, 16))     ; Index = 1
;~  _GUIImageList_Add($hImage, _GUICtrlListView_CreateSolidBitMap($hLV, 0x0000FF, 16, 16))     ; Index = 2
;~  _GUICtrlListView_SetImageList($hLV, $hImage, 1)

    ; --- StatusBar ---
    Local  $aStatusBarParts[5] = [220, 160, 160, 140, -1]
    Global $StatusBar = _GUICtrlStatusBar_Create($hWnd, $aStatusBarParts) ; $SBARS_SIZEGRIP

    ; --- DEBUG ---
    Local $Debug_1 = GUICtrlCreateButton("Check Random", 928, 383, 91, 25) ; 152
    Local $Debug_2 = GUICtrlCreateButton("Load From Array", 1024, 383, 91, 25) ; 152
    Local $Debug_3 = GUICtrlCreateButton("Load 15 Rows", 928, 412, 91, 25)
    Local $Debug_4 = GUICtrlCreateButton("Load 500 Rows", 1024, 412, 91, 25)
    Local $DebugLabel_1 = GUICtrlCreateLabel("DEBAG", 928, 444, 36, 17)


    GUIRegisterMsg($WM_NOTIFY, "WM_NOTIFY")
    GUIRegisterMsg($WM_SIZE, "WM_SIZE")
    GUIRegisterMsg($WM_GETMINMAXINFO, "MY_WM_GETMINMAXINFO") ; Restrict minimaze window below specified size [$gWinXMin, $gWinYMin]
    GUISetState(@SW_SHOW)


    ; =============================================================================================
    ; --- Load Test ---
    Local $sTestCsvFile = @ScriptDir & "\Test CSV 15.csv"
    _ListView_Load($sTestCsvFile, $aItems, True) ; Load Autosized
    ; =============================================================================================

    ; Message loop
    While 1
        Switch GUIGetMsg()
            Case $Debug_1
                ConsoleWrite(UBound($aItems, 1) & @CRLF)
                ConsoleWrite(UBound($aItems, 2) & @CRLF)
                For $i = 0 To UBound($aItems, 1) - 1
                    $aItems[$i][10] = 8192 ; Checked
                Next
                ConsoleWrite("- $i " & $i & @CRLF)
                GUICtrlSendMsg( $idLV, $LVM_SETITEMCOUNT, $i, 0 )
;~              $aItems[$i][10] = 4096 ; Unchecked
;~              $aItems[$k+$j][10] = 8192 ; Checked

            Case $Debug_2
                Local $iRows = 250
                Dim $aItems[$iRows+1][12]
                For $iRow = 0 To $iRows - 1
                    $aItems[$iRow][0] = $iRow + 1
                    $aItems[$iRow][1] = "AAA-" & Random(1000000, 9999999, 1)
                    $aItems[$iRow][2] = "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB"
                    $aItems[$iRow][3] = "CCC-" & Random(1000000, 9999999, 1)
                    $aItems[$iRow][4] = "DDD-" & Random(1000000, 9999999, 1)
                    $aItems[$iRow][10] = 4096 ; Unchecked
                Next
                GUICtrlSendMsg( $idLV, $LVM_SETITEMCOUNT, $iRow, 0 )
                _GUICtrlStatusBar_SetText($StatusBar, $iRow & " Rows Loaded", 1)

            Case $Debug_3
                GUICtrlSetData($Debug_3, "Load 15")
                $sTestCsvFile = @ScriptDir & "\Test CSV 15.csv"
                _ListView_Load($sTestCsvFile, $aItems, True) ; Load Autosized

            Case $Debug_4
                GUICtrlSetData($Debug_4, "Load 500")
                $sTestCsvFile = @ScriptDir & "\Test CSV 500.csv"
                _ListView_Load($sTestCsvFile, $aItems, True) ; Load Autosized

            Case $GUI_EVENT_CLOSE
                ExitLoop
        EndSwitch
    WEnd
    GUIDelete()
EndFunc   ;==>Example






;
;
; Return Test Count
Func _ListView_Load($sTestCsvFile, ByRef $aItems, $AutoSize = True)
    Local $iRow = 0, $iNumLen, $aData, $aName, $GroupName, $TestName, $TestIdx, $MinVal, $MaxVal, $iStatus;, $aName

    Local $sTestIniFile = StringRegExpReplace($sTestCsvFile, '\.[^.]*$', '.tcf') ; INI File


;~  _SendMessage($hLV, $LVM_DELETEALLITEMS) ; Delete All

    Local $aCsv = FileReadToArray($sTestCsvFile) ; 9.289ms
    If @error Then Return MsgBox($TopMost+16, "_ListView_Load()", "! CSV File Reading ERROR")
    Local $LinesCount = @extended
    Dim $aItems[$LinesCount][12]
    $iNumLen = StringLen($LinesCount)

    For $i = 1 To $LinesCount-1 ;
        If StringLeft($aCsv[$i], 5) = "TEST_" Then
            $aData = StringSplit($aCsv[$i], ",")
            If $aData[0] >= 5 Then
                $iRow += 1

                $aData[1] = StringReplace($aData[1], "TEST_", "", 1)
                $aData[1] = StringReplace(StringReplace($aData[1], "__", "|", 1), "_", " ", 1)
                $aName = StringSplit($aData[1], "|", 3)

                If @error Then
                    $GroupName = ""
                    $TestName = $aName[0]
                Else
                    $GroupName = $aName[0]
                    $TestName = StringReplace($aName[1], "_", " ") ; (all '_' replaced with ' ')
                EndIf

                $TestIdx = StringStripWS($aData[2], 7)
                $MinVal = StringStripWS($aData[3], 7)
                $MaxVal = StringStripWS( $aData[4], 7)

                $aItems[$iRow - 1][0] = StringFormat("%0" & $iNumLen & "i  ", $iRow)
                $aItems[$iRow - 1][1] = $GroupName
                $aItems[$iRow - 1][2] = $TestName
                $aItems[$iRow - 1][3] = $MinVal
                $aItems[$iRow - 1][4] = $MaxVal
                $aItems[$iRow - 1][10] = 4096 ; 4096-Unchecked, 8192-Checked
            EndIf
        Else
            ContinueLoop
        EndIf
    Next

    _GUICtrlListView_BeginUpdate($hLV)
    GUICtrlSendMsg($idLV, $LVM_SETITEMCOUNT, $iRow, 0) ; Update data [$aItems] to ListView

    _GUICtrlListView_SetColumnWidth($hLV, 0, -1)
    If $AutoSize Then
        For $i = 1 To 2
            _GUICtrlListView_SetColumnWidth($hLV, $i, $LVSCW_AUTOSIZE)
        Next
    EndIf

    _GUICtrlListView_EndUpdate($hLV)
    _GUICtrlStatusBar_SetText($StatusBar, $iRow & " Rows Loaded", 1)
    Return $iRow
EndFunc   ;==>_ListView_Load






Func WM_NOTIFY($hWnd, $iMsg, $wParam, $lParam)
    Local Static $tText = DllStructCreate("wchar[50]")
    Local Static $pText = DllStructGetPtr($tText)

    Local $tNMHDR, $hWndFrom, $iCode
    $tNMHDR = DllStructCreate($tagNMHDR, $lParam)
    $hWndFrom = HWnd(DllStructGetData($tNMHDR, "hWndFrom"))
    $iCode = DllStructGetData($tNMHDR, "Code")

    Switch $hWndFrom

        Case $hLV

            Switch $iCode

                Case $LVN_GETDISPINFOW
                    Local $tNMLVDISPINFO = DllStructCreate($tagNMLVDISPINFO, $lParam)
                    Local $iItem = DllStructGetData($tNMLVDISPINFO, "Item")
                    Local $iSubItem = DllStructGetData($tNMLVDISPINFO, "SubItem")
                    Local $iMask = DllStructGetData($tNMLVDISPINFO, "Mask")

                    ; Text
                    If BitAND($iMask, $LVIF_TEXT) Then
                        Local $sText = $aItems[$iItem][$iSubItem]
                        DllStructSetData($tText, 1, $sText)
                        DllStructSetData($tNMLVDISPINFO, "Text", $pText)
                        DllStructSetData($tNMLVDISPINFO, "TextMax", StringLen($sText))
                    EndIf

                    ; Checkbox in first column
                    If BitAND($iMask, $LVIF_STATE) And $iSubItem = 0 Then
                        DllStructSetData($tNMLVDISPINFO, "State", $aItems[$iItem][10])
                    EndIf

                    ; Icon in first column
                    ; Use proper $iSubItem value for other columns
                    If BitAND($iMask, $LVIF_IMAGE) And $iSubItem = 6 Then ; Status Column Icon
                        DllStructSetData($tNMLVDISPINFO, "Image", $aItems[$iItem][11])
                    EndIf


                Case $NM_CUSTOMDRAW
                    Local $tNMLVCUSTOMDRAW = DllStructCreate($tagNMLVCUSTOMDRAW, $lParam)
                    Local $dwDrawStage = DllStructGetData($tNMLVCUSTOMDRAW, "dwDrawStage")
                    Local $dwItemSpec = DllStructGetData($tNMLVCUSTOMDRAW, "dwItemSpec")

                    Switch $dwDrawStage      ; Holds a value that specifies the drawing stage
                        Case $CDDS_PREPAINT
                            ; Before the paint cycle begins
                            Return $CDRF_NOTIFYITEMDRAW ; Notify the parent window of any item-related drawing operations

                        Case $CDDS_ITEMPREPAINT
                            ; Before painting an item
                            If Mod($dwItemSpec, 2) = 1 Then
                                DllStructSetData($tNMLVCUSTOMDRAW, "ClrTextBk", 0xFFFFFF)
                            Else
                                DllStructSetData($tNMLVCUSTOMDRAW, "ClrTextBk", 0xEBF7FF) ; BGR
                            EndIf
                            Return $CDRF_NEWFONT ; $CDRF_NEWFONT must be returned after changing font or colors
                    EndSwitch


                Case $NM_CLICK
                    Local $tInfo = DllStructCreate($tagNMITEMACTIVATE, $lParam)
                    Local $iItem = DllStructGetData($tInfo, "Index")
                    Local $iSubItem = DllStructGetData($tInfo, "SubItem")
                    If $iSubItem = 0 Then
                        If $aItems[$iItem][10] = 4096 Then
                            $aItems[$iItem][10] = 8192 ; Checked
                        Else
                            $aItems[$iItem][10] = 4096 ; Unchecked
                        EndIf
                        _GUICtrlListView_RedrawItems($hLV, $iItem, $iItem)
                    EndIf


                Case $NM_DBLCLK ; User double-clicks an item with the left mouse button (No return value)
                    Local $tInfo = DllStructCreate($tagNMITEMACTIVATE, $lParam)
                    Local $iItem = DllStructGetData($tInfo, "Index")
                    Local $iSubItem = DllStructGetData($tInfo, "SubItem")
                    ConsoleWrite("--> DBLCLK Row : " & $iItem + 1 & @CRLF)
                    ConsoleWrite("-->       Name : " & $aItems[$iItem][$iSubItem] & @CRLF)
            EndSwitch

    EndSwitch

    Return $GUI_RUNDEFMSG
EndFunc   ;==>WM_NOTIFY



; Resize the status bar when GUI size changes
Func WM_SIZE($hWnd, $iMsg, $wParam, $lParam)
    #forceref $hWnd, $iMsg, $wParam, $lParam
;~  ConsoleWrite("ListView Width " & ControlGetPos($hWnd, "", $idLV)[2] & @CRLF)
    _GUICtrlStatusBar_Resize($StatusBar)
    Return $GUI_RUNDEFMSG
EndFunc   ;==>WM_SIZE



Func MY_WM_GETMINMAXINFO($hWnd, $Msg, $wParam, $lParam)
    Local $minmaxinfo = DllStructCreate("int;int;int;int;int;int;int;int;int;int",$lParam)
    DllStructSetData($minmaxinfo, 7, $gWinXMin); min X (+15)
    DllStructSetData($minmaxinfo, 8, $gWinYMin); min Y (+15)
    Return 0
EndFunc

Test CSV 15.csv Test CSV 500.csv Virt_LV_Garbage.au3

Edited August 18, 2022 by RAMzor

[pixelsearch]

This line seems to cause the issue :

Local Static $tText = DllStructCreate("wchar[50]")

Increasing the value seems to solve your problem.

 

[Nine]

Good point @pixelsearch.

By default any struct is filled with 0.  And string must be ended with a null character.  In you code, if you use a string larger than 50, the null char isn't there and it will continue searching for it,  grabbing garbage characters along the way.

So having a larger buffer will solve partly your problem.  You must be careful since your variable is static.  You need to make sure to add a null char at the end, otherwise you may end up with a different garbage issue. 

 

[RAMzor]

Thanks guys for the help!  🍻

Increasing "wchar" parameter and adding  a Null char at the end of each subitem solve the issue 

DllStructCreate("wchar[50]") ==> DllStructCreate("wchar[60]")

$aItems[$iRow][2] = $TestName & Null

 

[pixelsearch]

I remember we already discussed this parameter of [50] or [260] or [4096] in this link and that one.
Finally @jpm choosed 4096 in ArrayDisplayInternals.au3 when using a virtual listview in the last releases of AutoIt :

Case -177 ; $LVN_GETDISPINFOW
    Local Static $tText = DllStructCreate("wchar[4096]"), $pText = DllStructGetPtr($tText)

Note: it was also 4096 in AutoIt 3.3.14.5 when not using a virtual listview, so why not always creating the structure with 4096 to avoid possible issues like OP's one ?

Concerning the final NULL character that @Nine talked about, could we please have some complementary informations ?

Here is an example script where I created a structure of 32 length (instead of 4096) for cosmetic reasons (size of 2 pics below) :

#include <Array.au3>
#include <MsgBoxConstants.au3>

Local $tStruct = DllStructCreate("char[32]")
If @error Then Exit Msgbox(0, "DllStructCreate", "error " & @error)

;=======================
Local $sData = "12345678"
DllStructSetData($tStruct, 1, $sData)
If @error Then Exit Msgbox(0, "DllStructSetData 1", "error " & @error)

MsgBox($MB_TOPMOST, "Pointer of struct", DllStructGetPtr($tStruct))
; dump memory here

;=======================
Local $sData2 = "ABCD"
DllStructSetData($tStruct, 1, $sData2)
If @error Then Exit Msgbox(0, "DllStructSetData 2", "error " & @error)

MsgBox($MB_TOPMOST, "", "Close after dumping memory")
; dump memory here

;=======================
$tStruct = 0 ; release the resources used by the structure.

I didn't add by myself any NULL character at the end of the strings but we can see that the 2nd pic got an automatic NULL character [i.e Chr(0)] placed just after the "ABCD" string (0x41 to 0x44)

So my question is : why should we add by ourselves a NULL character in the script when it appears that AutoIt adds it automatically during the DllStructSetData() statement ?
Thanks

Edit: I just checked jpm's code in ArrayDisplayInternals.au3
He creates the structure with 4096 ...

Local Static $tText = DllStructCreate("wchar[4096]")

... then he keeps checking on and on that the string length isn't > 4095

If StringLen($sTemp) > 4095 Then $sTemp = StringLeft($sTemp, 4095)
    DllStructSetData($tText, 1, $sTemp)
    ...

Probably to keep 1 byte left (in fact 2 bytes when wchar, no big deal) for the NULL character automatically added by AutoIt at the 4096th position. If it's the correct reason, then we should do same every time, checking that the String length has a maximum length = structure length - 1

Edited August 19, 2022 by pixelsearch
Jpm's 4096 & 4095