DllStructCreate("wchar[50]") - does "50" represents number of characters?

[VAN0]

Hello.

I'm trying optimize virtual listview

And I stumble upon this code:

Local Static $tText = DllStructCreate( "wchar[50]" )
Local Static $pText = DllStructGetPtr( $tText )
DllStructSetData( $tText, 1, $sItem )
DllStructSetData( $tNMLVDISPINFO, "Text", $pText )
DllStructSetData( $tNMLVDISPINFO, "TextMax", StringLen( $sItem ) )

It works with short strings correctly, but long text gets corrupted. When I changed 50 to 255, long text displays fine too.

So, I figures if 50 is number of characters, than why not make it dynamic per each string:

Local $tText = DllStructCreate( "wchar[" & StringLen($sItem) "]" )
Local $pText = DllStructGetPtr( $tText )

This actually broke even short strings

So what does that "50" represent and what should I use to avoid any corruption on really long strings (file path + command line) and for very large list should it be dynamically changed?

 

Thank you.

[LarsJ]

50 is the number of characters but in addition, make sure there is room for a terminating zero character. Replace this line

Local $tText = DllStructCreate( "wchar[" & StringLen($sItem) & "]" )

with this line

Local $tText = DllStructCreate( "wchar[" & StringLen($sItem)+1 & "]" )

 

To optimize the code for speed, the fastest thing you can do is to define $tText and $pText at the top of the code as global variables in this way (the AutoIt code interpreter will not spend a single nano second on that line in the WM_NOTIFY function):

Global $tText = DllStructCreate( "wchar[1024]" ), $pText = DllStructGetPtr( $tText )

Make sure the $tText buffer is sufficiently large.

The second fastest thing you can do is to define $tText and $pText as local static variables in the WM_NOTIFY function (the AutoIt code interpreter will not spend much time on the line when the variables are initialized):

Local Static $tText = DllStructCreate( "wchar[1024]" ), $pText = DllStructGetPtr( $tText )

The slowest thing you can do is to create the variables dynamically every time you need them (the AutoIt code interpreter must interpret and execute these 3 commands (DllStructCreate, StringLen, DllStructGetPtr) each time a single cell in the listview needs to be updated).

[VAN0]

Thank you for the information.

I did try double, tripple, x10 times the string length, it still corrupted text

Local $tText = DllStructCreate( "wchar[" & (StringLen($sItem)*10) & "]" )

As "corrupted" I mean it shows some of the text than some garbage or even showed text from the source itself...

I'm just curious, though, why my "dynamic" allocation approach doesn't work?

P.S.

The reason why I thought it might be necessary is because my largest string could possibly be 32767 characters (command line limit) and hypothetically I might have over a million rows with 10 columns, that would be a lot memory to waste using wchar[32767] if it was used for each small or large string. I mean doesn't it allocate memory for that? That 32767 would actually be 65535 bytes * 10,000,000 fields = that would be insane amount of RAM.

[TheXman]

1 hour ago, VAN0 said:

I'm just curious, though, why my "dynamic" allocation approach doesn't work?

Your dynamic allocation looks fine.  The problem is most likely when you set "TextMax".  "TextMax" is the size of the buffer in bytes.  Wchar uses 2 bytes per character as oppose to Char which uses 1 byte per character.  You are setting it using StringLen, which gives you the length of the string in characters, not bytes needed to store the string.  What you really want is something that will give you the size of the buffer like:

DllStructSetData( $tNMLVDISPINFO, "TextMax", DllStructGetSize($tText) )

OR

$tNMLVDISPINFO.TextMax = DllStructGetSize($tText)

You were setting "TextMax" to half the size that it really needed to be.

 

The snippet below should make it clear:

$tWchar = DllStructCreate("wchar[50]")
$tChar  = DllStructCreate("char[50]")

MsgBox(0, "Test", _
       "Size of wchar[50] = " & DllStructGetSize($tWchar) & @CRLF & _
       "Size of char[50]  = " & DllStructGetSize($tChar))

 

Edited June 17, 2020 by TheXman

[VAN0]

It sure does make sense, however the original code in virtual listview uses StringLen for TextMax value and it works fine...In fact it seems to be working fine without TextMax line all together as long as $tText is set as static or global.

Just in case here is the code I'm playing with:
 

expandcollapse popup

#include <GUIConstants.au3>
#include <WindowsConstants.au3>
#include <GuiListView.au3>

Opt( "MustDeclareVars", 1 )

Global $hGui, $hLV
Global $rowNum = 100000000 ;number of rows
Global $charNum = 20 ;number of characters

Example()

Func Example()
    $hGui = GUICreate( "Virtual ListViews", 850, 400 )

    Local $idLV = GUICtrlCreateListView( "Col1|Col2|Col3", 20, 40, 850-40, 400-60, $LVS_OWNERDATA, BitOR( $WS_EX_CLIENTEDGE, $LVS_EX_DOUBLEBUFFER, $LVS_EX_FULLROWSELECT ) )
    $hLV = GUICtrlGetHandle( $idLV )

    GUICtrlSendMsg( $idLV, $LVM_SETCOLUMNWIDTH, 0, 100)
    GUICtrlSendMsg( $idLV, $LVM_SETCOLUMNWIDTH, 1, 300)
    GUICtrlSendMsg( $idLV, $LVM_SETCOLUMNWIDTH, 2, 300)
    GUICtrlSendMsg( $idLV, $LVM_SETITEMCOUNT, $rowNum , 0 )

    GUIRegisterMsg( $WM_NOTIFY, "WM_NOTIFY" )
    GUISetState( @SW_SHOW )

    While 1
        If GUIGetMsg() = $GUI_EVENT_CLOSE Then ExitLoop
    WEnd
    GUIDelete()
EndFunc

Func WM_NOTIFY( $hWnd, $iMsg, $wParam, $lParam )

    Local $tNMHDR, $hWndFrom, $iCode
    $tNMHDR = DllStructCreate( $tagNMHDR, $lParam )
    $hWndFrom = HWnd( DllStructGetData( $tNMHDR, "hWndFrom" ) )
    $iCode = DllStructGetData( $tNMHDR, "Code" )

    Switch $hWndFrom

        Case $hLV

            Switch $iCode

                Case $LVN_GETDISPINFOW
                    Local $tNMLVDISPINFO = DllStructCreate( $tagNMLVDISPINFO, $lParam )
                    If BitAND( DllStructGetData( $tNMLVDISPINFO, "Mask" ), $LVIF_TEXT ) Then

                        Local $sItem = $tNMLVDISPINFO.SubItem = 0 ? String($tNMLVDISPINFO.Item) : FillString($tNMLVDISPINFO.Item, $tNMLVDISPINFO.SubItem)
                        Local Static $tText = DllStructCreate( "wchar[" & $charNum + 1 & "]" )
                        Local Static $pText = DllStructGetPtr( $tText )

                        DllStructSetData( $tText, 1, $sItem )
                        DllStructSetData( $tNMLVDISPINFO, "Text", $pText )
                        ;do we need this line at all?
                        DllStructSetData( $tNMLVDISPINFO, "TextMax", StringLen( $sItem ) * 2 )
                    EndIf

            EndSwitch

    EndSwitch

    Return $GUI_RUNDEFMSG
EndFunc

Func FillString($row, $col)
    Local $n = 65 + Mod($row + (($col - 1) * $charNum),  26)
    Local $r = ""
    For $i = 1 To $charNum
        $r &= Chr($n)
        $n += 1
        If $n > 90 Then $n = 65
    Next
    Return $r
EndFunc

Remove Static from $tText and $pText and text in Col1 gets all messed up. At this point I'm just curious why this happening.

 

P.S.

This also showed that wchar buffer size starts affect memory usage after 8 digit number, if the string is less than a million characters dynamic allocation is not needed at all (and we are talking about 30MB ram usage with 1,000,000 characters buffer vs 44MB with 10,000,000 buffer, displaying 100,000,000 rows with 2 columns of 30 char long strings, in fact changing length of displayed strings doesn't affect memory usage at all)

[OFFTOPIC]

Spoiler

I didn't know we can use variable created by DllStructCreate() as an object, Is there any benefit using DllStructGetData()/DllStructSetData() instead of directly $myobject.mykey?

 

[TheXman]

<snipped snide remarks>

Do you not see the difference between your original line and the line below?  Do you also not see that the line below does the exact same thing as what I posted and for the reason that I said?  Do you not understand why the line below is multiplying the string's length by 2.  As in most languages, there is more than 1 way to do the same thing. 

DllStructSetData( $tNMLVDISPINFO, "TextMax", StringLen( $sItem ) * 2 )

That's the problem with "stumbling upon" other's code and trying to use it without taking the time to understand exactly what it is doing.  You are trying to take shortcuts in the learning process and not listening OR learning at all.  Trying to help you is a waste of my time and it will not happen again.  If you look at my "About Me" in my profile, and look at the last bullet point, you definitely fall into that category.

Good luck, you will certainly need it as far as scripting goes.

Edited June 17, 2020 by JLogan3o13

[VAN0]

Spoiler

Wow, someone got up on the wrong foot this morning...(or you are Valik's cousin?)

Your first reply only confirmed what I said before: wchar uses 2 bytes per it's number

Quote

That 32767 would actually be 65535 bytes

Second, the code I posted is not what I "use", that's how I LEARN how it works. If you do it differently somehow, well good for you.

Quote

Do you not understand why the line below is multiplying the string's length by 2.

I guess I don't, because it doesn't matter what number I set that line to, weather it's number of characters in string, or double, triple that, or even if I delete that line all together, the result is the same: it shows correct text at any length. So, no, I do not understand why that line is needed at all since TextMax is read-only anyway...

Quote

cchTextMax

Type: int

Number of TCHARs in the buffer pointed to by pszText, including the terminating NULL.

This member is only used when the structure receives item attributes. It is ignored when the structure specifies item attributes. For example, cchTextMax is ignored during LVM_SETITEM and LVM_INSERTITEM. It is read-only during LVN_GETDISPINFO and other LVN_ notifications.

https://docs.microsoft.com/en-us/windows/win32/api/commctrl/ns-commctrl-lvitema

 

Nobody is forcing you to reply, so keep your insults to yourself, nobody is appreciate them, at least I don't.

Edited June 17, 2020 by VAN0

[JLogan3o13]

@TheXman I snipped your unnecessary remarks above. Please refer to the forum rules on insulting others. If you are getting (understandably) frustrated with someone, you are more than welcome to walk away from the thread.

[LarsJ]

Some runnable code makes things a lot easier. Note that the $LVN_GETDISPINFOW notifications comes directly from the Windows operating system and that the $tagNMLVDISPINFO structures that you fill out as a response to these notifications are used by the operating system to display the text in the listview cells. This means that the $tText buffer must be valid until the operating system is completely finished using the buffer. It's not enough that the buffer is only valid in the WM_NOTIFY function. It must also be valid while the operating system uses the $tagNMLVDISPINFO structure and thus the buffer. Therefore, the buffer must be global or local static. It's not enough that it's just a local variable in the WM_NOTIFY function.

In fact, it doesn't seem necessary to fill in the "TextMax" field. So the code can be optimized a bit.

Note that in a listview cell, as far as I remember, there is a limitation on the length of the texts of a few thousand characters.