Optimized Array.au3

[Zedna]

@ultima, gary

good work guys on removing listview UDF dependency, smart idea!

I like it.

[randallc]

I managed to get _ArrayDisplay() to be entirely self-contained. It no longer relies on external helper functions, and does all of the Dll*() stuff internally. It's much faster too. For example, trying to add 10000 items with 3 subitems using the above-provided version used to take 3.4 seconds, but now takes 1.9 seconds (on my computer anyhow). Adding 40000 items with 3 subitems used to take 17 seconds, but now takes around 7 seconds. Basically, it's scaling up a lot more nicely because the redundancy in calling/creating the DLL-related stuff was minimized.

In total, the function adds 8.45KB uncompiled code to Array.au3 now, but that's obviously a huge improvement over the previous several-hundred-KB-compiled situation Test, test, test away!

@GaryFrost: I'm not sure if you're going to remove your script, but I'd say not to, just in case I screwed something up in the transition (so that we have a known, mostly-working-with-a-minor-bug copy on hand)

Hey, man,

C'est tres bon!

Mais oui, qu'est ce que c'est que ce la f..k!

In italian (ie Sopranos NJ style) "What ya goin da do!"

Fantasic.

1. Maintenant, 1st column width readable; n'est ce pas possibleh bleh bleeah ?

Je ne ce fais rein; now that's all the french I ever learnt at school.

2. Can we fix that in "array.au3" and still have my "_ArrayDebugDisplay" in "DeBug.au3"? ; eg to display more than 200 cols, 1D laterally, etc?

_ArrayDebugDisplay.au3

I know its not a big deal, though, congrats on fixing the "bloat"

Best, R

Edited December 24, 2007 by randallc

[randallc]

Hi again,

I have modified all your Display script (ie no "includes" or dependencies) again, to include column width fix; [at least for your entertainment, if not use!]

1. Displays 1D arrays with delimiters , as though 2D

2. Displays other 1D transposed if requested

3. Headings available, so makes 3D readable.

4. Will not corrupt display of columns over 250 etc

Best, randall

expandcollapse popup

; Ex. #1:
#include <ArrayUltDisplay.au3>

Dim $avArray[8]
$avArray[0] = 7
$avArray[1] = "Brian|Smith"
$avArray[2] = "Jon|Brown"
$avArray[3] = "Larry|Maddison"
$avArray[4] = "Christa|Heinrich"
$avArray[5] = "Rick|Richards"
$avArray[6] = "Jack|Tomas"
$avArray[7] = "Gregory|Thompson"

_ArrayUltDisplay($avArray, "_ArrayDisplay() Test [1D, delimitd strings, 'Replace' param same as dataSeparator]|Hi0|Hi1",1,0,"|","|")
; Ex. #2:
Dim $arGrid[2][4] = [["Paul", "Jim", "Richard", "Louis"], [485.44, 160.68, 275.16, 320.00]]
_ArrayUltDisplay($arGrid, "_ArrayDisplay() 2D Test") ; [2D]
_ArrayUltDisplay($arGrid, "_ArrayDisplay() 2D Test [2D, transposed]", 1, 1) ; [2D, transposed]

; Ex. #3:
Dim $avArray[8]
$avArray[0] = 7
$avArray[1] = "Brian"
$avArray[2] = "Jon"
$avArray[3] = "Larry"
$avArray[4] = "Christa"
$avArray[5] = "Rick"
$avArray[6] = "Jack"
$avArray[7] = "Gregory"
_ArrayUltDisplay($avArray, "_ArrayDisplay() Test[1D")
_ArrayUltDisplay($avArray, "_ArrayDisplay() Test[1D, transposed]",1,1)
; Ex. #5:
Dim $avArray[1000]
for $i=0 to 999
    $avArray[$i]="Item"&$i
Next
_ArrayUltDisplay($avArray, "_ArrayDisplay() Test[1D")
_ArrayUltDisplay($avArray, "_ArrayDisplay() Test[1D, transposed]",1,1)

Edited December 25, 2007 by randallc

[GaryFrost]

I managed to get _ArrayDisplay() to be entirely self-contained. It no longer relies on external helper functions, and does all of the Dll*() stuff internally. It's much faster too. For example, trying to add 10000 items with 3 subitems using the above-provided version used to take 3.4 seconds, but now takes 1.9 seconds (on my computer anyhow). Adding 40000 items with 3 subitems used to take 17 seconds, but now takes around 7 seconds. Basically, it's scaling up a lot more nicely because the redundancy in calling/creating the DLL-related stuff was minimized.

In total, the function adds 8.45KB uncompiled code to Array.au3 now, but that's obviously a huge improvement over the previous several-hundred-KB-compiled situation Test, test, test away!

@GaryFrost: I'm not sure if you're going to remove your script, but I'd say not to, just in case I screwed something up in the transition (so that we have a known, mostly-working-with-a-minor-bug copy on hand)

If we don't see any bug finds by the end of the day tomorrow I'm going to roll this into the next beta.

[Zedna]

If we don't see any bug finds by the end of the day tomorrow I'm going to roll this into the next beta.

I tested Ultima's arraydisplay function and I have these suggestions:

1) change default $sReplace = "~" to $sReplace = "|" so if we call _ArrayDisplay function without any parametres then we want to see unchanged content of array. Here can be also little speed optimization: If $sReplace is equal to $sSeparator then don't do StringReplace()

here is example:

#include <Array.au3>

Dim $avArray[8]
$avArray[0] = 7
$avArray[1] = "Brian|Smith"
$avArray[2] = "Jon|Brown"
$avArray[3] = "Larry|Maddison"
$avArray[4] = "Christa|Heinrich"
$avArray[5] = "Rick|Richards"
$avArray[6] = "Jack|Tomas"
$avArray[7] = "Gregory|Thompson"

_ArrayDisplay($avArray)

2) transpose on 1D array doesn't work for me. I don't know if it's OK or not. I expected that instead of 1 column and 7 rows there will be 7 columns and 1 row after 1D transpose.

note: 2D transpose works OK.

example:

#include <Array.au3>

Dim $avArray[8]
$avArray[0] = 7
$avArray[1] = "Brian"
$avArray[2] = "Jon"
$avArray[3] = "Larry"
$avArray[4] = "Christa"
$avArray[5] = "Rick"
$avArray[6] = "Jack"
$avArray[7] = "Gregory"
_ArrayDisplay($avArray, "_ArrayDisplay() Test[1D")
_ArrayDisplay($avArray, "_ArrayDisplay() Test[1D, transposed]",-1,1)

Tested UDF from Post #80 by examples from Post #83 with AutoIt 3.2.10 on WIN98SE

BTW: Thanks for your work on this guys!

Edited December 26, 2007 by Zedna

[GaryFrost]

I tested Ultima's arraydisplay function and I have these suggestions:

1) change default $sReplace = "~" to $sReplace = "|" so if we call _ArrayDisplay function without any parametres then we want to see unchanged content of array. Here can be also little speed optimization: If $sReplace is equal to $sSeparator then don't do StringReplace()

here is example:

#include <Array.au3>

Dim $avArray[8]
$avArray[0] = 7
$avArray[1] = "Brian|Smith"
$avArray[2] = "Jon|Brown"
$avArray[3] = "Larry|Maddison"
$avArray[4] = "Christa|Heinrich"
$avArray[5] = "Rick|Richards"
$avArray[6] = "Jack|Tomas"
$avArray[7] = "Gregory|Thompson"

_ArrayDisplay($avArray)

2) transpose on 1D array doesn't work for me. I don't know if it's OK or not. I expected that instead of 1 column and 7 rows there will be 7 columns and 1 row after 1D transpose.

note: 2D transpose works OK.

example:

#include <Array.au3>

Dim $avArray[8]
$avArray[0] = 7
$avArray[1] = "Brian"
$avArray[2] = "Jon"
$avArray[3] = "Larry"
$avArray[4] = "Christa"
$avArray[5] = "Rick"
$avArray[6] = "Jack"
$avArray[7] = "Gregory"
_ArrayDisplay($avArray, "_ArrayDisplay() Test[1D")
_ArrayDisplay($avArray, "_ArrayDisplay() Test[1D, transposed]",-1,1)

Tested UDF from Post #80 by examples from Post #83 with AutoIt 3.2.10 on WIN98SE

BTW: Thanks for your work on this guys!

This shoudl fix the 1D Transpose, -Ultima- will have to look it over, optimize if needed and make sure I didn't break anything else.

As for the Default I'll leave that to -Ultima- to look at.

Edited December 27, 2007 by GaryFrost
Removed attachment

[randallc]

Hi,

OK, thanks.

For your interest, I have taken Gary and Ultima's fast ArrayDisplay further; needs WinXP or better for fast vbs sort;

Sorts as fast [50%slower than] each time as the new fast display, no dependencies.[and twice [30% faster] as fast for clicking back on same column to reverse]

Best, randall

#include <ArrayUltDisplay.au3>
#include<_FileListToArrayFaster1g.au3>
Local $Message = "all              files    ?", $Exclude, $filter = "*", $Flag = 1, $Recurse = 1, $BasDir = 1
Local $timerstamp1 = TimerInit(), $sPath = @ScriptDir, $filter = '*.*', $filter = '*.*|*.doc'  ;, $filter = '*.doc'
ConsoleWrite("$sPath=" & $sPath & @LF)
$ar_Array = _FileListToArray3 ($sPath, $filter, $Flag, $Recurse, $BasDir);, "", 0) ;
ConsoleWrite($Message & "=" & UBound($ar_Array) - 1 & @TAB & " _FileListToArray3 " & @TAB & _
        ":" & @TAB & Round(TimerDiff($timerstamp1)) & "" & @TAB & " msec" & @LF)
_ArrayUltDisplay ($ar_Array, "Fast Array Display, Gary and  Ultima; Click column header to sort", 0)

[Removed due to bug... randall]

Re-posted separately here;.....Fast New ArrayDisplay with Fast Sort, Thanks to Ultima, GaryFrost

Edited December 27, 2007 by randallc

[-Ultima-]

Latest Changes

- _ArrayDisplay: $sSeparator now defaults to Chr(1)
- _ArrayDisplay: $sReplace now defaults to "|"
- _ArrayDisplay: Now transposes 1D arrays
- _ArrayDisplay: Fixed some cases where buffer size might be too small
- _ArrayDisplay: Sacrificed a small bit of performance in text array generation for code maintainability

Regarding buffer size... For one thing, I'd forgotten to add 1 to the max buffer size (for the null character). Additionally, I'd forgotten that the user could set $sReplace to be longer than $sSeparator, in which case the buffer size would also be calculated incorrectly. Both are fixed.

The small performance hit equates to around 8% slower than the previous version (which translates to approximately 2 seconds of difference when displaying arrays of around 100,000 items with 3 subitems each). I opted for the performance hit because it was getting annoying to remember to do some things in each and every case (like checking for max buffer length) when the only difference in each case was where to find the next piece of data to add to the text array.

I caved in on the 1D transposing because indeed, it is rather unintuitive that there should be any artificial limitation on which arrays can be transposed. Sorry for being stubborn about that point all this time, randallc

[ATTACHMENT REMOVED]

Edited December 28, 2007 by -Ultima-

[GaryFrost]

Latest Changes

- _ArrayDisplay: $sSeparator now defaults to Chr(1)
- _ArrayDisplay: $sReplace now defaults to "|"
- _ArrayDisplay: Now transposes 1D arrays
- _ArrayDisplay: Fixed some cases where buffer size might be too small
- _ArrayDisplay: Sacrificed a small bit of performance in text array generation for code maintainability

Regarding buffer size... For one thing, I'd forgotten to add 1 to the max buffer size (for the null character). Additionally, I'd forgotten that the user could set $sReplace to be longer than $sSeparator, in which case the buffer size would also be calculated incorrectly. Both are fixed.

The small performance hit equates to around 8% slower than the previous version (which translates to approximately 2 seconds of difference when displaying arrays of around 100,000 items with 3 subitems each). I opted for the performance hit because it was getting annoying to remember to do some things in each and every case (like checking for max buffer length) when the only difference in each case was where to find the next piece of data to add to the text array.

I caved in on the 1D transposing because indeed, it is rather unintuitive that there should be any artificial limitation on which arrays can be transposed. Sorry for being stubborn about that point all this time, randallc

Errors out on 2D transpose on line 290.

Also change line 277 from Dim to Local

[GaryFrost]

>_>

That's weird... I tested several 2D arrays, and they worked perfectly fine... *fix*

Here's a couple tests I ran, just to see what would happen.

#include <Array.au3>

Dim $avArray[8]
$avArray[0] = 7
$avArray[1] = "Brian"
$avArray[2] = "Jon"
$avArray[3] = "Larry"
$avArray[4] = "Christa"
$avArray[5] = "Rick"
$avArray[6] = "Jack"
$avArray[7] = "Gregory"
_ArrayDisplay($avArray, "_ArrayDisplay() Test[1D")
_ArrayDisplay($avArray, "_ArrayDisplay() Test[1D, transposed]",-1,1)

Dim $arGrid[2][4] = [["Paul", "Jim", "Richard", "Louis"], [485.44, 160.68, 275.16, 320.00]]
_ArrayDisplay($arGrid, "_ArrayDisplay() 2D Test") ; [2D]
_ArrayDisplay($arGrid, "_ArrayDisplay() 2D Test [2D, transposed]", -1, 1) ; [2D, transposed]

[-Ultima-]

Attempt 2!

Array.au3

Edited December 26, 2007 by -Ultima-

[GaryFrost]

Attempt 2!

Much better

[randallc]

Hi,

Thanks to you both for all the fantastic efforts.

@Ultima;

1. I know the 1D transpose is no big deal ; no need for apology!

2. After all your great efforts, the width of column1, used most for display, is still on default, and I think it is easy to fix and so does not let down the appearance?

See my script mods above, but all it needs, I think, is;

Local Const $LVM_SETCOLUMNWIDTH = (0x1000 + 30)
Local Const $LVSCW_AUTOSIZE = -1
GUICtrlSendMsg($hListView, $LVM_SETCOLUMNWIDTH, 1, $LVSCW_AUTOSIZE)

What do you think?

Best, randall

Edited December 27, 2007 by randallc

[GaryFrost]

Hi,

Thanks to you both for all the fantastic efforts.

@Ultima;

1. I know the 1D transpose is no big deal ; no need for apology!

2. After all your great efforts, the width of column1, used most for display, is still on default, and I think it is easy to fix and so does not let down the appearance?

See my script mods above, but all it needs, I think, is;

Local Const $LVM_SETCOLUMNWIDTH = (0x1000 + 30)
Local Const $LVSCW_AUTOSIZE = -1
GUICtrlSendMsg($hListView, $LVM_SETCOLUMNWIDTH, 1, $LVSCW_AUTOSIZE)

What do you think?

Best, randall

Close, but I think this works better. Edited December 27, 2007 by GaryFrost
Removed attachment

[-Ultima-]

Sorta, though you don't really need an $iHeaders variable, since $iSubMax already takes care of the number of columns (it's redundant to count the number of columns over). Anyhow, it's probably simpler (and faster?) to just do ControlSend($hGUI, "", $hListView, "^{NUMPADADD}") to get the same result.

I don't feel like the first column should get any priority over other columns just because it's the first column (order doesn't necessarily equate to importance of visibility), so resizing only the first column doesn't seem... right to me. At the same time, though, I'm still hesitant about resizing every single column because of the listview column width limitation. What does that come down to? Me sorta disagreeing that the columns need to be resized at all The default width is enough to get a good preview of what text each item/subitem contains -- if the user needs to see any more text, then the columns are resizable for a reason.

Edited December 27, 2007 by -Ultima-

[GaryFrost]

Sorta, though you don't really need an $iHeaders variable, since $iSubMax already takes care of the number of columns (it's redundant to count the number of columns over). Anyhow, it's probably simpler to just do ControlSend($hGUI, "", $hListView, "^{NUMPADADD}") to get the same result.

I don't feel like the first column should get any priority over other columns just because it's the first column, so resizing only the first column doesn't seem... right to me. At the same time, though, I'm still hesitant about resizing every single column because of the listview column width limitation. What does that come down to? Me sorta disagreeing that the columns need to be resized at all The default width is enough to get a good preview of what text each item/subitem contains -- if the user needs to see any more text, then the columns are resizable for a reason.

Default size is fine with me.

What I need is finalization on this so I can this in before the next beta goes out.

[-Ultima-]

I think the last version I posted is good to go for the beta. If any more changes need to be made, they could always be made and added for the next beta after that.

The example scripts can be found in the first post.

Edited December 27, 2007 by -Ultima-

[GaryFrost]

I think the last version I posted is good to go for the beta. If any more changes need to be made, they could always be made and added for the next beta after that.

The example scripts can be found in the first post.

_ArrayDelete header needs to be fixed

Fixed my copy, the Syntax line is wrong.

Edited December 27, 2007 by GaryFrost

[randallc]

I don't feel like the first column should get any priority over other columns just because it's the first column (order doesn't necessarily equate to importance of visibility), so resizing only the first column doesn't seem... right to me

Hi,

It seems to me that 80% of people will still be using this for 1D array, and so this first column is what most people will see (and it then looks weird to not see total width). Hence my suggestion of just handling column1.

[another 10% probably only need to read col1 in 2D array anyway, for a quick debug glance?...]

[;and, I pefer to have at least one column readable in 2D, and I personally believe it looks better, often shows enough at first glance to know what was needed, without looking further,changing col width etc]

Best, randall

Edited December 27, 2007 by randallc

[-Ultima-]

I don't see the default column width any differently than I see the default window size. As long as the initial window size gives a good preview, the rest can be handled by either resizing (the window) or scrolling the scrollbar.

If the user has to scroll to see all of the data, we (obviously) wouldn't resize the window to try and prevent the need to scroll. In such cases, trying to let the user see things all of the text at first glace without changing something in the listview would (IMO) be a wasted effort, since scrolling would be unavoidable. Scrolling a listview takes just about the same amount of effort as resizing a column. Also, if the user happens to have a first column full of long sentences, phrases, or whatnot -- long enough to force a horizontal scroll -- and we automatically resize the column to fit all of that text, then we're back to square one; the user won't be able to see all of the text without manipulating the listview in some way or another.

Edited December 28, 2007 by -Ultima-