Jump to content

_ArrayDelEmptyRows

Deye

By Deye,
in AutoIt Example Scripts

 Share
Go to solution Solved by Deye,

Recommended Posts

Deye Universalist

Deye

Posted
Posted (edited)

just needed this post to get a clue of how well will this do as an alternative to searching the the array @deleting empty rows 1D & 2D speed wise

Note: for the 1D the method doesn't work if the chosen delimiter exists in any of the array elements ..(not about the "|" one) 

So Here is an all around fast combo that includes @Inpho's _ArrayFindEmptyRows()  that proved to be a very fast way for removing 2D empty rows 

Spoiler 
; Posted in AutoIt Example Scripts forum
; https://www.autoitscript.com/forum/topic/200038-_arraydelemptyrows/

#include <File.au3>
#include <Timers.au3>

Local $Array = _GenRandomArray(10000)
$aTmp = $Array

$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayDelEmptyRows($aTmp, 1)        ;2D
ConsoleWrite("2D : " & _Timer_Diff($hTimer) & @CRLF)
_ArrayDisplay($aArrayRemoveBlanks, "2D")

$aTmp = _ArrayExtract($Array, Default, Default, 1, 1)    ;1D
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayDelEmptyRows($aTmp, 1)
ConsoleWrite("1D : " & _Timer_Diff($hTimer) & @CRLF)
_ArrayDisplay($aArrayRemoveBlanks, "1D")

Func _GenRandomArray($icount)
    Local $aArrayReturn[$icount][4]
    For $i = 0 To UBound($aArrayReturn) - 1
        If Mod($i, 5) = 0 Then $aArrayReturn[$i][1] = Random(1, 100)
    Next
    Return $aArrayReturn
EndFunc   ;==>_GenRandomArray

Func _ArrayDelEmptyRows(ByRef $aArray, $bUBound = False, $sDelim = "'")
    If UBound($aArray, 2) >= 1 Then
        Local $ai = _ArrayFindEmptyRows($aArray)
        If Not IsArray($ai) Then Return $aArray
        _ArrayDelete($aArray, _ArrayToString($ai, ";"))
        If $bUBound Then _ArrayInsert($aArray, 0, UBound($aArray))
        Return $aArray
    Else
        Local $pattern = $sDelim & "[" & $sDelim & "]*[" & $sDelim & "]"
        $str = StringRegExpReplace($sDelim & _ArrayToString($aArray, $sDelim), $pattern, $sDelim)
        $sString = StringRegExpReplace($str, $sDelim & "$", "")
        Return StringSplit(StringTrimLeft($sString, 1), $sDelim, $bUBound ? "" : 3)
    EndIf
EndFunc   ;==>_ArrayDelEmptyRows

Func _ArrayFindEmptyRows(ByRef Const $aArray, $iStartRow = 0)
    If Not IsArray($aArray) Then Return (SetError(-1, 0, ""))
    If UBound($aArray) < 2 Then Return (SetError(-2, 0, ""))
    Local $i, $ii
    Local $sResults = ""
    Local $aReturn
    Local $iArrayRows = UBound($aArray) - 1
    If $iStartRow > $iArrayRows Then Return (SetError(-3, 0, ""))
    Local $iArrayColumns = UBound($aArray, 2)
    If @error = 2 Then $iArrayColumns = 1
    If $iArrayColumns = 1 Then
        For $i = $iStartRow To $iArrayRows
            If $aArray[$i] = "" Then $sResults &= $i & ";"
        Next
    ElseIf $iArrayColumns > 1 Then
        For $i = $iStartRow To $iArrayRows
            For $ii = 0 To $iArrayColumns - 1
                If $aArray[$i][$ii] <> "" Then ExitLoop
                If $ii = $iArrayColumns - 1 Then $sResults &= $i & ";"
            Next
        Next
    EndIf
    If $sResults <> "" Then
        $sResults = StringTrimRight($sResults, 1)
    Else
        Return (SetError(-4, 0, ""))
    EndIf
    $aReturn = StringSplit($sResults, ";", 3)
    Return ($aReturn)
EndFunc   ;==>_ArrayFindEmptyRows

 

 

Edited by Deye
Inpho Wayfarer

Inpho

Posted
Posted (edited) 
Global $sFile = @TempDir & "\TestArray.txt"
Global $aArray, $aArrayRemoveBlanks
Global $i, $ii
Global $hTimer

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayFindEmptyRows($aArray)
_ArrayDelete($aArray, $aArrayRemoveBlanks)
ConsoleWrite("_ArrayFindEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayFindEmptyRows($aArray)
_ArrayDelete($aArray, $aArrayRemoveBlanks)
ConsoleWrite("_ArrayFindEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayFindEmptyRows($aArray)
_ArrayDelete($aArray, $aArrayRemoveBlanks)
ConsoleWrite("_ArrayFindEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayFindEmptyRows($aArray)
_ArrayDelete($aArray, $aArrayRemoveBlanks)
ConsoleWrite("_ArrayFindEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayDelEmptyRows($aArray)
ConsoleWrite("_ArrayDelEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayDelEmptyRows($aArray)
ConsoleWrite("_ArrayDelEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayDelEmptyRows($aArray)
ConsoleWrite("_ArrayDelEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayDelEmptyRows($aArray)
ConsoleWrite("_ArrayDelEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

Exit


; #FUNCTION# ====================================================================================================================
; Name ..........: _ArrayFindEmptyRows
; Description ...: Returns an array of indexes (similar to _ArrayFindAll) ready to feed straight into _ArrayDelete.
; Syntax ........: _ArrayFindEmptyRows(Const Byref $aArray[, $iStartRow = 1])
; Parameters ....: $aArray              - [in/out and const] The array to search for empty rows
;                  $iStartRow           - [optional] an integer value. The index to start searching from
; Return values .: Success:             - An array of indexes
;                  Failure              - Empty string and @error flag as follows:
;                                       1 - $aArray is not an array
;                                       2 - $aArray contains one row
;                                       3 - $iStartRow is out of bounds
;                                       4 - No results (unable to find any blank rows)
; Author ........: Sam Coates
; ===============================================================================================================================
Func _ArrayFindEmptyRows(ByRef Const $aArray, $iStartRow = 1)

    If Not IsArray($aArray) Then Return(SetError(-1, 0, "")) ;; Array isn't an array
    If UBound($aArray) < 2 Then Return(SetError(-2, 0, "")) ;; Array only contains one row

    Local $i, $ii
    Local $sResults = ""
    Local $aReturn
    Local $iArrayRows = UBound($aArray) - 1 ;; hold the number of rows
    If $iStartRow > $iArrayRows Then Return(SetError(-3, 0, "")) ;; Check if StartRow isn't out of bounds

    Local $iArrayColumns = UBound($aArray, 2) ;; hold the number of columns
    If @error = 2 Then $iArrayColumns = 1 ;; if error, then 1d array

    If $iArrayColumns = 1 Then ;; if 1d array

        For $i = $iStartRow To $iArrayRows ;; loop through rows
            If $aArray[$i] = "" Then $sResults &= $i & ";" ;; if its blank, save the index
        Next

    ElseIf $iArrayColumns > 1 Then ;; if 2d array

        For $i = $iStartRow To $iArrayRows ;; loop through rows
            For $ii = 0 To $iArrayColumns - 1 ;; loop through columns
                If $aArray[$i][$ii] <> "" Then ExitLoop ;; if a non-blank is found in any cell on a row, skip to next row
                If $ii = $iArrayColumns - 1 Then $sResults &= $i & ";" ;; if we reach the end of the columns and still havent found a non-blank, save the index
            Next
        Next

    EndIf

    If $sResults <> "" Then ;; if we made changes
        $sResults = StringTrimRight($sResults, 1) ;; strip the final semi-colon
    Else ;; if we made no changes
        Return(SetError(-4, 0, "")) ;; No results
    EndIf

    $aReturn = StringSplit($sResults, ";") ;; split the final string

    Return($aReturn) ;; return it

EndFunc


Func _ArrayDelEmptyRows(ByRef $Array, $bUBound = False, $sDelim = "'") ; $bUBound return the $Array[0] with elements count
    Local Const $pattern = $sDelim & "[|]*[|]"
    Local $str
    If UBound($Array, 2) Then
        Local $sString, $ai
        ;If $Array[0][0] = UBound($Array) Then _ArrayDelete($Array, 0)
        $str = _ArrayToString($Array, Default, Default, Default, $sDelim)
        $sString = StringRegExpReplace($sDelim & $str, $pattern, $sDelim)
        $ai = _ArrayFindAll(StringSplit(StringTrimLeft($sString, 1), $sDelim, 3), "")
        If Not IsArray($ai) Then Return $Array
        Local $UBoundCol = UBound($Array, 2) - 1, $aTemp[UBound($Array) - UBound($ai)][$UBoundCol + 1]
        Local $sAi = "," & _ArrayToString($ai, ",") & ","
        Local $x = -1
        For $i = 0 To UBound($Array) - 1
            If StringInStr($sAi, "," & $i & ",") Then ContinueLoop
            $x += 1
            For $j = 0 To $UBoundCol
                $aTemp[$x][$j] = $Array[$i][$j]
            Next
        Next
        If $bUBound Then _ArrayInsert($aTemp, 0, UBound($aTemp))
        Return $aTemp
    Else
        $str = StringRegExpReplace($sDelim & _ArrayToString($Array, $sDelim), $pattern, $sDelim)
        $sString = StringRegExpReplace($str, $sDelim & "$", "")
        Return StringSplit(StringTrimLeft($sString, 1), $sDelim, $bUBound ? "" : 3)
    EndIf
EndFunc   ;==>_ArrayDelEmptyRows

Func _GenRandomArray()
    Local $aArrayReturn[20000][10]
    If FileExists($sFile) = 0 Then
        For $i = 1 To UBound($aArrayReturn) - 1
            ;(Mod($i, 10) = 0) ? ($aArray[$i][0] = (_StringRandom(20, 4)) : ($aArray[$i][0] = "") ;; why won't this work lel
            If Mod($i, 10) <> 0 Then $aArrayReturn[$i][0] = _StringRandom(20, 4)
        Next
        _FileWriteFromArray($sFile, $aArrayReturn)
    Else
        _FileReadToArray($sFile, $aArrayReturn, 1, "|")

    EndIf

    Return ($aArrayReturn)

EndFunc   ;==>_GenRandomArray

; #FUNCTION# ====================================================================================================================
; Name ..........:  _StringRandom
; Description ...:  Returns a string of random characters
; Syntax ........:  _StringRandom($iAmount[, $iType = 1])
; Parameters ....:  $iAmount            - an integer value. Length of returned string
;                   $iType              - [optional] an integer value. Default is 1.
;                                       1 - Return digits (0-9)
;                                       2 - Return hexadecimal (0-9, A - F)
;                                       3 - Return Alphanumeric upper (0-9, A - Z)
;                                       4 - Return Alphanumeric (0-9, A - Z, a - z)
;                                       5 - Return Alpha upper (A - Z)
;                                       6 - Return Alpha (A - Z, a - z)
; Return values .:  Success             - String
;                   Failure             - Empty string and @error flag as follows:
;                   @error :            1 - $iAmount is not a positive integer
;                                       2 - $iType is out of bounds
; Author ........:  Sam Coates
; ===============================================================================================================================
Func _StringRandom($iAmount, $iType = 1)

    If $iAmount < 1 Or IsInt($iAmount) = 0 Then Return(SetError(-1, 0, ""))

    Local $sString = ""
    Local $iRandom = 0, $iRandomLow = 1, $iRandomHigh = 62
    Local $aCharId[63]

    If $iType = 1 Then ;; digits: 1 - 10
        $iRandomHigh = 10
    ElseIf $iType = 2 Then ;; hexadecimal: 1 - 16
        $iRandomHigh = 16
    ElseIf $iType = 3 Then ;; alnumupper: 1 - 36
        $iRandomHigh = 36
    ElseIf $iType = 4 Then ;; alnum: 1 - 62
        $iRandomHigh = 62
    ElseIf $iType = 5 Then ;; alphaupper: 11 - 36
        $iRandomLow = 11
        $iRandomHigh = 36
    ElseIf $iType = 6 Then ;; alpha: 11 = 62
        $iRandomLow = 11
        $iRandomHigh = 62
    Else
        Return(SetError(-2, 0, ""))
    EndIf

    For $i = 1 To 10 ;; loop through our array, assigning ascii values to each element
        $aCharId[$i] = Chr(47 + $i)
    Next

    For $i = 11 to 36
        $aCharId[$i] = Chr(54 + $i)
    Next

    For $i = 37 To 62
        $aCharId[$i] = Chr(60 + $i)
    Next

    For $i = 1 To $iAmount
        $iRandom = Random($iRandomLow, $iRandomHigh, 1) ;; random interger between $iRandomLow and $iRandomHigh
        $sString &= $aCharId[$iRandom] ;; append string with corresponding character from ascii array
    Next

    Return ($sString)

EndFunc
_ArrayFindEmptyRows() : 539.754431033249
_ArrayFindEmptyRows() : 548.060110904121
_ArrayFindEmptyRows() : 553.839503139588
_ArrayFindEmptyRows() : 547.405446928677
_ArrayDelEmptyRows() : 31038.761705423
_ArrayDelEmptyRows() : 31228.4033974106
_ArrayDelEmptyRows() : 31289.9452321491
_ArrayDelEmptyRows() : 31734.444344232

Unfortunately the bigger the 2D array, the slower it gets :(

Edited by Inpho
Inpho Wayfarer

Inpho

Posted
Posted 
#include <Timers.au3>
#include <Array.au3>
#include <File.au3>

Opt("MustDeclareVars", 1)

Global $sFile = @TempDir & "\TestArray.txt"
Global $aArray, $aArrayRemoveBlanks
Global $i, $ii
Global $hTimer

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayFindEmptyRows($aArray)
_ArrayDelete($aArray, $aArrayRemoveBlanks)
ConsoleWrite("_ArrayFindEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayFindEmptyRows($aArray)
_ArrayDelete($aArray, $aArrayRemoveBlanks)
ConsoleWrite("_ArrayFindEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayFindEmptyRows($aArray)
_ArrayDelete($aArray, $aArrayRemoveBlanks)
ConsoleWrite("_ArrayFindEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayFindEmptyRows($aArray)
_ArrayDelete($aArray, $aArrayRemoveBlanks)
ConsoleWrite("_ArrayFindEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayDelEmptyRows($aArray)
ConsoleWrite("_ArrayDelEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayDelEmptyRows($aArray)
ConsoleWrite("_ArrayDelEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayDelEmptyRows($aArray)
ConsoleWrite("_ArrayDelEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayDelEmptyRows($aArray)
ConsoleWrite("_ArrayDelEmptyRows() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayRemoveBlanks($aArray)
ConsoleWrite("_ArrayRemoveBlanks() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayRemoveBlanks($aArray)
ConsoleWrite("_ArrayRemoveBlanks() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayRemoveBlanks($aArray)
ConsoleWrite("_ArrayRemoveBlanks() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

$aArray = _GenRandomArray()
$hTimer = _Timer_Init()
$aArrayRemoveBlanks = _ArrayRemoveBlanks($aArray)
ConsoleWrite("_ArrayRemoveBlanks() : " & _Timer_Diff($hTimer) & @CRLF)
$aArray = 0

Exit

; #FUNCTION# ====================================================================================================================
; Name ..........: _ArrayFindEmptyRows
; Description ...: Returns an array of indexes (similar to _ArrayFindAll) ready to feed straight into _ArrayDelete.
; Syntax ........: _ArrayFindEmptyRows(Const Byref $aArray[, $iStartRow = 1])
; Parameters ....: $aArray              - [in/out and const] The array to search for empty rows
;                  $iStartRow           - [optional] an integer value. The index to start searching from
; Return values .: Success:             - An array of indexes
;                  Failure              - Empty string and @error flag as follows:
;                                       1 - $aArray is not an array
;                                       2 - $aArray contains one row
;                                       3 - $iStartRow is out of bounds
;                                       4 - No results (unable to find any blank rows)
; Author ........: Sam Coates
; ===============================================================================================================================
Func _ArrayFindEmptyRows(ByRef Const $aArray, $iStartRow = 1)

    If Not IsArray($aArray) Then Return(SetError(-1, 0, "")) ;; Array isn't an array
    If UBound($aArray) < 2 Then Return(SetError(-2, 0, "")) ;; Array only contains one row

    Local $i, $ii
    Local $sResults = ""
    Local $aReturn
    Local $iArrayRows = UBound($aArray) - 1 ;; hold the number of rows
    If $iStartRow > $iArrayRows Then Return(SetError(-3, 0, "")) ;; Check if StartRow isn't out of bounds

    Local $iArrayColumns = UBound($aArray, 2) ;; hold the number of columns
    If @error = 2 Then $iArrayColumns = 1 ;; if error, then 1d array

    If $iArrayColumns = 1 Then ;; if 1d array

        For $i = $iStartRow To $iArrayRows ;; loop through rows
            If $aArray[$i] = "" Then $sResults &= $i & ";" ;; if its blank, save the index
        Next

    ElseIf $iArrayColumns > 1 Then ;; if 2d array

        For $i = $iStartRow To $iArrayRows ;; loop through rows
            For $ii = 0 To $iArrayColumns - 1 ;; loop through columns
                If $aArray[$i][$ii] <> "" Then ExitLoop ;; if a non-blank is found in any cell on a row, skip to next row
                If $ii = $iArrayColumns - 1 Then $sResults &= $i & ";" ;; if we reach the end of the columns and still havent found a non-blank, save the index
            Next
        Next

    EndIf

    If $sResults <> "" Then ;; if we made changes
        $sResults = StringTrimRight($sResults, 1) ;; strip the final semi-colon
    Else ;; if we made no changes
        Return(SetError(-4, 0, "")) ;; No results
    EndIf

    $aReturn = StringSplit($sResults, ";") ;; split the final string

    Return($aReturn) ;; return it

EndFunc

Func _ArrayDelEmptyRows(ByRef $Array, $bUBound = False, $sDelim = "'") ; $bUBound return the $Array[0] with elements count
    Local Const $pattern = $sDelim & "[|]*[|]"
    Local $str, $sString
    If UBound($Array, 2) Then
        Local $ai
        ;If $Array[0][0] = UBound($Array) Then _ArrayDelete($Array, 0)
        $str = _ArrayToString($Array, Default, Default, Default, $sDelim)
        $sString = StringRegExpReplace($sDelim & $str, $pattern, $sDelim)
        $ai = _ArrayFindAll(StringSplit(StringTrimLeft($sString, 1), $sDelim, 3), "")
        If Not IsArray($ai) Then Return $Array
        Local $UBoundCol = UBound($Array, 2) - 1, $aTemp[UBound($Array) - UBound($ai)][$UBoundCol + 1]
        Local $sAi = "," & _ArrayToString($ai, ",") & ","
        Local $x = -1
        For $i = 0 To UBound($Array) - 1
            If StringInStr($sAi, "," & $i & ",") Then ContinueLoop
            $x += 1
            For $j = 0 To $UBoundCol
                $aTemp[$x][$j] = $Array[$i][$j]
            Next
        Next
        If $bUBound Then _ArrayInsert($aTemp, 0, UBound($aTemp))
        Return $aTemp
    Else
        $str = StringRegExpReplace($sDelim & _ArrayToString($Array, $sDelim), $pattern, $sDelim)
        $sString = StringRegExpReplace($str, $sDelim & "$", "")
        Return StringSplit(StringTrimLeft($sString, 1), $sDelim, $bUBound ? "" : 3)
    EndIf
EndFunc   ;==>_ArrayDelEmptyRows

Func _ArrayRemoveBlanks(ByRef $aArray)

    Local $i, $j = 0

    For $i = 1 To UBound($aArray) - 1
        If $aArray[$i] <> "" Then $j += 1
    Next

    Local $aArrayReturn[$j + 1]
    $j = 1

    For $i = 1 To UBound($aArray) - 1
        If $aArray[$i] <> "" Then
            $aArrayReturn[$j] = $aArray[$i]
            $j += 1
        EndIf
    Next

    $aArrayReturn[0] = UBound($aArrayReturn) - 1
    Return ($aArrayReturn)

EndFunc   ;==>_ArrayRemoveBlanks

Func _GenRandomArray()
    Local $aArrayReturn[100000];[10]
    If FileExists($sFile) = 0 Then
        For $i = 1 To UBound($aArrayReturn) - 1
            ;(Mod($i, 10) = 0) ? ($aArray[$i][0] = (_StringRandom(20, 4)) : ($aArray[$i][0] = "") ;; why won't this work lel
            If Mod($i, 10) <> 0 Then $aArrayReturn[$i] = _StringRandom(20, 4)
        Next
        _FileWriteFromArray($sFile, $aArrayReturn)
    Else
        _FileReadToArray($sFile, $aArrayReturn, 1, "|")

    EndIf

    Return ($aArrayReturn)

EndFunc   ;==>_GenRandomArray

; #FUNCTION# ====================================================================================================================
; Name ..........:  _StringRandom
; Description ...:  Returns a string of random characters
; Syntax ........:  _StringRandom($iAmount[, $iType = 1])
; Parameters ....:  $iAmount            - an integer value. Length of returned string
;                   $iType              - [optional] an integer value. Default is 1.
;                                       1 - Return digits (0-9)
;                                       2 - Return hexadecimal (0-9, A - F)
;                                       3 - Return Alphanumeric upper (0-9, A - Z)
;                                       4 - Return Alphanumeric (0-9, A - Z, a - z)
;                                       5 - Return Alpha upper (A - Z)
;                                       6 - Return Alpha (A - Z, a - z)
; Return values .:  Success             - String
;                   Failure             - Empty string and @error flag as follows:
;                   @error :            1 - $iAmount is not a positive integer
;                                       2 - $iType is out of bounds
; Author ........:  Sam Coates
; ===============================================================================================================================
Func _StringRandom($iAmount, $iType = 1)

    If $iAmount < 1 Or IsInt($iAmount) = 0 Then Return(SetError(-1, 0, ""))

    Local $sString = ""
    Local $iRandom = 0, $iRandomLow = 1, $iRandomHigh = 62
    Local $aCharId[63]

    If $iType = 1 Then ;; digits: 1 - 10
        $iRandomHigh = 10
    ElseIf $iType = 2 Then ;; hexadecimal: 1 - 16
        $iRandomHigh = 16
    ElseIf $iType = 3 Then ;; alnumupper: 1 - 36
        $iRandomHigh = 36
    ElseIf $iType = 4 Then ;; alnum: 1 - 62
        $iRandomHigh = 62
    ElseIf $iType = 5 Then ;; alphaupper: 11 - 36
        $iRandomLow = 11
        $iRandomHigh = 36
    ElseIf $iType = 6 Then ;; alpha: 11 = 62
        $iRandomLow = 11
        $iRandomHigh = 62
    Else
        Return(SetError(-2, 0, ""))
    EndIf

    For $i = 1 To 10 ;; loop through our array, assigning ascii values to each element
        $aCharId[$i] = Chr(47 + $i)
    Next

    For $i = 11 to 36
        $aCharId[$i] = Chr(54 + $i)
    Next

    For $i = 37 To 62
        $aCharId[$i] = Chr(60 + $i)
    Next

    For $i = 1 To $iAmount
        $iRandom = Random($iRandomLow, $iRandomHigh, 1) ;; random interger between $iRandomLow and $iRandomHigh
        $sString &= $aCharId[$iRandom] ;; append string with corresponding character from ascii array
    Next

    Return ($sString)

EndFunc
_ArrayFindEmptyRows() : 0.0637558740930338
_ArrayFindEmptyRows() : 0.0628228613014284
_ArrayFindEmptyRows() : 0.0665549124678499
_ArrayFindEmptyRows() : 0.0631338655652969
_ArrayDelEmptyRows() : 0.10480843692367
_ArrayDelEmptyRows() : 0.0727749977452191
_ArrayDelEmptyRows() : 0.0730860020090875
_ArrayDelEmptyRows() : 0.0718419849536137
_ArrayRemoveBlanks() : 0.0398085457751626
_ArrayRemoveBlanks() : 0.0373205116642149
_ArrayRemoveBlanks() : 0.0404305543028995
_ArrayRemoveBlanks() : 0.0413635670945049

A very nooby but even faster way with large 1D arrays; _ArrayRemoveBlanks()

jchd Universalist

jchd

Posted
Posted

These last timing denote an error, or you have a really very fast machine!

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Inpho Wayfarer

Inpho

Posted
Posted

@jchd My brain doesn't work like most others...

Was a problem in _GenRandomArray; fixed below. 

Func _GenRandomArray()
    Local $aArrayReturn[100000];[10]
    If FileExists($sFile) = 0 Then
        For $i = 1 To UBound($aArrayReturn) - 1
            ;(Mod($i, 10) = 0) ? ($aArray[$i][0] = (_StringRandom(20, 4)) : ($aArray[$i][0] = "") ;; why won't this work lel
            If Mod($i, 10) <> 0 Then $aArrayReturn[$i] = _StringRandom(20, 4)
        Next
        _FileWriteFromArray($sFile, $aArrayReturn)
    Else
        _FileReadToArray($sFile, $aArrayReturn, 1) ;; for 1d array
        ;_FileReadToArray($sFile, $aArrayReturn, 1, "|") ;; for 2d array

    EndIf

    Return ($aArrayReturn)

EndFunc   ;==>_GenRandomArray
_ArrayFindEmptyRows() : 613.280193071447
_ArrayFindEmptyRows() : 597.97971630191
_ArrayFindEmptyRows() : 595.951657497224
_ArrayFindEmptyRows() : 595.835030898274
_ArrayDelEmptyRows() : 211.501870690647
_ArrayDelEmptyRows() : 205.61238294577
_ArrayDelEmptyRows() : 205.040135100252
_ArrayDelEmptyRows() : 207.841972513443
_ArrayRemoveBlanks() : 471.809951514435
_ArrayRemoveBlanks() : 469.115099568015
_ArrayRemoveBlanks() : 470.167537996946
_ArrayRemoveBlanks() : 470.371867798308

As yesterday, Deye's function proves faster in this case (1d array). My _ArrayRemoveBlanks is faster than my _ArrayFindEmptyRows but again, at the expense of memory.

 

Inpho Wayfarer

Inpho

Posted
Posted

Unnecessary data conversion makes it slow. Changed one line and made it faster. No ping pong with conversion :) 

#include <Timers.au3>
#include <Array.au3>

Local $Array = _GenRandomArray(100000)
$aTmp = $Array


For $i = 1 To 9
    $hTimer = _Timer_Init()
    $aArrayRemoveBlanks = _ArrayDelEmptyRowsOld($aTmp)        ;2D
    ConsoleWrite("2D : " & _Timer_Diff($hTimer) & @CRLF)
    $aArrayRemoveBlanks = 0
    $aTmp = $Array
Next

For $i = 1 To 9
    $hTimer = _Timer_Init()
    $aArrayRemoveBlanks = _ArrayDelEmptyRows($aTmp)        ;2D
    ConsoleWrite("2D : " & _Timer_Diff($hTimer) & @CRLF)
    $aArrayRemoveBlanks = 0
    $aTmp = $Array
Next

Func _GenRandomArray($icount)
    Local $aArrayReturn[$icount][4]
    For $i = 0 To UBound($aArrayReturn) - 1
        If Mod($i, 5) = 0 Then $aArrayReturn[$i][1] = Random(1, 100)
    Next
    Return $aArrayReturn
EndFunc   ;==>_GenRandomArray

Func _ArrayDelEmptyRows(ByRef $aArray, $bUBound = False, $sDelim = "'")
    If UBound($aArray, 2) >= 1 Then
        Local $ai = _ArrayFindEmptyRows($aArray)
        If Not IsArray($ai) Then Return $aArray
        _ArrayDelete($aArray, $ai) ;; changed this line to remove the unnecessary _ArrayToString
        If $bUBound Then _ArrayInsert($aArray, 0, UBound($aArray))
        Return $aArray
    Else
        Local $pattern = $sDelim & "[" & $sDelim & "]*[" & $sDelim & "]"
        $str = StringRegExpReplace($sDelim & _ArrayToString($aArray, $sDelim), $pattern, $sDelim)
        $sString = StringRegExpReplace($str, $sDelim & "$", "")
        Return StringSplit(StringTrimLeft($sString, 1), $sDelim, $bUBound ? "" : 3)
    EndIf
EndFunc   ;==>_ArrayDelEmptyRows

; #FUNCTION# ====================================================================================================================
; Name ..........: _ArrayFindEmptyRows
; Description ...: Returns an array of indexes (similar to _ArrayFindAll) ready to feed straight into _ArrayDelete.
; Syntax ........: _ArrayFindEmptyRows(Const Byref $aArray[, $iStartRow = 1])
; Parameters ....: $aArray              - [in/out and const] The array to search for empty rows
;                  $iStartRow           - [optional] an integer value. The index to start searching from
; Return values .: Success:             - An array of indexes
;                  Failure              - Empty string and @error flag as follows:
;                                       1 - $aArray is not an array
;                                       2 - $aArray contains one row
;                                       3 - $iStartRow is out of bounds
;                                       4 - No results (unable to find any blank rows)
; Author ........: Sam Coates
; ===============================================================================================================================
Func _ArrayFindEmptyRows(ByRef Const $aArray, $iStartRow = 1)

    If Not IsArray($aArray) Then Return(SetError(-1, 0, "")) ;; Array isn't an array
    If UBound($aArray) < 2 Then Return(SetError(-2, 0, "")) ;; Array only contains one row

    Local $i, $ii
    Local $sResults = ""
    Local $aReturn
    Local $iArrayRows = UBound($aArray) - 1 ;; hold the number of rows
    If $iStartRow > $iArrayRows Then Return(SetError(-3, 0, "")) ;; Check if StartRow isn't out of bounds

    Local $iArrayColumns = UBound($aArray, 2) ;; hold the number of columns
    If @error = 2 Then $iArrayColumns = 1 ;; if error, then 1d array

    If $iArrayColumns = 1 Then ;; if 1d array

        For $i = $iStartRow To $iArrayRows ;; loop through rows
            If $aArray[$i] = "" Then $sResults &= $i & ";" ;; if its blank, save the index
        Next

    ElseIf $iArrayColumns > 1 Then ;; if 2d array

        For $i = $iStartRow To $iArrayRows ;; loop through rows
            For $ii = 0 To $iArrayColumns - 1 ;; loop through columns
                If $aArray[$i][$ii] <> "" Then ExitLoop ;; if a non-blank is found in any cell on a row, skip to next row
                If $ii = $iArrayColumns - 1 Then $sResults &= $i & ";" ;; if we reach the end of the columns and still havent found a non-blank, save the index
            Next
        Next

    EndIf

    If $sResults <> "" Then ;; if we made changes
        $sResults = StringTrimRight($sResults, 1) ;; strip the final semi-colon
    Else ;; if we made no changes
        Return(SetError(-4, 0, "")) ;; No results
    EndIf

    $aReturn = StringSplit($sResults, ";") ;; split the final string

    Return($aReturn) ;; return it

EndFunc

Func _ArrayDelEmptyRowsOld(ByRef $aArray, $bUBound = False, $sDelim = "'")
    If UBound($aArray, 2) >= 1 Then
        Local $ai = _ArrayFindEmptyRowsOld($aArray)
        If Not IsArray($ai) Then Return $aArray
        _ArrayDelete($aArray, _ArrayToString($ai, ";"))
        If $bUBound Then _ArrayInsert($aArray, 0, UBound($aArray))
        Return $aArray
    Else
        Local $pattern = $sDelim & "[" & $sDelim & "]*[" & $sDelim & "]"
        $str = StringRegExpReplace($sDelim & _ArrayToString($aArray, $sDelim), $pattern, $sDelim)
        $sString = StringRegExpReplace($str, $sDelim & "$", "")
        Return StringSplit(StringTrimLeft($sString, 1), $sDelim, $bUBound ? "" : 3)
    EndIf
EndFunc   ;==>_ArrayDelEmptyRows

Func _ArrayFindEmptyRowsOld(ByRef Const $aArray, $iStartRow = 0)
    If Not IsArray($aArray) Then Return (SetError(-1, 0, ""))
    If UBound($aArray) < 2 Then Return (SetError(-2, 0, ""))
    Local $i, $ii
    Local $sResults = ""
    Local $aReturn
    Local $iArrayRows = UBound($aArray) - 1
    If $iStartRow > $iArrayRows Then Return (SetError(-3, 0, ""))
    Local $iArrayColumns = UBound($aArray, 2)
    If @error = 2 Then $iArrayColumns = 1
    If $iArrayColumns = 1 Then
        For $i = $iStartRow To $iArrayRows
            If $aArray[$i] = "" Then $sResults &= $i & ";"
        Next
    ElseIf $iArrayColumns > 1 Then
        For $i = $iStartRow To $iArrayRows
            For $ii = 0 To $iArrayColumns - 1
                If $aArray[$i][$ii] <> "" Then ExitLoop
                If $ii = $iArrayColumns - 1 Then $sResults &= $i & ";"
            Next
        Next
    EndIf
    If $sResults <> "" Then
        $sResults = StringTrimRight($sResults, 1)
    Else
        Return (SetError(-4, 0, ""))
    EndIf
    $aReturn = StringSplit($sResults, ";", 3)
    Return ($aReturn)
EndFunc   ;==>_ArrayFindEmptyRows
2D : 2406.28075064223
2D : 2304.23184817969
2D : 2299.91851662945
2D : 2317.10839776325
2D : 2305.10794991572
2D : 2300.56043142646
2D : 2309.56527688796
2D : 2316.27894681189
2D : 2311.45867673494
2D : 1639.29675497142
2D : 1644.15527869179
2D : 1636.44670303355
2D : 1640.72364697174
2D : 1637.47146527005
2D : 1646.61346403847
2D : 1645.56444339394
2D : 1649.99160285876
2D : 1636.63206215129

 

Deye Universalist

Deye

Posted
  • Author
Posted (edited)

Comes to show how _arraytostring  is slow @ degrading speed 

- Sending a ready made array to arraydelete will get it processed faster than sending it just a string, so  using  stringsplit priory will get it go faster than arraydelete will do it,  strange thing ..

Edited by Deye
Inpho Wayfarer

Inpho

Posted
Posted

Yep, I see why; looking at the _ArrayDelete function, it uses StringSplit on the string that gets fed to it to turn it into an array. Nice to know we can save some roundabouts.

I'm going to scrutinise functions that accept multiple datatypes as an input from now on - if there's a fastest datatype , i want to use it.

 

  • Solution
Deye Universalist

Deye

Posted
  • Author
  • Solution
Posted (edited)

Yet an extra shave off

all so if there are no empty rows found  it will return faster than previous examples .. 

Func _ArrayDelEmptyRows(ByRef $aArray, $bUBound = False, $sDelim = "'")
    Local $iArrayColumns = UBound($aArray, 2)
    If $iArrayColumns >= 1 Then
        Local $iCopyTo_Index = 0
        For $i = 0 To UBound($aArray) - 1
            For $j = 0 To $iArrayColumns - 1
                If $aArray[$i][$j] <> "" Then ExitLoop
                If $j = $iArrayColumns - 1 Then ContinueLoop 2
            Next
            If $i <> $iCopyTo_Index Then
                For $j = 0 To $iArrayColumns - 1
                    $aArray[$iCopyTo_Index][$j] = $aArray[$i][$j]
                Next
            EndIf
            $iCopyTo_Index += 1
        Next
        If UBound($aArray) > $iCopyTo_Index Then ReDim $aArray[$iCopyTo_Index][$iArrayColumns]
        If $bUBound Then _ArrayInsert($aArray, 0, UBound($aArray))
        Return ($aArray)
    Else
        Local $pattern = $sDelim & "[" & $sDelim & "]*[" & $sDelim & "]"
        $str = StringRegExpReplace($sDelim & _ArrayToString($aArray, $sDelim), $pattern, $sDelim)
        $sString = StringRegExpReplace($str, $sDelim & "$", "")
        Return StringSplit(StringTrimLeft($sString, 1), $sDelim, $bUBound ? "" : 3)
    EndIf
EndFunc

 

Edited by Deye
Inpho Wayfarer

Inpho

Posted
Posted

Nice, another 30% speed gained in my tests. 

Please comment your code (also, please don't strip comments from mine; I'd like them to remain personally if it's not an unfavourable request :)).

Deye Universalist

Deye

Posted
  • Author
Posted

Inpho,

Very favorable, Yet IMO comments go better with udfs or larger functions, Less with small examples such as these
As this thread is now (better left untouched) over packed with examples

Any one reading the thread will see your remarks 
Thanks for the heads up and ideas into looking for faster ways..

Deye

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...