Array Loop Help

[xcaliber13]

Hello,

       I have a very large file (over 200,000) lines.  I read it to an array and once in the array If 3 cells match the same three cells in the next line down then copy another cell to the first line.  And do this for every line that matches.   Here is the code I am using. It seems to be working but it has been running for over 12 hours and has not completed.  Is there a better way for me to do this?   Thank you

Local $aArray
_FileReadToArray('C:\RegTemp\temp1.txt', $aArray, $FRTA_NOCOUNT, @TAB)
Sleep(3000)
for $i = 1 to UBound($aArray)-1 step 1
    $origRow = $aArray[$i][3] & $aArray[$i][7] & $aArray[$i][12]
    $1Row = $aArray[$i+1][3] & $aArray[$i+1][7] & $aArray[$i+1][12]
    $2Row = $aArray[$i+2][3] & $aArray[$i+2][7] & $aArray[$i+2][12]
    $3Row = $aArray[$i+3][3] & $aArray[$i+3][7] & $aArray[$i+3][12]
    $4Row = $aArray[$i+4][3] & $aArray[$i+4][7] & $aArray[$i+4][12]
    $5Row = $aArray[$i+5][3] & $aArray[$i+5][7] & $aArray[$i+5][12]
    Sleep(5000)
    If $origRow = $1Row Then
        $aArray[$i][10] = $aArray[$i][10] &" "& $aArray[$i+1][10]
        $aArray[$i+1][10] = ""
        If $origRow = $2Row Then
        $aArray[$i][10] = $aArray[$i][10] &" "& $aArray[$i+2][10]
        $aArray[$i+2][10] = ""
        If $origRow = $3Row Then
        $aArray[$i][10] = $aArray[$i][10] &" "& $aArray[$i+3][10]
        $aArray[$i+3][10] = ""
        If $origRow = $4Row Then
        $aArray[$i][10] = $aArray[$i][10] &" "& $aArray[$i+4][10]
        $aArray[$i+4][10] = ""
        If $origRow = $5Row Then
        $aArray[$i][10] = $aArray[$i][10] &" "& $aArray[$i+5][10]
        $aArray[$i+5][10] = ""
        EndIf
        EndIf
        EndIf
        EndIf
   EndIf
Next
_FileWriteFromArray('C:\RegTemp\temp1.txt\temp2.txt', $aArray, Default, Default)
_ArrayDisplay($aArray)

 

[water]

I would remove the Sleep statement. 5 Seconds x 200,000 times equals to about 278 hours

[TheXman]

REALLY?!?  You have a 5 second sleep inside the loop.  If you plan to go through the loop 200,000 times, that is over 277 hours of sleep time. 

Edited October 30, 2020 by TheXman

[xcaliber13]

Thanks for pointing that out but if I do not have a sleep I am getting this error:

"C:\Users\user82\Desktop\WorkingScripts\AdvocateReg.au3" (45) : ==> Array variable has incorrect number of subscripts or subscript dimension range exceeded.:

No error if there is a sleep.   So any other way around this?  Am I just missing something else?   Thank you   And yeah I did not see how long the sleep would add up. 

[TheXman]

First of all, I find it extremely difficult to believe that the sleep has anything to do with the error that your receiving other than, without it, it allows you to get to the line with the error sooner.

That error message would lead me to believe that all of your tab-separated value lines do not have at least 13 tab-separated values.  Your code makes the assumption that every line has the at least 13 values.  If any lines have less than 13 values, that is why you would get that error.

Edit:

I think @Dan_555's comment about how you are referencing "$i+x" is the more likely reason you're getting the error.  Removing the sleep just allowed you to get to the end of the file, where it is happening, sooner.

Edited October 30, 2020 by TheXman

[Dan_555]

Hmm

looks like the array can go out of bounds.

at least logically

for $i = 1 to UBound($aArray)-1 step 1
    $origRow = $aArray[$i][3] & $aArray[$i][7] & $aArray[$i][12]
    $1Row = $aArray[$i+1][3] & $aArray[$i+1][7] & $aArray[$i+1][12]
    $2Row = $aArray[$i+2][3] & $aArray[$i+2][7] & $aArray[$i+2][12]
    $3Row = $aArray[$i+3][3] & $aArray[$i+3][7] & $aArray[$i+3][12]
    $4Row = $aArray[$i+4][3] & $aArray[$i+4][7] & $aArray[$i+4][12]
    $5Row = $aArray[$i+5][3] & $aArray[$i+5][7] & $aArray[$i+5][12]

You are using the for loop to check to the upper bound -1, but you are increasing the $i up to +5 in the 5th row. Here, the number should be, at least, defined as Ubound($aArray)-6

 

The other source of array error may be that the first lines from the text are not separated enough by the delimiter, so that the array is created with less than needed number of delimiters.

I'm reffering to $aArray[lines][delimiters]  

 

Edited October 30, 2020 by Dan_555

[xcaliber13]

TheXMan   thank you for taking the time to help.  But if that were the case would I not be able to do a _ArrayDisplay right after _FileReadtoArray?   The _ArrayDisplay is showing 238303 row 13 columns.  I checked each line or at least to the limit that _ArrayDisplay is showing (65524 lines) and each line is 13 cells.  Col1 - Col12

Hope I am making some since.  Just trying to understand this.  Again thank you

[TheXman]

Yes, I updated my previous post to agree with Dan_555.  The way you are accessing rows, by using $i+x, makes the assumption that those lines exist.  For example, if you had only 1 row in the array, you can see how you would try to reference rows that are outside the bounds of the array.  You would have to be lucky for your code not to get that error.  The number of rows would have to align perfectly with the expected bounds.  If you are going to do it that way, you should add code to make sure that you are not exceeding the bounds of the array.  Something like:

If $i+x < ubound($aArray) Then Do something with $aArray[$i+x][y]

 

Edited October 30, 2020 by TheXman

[xcaliber13]

OK!   Now that makes perfect since.  Thank  you   I will continue to work on this script.