For Next Loops Speed

[step887]

So I have array of data and an index of lines that need to some follow up data, some of the lines maybe duplicated so I am doing stringregexp 3 and then returning data and then doing another FOR NEXT with the result of that, Some lines are not duplicated so the Loop is 0 to 0, so it runs through once, but I noticed the less loops the more time needed. Which I do not understand. So why the increase time, and any ideas for work arounds3w3fes5245w vcre45tfgdeedefdfdffdfr343w3w5edvggegsdacZ

Here is some times from script 

first number is the second number in for statement 

Ex:For $i = 0 to 0

0 - 2954.91724688962

2 - 6.81581250237796

2 - 6.30711866986265

0 - 2905.05535897729

0 - 2900.23817676825

I made example code and here the results

3 - 34.9009790513521

2 - 70.7139274168353

0 - 233.322919074548

0 - 231.709153067621

0 - 231.155909447909

1 - 122.447368578827

 

Here is my example code 

#include <array.au3>
Global $main[1600][8],$index = '<>', $num
For $i = 1 to 1000
    $main[$i][0] = 'somedata'
    $main[$i][1] = 'somedata'
    _randomdata($i)
    $main[$i][3] ='somedata'
    $main[$i][4] ='somedata'
    $main[$i][5] ='somedata'
    $main[$i][6] ='somedata'
Next
_ArrayDisplay($main)
$num = StringSplit($num, '|')
MsgBox(0,'',$index)
For $i = 1 to $num[0]
    $st = TimerInit()
    $arr = StringRegExp($index,'.+?'&$num[$i]&'\<(\d+?)\>',3)
    If Not IsArray($arr) Then ContinueLoop
    $r = Ubound($arr)-1
    For $f = 0 to $r
        $main[$arr[$f]][7] = $main[$arr[$f]][2]
    Next
ConsoleWrite($r & ' - ' & TimerDiff($st) & @CRLF)
Next
_ArrayDisplay($main)
Func _Randomdata($n)
    $x = Random(1,2,1)
    If $x = 2 Then
         $r = Random(100000,100500,1)
         $main[$n][2] = $r
        $index &= $r & '<'&$n&'>'
        If Not StringInStr($num,$r) Then $num &= $r & '|'
    EndIf
EndFunc

 

[Melba23]

step887,

As clear as mud - and the code is no help. Try explaining again and adding some comments to the script so we can get an idea of what is supposed to be happening.

M23

[kylomas]

Should this line $num = StringSplit($num, '|') be  $num = StringSplit($main, '|')  ??

edit: Ignore this, I see it, set in _Randomdata

Edited August 30, 2014 by kylomas

[Jos]

It is a bit of a complex way of showing that  StringRegExp() takes longer when nothing is found than when it returns something.

Jos

[Jos]

easier example:

$num = "<>100231<4>100159<7>100480<8>100407<15>100423<16>100372<17>100090<18>100202<21>100330<23>100091<24>100352<25>100430<26>100238<28>100221"
For $x = 1 to 6
        $num &= $num
Next
$st = TimerInit()
$arr = StringRegExp($num,'.+?100423\<(\d+?)\>',3)
ConsoleWrite(Ubound($arr) & ' - ' & TimerDiff($st) & @CRLF)

$st = TimerInit()
$arr = StringRegExp($num,'.+?999999\<(\d+?)\>',3)
ConsoleWrite(Ubound($arr) & ' - ' & TimerDiff($st) & @CRLF)

[Jos]

I am not a RegEx person, but does this give the same result but faster?

$arr = StringRegExp($index,'.*?'&$num[$i]&'\<(\d*?)\>',3)

Jos

[step887]

step887,

As clear as mud - and the code is no help. Try explaining again and adding some comments to the script so we can get an idea of what is supposed to be happening.

M23

Sorry I try to clear, basically I have array of files and column 2 is MD5 and if that is populated I use to look up data on that file and put in column 7 (8)

but sometimes there is same md5 in the array of files, so I will populate the same data to all lines with the same md5.  I have commented on the code below.

 

Should this line $num = StringSplit($num, '|') be  $num = StringSplit($main, '|')  ??

No $num is a lists of numbers I used in column 2 and need go through the array and add data to column 7

It is a bit of a complex way of showing that  StringRegExp() takes longer when nothing is found than when it returns something.

Jos

Actually StringregExp flag 3 is returning at least one result, and since you mentioned this I updated the code and it is Stringregexp that is taken the time.  Is there a workaround for this?

first number is num of returns - 1 because I am using ubound, SRE is time difference after the reg exp and total is after the For  

1 SRE: 202.508481847547 total: 202.540124318271

0 SRE: 222.228402681416 total: 222.256714365748

0 SRE: 221.022324928871 total: 221.050303534564

1 SRE: 22.6979769469612 total: 22.7189609012309

0 SRE: 218.70642915051 total: 218.735073913481

 

Updated code with explanation 

expandcollapse popup

#include <array.au3>
Global $main[1600][8],$index = '<>', $num
For $i = 1 to 1000
    $main[$i][0] = 'somedata'
    $main[$i][1] = 'somedata'
    _randomdata($i)
    $main[$i][3] ='somedata'
    $main[$i][4] ='somedata'
    $main[$i][5] ='somedata'
    $main[$i][6] ='somedata'
Next
;populating data in my array
;if something is column 2, I need to follow up and add more data into column 7
_ArrayDisplay($main)
$num = StringSplit($num, '|');this is a index of the numbers used on column 2
MsgBox(0,'',$index); here is the index
For $i = 1 to $num[0]; so this goes through all the numbers used in column 2
    $st = TimerInit()
    $arr = StringRegExp($index,'.+?'&$num[$i]&'\<(\d+?)\>',3); checks the index and returns everytime a number is used in main

    If Not IsArray($arr) Then ContinueLoop; no matches
    $r = Ubound($arr)-1; number of returns -1
        ConsoleWrite($r & ' SRE: '  & TimerDiff($st))
    For $f = 0 to $r
        $main[$arr[$f]][7] = $main[$arr[$f]][2];writes data the to all numbers used.
    Next
ConsoleWrite(' total: ' & TimerDiff($st) & @CRLF); so this writes how many times
Next
_ArrayDisplay($main)
Func _Randomdata($n)
    $x = Random(1,2,1);use this to writ 50% of the clls
    If $x = 2 Then
         $r = Random(100000,100500,1); assign one of 500 numbers
         $main[$n][2] = $r
        $index &= $r & '<'&$n&'>'
        If Not StringInStr($num,$r) Then $num &= $r & '|'; keep a list of numbers used
    EndIf
EndFunc

2 SRE: 16.0430657357341 total: 16.0647158472821

[Jos]

 

.  Is there a workaround for this?

 

Did you see the post above your last one?

Edited August 29, 2014 by Jos

[step887]

I am not a RegEx person, but does this give the same result but faster?

$arr = StringRegExp($index,'.*?'&$num[$i]&'\<(\d*?)\>',3)

Jos

that works

0 SRE: 0.315758549962129 total: 0.380042727327837

3 SRE: 0.272791405505257 total: 0.307098505342914

1 SRE: 0.263798282246842 total: 0.287446865630082

2 SRE: 0.253472844431625 total: 0.279119899650068

3 SRE: 0.273124484144458 total: 0.303767718950909

 

Thanks a lot Jos, so I guess the question is what is the difference between + and * in regex?

[step887]

Did you see the post above your last one?

 

You posted while I typing up my previous response 

[JLogan3o13]

Regex generally makes my eyes bleed, but I thought + requires at least one character to match, where * does not.

[Jos]

Thanks a lot Jos, so I guess the question is what is the difference between + and * in regex?

The difference is explained in the Helpfile: +? = 1 or more Lazy; *? = 0 or more Lazy, but I have no idea why the difference is so big.

I am sure somebody will stop by that understands

[step887]

I do not understand it either, but I went from 480 secs to less then one sec making that one change

Thanks for the help. 

[jchd]

Your sample code is overly complex to show your point.

Notice that neither .+? nor .*? are necessary: these sub-pattern just waste cycles for no benefit.Worse, .+? may cause duplicates to be missed.

Now to the point: PCRE (the regexp engine used by AutoIt) performs complex initial optimizations by default. It is possible to obtain the bytecode generated by the internal pattern compiler but it isn't worse the pain in this case.

As a general rule of thumb, avoid .+ and .* as much as possible for efficiency.

Try this version:

$arr = StringRegExp($index, $num[$i] & '\<(\d+)\>', 3); checks the index and returns everytime a number is used in main