How to write a more efficient script?

[clearguy]

Hi,

as said in titel,what are the hints to write a more efficient script?

I ask this because of my project SOW Project which is ver,very slow.

Thanx

[bluebearr]

Your script is so slow because you are reading from files and writing to files only one character at a time. For each operation, you are opening the file, doing the read/write, and then closing the file again.

One way to speed up the file operations is to use file handles, so that you keep a file open until you are done with it. You can also speed up you file operations by reading and writing larger chunks of data.

I used this latter technique to speed up your GenHexSource function as shown below. In this version, I build a variable ($newfile) and only write it to the actual file every 100 lines. This is probably a 100 times faster than your version. However, my version may add an extra @CRLF at the end of the file (you should check this and remove it if this is the case).

Func GenHexSource();~~ generates a file with the characters converted in hexa,at each line
    GUICtrlSetData($stat,"Generating HexSource")  ;~~ updates data in status bar
    $code = FileRead($filepath)   ;~~ reads the file to encrypt
    $length = StringLen(FileRead($filepath)) ;~~ length of the string of the file to encrypt
    Dim $newfile
    For $start = 1 To $length   ;~~ starts to making the HexSource file(HS)
        $string = StringMid($code,$start,1) ;~~
        $string = _StringToHex($string) ;~~
        $newfile = $newfile & $string & @CRLF ; add data to the $newfile variable
        If Int($start/100) = $start/100 Then  ; write to file every 100 lines
            FileWriteLine(@WorkingDir&"\generated.hs",$newfile) ;~~ a temp file
            $newfile = ''
        EndIf
        $percent = (100*$start)/$length ;~~
        GUICtrlSetData($percentage,$percent&"%") ;~~updates progress
        GUICtrlSetData($progressbar,$percent) ;~~updates progress
    Next ;~~ end of HexSource generation
    if $newfile <> '' Then  ; write whatever lines are left that we haven't written yet
        FileWriteLine(@WorkingDir&"\generated.hs",$newfile)
    EndIf
    GUICtrlSetData($progressbar,0) ;~~
EndFunc ;~~

Edited August 21, 2006 by bluebearr

[clearguy]

Thank you so much,you've explained me a real great thing,you should make so a kinda "better efficient" topic which will be sticked

[clearguy]

i've tryed it,I've put fileopen and used a filehandle in filewrite(),but I don't feel it's 100 times faster,I don't see any chage,I think my script is just badly written, ,isn't it?

[bluebearr]

i've tryed it,I've put fileopen and used a filehandle in filewrite(),but I don't feel it's 100 times faster,I don't see any chage,I think my script is just badly written, ,isn't it?

As I said in my original reply, I think the main issue is that you write to or read from the files for every character (or its hexadecimal equivalent). Since writing to memory is about a million times faster than writing to disk, if you do most of your work in memory (writing to variables) and only write to disk infrequently, it will greatly speed up your script.

[strate]

$code = FileRead($filepath)  ;~~ reads the file to encrypt
    $length = StringLen(FileRead($filepath));~~ length of the string of the file to encrypt

Could be placed with

$code = FileRead($filepath)  ;~~ reads the file to encrypt
    $length = StringLen( $code );~~ length of the string of the file to encrypt

[strate]

This is completely untested.

Func GenHexSource($filepath);~~ generates a file with the characters converted in hexa,at each line
    Local $Code = '', $newfile = ''
    GUICtrlSetData($stat,"Generating HexSource");~~ updates data in status bar
    $code = FileRead($filepath) ;~~ reads the file to encrypt
    FileWriteLine(@WorkingDir&"\generated.hs",_StringToHex($code));~~ a temp file
EndFunc;~~  :mellow:  :mellow:

EDIT: and I have no idea what I'm doing....

Edited August 21, 2006 by strate

[clearguy]

This is completely untested.

Func GenHexSource($filepath);~~ generates a file with the characters converted in hexa,at each line
    Local $Code = '', $newfile = ''
    GUICtrlSetData($stat,"Generating HexSource");~~ updates data in status bar
    $code = FileRead($filepath);~~ reads the file to encrypt
    FileWriteLine(@WorkingDir&"\generated.hs",_StringToHex($code));~~ a temp file
EndFunc;~~  :mellow:  :mellow:

EDIT: and I have no idea what I'm doing....

I see lol.

Look at here,I just work on memory,with your $newfile variable,but it takes alot of time too:

For $start = 1 To $length   ;~~ starts to making the HexSource file(HS)
        $string = _StringToHex(StringMid($code,$start,1)) ;~~
        $newfile = $newfile & $string & @CRLF ;variable that takes all information => NO disk write
        ;$string = _StringToHex($string) ;~~not used
        ;FileWriteLine($handle,$string) ;~~  not used
        $percent = (100*$start)/$length ;~~
        GUICtrlSetData($percentage,$percent&"%") ;~~updates progress
        GUICtrlSetData($progressbar,$percent) ;~~updates progress
    Next ;~~ end of HexSource generation

[bluebearr]

I just work on memory,with your $newfile variable,but it takes alot of time too

The next thing that I would try to optimize would be the percent display. You don't need to update the display on every character. Maybe something like this:

$percentold = 0
    For $start = 1 To $length   ;~~ starts to making the HexSource file(HS)
        $string = _StringToHex(StringMid($code,$start,1)) ;~~
        $newfile = $newfile & $string & @CRLF ;variable that takes all information => NO disk write
        ;$string = _StringToHex($string) ;~~not used
        ;FileWriteLine($handle,$string) ;~~  not used
        $percent = Round((100*$start)/$length, 0) ;~~
        if $percent <> $percentold Then
            GUICtrlSetData($percentage,$percent&"%") ;~~updates progress
            GUICtrlSetData($progressbar,$percent) ;~~updates progress
            $percentold = $percent
        EndIf
    Next ;~~ end of HexSource generation

This won't help as much as not writing to the file on every character, but it may do something.

However, you may want to also rethink your whole strategy of creating the HexSource file. If this file is just the original file converted to hex values and with a @CRLF between each original character, this is something you could do in memory on the fly as you create your encrypted file. This would also speed things up.

[clearguy]

Yes 200% more faster,thank you!

Now it generates directly the encrypted file,which is an important gain of time.And with the update of the progressbar,I noticed that without it were faster,but didn't know how to regulate this...until you gave me THE idea