Clear memory

[xdp22]

Hello all users, sorry for bad english. I got problem.

I using script, what checking for me 10 000 lines of text, it's usefully for me to make some things to work.

But i get error after some minutes of script work, this error is - Error Allocating Memory, it's because of allocating virtual memory.

What i can do? i make it bigger in control panel, but didn't help.

My computer is not bad for autoit i think.. it's :

INTEL Core i5 760

4 GB RAM DDR3

GeForce GTX 460

1 Terabyte Harddrive

Windows XP

How i can fix that error? meybe there is some thing for clean memory? function in autoit?

Thanks

[kaotkbliss]

This function can be found on the forums here.

If you have a loop in your script that increases the ram each time it loops. Put a call to this function in the loop to keep ram usage down.

Func _ReduceMemory($i_PID = -1)
    If $i_PID <> -1 Then
        Local $ai_Handle = DllCall("kernel32.dll", 'int', 'OpenProcess', 'int', 0x1f0fff, 'int', False, 'int', $i_PID)
        $ai_Return = DllCall("psapi.dll", 'int', 'EmptyWorkingSet', 'long', $ai_Handle[0])
        DllCall('kernel32.dll', 'int', 'CloseHandle', 'int', $ai_Handle[0])
    Else
        $ai_Return = DllCall("psapi.dll", 'int', 'EmptyWorkingSet', 'long', -1)
    EndIf

    Return $ai_Return[0]
EndFunc   ;==>_ReduceMemory

[xdp22]

Yeah i forgot to add i using that function, it give me some more time script working, but error come :/

[guinness]

Probably best to post an Example which can replicate the error. I was going to suggest the 'EmptyWorkingSet' too!

[kaotkbliss]

I think that function should be in the FAQ

[xdp22]

So, what i need to do? still error, i think it's because it works on 10 000 lines of text, and all lines is on memory, i need to made, to script forgot other lines, remember only this line what using now, how to do that :/

Edited March 18, 2011 by xdp22

[guinness]

I'm a little skeptical because I just did a test of my own creating about 90,000 lines and I was no where near running out of memory.

[MHz]

You may need to show some code. This could be a excessive 10000 plus handles open or a excessive 10000 plus script execution problem or anything else possible, eliminating the impossible of course.

10000 lines is not anywhere near 4GB unless you have enormous sized lines.

[xdp22]

How i should configure my Virtual memory? 4gb ram

[MHz]

How i should configure my Virtual memory? 4gb ram

The error is possibly flagged about memory address being exceeded. AFAIK, virtual memory operates within your memory address range. If you have 1 GB ram for a 32bit system then you could have 3GB virtual memory allocated to make 4GB of ram (1GB physical + 3GB virtual). I am not a memory allocation expert of Windows as it is closed source so take it on crazy faith.

[AdmiralAlkex]

How i should configure my Virtual memory? 4gb ram

The 2GB you have per default on x86 Windows should be enough. Now why don't you post a reproducer as guinness said and then maybe someone can suggest another way to do it that doesn't use so much memory.

[kaotkbliss]

It could be as MHz suggested and every line opens a new handle and the handles are never closed creating thousands of open handles, which would be a code problem and not a memory problem.

[xdp22]

Here is my code, so if someone can help, thanks

expandcollapse popup

#include <array.au3>
#Include <File.au3>
#include <string.au3>

Global $file1 = "excel_tables.txt"
Global $file2 = "result.txt"

Func work()
Dim $array

_FileReadToArray ($file1, $array)
_arraydelete ($array , 0)
$string = _ArrayToString ($array, ":")
$Barray = stringsplit ($string , ":" , 1)
_arraydelete ($Barray , 0)
$i = 0

                $1 = $Barray[$i]
                $2 = $Barray[$i + 1]
                
If StringIsDigit( $1) Then
    If Not StringInStr( $file2, $1) Then
        FileWriteLine( $file2, $1 & ":" & $2)
    Else
        delete()
    EndIf
Else
    delete()
EndIf
EndFunc ;===> work



func delete()
$filee = FileReadLine($file1, 1)
Dim $aRecords
_FileReadToArray($file1,$aRecords)
For $x = 1 to $aRecords[0]
If stringinstr($aRecords[$x], $filee) then _FileWriteToLine($file1, $x, "", 1)
FileClose( $file1)
FileClose( $file2)
_ReduceMemory()
work()
Next
EndFunc ;===> delete


Func _ReduceMemory($i_PID = -1)
    If $i_PID <> -1 Then
        Local $ai_Handle = DllCall("kernel32.dll", 'int', 'OpenProcess', 'int', 0x1f0fff, 'int', False, 'int', $i_PID)
        Local $ai_Return = DllCall("psapi.dll", 'int', 'EmptyWorkingSet', 'long', $ai_Handle[0])
        DllCall('kernel32.dll', 'int', 'CloseHandle', 'int', $ai_Handle[0])
    Else
        Local $ai_Return = DllCall("psapi.dll", 'int', 'EmptyWorkingSet', 'long', -1)
    EndIf

    Return $ai_Return[0]
EndFunc;==> _ReduceMemory()

[guinness]

Could you provide Example text files too, because this code could be the problem. For example you are using FileClose() without using FileOpen(), plus I believe the UDF Functions deal with opening and closing of the files. Another point to add it is I have read somewhere that only Local and Global should be used nowadays instead of Dim.

[Bowmore]

Your code as posted does not do anything. None of the functions are ever called, I can see lots of problems with the functions Work() and Delete() as they recursively call each other whilst in the middle of processing arrays the create, The work() function is also doing a lot of unnecessary work creating arrays, converting to a string then back to array creating new copies of the data each time. This will soon use up any amount of available memory. I think that they way you are trying do it, whatever it is, is probably wrong and this is what is causing you to run out of memory. I would recommend that you stop trying to fix the bugs in your current code and start again with a new design.

To help us help you, you need to clear state what you want to do with the file you are processing. Provide a small sample file of the data to be processed and another file containing the data as you want it to be after processing. A few lines should be enough because the size of the file is not the problem.

[MHz]

I am going to guess that work() is supposed to be called first though you seem to miss it in your script.

work() calls delete() and delete() calls work(). The memory usage IMO is caused from function recursion which is creating 1 or 2 arrays with each recurring call so you probably get up to 50000 arrays by the time you get the low memory error message.

I suggest that you recreate the script and avoid function recursion by ensuring that you return from a function call each time.