Accumulating Memory Usage

[zbatev]

Hi All,

I've got an application that starts 8,000K memory usage according to the Task Manager but as time goes by it grows higher and higher.

The structure is like this(this is not the actual code):

I was monitoring the memory usage at random times and its like this:

9,856 K

10,220 K

11,216 K

11,220 K

12,156 K

12,984 K

12,988 K

13,860 K

...and this will continue to accumulate in time

While 1 ;this is because this application run 24 hours
 $valueA = False
  While $valueA = False
   for $x = 1 to 10
     if $monitorThis = $requiredValue then ;this portion will monitor the time 
       $valueA = True
       exitloop
     endif
   next
  Wend

  if $valueA = true then
    ;do some stuff
    sleep(840000) ;14 minutes
  endif
Wend

Edited July 28, 2011 by zbatev

[guinness]

Search for WinAPIEx.au3 by Yashied and the Function _WinAPI_EmptyWorkingSet() which "Removes as many pages as possible from the working set of the specified process."

OR

Call this Function in you .au3 to reduce the memory of the current process.

Func _ReduceMemory()
    Local $aReturn = DllCall("psapi.dll", "int", "EmptyWorkingSet", "long", -1)
    If @error = 1 Then
        Return SetError(1, 0, 0)
    EndIf
    Return $aReturn[0]
EndFunc   ;==>_ReduceMemory

Edited July 28, 2011 by guinness

[zbatev]

Wow, thanks a lot... Let me try that and I'll get back to you as soon as I'm done monitoring for improvements

[guinness]

Is it a program you've designed or another process e.g. Opera.exe?

[zbatev]

Is it a program you've designed or another process e.g. Opera.exe?

Actually it is an automatic screen capture program that I made. We dont want to buy commercial version and the free version does not meet the requirements.

My boss wants to get screen shots at these times in (mm:ss):

59:55, 00:00, 00:05

14:55, 15:00, 15:05

29:55, 30:00, 30:05

44:55, 45:00, 45:05

I'm done with that but the problem is as I've mentioned the memory size accumulating over time.

I'm currently testing your suggestions and see which one will work...

Edited July 28, 2011 by zbatev

[monoscout999]

Hi All,

I've got an application that starts 8,000K memory usage according to the Task Manager but as time goes by it grows higher and higher.

The structure is like this(this is not the actual code):

I was monitoring the memory usage at random times and its like this:

9,856 K

10,220 K

11,216 K

11,220 K

12,156 K

12,984 K

12,988 K

13,860 K

...and this will continue to accumulate in time

While 1 ;this is because this application run 24 hours
 $valueA = False
  While $valueA = False
   for $x = 1 to 10
     if $monitorThis = $requiredValue then ;this portion will monitor the time 
       $valueA = True
       exitloop
     endif
   next
  Wend

  if $valueA = true then
    ;do some stuff
    sleep(840000) ;14 minutes
  endif
Wend

For me, one problem is the bad design of the loop, maybe it stucks in the "While $valueA = False" loop that have no sleep() this affect the CPU usage, i never run a script for so long time as a entire day, but you can try another method of scripting because there is something, maybe in the rest of the script that is not properly managed.

My boss wants to get a screen shots at these times(in mm:ss):

59:55, 00:00, 00:05

14:55, 15:00, 15:05

29:55, 30:00, 30:05

44:55, 45:00, 45:05

Are these times taken for the real clock time or times from the script start to run? Edited July 28, 2011 by monoscout999

[guinness]

Another tip to add what monoscout999 said is look at AdlibRegister() if you want to call a Function every 14 minutes instead of Sleeping.

[zbatev]

Hi guinness,

I think it's working the memory usage has dramatically lowered to 800K.

I also noticed one thing, each time I show GUISetState(@SW_SHOW) a hidden form GUISetState(@SW_HIDE), the memory usage accumulates.

What i did now is applied the same concept as you suggested like:

GUISetState(@SW_HIDE)
_ReduceMemory()

But I'm guessing I'm using the wrong flags @SW_SHOW and @SW_HIDE? Because I was thinking more like the visual basic Load 'Form' Unload 'Form'.

[zbatev]

Another tip to add what monoscout999 said is look at AdlibRegister() if you want to call a Function every 14 minutes instead of Sleeping.

'AdlibRegister()' --- I'm looking into this right now.

[guinness]

A quick Example so you get an idea >>

Global $iCount = 0

AdlibRegister("_Function", 3000)
While 1
    Sleep(100)
    If $iCount = 5 Then ; Just for testing purposes, this will Exit after 5 MsgBox()'s have been displayed.
        AdlibUnRegister("_Function")
        Exit
    EndIf
WEnd

Func _Function()
    MsgBox(64, "MsgBox() - " & $iCount + 1, "This is showing because the Function is called every 3 seconds.")
    $iCount += 1
EndFunc   ;==>_Function

[zbatev]

Hi monoscout999,

Thank you for the criticism I appreciate it.

For me, one problem is the bad design of the loop, maybe it stucks in the "While $valueA = False" loop that have no sleep() this affect the CPU usage, i never run a script for so long time as a entire day, but you can try another method of scripting because there is something, maybe in the rest of the script that is not properly managed.

Are these times taken for the real clock time or times from the script start to run?

Yes I was hesitant to put Sleep() function since I might miss the time requirement, which is to the second:

59:55, 00:00, 00:05

14:55, 15:00, 15:05

29:55, 30:00, 30:05

44:55, 45:00, 45:05

And yes this are real clock time NOT from when the script start to run.

Do you have any other suggestion where I can hit those specified time without missing one? The only solution I could think of is not pushing a Sleep function even for a fraction of the time, and it is actually working(sacrificing PC workload I think).

Edited July 28, 2011 by zbatev

[zbatev]

Hi guinness & monoscout999,

I think I've manage to eliminate the weakness of my design by using the _ReduceMemory() and AdlibRegister(), I've reconstructed my design into this form:

CaptureScreen()

Func CaptureScreen()
While 1
 $valueA = False
  While $valueA = False
   for $x = 1 to 10
     if $monitorThis = $requiredValue then ;this portion will monitor the time 
       $valueA = True
       exitloop
     endif
   next
  Wend

  if $valueA = true then
    ;do some stuff
    _ReduceMemory()
    AdlibRegister("CaptureScreen",840000) ;calls itself after 14 minutes
  endif
Wend
EndFunc

I still have an infinite loop without a Sleep() or similar function BUT that infinite loop will only run for a maximum of 1 minute every 15th minute, the _ReduceMemory() will eliminate unecessary/unused resources and AdlibRegister tooks care of the PC workload.

Thank you very much for scouting problems like this and sharing your genius!

Edited July 28, 2011 by zbatev

[monoscout999]

i think the weaknes is still there but if it work well, then it is ok

[guinness]

Those loops do look a little excessive.

[JohnOne]

I doubt the loops are causing the problem, and the reduce memory workaround is really a cheap imitation of a fix.

I suspect the problem lies elsewhere in your code, probably your screencapture setup.

[smartee]

I agree with JohnOne I keep imagining some bitmap and graphics objects left undestroyed, I think the puppet is still under the table. Maybe you want to show us the puppet the rest of your code ie. ?

[monoscout999]

I agree with JohnOne I keep imagining some bitmap and graphics objects left undestroyed, I think the puppet is still under the table. Maybe you want to show us the puppet the rest of your code ie. ?

I bet for this.. the Bitmap object created by the screenshot is not being deleted.

[zbatev]

Thanks guys actually everything is working fine right now but I will still do some more math so I will eliminate 1 or 2 loops... I'll keep this thread updated when I've got something new....

And yes I still think the puppet is still under the table but for now they have to settle with it while I'm working on improving the application.

There are some things that need to be fix like the AdlibRegister is inside a loop, I think I have to let it out. From the looks of it, its really messy I've got some cleaning to do.

Edited July 29, 2011 by zbatev

[ChrisL]

Actually it is an automatic screen capture program that I made. We dont want to buy commercial version and the free version does not meet the requirements.

My boss wants to get screen shots at these times in (mm:ss):

59:55, 00:00, 00:05

14:55, 15:00, 15:05

29:55, 30:00, 30:05

44:55, 45:00, 45:05

I'm done with that but the problem is as I've mentioned the memory size accumulating over time.

I'm currently testing your suggestions and see which one will work...

Empty working set pushes the memory on to the pagefile, if you select show pagefile in task manager you'll probably notice this getting bigger rather than the physical memory, any memory items required then have to be pulled back out from the pagefile when needed, seeing as you say its a screen grabber then there will be a GDI or GDI+ object that your not disposing of which is taking up the memory, I have some GDI+ scripts that create hundreds of objects and destroy them over a period of time and the memory is totally stable if you dispose of everything propperly

[masvil]

The _WinAPI_EmptyWorkingSet() method works good here. Is there a similar way to empty/reduce the virtual memory size as well?