CleanScript

[peethebee]

[English]

Hi!

To all who love tiny small scripts and exe files:

In cooperation with Markus I wrote my first real C++ script. It parses AutoIt scripts independently from the compiler so it is a kind of precompiler.

At the beginning it includes the #included files into the script.

Afterwards it can delete comments (all styles) and - which is a very powerful function - unused (uncalled) functions.

The last one is the really new thing - besides of the removement of the comments using C++ is incredibly much faster than with (AutoIt)Tidy.exe.

It is useful because you never use all functions of your include file, but just some few.

It goes through the source three-times to uncover dependencies between functions.

Now CleanScript erases all others from the script and therefore from the resulting exe which means both are smaller by a remarkable amount of lines/bytes.

At the end all empty lines are removed.

An example with Galenda (http://www.autoitscript.com/forum/index.php?showtopic=28336&hl=galenda):

Script size before (with includes): 575 KB

After using CleanScript: 217 KB - with the same functionality...

Compiled Script before: 310 KB

After using CleanScript: 235 KB (around 25% less)

What you have to "obey":

- #include may only contain one space in the line - between the E char and < or "

- If ; have another meaning than a comment (e.g. in MsgBoxes) you have to fix the Dev-C++ 4.9.9.2 source

- Call and Execute can make the program delete funcs that are called that way. A workaround is to call the Funcs anywhere once.

Please test it and give me your input on it. Perhaps a C++ geek could help me to extend it and improve effeciency even more.

Than it perhaps could be part of the official AutoIt compiler one time perhaps.

I think of integrating a scan for unused Const and Vars in the future (espacially for GUIConstants.au3).

Another intersting idea is to put all the UDFs together and just #include <all.au3>. This leads in to a long time for scanning your script in CleanScript, but you never have to worry again if you included all that's necessary.

You can grab it: here:http://www.autoitscript.com/forum/index.php?s=&showtopic=30573&view=findpost&p=240018

or in the attachment.

peethebee

[/English]

[German]

Hallo!

An alle, die gerne kleine Skripte und vor allem kleine Exe-Dateien haben, an die 56k-Modem-Nutzer und 20-GB-Festplatten-Besitzer (Ich liebe euch alle )

Zusammen mit Markus habe ich ein Programm in C++ geschrieben, das AutoIt-Skript vor dem Kompilieren selbst parst.

Dabei werden zunächst die Includes eingebaut. Dann werden je nach Einstellung in der zugehörigen AutoIt-GUI (GUIS in C++ sind einfach nur pain) Kommentare und/oder ungenutzte Funktionen entfernt. Das Entfernen von Kommentaren beherrscht auch Tidy.exe aus dem SciTE-Lieferumfang. Leider wird das aber in AutoIt gemacht, sodass die Performance bescheiden ist. "CleanScript" erledigt die Kommentarentfernung bei Galenda in weniger als einer halben Sekunde, wobei die meiste Zeit durch Ausgaben auf der Konsole verloren geht. Tidy braucht (zugegeben inklusive Einrücken) 45 Sekunden.

Das Highlight in unserem Programm ist aber zweifellos die Möglichkeit, ungenutzte Funktionen entfernen zu können. Dazu wird der gesamte Quelltext inklusive Include-Dateien auf Funktionen und deren Aufrufe hin untersucht. Funktionen, die definiert, aber nie aufgerufen werden (i.d.R. die große Mehrheit derer, die aus den Includes kommen), werden entfernt.

Zu guter Letzt werden auch die leeren Zeilen noch entfernt.

Mal ein Beispiel anhand der aktuellen Galenda-Version:

Ursprungsgröße Skript (mit Includes): 575 KB

Nach Behandlung mit CleanScript: 217 KB - bei gleicher Funktionalität...

Ursprungsgröße kompiliertes Skript: 310 KB

Nach Behandlung mit CleanScript: 235 KB (knapp 25% Einsparung)

Das kann sich schon sehen lassen, denke ich.

Was man beachten muss:

- Bei #includes, darf nur ein Leerzeichen in der Zeile stehen und zwar zwischen "#include" und < oder ".

- Bei Strichpunkten, die keine Kommentare anzeigen (z.B. in MsgBoxen oder Strings) kann es zu unerwünschten Resultaten führen, wenn man die Kommentare entfernen lässt. Dazu müsste man den Dev-C++ 4.9.9.2-Quelltext leicht anpassen.

- Call und Execute können bei der Funktionsenternung Probleme machen, da Aufrufe der Funktionen dann nicht erkannt werden. Abhilfe kann ein Aufruf an beliebiger anderer Stelle schaffen.

Bitte ausführlich testen und vielleicht auch die ein oder andere Einsparungszahl posten...

In meinem deutschen Forum kann man es downloaden (kurz: http://www.autoit.de): http://www.autoitscript.com/forum/index.ph...st&p=240018

peethebee & Markus

[/German]

Edited September 22, 2006 by peethebee

[peethebee]

Hi againg!

Just 57 views and no single answer is not much...

Did anyone at least test it?

I think it is already very useful and it will be still improved.

peethebee

[Jon]

Sounds pretty cool. Not had a chance to test yet though.

[peethebee]

The performance is not that good (in C++ thinking, it is veeery fast, compared to AutoIt), because we coded it inefficiently as it was our first script in C++, but we try to improve it.

Moreover Markus is atm including a feature to strip unused Consts and vars.

peethebee

Edited August 10, 2006 by peethebee

[JSThePatriot]

I think this has the potential to be a great utility. I have already linked a couple of people to it. I havent tested it myself yet, as I have never been concerned about that.

I hope to be able to test it and look at the code. I have already downloaded the source.

JS

[peethebee]

We would like to have some C++ geeks to have a look at it to improve the speed which is definately possible.

We moreover hope too that it will be great ;-)

peethebee

[Kickassjoe]

i might try it tomorrow if i get around to it, and can find ur post again

[markloman]

I've tested it and it crashed. I don't think it had done anything. I must say I have over 55 own includes and use almost every AutoIt include in my program.

[peethebee]

I tested it with a 4000 lines script my mine (Galenda) and it worked perfectly.

Did you use the AutoIt-GUI to give the right parameters?

peethebee

[Cyberworld]

I tested your program. But it does not save any optimized version on my script. It just show some info in German in the console window.

"Quelldatei: C:\DOCUME~1\ADMINI~1\Desktop\Testprogram.au3

Zieldatei: C:\Documents

Aufgabentyp: CF-

Datei in Array einlesen

Datei in Array eingelesen

Includes einbauen

Includes eingebaut

Datei schreiben ohne Optimierungen

Datei geschrieben.

Entferne Kommentare.

Kommentare entfernt.

Durchgang: 1

Funktionsaufrufe untersuchen...

Funktionsaufrufe aufgelistet.

Durchgang: 2

Funktionsaufrufe untersuchen...

Funktionsaufrufe aufgelistet.

Durchgang: 3

Funktionsaufrufe untersuchen...

Funktionsaufrufe aufgelistet.

Leere Zeilen entfernen.

Leere Zeilen entfernt.

Ergebnis in Datei schreiben.

Ergebnis in Datei geschrieben.

Dateigroesse nicht optimiert: 13130 Bytes

Dateigroesse optimiert: 7786 Bytes

Einsparung: 40.7007%

Fertig.

Press any key to continue . . .

"

I did test both "CleanScript GUI.exe" & CleanScript GUI.au3.

Selected Options:

"Remove comments"

"Remove unused functions"

"Use beta includes"

(Tested on Windows XP SP2.)

I also tried without the Beta includes option but that didn't make any difference. The finished file can't be found anywhere.

UPDATE: I found the bug. If the destination path does include a "space" then the file will not be written!

Edited August 18, 2006 by Cyberworld

[Cyberworld]

..But the compiled program is still the same size. Does it matter if Autoit is installed in a different path than the default? I think your program did not find the includes because the "# <include>" lines are still in my source...

[Markus]

UPDATE: I found the bug. If the destination path does include a "space" then the file will not be written!

Ah right, thank you, our mistake:

Because of possible "space", we make "FileGetShortName" for the source path. If there is no dest path, we just cut 4 letters from that string and add _clean.au3 --> bug. I'll fix that.

..But the compiled program is still the same size. Does it matter if Autoit is installed in a different path than the default?

No, it shouldn't matter, as we search the install dir in the registry.

I think your program did not find the includes because the "# <include>" lines are still in my source...

That's possible.

So far we can't find all #include lines in your script. We only find #include if it's written at the start of the line with no space and we only find

#Include

and

#include

as C++ is very strict with uppercase and lowercase written strings.

But i'll change that, that we can also find

#iNclUdE

in your script, not that hard.

But there shouldn't be a space between start of line or anywhere else in the line, except between #include and < or "

[Markus]

New Version:

- "space"-bug fixed

- CleanScript also finds "#iNclUdE" now

- you can decide between full and less output in the gui and give the number of the runs for "delete unused functions"

- CleanScript detects all var-declarations now, and deletes them, if unused (very effective for GUI-Constants for example)

Markus

Edited September 22, 2006 by Markus

[Dickb]

New Version:

- "space"-bug fixed

- CleanScript also finds "#iNclUdE" now

- you can decide between full and less output in the gui and give the number of the runs for "delete unused functions"

- CleanScript detects all var-declarations now, and deletes them, if unused (very effective for GUI-Constants for example)

Markus

Hi,

When I use it on one of my scripts (the first one I tested) I get an error about a missing variable.

I use AutoIt v. 3.2.0.1 (Removed the beta already).

The problem is that when an included script includes another script for the constants, they are not added in the cleaned script.

To clearify what I mean I have created a small example. (see attachment)

Main.au3 = the original script, Main_cleaned.au3 is the script alter cleanup. (Obvious I guess)

The include file that is causing the problem in my tested script is GuiCombo.au3, which is including ComboConstants.au3

Hope this can be solved.

For the rest, I like this program. It is fast and usefull.

Thanks, Dick

Note: Sorry, confusing name for my attachment. It contains the DEMO scripts, not the cleanup program.

CleanScript.zip

Edited August 18, 2006 by Dickb

[Cyberworld]

But there shouldn't be a space between start of line or anywhere else in the line, except between #include and < or "

I see, That's why it doesn't work for me. I use "TAB"s before my #include.

Would it be difficult to make the program to strip the leading "Space"s and "TAB"s before it check the lines for #includes?

Edited August 19, 2006 by Cyberworld

[taurus905]

peethebee,

When I run your CleanScript GUI.au3 it does not produce an outputted script.

Am I missing something?

Thanks,

taurus905

[peethebee]

But there shouldn't be a space between start of line or anywhere else in the line, except between #include and < or "

I think Markus can fix this right when allowing all cases of incLuDE ;-)

@taurus905: Did you try it with the new version? Could you give the script's output here?

peethebee

Edited August 19, 2006 by peethebee

[Zedna]

@peethebee

Very nice idea!

I think new options could be added:

1) Remove leading spaces/tabs

2) Remove trailing spaces/tabs

3) Remove empty lines

1) and 2)

If I want achieve absolutely minimum size of output EXE no matter of readability decompiled script

this option will be usefull. Note that I have original sources for reading, so script_clean.au3 is only temporary file for compile and not for reading.

3)

- It can be optional (if it's now hardcoded)

- At the end of my cleaned script are still two empty lines (@CRLF@CRLF)

***

And I also think some sort of these things could be added to Aut2Exe as compiler options (for optimize size of EXE)

- for example as one new option with name:

Strip unused stuff from script or

Optimize script for minimum EXE size or

something similar ...

[peethebee]

Thanks for your cheering!

We are working on one big thing, which is calling the new C-style Tidy2 and the aut2exe right out of the cleanscript.au3. This would not require additions or changes to the stable aut2exe.

point 1) and 2) could be done by tidy before running cleanscript over it.

point 3) is now hardcoded as there would be many empty lines otherwise. The tow lines should be possible to take out by script. We'll change that.

peethebee

[Jos]

point 1) and 2) could be done by tidy before running cleanscript over it.

point 3) is now hardcoded as there would be many empty lines otherwise. The tow lines should be possible to take out by script. We'll change that.

Tidy can also remove empty lines...

*** Remove Empty lines from Script Source

Remove_Empty_Lines=0