Smaller x64 exe scripts

[BullGates]

I was looking for a way of getting smaller compiled scripts for x64, this seems to work fine and I get exe's around 340Kb:

1. Download the latest mpress from here

2. Compress C:\Program Files (x86)\AutoIt3\Aut2Exe\AutoItSC_x64.bin using mpress with switch /s

3. Compile the script as normal...

If I use mpress like upx after the conversion, it wont execute the script.

Have fun.

[UEZ]

Thanks for the hint!

There is also an alternative packer: LARP64Free

Br,

UEZ

Edited November 16, 2010 by UEZ

[trancexx]

mpress is one of the most advanced compressors out there. LARP64 can't be measured with it.

[KaFu]

That's what I thought seeing it being used for some AutoItObject stuff ... Downloaded already but not tested yet .

[Mobius]

If I use mpress like upx after the conversion, it wont execute the script.

Have fun.

That is because mpress does not support overlays (A3x asbo), that is why you are having to compress the bin first.

a more descriptive suggestion of your workaround can be found Here

Edited November 17, 2010 by Mobius

[BullGates]

I don't really see the point of using it if it can't replace upx, but I only compile x86 scrips.

AutoItSC.bin compressed from 631 KB to 271 KB (43%), but that only reduced the size of the compiled script by 0.01 MB.

We are talking about x64 so if you want your x64 scripts reduced you should compress AutoItSC_x64.bin not AutoItSC.bin.

[Mobius]

Nice chart. It seems obvious to me that upx is the best compressor available for AutoIt. This explains why I don't see the point of using mpress - there isn't one.

Upx does not support 64bit binaries is one point, oh and even if you don't use it for x64 bins, it is still a very nice alternative to upx, (although it isn't much harder than upx to reverse, its compression is equal or above in some cases.)

[JFX]

Nice chart. It seems obvious to me that upx is the best compressor available for AutoIt. This explains why I don't see the point of using mpress - there isn't one.

Think you missed something. UPX is the best compressor, maybe.

But does it support x64 executables?

Edit: ahh, Mobius was faster

Edited November 17, 2010 by JFX

[BullGates]

Okay, I didn't know that. That changes things a bit and now this topic makes more sense. In regards to x64, I now see how mpress would be a useful alternative until upx is updated. As I don't use or support x64, this isn't an issue that I will have to deal with any time soon. By then I'm sure this issue will have been resolved, good luck.

This was the point of my post. We have x64 for some time now and still no compressor is available. x64 is widelly used now with Windows 7 and still upx does not support x64 executables.

[wraithdu]

Nice find! Bonus: you can pre-pack the x86 AutoItSC.bin file and forget having to use UPX ever again. It actually compresses the bin smaller than UPX, and all the resource update functions from AutoIt3Wrapper still work (color me happy since I wrote them!).

With UPX, trying to pre-pack the bin file would cause the compiled script to crash. But with mpress it works just fine. I'm a convert!

Edited November 17, 2010 by wraithdu

[Ascend4nt]

Very nice. I've already recompiled all my x64 scripts, and as wraithdu points out - it works great for x86 scripts also. Sweet. Maybe the 3.3.6.2 release (eta: 2020?) will convert to this instead?

[Zedna]

With UPX, trying to pre-pack the bin file would cause the compiled script to crash. But with mpress it works just fine. I'm a convert!

VERY GOOD FIND!!

Maybe this is worth of posting in BugTrack as request - precompile bin files in standard instalation with mpress.

[JohnOne]

Wow, thats pretty good.

I've heard the reasoning before about 600KB odd being reletively tiny anyway, but if you have 20 tools in your ebag, then thats over 10MB saved.

Im impressed and have now redone a few of my x32 scripts.

[wraithdu]

Here's a script using my functions from AutoIt3Wrapper to compress a compiled AutoIt script with MPRESS (_Main() function is at the bottom). It gets the extra script data, compresses the stub, then appends the saved data. Compile this script as a CUI and drop any compiled script file on it. FYI, MPRESS will try to compress a UPX packed script, but the result won't run. Just decompress the UPX packed script first. Sometimes MPRESS will hang, I'm not sure why. Just close the script (and MPRESS) by pressing Ctrl+C in the console window.

expandcollapse popup

#NoTrayIcon
#AutoIt3Wrapper_Res_requestedExecutionLevel=asInvoker
#AutoIt3Wrapper_Au3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6
#AutoIt3Wrapper_Run_After=del "%scriptdir%\%scriptfile%_Obfuscated.au3"
#AutoIt3Wrapper_Run_Obfuscator=y
#Obfuscator_Parameters=/sf /sv /om /cs=0 /cn=0
#AutoIt3Wrapper_Change2CUI=y
Opt("MustDeclareVars", 1)

#include <WinAPI.au3>

;
; RESOURCE CONSTANTS
;
Global Const $IMAGE_DOS_SIGNATURE = 0x5A4D
Global Const $tagIMAGE_SECTION_HEADER = _
        "byte Name[8];" & _
        "dword Misc;" & _
        "dword VirtualAddress;" & _
        "dword SizeOfRawData;" & _
        "dword PointerToRawData;" & _
        "dword PointerToRelocations;" & _
        "dword PointerToLinenumber;" & _
        "ushort NumberOfRelocations;" & _
        "ushort NumberOfLinenumbers;" & _
        "dword Characteristics"
Global Const $tagIMAGE_DOS_HEADER = _
        "ushort emagic;" & _
        "ushort ecblp;" & _
        "ushort ecp;" & _
        "ushort ecrlc;" & _
        "ushort ecparhdr;" & _
        "ushort eminalloc;" & _
        "ushort emaxalloc;" & _
        "ushort ess;" & _
        "ushort esp;" & _
        "ushort ecsum;" & _
        "ushort eip;" & _
        "ushort ecs;" & _
        "ushort elfarlc;" & _
        "ushort eovno;" & _
        "ushort eres[4];" & _
        "ushort eoemid;" & _
        "ushort eoeminfo;" & _
        "ushort eres2[10];" & _
        "long elfanew"
Global Const $tagIMAGE_FILE_HEADER = _
        "ushort Machine;" & _
        "ushort NumberOfSections;" & _
        "dword TimeDateStamp;" & _
        "dword PointerToSymbolTable;" & _
        "dword NumberOfSymbols;" & _
        "ushort SizeOfOptionalHeader;" & _
        "ushort Characteristics"
Global $tagTEMP = _
        "ushort Magic;" & _ ; Standard fields
        "byte MajorLinkerVersion;" & _
        "byte MinorLinkerVersion;" & _
        "dword SizeOfCode;" & _
        "dword SizeOfInitializedData;" & _
        "dword SizeOfUninitializedData;" & _
        "dword AddressOfEntryPoint;" & _
        "dword BaseOfCode;" & _
        "dword BaseOfData;" & _
        "dword ImageBase;" & _ ; NT additional fields
        "dword SectionAlignment;" & _
        "dword FileAlignment;" & _
        "ushort MajorOperatingSystemVersion;" & _
        "ushort MinorOperatingSystemVersion;" & _
        "ushort MajorImageVersion;" & _
        "ushort MinorImageVersion;" & _
        "ushort MajorSubsystemVersion;" & _
        "ushort MinorSubsystemVersion;" & _
        "dword Win32VersionValue;" & _
        "dword SizeOfImage;" & _
        "dword SizeOfHeaders;" & _
        "dword CheckSum;" & _
        "ushort Subsystem;" & _
        "ushort DllCharacteristics;" & _
        "dword SizeOfStackReserve;" & _
        "dword SizeOfStackCommit;" & _
        "dword SizeOfHeapReserve;" & _
        "dword SizeOfHeapCommit;" & _
        "dword LoaderFlags;" & _
        "dword NumberOfRvaAndSizes"
; assign indexes to each IMAGE_DATA_DIRECTORY struct so we can find them later
For $i = 0 To 15
    $tagTEMP &= ";ulong VirtualAddress" & $i & ";ulong Size" & $i
Next
Global Const $tagIMAGE_OPTIONAL_HEADER = $tagTEMP
$tagTEMP = 0
Global Const $tagIMAGE_NT_HEADERS = _
        "dword Signature;" & _
        $tagIMAGE_FILE_HEADER & ";" & _ ; offset = 4
        $tagIMAGE_OPTIONAL_HEADER ; offset = 24

Func _FIELD_OFFSET(ByRef $s, $element)
    If (Not IsDllStruct($s)) Or (Not IsString($element)) Then Return SetError(1, 0, 0)
    Local $s_ptr = DllStructGetPtr($s, 1) ; ptr to first byte of struct
    If @error Then Return SetError(2, 0, 0)
    Local $f_ptr = DllStructGetPtr($s, $element) ; ptr to first byte of element
    If @error Then Return SetError(3, 0, 0)
    Return Number($f_ptr - $s_ptr) ; offset of element in struct
EndFunc   ;==>_FIELD_OFFSET

Func _getFileExtraData($fname, ByRef $extraData, ByRef $dwSize)
    ; get target file as binary struct
    Local $hFile = FileOpen($fname, 16)
    If $hFile = -1 Then Return SetError(1, 0, 0)
    Local $bin = FileRead($hFile)
    FileClose($hFile)
    Local $dataSize = BinaryLen($bin)
    Local $sdata = DllStructCreate("byte[" & $dataSize & "]")
    Local $data = DllStructGetPtr($sdata) ; ptr to data
    DllStructSetData($sdata, 1, $bin)

    ; read the DOS header to make sure we have a valid file
    Local $dos_header = DllStructCreate($tagIMAGE_DOS_HEADER, $data)
    If DllStructGetData($dos_header, "emagic") <> $IMAGE_DOS_SIGNATURE Then
        ConsoleWrite("Not a valid executable file." & @CRLF)
        Return SetError(2, 0, 0)
    EndIf

    ; parse sections to calculate real data size
    Local $endOfImage = 0
    Local $psec = _IMAGE_FIRST_SECTION(DllStructGetPtr($dos_header) + DllStructGetData($dos_header, "elfanew")) ; ptr to first section
    Local $section = DllStructCreate($tagIMAGE_SECTION_HEADER, $psec) ; SECTION header
    Local $headers = DllStructCreate($tagIMAGE_NT_HEADERS, DllStructGetPtr($dos_header) + DllStructGetData($dos_header, "elfanew")) ; NT header
    Local $pRawData, $sizeRawData

    ; loop through sections to get the end of image
    For $i = 0 To DllStructGetData($headers, "NumberOfSections") - 1
        $pRawData = DllStructGetData($section, "PointerToRawData")
        $sizeRawData = DllStructGetData($section, "SizeOfRawData")
        If $endOfImage < ($pRawData + $sizeRawData) Then _
                $endOfImage = $pRawData + $sizeRawData
        ; get next section
        $psec += DllStructGetSize($section)
        $section = DllStructCreate($tagIMAGE_SECTION_HEADER, $psec)
    Next

    ; copy extra data
    If $dataSize > $endOfImage Then
        ; for some reason, SizeOfImage has a value that doesn't work for us, so we us the binary data length
        $dwSize = $dataSize - $endOfImage
        $extraData = DllStructCreate("byte[" & $dwSize & "]")
        ; ptr to end of image - extra data
        _memcpy(DllStructGetPtr($extraData), $data + $endOfImage, $dwSize)
    Else
        Return SetError(3, 0, 0)
    EndIf

    Return 1
EndFunc   ;==>_getFileExtraData

Func _appendFileExtraData($fname, ByRef $extraData, ByRef $dwSize)
    Local $bytesWritten = 0
    If Not IsDllStruct($extraData) Or $dwSize = 0 Then Return SetError(1, 0, 0)
    Local $pextraData = DllStructGetPtr($extraData)

    Local $hFile = _WinAPI_CreateFile($fname, 2, 4) ; open for writing
    If Not $hFile Then Return SetError(2, 0, 0)

    Local $pos = _WinAPI_SetFilePointer($hFile, 0, 2) ; set pointer to end of file
    If @error Or $pos = -1 Then
        _WinAPI_CloseHandle($hFile)
        Return SetError(3, 0, 0)
    EndIf

    Local $ret = _WinAPI_WriteFile($hFile, $pextraData, $dwSize, $bytesWritten, 0) ; write the extra data
    _WinAPI_CloseHandle($hFile)
    $extraData = 0 ; release extra data

    If Not $ret Or $bytesWritten <> $dwSize Then Return SetError(4, 0, 0)
    Return 1
EndFunc   ;==>_appendFileExtraData

;; #define IMAGE_FIRST_SECTION( ntheader ) ((PIMAGE_SECTION_HEADER)       \
;;    ((ULONG_PTR)ntheader +                                              \
;;     FIELD_OFFSET( IMAGE_NT_HEADERS, OptionalHeader ) +                 \
;;     ((PIMAGE_NT_HEADERS)(ntheader))->FileHeader.SizeOfOptionalHeader   \
;;    ))
Func _IMAGE_FIRST_SECTION($h)
    Local $header = DllStructCreate($tagIMAGE_NT_HEADERS, $h)
    ; _FIELD_OFFSET of OptionalHeader ('Magic' field) in IMAGE_NT_HEADERS stucture = 24
    Return ($h + _FIELD_OFFSET($header, "Magic") + DllStructGetData($header, "SizeOfOptionalHeader"))
EndFunc   ;==>_IMAGE_FIRST_SECTION

Func _memcpy($dest, $src, $size)
    Local $ret = DllCall("msvcrt.dll", "ptr:cdecl", "memcpy", "ptr", $dest, "ptr", $src, "uint", $size)
    Return $ret[0]
EndFunc   ;==>_memcpy

Func _Main($file)
    Local $data, $size = 0

    ConsoleWrite("Getting script data..." & @CRLF)
    _getFileExtraData($file, $data, $size)
    If Not @error Then
        ConsoleWrite("Script size: " & $size & @CRLF & @CRLF)
        Local $ret = RunWait('"mpress.exe" -s -b "' & $file & '"', "", @SW_HIDE)
        ConsoleWrite(@CRLF & "MPRESS return: " & $ret & @CRLF)
        If $ret = 0 Then
            ConsoleWrite("Appending script..." & @CRLF)
            _appendFileExtraData($file, $data, $size)
            If @error Then ConsoleWrite("Error: " & @error & @CRLF)
        EndIf
    Else
        ConsoleWrite("Error: " & @error & @CRLF)
    EndIf
    ConsoleWrite(@CRLF & "Press any key to exit...")
    DllCall("msvcrt.dll", "int:cdecl", "_getwch")
EndFunc

;; MAIN SCRIPT
If ($CmdLine[0] <> 0) And FileExists($CmdLine[1]) Then _Main($CmdLine[1])

Edited November 18, 2010 by wraithdu

[wraithdu]

Oh, and make sure mpress.exe is in your PATH (or edit the script to hard code the path).

[Jon]

Remember, if you have 20 compiled scripts and you are worried about space you can just compile them as .a3x and distribute autoit3.exe to run them. Or even just compile a single script to exe (leaving the rest as .a3x) and then use that compiled exe to run the rest.

[wraithdu]

I also wanted to point out that the two lines of code I quoted you on are redundant, as far as I know. The first line effectively enforces the "MustDeclareVars" option making it unnecessary to add it separately.

Not really. The AutoIt3Wrapper line is strictly for the pre-compile au3check parameters, while the Opt() function is for the actual AutoIt runtime. I've had instances where au3check doesn't catch everything.

[wraithdu]

I was trying to think of an example yesterday, but I couldn't remember. It's a very infrequent occurrence.

[wraithdu]

Here's an example

#AutoIt3Wrapper_Au3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

_Main()

Func _Main()
    If False Then
        Local $ret = "Error time..."
    EndIf
    MsgBox(0, "", $ret)
EndFunc

[wraithdu]

On a bigger note... don't pre-compress the bin files with MPRESS. I've already run into some bad juju and crashes when trying to run certain functions. It will have to be post-compressed with a script like mine from above. I'll come up with a simple example when I get some time. Consider this a WARNING!!!