Search the Community

PE File Overlay Extraction (and Certificate info) Executable and other PE files can contain "overlays", which is data that is appended to the end of the file. This data can be important, such as setup packages, Authenticode signatures*, and overlays for AutoIt scripts. Or it could just be extra unneeded cruft (sometimes). Whatever the case is, I wanted to find a way to detect if this data was present. This project is actually a result of dealing with so-called 'File Optimizer' programs that would strip Overlay information from Executables (leaving compiled AutoIt scripts crippled!). And also a legit >answer to my topic in Help and Support. While future versions of AutoIt (new beta releases and any official release after v3.3.8.1) are putting tokenized scripts into a resource within the executable, all current compiled scripts are still put together with the tokenized script appended as an overlay. The UDF here allows you to detect any overlay a PE (Portable Executable) file may have, and allows you to extract the Overlay into a separate file - or alternatively extract the exe without the overlay. You can actually extract AutoIt scripts and write them to .A3X files using this method, if you so desire. But don't be a hacker! Mommy will scold you.. If you separate both the exe and overlay, you can combine them again using a simple file-append, something like: copy /b stripped.exe+script.a3x myscript.exe _ Anyway, the method to detect overlays is relatively simple - we need to look through the PE file's various headers and find out where the last section of data/code is and its size. If that last section doesn't reach the end of the file, then you will find an Overlay waiting at the end of the final section. However, there's an issue with Certificate Tables (or signatures) which makes it a bit more tricky to detect - basically the end of the last section and the beginning of the Certificate must be examined to find the sandwiched-in overlay. *Auhenticode signatures note: These and other certificates are actually linked to in the PE Data Directory, which I had missed in earlier versions. Now they are accounted for however, and not considered overlays nor are they allowed to be extracted (well, you could extract them but the signature is bound to the unique checksum of the file and needs to be referenced from the Data Directory). IMPORTANT: The example now queries which part to save, and "No" button means 'yes' to Exe extract. (I didn't want to mess around with creating dialog windows, sorry). So, here's the UDF with a working example (note the 128MB limit can easily be worked around): ; ======================================================================================================== ; <FilePEOverlayExtract.au3> ; ; UDF and Example of getting Overlay info and optionally extracting that info to a file. ; ; NOTE that this isn't intended to be used to hack or decompile AutoIt executables!! ; It's main purpose is to find Overlays and Certificates and extract/save or just report the info ; ; Functions: ; _PEFileGetOverlayInfo() ; Returns a file offset for overlay data (if found), and the size ; ; Author: Ascend4nt ; ======================================================================================================== ; Arry indexing Global Enum $PEI_OVL_START = 0, $PEI_OVL_SIZE, $PEI_CERT_START, $PEI_CERT_SIZE, $PEI_FILE_SIZE ; ---------------------- MAIN CODE ------------------------------- Local $sFile, $sLastDir, $sLastFile, $aOverlayInfo $sLastDir = @ScriptDir While 1 $sFile=FileOpenDialog("Select PE File To Find Overlay Data In",$sLastDir,"PE Files (*.exe;*.dll;*.drv;*.scr;*.cpl;*.sys;*.ocx;*.tlb;*.olb)|All Files (*.*)",3,$sLastFile) If @error Or $sFile="" Then Exit $sLastFile=StringMid($sFile,StringInStr($sFile,'\',1,-1)+1) $sLastDir=StringLeft($sFile,StringInStr($sFile,'\',1,-1)-1) $aOverlayInfo = _PEFileGetOverlayInfo($sFile) If $aOverlayInfo[$PEI_OVL_START] = 0 Then ConsoleWrite("Failed Return from _PEGetOverlayOffset(), @error = " & @error & ", @extended = " & @extended & @CRLF) MsgBox(64, "No Overlay Found", "No overlay found in " & $sLastFile) ContinueLoop EndIf ConsoleWrite("Return from _PEFileGetOverlayInfo() = " & $aOverlayInfo[$PEI_OVL_START] & ", @extended = " & $aOverlayInfo[$PEI_OVL_SIZE] & @CRLF) If $aOverlayInfo[$PEI_OVL_START] Then Local $hFileIn = -1, $hFileOut = -1, $sOutFile, $iMsgBox, $bBuffer, $bSuccess = 0 $iMsgBox = MsgBox(35, "Overlay found in " & $sLastFile, "Overlay Found. File size: " & $aOverlayInfo[$PEI_FILE_SIZE] & ", Overlay size: " & $aOverlayInfo[$PEI_OVL_SIZE] & @CRLF & @CRLF & _ "Would you like to:" & @CRLF & _ "[Yes]: extract and save Overlay" & @CRLF & _ "[No]: extract Exe without Overlay" & @CRLF & _ "[Cancel]: Do Nothing") If $iMsgBox = 6 Then If $aOverlayInfo[$PEI_OVL_SIZE] > 134217728 Then MsgBox(48, "Overlay is too huge", "Overlay is > 128MB, skipping..") ContinueLoop EndIf $sOutFile = FileSaveDialog("Overlay - SAVE: Choose a file to write Overlay data to (from " & $sLastFile&")", $sLastDir, "All (*.*)", 2 + 16) If Not @error Then While 1 $hFileOut = FileOpen($sOutFile, 16 + 2) If $hFileOut = -1 Then ExitLoop $hFileIn = FileOpen($sFile, 16) If $hFileIn = -1 Then ExitLoop If Not FileSetPos($hFileIn, $aOverlayInfo[$PEI_OVL_START], 0) Then ExitLoop ; AutoIt 2/3 Signature check requires 32 bytes min. If $aOverlayInfo[$PEI_FILE_SIZE] > 32 Then $bBuffer = FileRead($hFileIn, 32) If @error Then ExitLoop ; AutoIt2 & AutoIt3 signatures If BinaryMid($bBuffer, 1, 16) = "0xA3484BBE986C4AA9994C530A86D6487D" Or _ BinaryMid($bBuffer, 1 + 16, 4) = "0x41553321" Then ; "AU3!" ConsoleWrite("AutoIt overlay file found" & @CRLF) EndIf FileWrite($hFileOut, $bBuffer) ; subtract amount we read in above $bSuccess = FileWrite($hFileOut, FileRead($hFileIn, $aOverlayInfo[$PEI_OVL_SIZE] - 32)) Else $bSuccess = FileWrite($hFileOut, FileRead($hFileIn, $aOverlayInfo[$PEI_OVL_SIZE])) EndIf ExitLoop WEnd If $hFileOut <> -1 Then FileClose($hFileOut) If $hFileIn <> -1 Then FileClose($hFileIn) EndIf ElseIf $iMsgBox = 7 Then If $aOverlayInfo[$PEI_FILE_SIZE] - $aOverlayInfo[$PEI_OVL_SIZE] > 134217728 Then MsgBox(48, "EXE is too huge", "EXE (minus overlay) is > 128MB, skipping..") ContinueLoop EndIf $sOutFile = FileSaveDialog("EXE {STRIPPED} - SAVE: Choose a file to write EXE (minus Overlay) to. (from " & $sLastFile&")", $sLastDir, "All (*.*)", 2 + 16) If Not @error Then $bSuccess = FileWrite($sOutFile, FileRead($sFile, $aOverlayInfo[$PEI_OVL_START])) EndIf Else ContinueLoop EndIf If $bSuccess Then ShellExecute(StringLeft($sOutFile,StringInStr($sOutFile,'\',1,-1)-1)) Else MsgBox(64, "Error Opening or writing to file", "Error opening, reading or writing overlay info") EndIf EndIf WEnd Exit ; ------------------------ UDF Function ---------------------------- ; =================================================================================================================== ; Func _PEFileGetOverlayInfo($sPEFile) ; ; Returns information on Overlays present in a Windows PE file (.EXE, .DLL etc files), as well as Certificate Info. ; ; Only certain executables contain Overlays, and these are always located after the last PE Section, ; and most times before any Certificate info. Setup/install programs typically package their data in Overlays, ; and AutoIt compiled executables (at least up to v3.3.8.1) contain an overlay in .A3X tokenized format. ; ; Certificate info is available with or without an overlay, and comes after the last section and typically after ; an Overlay. Certificates are included with signed executables (such as Authenticode-signed) ; ; The returned info can be used to examine or extract the Overlay or Certificate, or just to examine the data ; (for example, to see if its an AutoIt tokenized script). ; ; NOTE: Any Overlays packaged into Certificate blocks are ignored, and the methods to extract this info may ; fail if the Certificate Table entries have their sizes modified to include the embedded Overlay. ; ; The returned information can be useful in preventing executable 'optimizers' from stripping the Overlay info, ; which was the primary intent in creating this UDF. ; ; ; Returns: ; Success: A 5-element array, @error = 0 ; [0] = Overlay Start (if any) ; [1] = Overlay Size ; [2] = Certificate Start (if any) ; [3] = Certificate Size ; [4] = File Size ; ; Failure: Same 5-element array as above (with all 0's), and @error set: ; @error = -1 = Could not open file ; @error = -2 = FileRead error (most likely an invalid PE file). @extended = FileRead() @error ; @error = -3 = FileSetPos error (most likely an invalid PE file) ; @error = 1 = File does not exist ; @error = 2 = 'MZ' signature could not be found (not a PE file) ; @error = 3 = 'PE' signature could not be found (not a PE file) ; @error = 4 = 'Magic' number not recognized (not PE32, PE32+, could be 'ROM (0x107), or unk.) @extended=number ; ; Author: Ascend4nt ; =================================================================================================================== Func _PEFileGetOverlayInfo($sPEFile) ;~ If Not FileExists($sPEFile) Then Return SetError(1,0,0) Local $hFile, $nFileSize, $bBuffer, $iOffset, $iErr, $iExit, $aRet[5] = [0, 0, 0, 0] Local $nTemp, $nSections, $nDataDirectories, $nLastSectionOffset, $nLastSectionSz Local $iSucces=0, $iCertificateAddress = 0, $nCertificateSz = 0, $stEndian = DllStructCreate("int") $nFileSize = FileGetSize($sPEFile) $hFile = FileOpen($sPEFile, 16) If $hFile = -1 Then Return SetError(-1,0,$aRet) ; A once-only loop helps where "goto's" would be helpful Do ; We keep different exit codes for different operations in case of failure (easier to track down what failed) ; The function can be altered to remove these assignments of course $iExit = -2 $bBuffer = FileRead($hFile, 2) If @error Then ExitLoop $iExit = 2 ;~ 'MZ' in hex (endian-swapped): If $bBuffer <> 0x5A4D Then ExitLoop ;ConsoleWrite("MZ Signature found:"&BinaryToString($bBuffer)&@CRLF) $iExit = -3 ;~ Move to Windows PE Signature Offset location If Not FileSetPos($hFile, 0x3C, 0) Then ExitLoop $iExit = -2 $bBuffer = FileRead($hFile, 4) If @error Then ExitLoop $iOffset = Number($bBuffer) ; Though the data is in little-endian, because its a binary variant, the conversion works ;ConsoleWrite("Offset to Windows PE Header="&$iOffset&@CRLF) $iExit = -3 ;~ Move to Windows PE Header Offset If Not FileSetPos($hFile, $iOffset, 0) Then ExitLoop $iExit = -2 ;~ Read in IMAGE_FILE_HEADER + Magic Number $bBuffer = FileRead($hFile, 26) If @error Then ExitLoop $iExit = 3 ; "PE/0/0" in hex (endian swapped) If BinaryMid($bBuffer, 1, 4) <> 0x00004550 Then ExitLoop ; Get NumberOfSections (need to use endian conversion) DllStructSetData($stEndian, 1, BinaryMid($bBuffer, 6 + 1, 2)) $nSections = DllStructGetData($stEndian, 1) ; Sanity check If $nSections * 40 > $nFileSize Then ExitLoop ;~ ConsoleWrite("# of Sections: " & $nSections & @CRLF) $bBuffer = BinaryMid($bBuffer, 24 + 1, 2) ; Magic Number check (0x10B = PE32, 0x107 = ROM image, 0x20B = PE32+ (x64) If $bBuffer = 0x10B Then ; Adjust offset to where "NumberOfRvaAndSizes" is on PE32 (offset from IMAGE_FILE_HEADER) $iOffset += 116 ElseIf $bBuffer = 0x20B Then ; Adjust offset to where "NumberOfRvaAndSizes" is on PE32+ (offset from IMAGE_FILE_HEADER) $iOffset += 132 Else $iExit = 4 SetError(Number($bBuffer)) ; Set the error (picked up below and set in @extended) to the unrecognized Number found ExitLoop EndIf ;~ 'Optional' Header Windows-Specific fields $iExit = -3 ;~ -> Move to "NumberOfRvaAndSizes" at the end of IMAGE_OPTIONAL_HEADER If Not FileSetPos($hFile, $iOffset, 0) Then ExitLoop $iExit = -2 ;~ Read in NumberOfRvaAndSizes $nDataDirectories = Number(FileRead($hFile, 4)) ; Sanity and error check If $nDataDirectories <= 0 Or $nDataDirectories > 16 Then ExitLoop ;~ ConsoleWrite("# of IMAGE_DATA_DIRECTORY's: " & $nDataDirectories & @CRLF) ;~ Read in IMAGE_DATA_DIRECTORY's (also moves file position to IMAGE_SECTION_HEADER) $bBuffer = FileRead($hFile, $nDataDirectories * 8) If @error Then ExitLoop ;~ IMAGE_DIRECTORY_ENTRY_SECURITY entry is special - it's "VirtualAddress" is actually a file offset If $nDataDirectories >= 5 Then DllStructSetData($stEndian, 1, BinaryMid($bBuffer, 4 * 8 + 1, 4)) $iCertificateAddress = DllStructGetData($stEndian, 1) DllStructSetData($stEndian, 1, BinaryMid($bBuffer, 4 * 8 + 4 + 1, 4)) $nCertificateSz = DllStructGetData($stEndian, 1) If $iCertificateAddress Then ConsoleWrite("Certificate Table address found, offset = " & $iCertificateAddress & ", size = " & $nCertificateSz & @CRLF) EndIf ; Read in ALL sections $bBuffer = FileRead($hFile, $nSections * 40) If @error Then ExitLoop ;~ DONE Reading File info.. ; Now to traverse the sections.. ; $iOffset Now refers to the location within the binary data $iOffset = 1 $nLastSectionOffset = 0 $nLastSectionSz = 0 For $i = 1 To $nSections ; Within IMAGE_SECTION_HEADER: RawDataPtr = offset 20, SizeOfRawData = offset 16 DllStructSetData($stEndian, 1, BinaryMid($bBuffer, $iOffset + 20, 4)) $nTemp = DllStructGetData($stEndian, 1) ;ConsoleWrite("RawDataPtr, iteration #"&$i&" = " & $nTemp & @CRLF) ; Is it further than last section offset? ; AND - check here for rare situation where section Offset may be outside Filesize bounds If $nTemp > $nLastSectionOffset And $nTemp < $nFileSize Then $nLastSectionOffset = $nTemp DllStructSetData($stEndian, 1, BinaryMid($bBuffer, $iOffset + 16, 4)) $nLastSectionSz = DllStructGetData($stEndian, 1) EndIf ; Next IMAGE_SECTION_HEADER $iOffset += 40 Next ;~ ConsoleWrite("$nLastSectionOffset = " & $nLastSectionOffset & ", $nLastSectionSz = " & $nLastSectionSz & @CRLF) $iSucces = 1 ; Everything was read in correctly Until 1 $iErr = @error FileClose($hFile) ; No Success? If Not $iSucces Then Return SetError($iExit, $iErr, $aRet) ;~ Now to calculate the last section offset and size to get the 'real' Executable end-of-file ; [0] = Overlay Start $aRet[0] = $nLastSectionOffset + $nLastSectionSz ; Less than FileSize means there's Overlay info If $aRet[0] And $aRet[0] < $nFileSize Then ; Certificate start after last section? It should If $iCertificateAddress >= $aRet[0] Then ; Get size of overlay IF Certificate doesn't start right after last section ; 'squeezed-in overlay' $aRet[1] = $iCertificateAddress - $aRet[0] Else ; No certificate, or < last section - overlay will be end of last section -> end of file $aRet[1] = $nFileSize - $aRet[0] EndIf ; Size of Overlay = 0 ? Reset overlay start to 0 If Not $aRet[1] Then $aRet[0] = 0 EndIf $aRet[2] = $iCertificateAddress $aRet[3] = $nCertificateSz $aRet[4] = $nFileSize Return $aRet EndFunc FilePEOverlayExtract.au3 ~prev downloads: 34 Updates: 2013-08-09-rev2: Fixed: Didn't detect 'sandwiched-in' Overlays - Overlays appearing between the end of code/data and before a Certificate section Changed: UDF now returns an array of information: Overlay offset and size, Certificate offset and size, and filesize Fixed: Overlays < 32 bytes may have been written incorrectly 2013-08-09: Fix: Certificate Table now identified and excluded from false detection as Overlay. 2013-08-07: Fix: Section Offsets that start beyond the filesize are now accounted for. I'm not sure when this happens, but it's been reported to happen on other sites. Modified: A more reasonable filesize limit. Modified: Option to Extract just the Executable without Overlay, or the Overlay itself 2013-08-03: Fixed: Calculation of FileRead data was off by 16 (which still worked okay, but was not coded correctly!) Fixed: @extended checking after calls to other code

A3X Script Extract With newer versions of AutoIT (v3.3.10.0+), the compiled script is no longer appended to files as an overlay, and instead is embedded as a binary resource. This leads my previous AutoIt-script detection UDF lacking. However, since that UDF (>PE File Overlay Extraction) was targeted towards overlays in general, its still a worthwhile tool to have. This UDF on the other hand is pretty specific - it lets you detect and optionally extract A3X scripts from a compiled executable. The method is rather straightforward - it looks for a resource type of RT_RCDATA with a resource name of "SCRIPT", and then extracts the binary, testing for the A3X signature along the way. The main UDF has two functions: _FileContainsScriptResource() _FileContainsA3XScript() The first of these functions is there only for checking if a resource with "SCRIPT" exists. Its main purpose is to report on possible embedded scripts. I made this a separate function primarily because of issues with compressed executables. Tools like UPX and MPRESS can compress the resources as well as the rest of the executable, so any detection of the A3X signature will fail in those circumstances. (Note that UPX and MPRESS don't compress overlay data, which is why the older Overlay-Extraction A3X-detection worked regardless) The second function will both check for and optionally return the A3X script resource as binary. It also does a signature check for verification. Anyway, here's an example of its usage. The main UDF is attached below. #include <_FileContainsA3XScript.au3> ; ======================================================================================================== ; <A3XScriptExtract.au3> ; ; Example of detecting AutoIt Scripts embedded as binary resources (in AutoIt v3.3.10.0+ exe's), and ; extracting them to .A3X files. ; ; This script can be invoked in interactive or command-line mode. ; Passing an executable as a parameter will extract an A3X resource, writing it out to ; an A3X file with the same base name as the executable. ; ; Uses <_FileContainsA3XScript.au3> functions ; ; Author: Ascend4nt ; ======================================================================================================== ; ---------------------- MAIN CODE ------------------------------- Local $sFile, $sLastDir, $sLastFile, $binA3X, $iErr, $nA3XSize ; Command-line parameter received? Simply do a direct A3X extraction If $CmdLine[0] Then $sFile = $CmdLine[1] If Not FileExists($sFile) Then Exit 1 $binA3X = _FileContainsA3XScript($sFile, True) If @error Then Exit @error $nA3XSize = @extended Local $nExt = StringInStr($sFile, '.', 0, -1) If $nExt Then $sFile = StringLeft($sFile, $nExt - 1) EndIf $sFile &= '.a3x' Exit Not FileWrite($sFile, $binA3X) EndIf ; No command-line parameters, query the user in interactive mode $sLastDir = @ScriptDir While 1 $sFile=FileOpenDialog("Select PE File To Look for Embedded A3X scripts In",$sLastDir,"PE Files (*.exe;*.dll;*.scr)|All Files (*.*)",3,$sLastFile) If @error Or $sFile="" Then Exit $sLastFile = StringMid($sFile, StringInStr($sFile, '\', 1, -1) + 1) $sLastDir = StringLeft($sFile, StringInStr($sFile, '\', 1, -1) - 1) ; Resource 'Script' check $bScriptResourceFound = _FileContainsScriptResource($sFile) ConsoleWrite("_FileContainsScriptResource() return: " & $bScriptResourceFound & ", @error = " & @error & ", @extended = " & @extended & @CRLF) ; Actual A3X script resource check. True to return the A3X script as binary $binA3X = _FileContainsA3XScript($sFile, True) $iErr = @error $nA3XSize = @extended ; No A3X script found? If $iErr Or $nA3XSize = 0 Then ConsoleWrite("Failed Return from _FileContainsA3XScript(), @error = " & $iErr & ", @extended = " & $nA3XSize & @CRLF) MsgBox(64, "No AutoIt A3X script resource Found", "AutoIt A3X script resource not Found!" & @CRLF & _ ($bScriptResourceFound ? "However, *A* script resource was found" : "No 'Script' resource found either") & @CRLF & _ "for '" & $sLastFile & "'") ContinueLoop EndIf ConsoleWrite("_FileContainsA3XScript() return type: " & VarGetType($binA3X) & ", value = " & (IsBinary($binA3X) ? "[A3X_Binary]" : $binA3X) & @CRLF) ConsoleWrite(@TAB & "@error = " & $iErr & ", @extended = " & $nA3XSize & @CRLF) Local $hFileOut = -1, $sOutFile, $iMsgBox, $bSuccess = 0 $iMsgBox = MsgBox(35, "A3X script resource found in " & $sLastFile, "A3X script resource found. File size: " & FileGetSize($sLastFile) & _ ", A3X script size: " & $nA3XSize & @CRLF & @CRLF & _ "Would you like to Extract and save A3X file?") If $iMsgBox = 6 Then ;~ If $nA3XSize > 134217728 Then ;~ MsgBox(48, "A3X script is too huge", "A3X script size is > 128MB, skipping..") ;~ ContinueLoop ;~ EndIf $sOutFile = FileSaveDialog("A3X script - SAVE: Choose a file to write A3X script data to (from " & $sLastFile&")", _ $sLastDir, "AutoIt Compiled Sript (*.a3x)|All (*.*)", 2 + 16) If @error Then ContinueLoop ; Simple check for extension - if none, add .a3x If StringInStr($sOutFile, '.') = 0 Then $sOutFile &= ".a3x" ; Else $hFileOut = FileOpen($sOutFile, 16 + 2) If $hFileOut <> - 1 Then $bSuccess = FileWrite($hFileOut, $binA3X) FileClose($hFileOut) EndIf Else ContinueLoop EndIf If $bSuccess Then ShellExecute(StringLeft($sOutFile,StringInStr($sOutFile,'\',1,-1)-1)) Else MsgBox(64, "Error Opening or writing to file", "Error opening, reading, or saving A3X file") EndIf WEnd Exit _FileContainsA3XScript.au3 *edit: Modified example: Now alternatively accepts a command-line parameter for automated script extraction