How to get subsystem info of a exe file

[Sundance]

Hiho,

as allways i've searched the forum but this time i haven't found the answer to my question. I need to know if a specific file is a cui or a gui app. I looked at msdn and found a simple script which should fit.

http://support.microsoft.com/kb/90493/en-us?fr=1

Unfortunately my c++ knowledge is very basic and loooong time ago. Does anyone know of a way to get this information through AutoIt?

thx in advance

Sundance

[KaFu]

A little research brought up a post of piccaso about changing the subsystem of a file, the rest was trivial. Gona be a nice addition to SMF ...

expandcollapse popup

; http://www.autoitscript.com/forum/index.php?showtopic=47809&view=findpost&p=369442
; piccaso

; HOWTO: How To Determine Whether an Application is Console or GUI
; http://support.microsoft.com/kb/90493/en-us?fr=1

$message = "Select Exe to analyse Subsystem from..."
$sFile = FileOpenDialog($message, @ScriptDir & "\", "Exe (*.exe)", 1)

If @error Then
    MsgBox(4096, "", "No File(s) chosen")
    Exit
Else
    $sFile = StringReplace($sFile, "|", @CRLF)
EndIf

MsgBox(0, "", "File Subsystem is: " & _GetExeSubsystem($sFile))


;===============================================================================
;
; Function Name:    _GetExeSubsystem
; Description:      Get's the subsystem byte in specified executeable
; Parameter(s):     $sExeFile, Relative or absolute path to exe file
;
; Requirement(s):
; Return Value(s):  True on Success or sets @error to:
;                       1 - Error calling MapAndLoad
;                       2 - No PE file
;                       3 - Error calling UnMapAndLoad
; Author(s):        piccaso, KaFu
;
;===============================================================================

Func _GetExeSubsystem($sExeFile)
    Local Const $gs_SetPeSubsystem_IMAGE_NT_HEADERS = "DWORD Signature;SHORT Machine;SHORT NumberOfSections;DWORD TimeDateStamp;DWORD PointerToSymbolTable;DWORD NumberOfSymbols;SHORT SizeOfOptionalHeader;SHORT Characteristics;SHORT Magic;BYTE MajorLinkerVersion;BYTE MinorLinkerVersion;DWORD SizeOfCode;DWORD SizeOfInitializedData;DWORD SizeOfUninitializedData;DWORD AddressOfEntryPoint;DWORD BaseOfCode;DWORD BaseOfData;DWORD ImageBase;DWORD SectionAlignment;DWORD FileAlignment;SHORT MajorOperatingSystemVersion;SHORT MinorOperatingSystemVersion;SHORT MajorImageVersion;SHORT MinorImageVersion;SHORT MajorSubsystemVersion;SHORT MinorSubsystemVersion;DWORD Win32VersionValue;DWORD SizeOfImage;DWORD SizeOfHeaders;DWORD CheckSum;SHORT Subsystem;SHORT DllCharacteristics;DWORD SizeOfStackReserve;DWORD SizeOfStackCommit;DWORD SizeOfHeapReserve;DWORD SizeOfHeapCommit;DWORD LoaderFlags;DWORD NumberOfRvaAndSizes"
    Local Const $gs_SetPeSubsystem_LOADED_IMAGE = "PTR ModuleName;PTR hFile;PTR MappedAddress;PTR FileHeader;PTR LastRvaSection;UINT NumberOfSections;PTR Sections;UINT Characteristics;BYTE fSystemImage;BYTE fDOSImage;BYTE fReadOnly;UBYTE Version;BYTE Links[8];UINT SizeOfImage"

    Local Const $gi_SetPeSubsystem_dwSignature = 17744 ; "PE"

    Local Const $IMAGE_SUBSYSTEM_NATIVE = 1
    Local Const $IMAGE_SUBSYSTEM_WINDOWS_GUI = 2
    Local Const $IMAGE_SUBSYSTEM_WINDOWS_CUI = 3
    Local Const $IMAGE_SUBSYSTEM_OS2_CUI = 5
    Local Const $IMAGE_SUBSYSTEM_POSIX_CUI = 7
    Local Const $IMAGE_SUBSYSTEM_NATIVE_WINDOWS = 8
    Local Const $IMAGE_SUBSYSTEM_WINDOWS_CE_GUI = 9

    Local $aTmp, $LOADED_IMAGE, $IMAGE_NT_HEADERS, $iStubsystem, $sStubsystem
    $LOADED_IMAGE = DllStructCreate($gs_SetPeSubsystem_LOADED_IMAGE)
    $aTmp = DllCall("imagehlp.dll", "int", "MapAndLoad", "str", $sExeFile, "str", "", "ptr", DllStructGetPtr($LOADED_IMAGE), "int", 0, "int", 0)
    If @error Then Return SetError(1, 0, False)
    If $aTmp[0] = 0 Then Return SetError(1, 0, False)
    $IMAGE_NT_HEADERS = DllStructCreate($gs_SetPeSubsystem_IMAGE_NT_HEADERS, DllStructGetData($LOADED_IMAGE, "FileHeader"))
    If DllStructGetData($IMAGE_NT_HEADERS, "Signature") = $gi_SetPeSubsystem_dwSignature Then
        $iStubsystem = DllStructGetData($IMAGE_NT_HEADERS, "Subsystem")
    Else
        DllCall("imagehlp.dll", "int", "UnMapAndLoad", "ptr", DllStructGetPtr($LOADED_IMAGE))
        Return SetError(2, 0, False)
    EndIf
    $aTmp = DllCall("imagehlp.dll", "int", "UnMapAndLoad", "ptr", DllStructGetPtr($LOADED_IMAGE))
    If Not @error Then
        If $aTmp[0] <> 0 Then
            Switch $iStubsystem
                Case $IMAGE_SUBSYSTEM_NATIVE
                    $sStubsystem = "Native"
                Case $IMAGE_SUBSYSTEM_WINDOWS_GUI
                    $sStubsystem = "Windows GUI"
                Case $IMAGE_SUBSYSTEM_WINDOWS_CUI
                    $sStubsystem = "Windows CUI"
                Case $IMAGE_SUBSYSTEM_OS2_CUI
                    $sStubsystem = "OS2 CUI"
                Case $IMAGE_SUBSYSTEM_POSIX_CUI
                    $sStubsystem = "POSIX CUI"
                Case $IMAGE_SUBSYSTEM_NATIVE_WINDOWS
                    $sStubsystem = "Native Windows"
                Case $IMAGE_SUBSYSTEM_WINDOWS_CE_GUI
                    $sStubsystem = "Windows CE GUI"
            EndSwitch
            Return $sStubsystem
        EndIf
    EndIf
    Return SetError(3, 0, False)
EndFunc   ;==>_GetExeSubsystem




#cs
;===============================================================================
;
; Function Name:    _SetExeSubsystem
; Description:      Changes the subsystem byte in specified executeable
; Parameter(s):     $sExeFile, Relative or absolute path to exe file
;                   $nNewSubSystem, New Value for Subsystem
; Requirement(s):
; Return Value(s):  True on Success or sets @error to:
;                       1 - Error calling MapAndLoad
;                       2 - No PE file
;                       3 - Error calling UnMapAndLoad
; Author(s):        piccaso
;
;===============================================================================
;

Func _SetExeSubsystem($sExeFile, $nNewSubsytem)
    Local $aTmp, $LOADED_IMAGE, $IMAGE_NT_HEADERS, $nOldStubsystem
    $LOADED_IMAGE = DllStructCreate($gs_SetPeSubsystem_LOADED_IMAGE)
    $aTmp = DllCall("imagehlp.dll", "int", "MapAndLoad", "str", $sExeFile, "str", "", "ptr", DllStructGetPtr($LOADED_IMAGE), "int", 0, "int", 0)
    If @error Then Return SetError(1, 0, False)
    If $aTmp[0] = 0 Then Return SetError(1, 0, False)
    $IMAGE_NT_HEADERS = DllStructCreate($gs_SetPeSubsystem_IMAGE_NT_HEADERS, DllStructGetData($LOADED_IMAGE, "FileHeader"))
    If DllStructGetData($IMAGE_NT_HEADERS, "Signature") = $gi_SetPeSubsystem_dwSignature Then
        $nOldStubsystem = DllStructGetData($IMAGE_NT_HEADERS, "Subsystem")
        DllStructSetData($IMAGE_NT_HEADERS, "Subsystem", $nNewSubsytem)
    Else
        DllCall("imagehlp.dll", "int", "UnMapAndLoad", "ptr", DllStructGetPtr($LOADED_IMAGE))
        Return SetError(2, 0, False)
    EndIf
    $aTmp = DllCall("imagehlp.dll", "int", "UnMapAndLoad", "ptr", DllStructGetPtr($LOADED_IMAGE))
    If Not @error Then
        If $aTmp[0] <> 0 Then Return SetError(0, $nOldStubsystem, True)
    EndIf
    Return SetError(3, 0, False)
EndFunc   ;==>_SetExeSubsystem

;------------
;  Example
;------------

_SetExeSubsystem("test.exe", $IMAGE_SUBSYSTEM_WINDOWS_CUI)
Switch @error
    Case 1
        ConsoleWrite("Error calling MapAndLoad (imagehlp not present, exe not found)" & @CRLF)
    Case 2
        ConsoleWrite("Not a PE file" & @CRLF)
    Case 3
        ConsoleWrite("Error calling UnMapAndLoad" & @CRLF)
    Case 0
        Switch @extended
            Case $IMAGE_SUBSYSTEM_WINDOWS_CUI
                ConsoleWrite("Changed, old subsytem was: CUI" & @CRLF)
            Case $IMAGE_SUBSYSTEM_WINDOWS_GUI
                ConsoleWrite("Changed, old subsytem was: GUI" & @CRLF)
            Case Else
                ConsoleWrite("Changed, old subsytem value is unknown")
        EndSwitch
        ConsoleWrite(@CRLF)
EndSwitch
#ce

Edit: Return SetError(3, 0, False) has to be behind last endif...

Edited May 26, 2010 by KaFu

[Sundance]

Hiho,

i don't believe i haven't found this post. I really need some advice in using the search function

I think that was the information i needed.

Thanks/Danke

Sundance

[KaFu]

I read the linked article you posted and just used the first const keyword "IMAGE_SUBSYSTEM_NATIVE" as the search term... Gern geschehen ...

[Robinson1]

Attention that sample code has a great tendency to crash >_< when it comes to call

DllCall("imagehlp.dll","UnMapAndLoad"... )

• Well one quick and dirty workaround would be to just drop/delete all UnMapAndLoad calls since we just read data. Nothing is needed to write back.

But well that's somehow unsatisfying and so we've learn nothing from that.

Well it took me some time to find out the reason for the crash that somehow occurs inside imagehlp!UnMapAndLoad();FreeModuleName during the KERNEL32.HeapFree call.

First i though there is something wrong with some unproper use of "ptr" or DllStructGetPtr() well but this seem to be fine.

However to crashes here:

typedef struct _LOADED_IMAGE {

PSTR ModuleName;

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

...

PUSH [DWORD EBP+8] ; /pMemory <-PSTR ModuleName;

PUSH 0 ; |Flags = 0

PUSH [DWORD 6CC851F4] ; |hHeap = 01420000

CALL [<&KERNEL32.HeapFree>]

when it tries to free the memory MapAndLoad allocated for the path of the ModuleName.

And finally I found it - the is problem is that UnMapAndLoad uses some other hHeap address than MapAndLoad used.So why does that value changes ?

->Well because Autoit loads and unload imagehlp.dll on each call when you use the dll parameter as a String.

• So instead doing it like this

  DllCall("imagehlp.dll","MapAndLoad"... ) DllCall("imagehlp.dll","UnMapAndLoad"... )Do it like this:

  $himagehlp_dll = DllOpen("imagehlp.dll")
  DllCall(himagehlp_dll, "int", "MapAndLoad"...)
  DllCall(himagehlp_dll, "int", "UnMapAndLoad"...)

  and everything will be alright.

Well base on the example i made my own code that is targeting the Dll-Flag in the PE-Characteristics

Func Main()

    ; Get TargetFile
    Local $PE_FileName = GetPEFile()
    If $PE_FileName = False Then Exit

    ; verify TargetFile
    If OpenPEFile($PE_FileName) = False Then Exit
    PEFile_UnMap()

    ; Copy *.* to *.dll and open it
    Local $Dll_FileName = $PE_FileName
    ChangeFileExt ($Dll_FileName,"dll")

    FileCopy ($PE_FileName, $Dll_FileName, 1)
    OpenPEFile($Dll_FileName)

    ; Set Dll Flag in PE_Header/Characteristics [via BitOR]
    NT_hdr_set("Characteristics", BitOR(NT_hdr_get("Characteristics"), _
            $Characteristics_IMAGE_FILE_DLL) )

    ; Save Changes
    PEFile_UnMap()

    Logtxt($Dll_FileName & " created.")

EndFunc   ;==>Main

^-that's just the roadmap - DL ExeToDll.au3 for real use....and why convert a Exe to an DLL?Na well - just for fun. ... or to inject it into another Exe - to get some code (or au3-Script ) in, to extract some 'virtual'/'bundled' files or data from that Process.

ExeToDll.au3

Edited March 9, 2013 by Melba23
Reposted from below

[UEZ]

@Robinson1: good catch!

Br,

UEZ

[Melba23]

Robinson1,

AutoIt3.dll is the AutoIt3.exe with the Dll-flag set in the header.

1. Call DecompileMe.exe with MyDumpScript.au3

DecompileMe.exe will ignore that Commandline parameter however it'll be set.

2. injectdll.exe AutoIt3.dll -> DecompileMe.exe

3. AutoIt3.dll will now run inside DecompileMe.exe and use the Commandline

and so run MyDumpScript.au3

4. MyDumpScript.au3 has also access to the virtual files

Please explain why this is not reverse-engineering of an AutoIt executable and also not breaking the Forum rule which states:

"Do not ask for help with AutoIt scripts, post links to, or start discussion topics on the following subjects:

[...]

Running or injecting any code (in any form) intended to alter the original functionality of another process.

[...]"

Until you do, I have removed the links above. My apologies in advance if I have erred on the side of caution.

M23

[Robinson1]

Robinson1,

Please explain why this is not reverse-engineering of an AutoIt executable and also not breaking the Forum rule which states:

"Do not ask for help with AutoIt scripts, post links to, or start discussion topics on the following subjects:

[...]

Running or injecting any code (in any form) intended to alter the original functionality of another process.

[...]"

Until you do, I have removed the links above. My apologies in advance if I have erred on the side of caution.

M23

Oh dear I fell a little like surround by mimosas in here.

However I also love&care about mimosa's and so I'll pay a little more attentions and be more cautiously.

To be pedantic - there is nothing altered or patch in the other processes program code - I just run some code in there and so make use of the resources in there.

Changing to Dll Flag is like changing an GUI to an Console App like this:

$IMAGE_SUBSYSTEM_WINDOWS_GUI = 2 ->

$IMAGE_SUBSYSTEM_WINDOWS_CUI = 3

It's like writing back what that PE-Subsystem example reads out.

Well what is 'reverse-engineering of an Auto It executable' and what not?

A Question for politicians (or a l(i)aywer) Best is may to see what it feels like and then to decide for ya own. So if you ask me - it is not.

However sorry about scaring ya with that inject crap. I mentioned it just for example.

... you can easily do it the other way like changing Dll->Exe.

^-Use it once on some temporary setup file that set the dll-flag to avoid that you run it just like clicking on it.

Na well enough of this 'hairsplitting'

What's a little pity is, that you also delete the attached *.au3 - since to code in the box is just the main function and don't run without the sub functions.

I reupped it.

(^-Delete it again if ya fell like and i'll keep with that.)

Edited March 9, 2013 by Robinson1

[Melba23]

Robinson1,

Thank you for your lengthy reply to my earlier enquiry.

As a mere "mimosa" (but charged with the overall supervision of the forum) I am delighted that you saw fit to put me back in my place in such a condescending manner. From what I can understand of the slang phrases and broken English of your post I am content for you leave the .au3 script in place - but I would point out that it might have been more polite on your behalf to await clearance before reposting it.

I am looking forward to our paths crossing again on the forum - but hopefully under different circumstances.

M23

[Robinson1]

<snip>

____________________________

To the Moderator: can you please shift that post to the old place.

(I unintentionally hit the 'delete' button + managed it to hit the confirm button in the same manner. It unfortunately appeared at the same place the mouse was on.

Whoa sometimes I'm surprised myself how quick thing like that can happen.

...and yes that 'Toggle editing Mode' can greatly mess up the post - but ya probably already know. Well for now I prefer 'Text mode'

)

Edited March 9, 2013 by Melba23
Code reposted above to keep the thread readable

[Melba23]

Robinson1,

I have reposted your code as requested to keep the thread readable.

Please try to limit the inadvertant mouse-clicks in future.

M23

[Robinson1]

I have reposted your code as requested to keep the thread readable.

Wow Thanks !!! :rolleyes:

Sorry for upsetting ya.

You're doing a great job, and

Please try to limit the inadvertant mouse-clicks in future.

...and well I'm laying there comfortable -as always- on an kind of camp bed.

And yes with the blanket and the mouse it sometimes happens that the blanket hits the mousebutton and ... na anyway, doesn't matters.

Edited March 9, 2013 by Robinson1

[KaFu]

Ascend4nt's nice my also be interesting for you...

[Robinson1]

What if there's no imagehlp.dll ?

So here a more simple noAPI solution that just uses AutoIT 'FileRead':

Func _FileCheckWinPEFormat($sPEFile)
    $hFile=FileOpen($sPEFile,16)
; Move to Windows PE Signature Offset location
    FileSetPos($hFile,0x3C,0
    $iOffset=FileRead($hFile,4)

; Move to Windows PE Header Offset
    FileSetPos($hFile,$iOffset,0) 
    $iOffset=Number($bBuffer)

; Read in data from Major O/S version up to Subsystem info
    $bBuffer=FileRead($hFile,30)
;...
    $aPEData[4]=Number(BinaryMid($bBuffer,29,2))    ; Subsystem
;....

->https://sites.google.com/site/ascend4ntscode/filecheckwinpeformat

^-But yeah imagehlp_dll is standard since WinXP. ...Or well i just checked Win98 setup-files and even there it found it! -> WIN98_29.CAB.

Beside imagehlp_dll!MapAndLoad I also like the way open using kernel32.dll!LoadLibraryExW. With the parameter LOAD_LIBRARY_AS_DATAFILE 0x00000002 you make it just to load/map the file into Memory without execution of DllMain. And beside dll it'll also work on exe(, sys, pdb, sym...) or any other PE-file.

expandcollapse popup

; LOAD_LIBRARY_AS_DATAFILE 0x00000002
    ;If this value is used, the system maps the file into the calling process's virtual address space as if it were a data file. Nothing is done to execute or prepare to execute the mapped file.

    ;LOAD_LIBRARY_AS_IMAGE_RESOURCE 0x00000020 (supported since Vista)
    ;the loader does not load the static imports or perform the other usual initialization steps.
    $a_hCall = DllCall("kernel32.dll", _
                "hwnd", "LoadLibraryExW", _
                "wstr", $sModule, _
                "hwnd", 0, _
    $pPointer = BitAND($a_hCall[0], 0xFFFF0000) ; Align to 0x10000 

    ;
    ; ==== MZ-Header ===
    ;
    Local $tIMAGE_DOS_HEADER = DllStructCreate( _
            "char Magic[2];" & _
            "ushort BytesOnLastPage;" & _
            "ushort Pages;" & _
            "ushort Relocations;" & _
            "ushort SizeofHeader;" & _
            "ushort MinimumExtra;" & _
...
            "dword AddressOfNewExeHeader", _
            $pPointer)

    $pPointer += DllStructGetData($tIMAGE_DOS_HEADER, "AddressOfNewExeHeader")
    ;
    ; ==== PE-Header ===
    ;
    Local $tIMAGE_FILE_HEADER = DllStructCreate("ushort Machine;" & _
            "ushort NumberOfSections;" & _
            "dword TimeDateStamp;" & _
            "dword PointerToSymbolTable;" & _
            "dword NumberOfSymbols;" & _
            "ushort SizeOfOptionalHeader;" & _
            "ushort Characteristics", _
            $pPointer)
...

All da bloody details are in the attached sample

...and well just for learning there is also the Func MapFile that make use of the API's CreateFileMapping and MapViewOfFile.

It's btw the same what MapAndLoad does under the hood.

Yeah yeah beside the 'boring' memory intensive ReadFile, WriteFile(when it comes to open bigger files), MemoryMapping is create thing. No need to write back changes. Change in the memory => Change in the File.

It's good to know about and to make use of it.

Well comment in this to enable it:

;Well beside LoadLibraryExW consider the use of 'FileMapping' like this:
   ; $pPointer= MapFile($sModule)
   ; ^^... so we've also have already the overlay in mem.

When using it the closing is DIY. Write the UnmapViewOfFile and CloseHandle your own. However as long we're dealing in view/readonly mode you may not care about a proper closing)

EOF_Exe.au3

Edited March 11, 2013 by Robinson1