Kernel Objects Information

[Ascend4nt]

Kernel Objects Information

---

Sample output of Object Handles probing

_

I've assembled a number of UDF's which use "undocumented" features of the O/S over the years. And this here would be the latest, and possibly the last (I hope?).

The purpose of this UDF is to query kernel objects in the system.  It's actually a pretty big UDF that ties together a lot of functionality, and hopefully makes it more accessible.

With the UDF you can:

• Query a Kernel Object for 'hidden' information using its handle:

  Object Type and stats (_ObjectGetTypeInfoUD), Attributes and Access (_ObjectGetBasicInfoUD), Kernel Object Name (_ObjectGetNameUD), etc

• Query certain Kernel Event Objects for current states:

  Event, IoCompletion and Mutex ("Mutant") signal states (and more), Semaphore counts, Timer's remaining time, etc

• Get a list of opened File handles and filenames (there's already a few UDF's dedicated to that, though)
• Collect all the current handles held by the O/S and its processes, using specific filters, and get information on what the object is and its current state

 

Kernel Objects Inspector script

_

What's an Object you say? Whats a Kernel? Whats an NT? Gosh, maybe you shouldn't be here - go read Youtube.

As Windows programmers, we make use of these Kernel Objects all the time...

Object Types List

Some of the most common System Objects: Token, Process, Thread, Event, Mutant (Mutex), Semaphore, Timer, File (includes NamedPipe and Mailslot), Key (Registry Key)

 

Anytime you work with these objects, you are generating new objects at the kernel level.  Luckily, the O/S allows above 16 million handles per process (see Pushing the Limits of Windows: Handles by Mark Russinovich), so this isn't a concern. However, if an individual process has in excess of 16K handles, there will be some trunacted values returned from the NT API call as it only returns 16-bit values for handles.  See >this post where I try to describe this in better detail.  However, this is no longer a problem with the latest update, which restores the upper bits of handles through a simple wraparound detection technique.

There's more to say, but perhaps its best to show what functions are available. From the NTKernelObjectsInfo UDF Header:

expandcollapse popup

;  Process Objects Functions:
;   _ProcessUDGetBasicInfo()     ; Gets PID, Parent PID#, PEB, Affinity, Priority, Exit Status
;   _ProcessUDGetSessionID()     ; Gets Session ID
;   _ProcessUDGetHandleCount()   ; Gets Handle Count for Process
;   _ProcessGetFilenameByPID()   ; Gets Filename based on PID #
;   _ProcessGetPathname()        ; Gets Pathname for Process
;
;  *INTERNAL* Process Functions:
;   __PUDQueryProcess()               ; Used by _ProcessUDGet:SessionID()/HandleCount()
;   __PFDeviceToDriveXlationArray()   ; Builds a 'DOS-device' translation array
;   __PFXlateDevicePathname()         ; Translates an NT device path to a Windows filesystem path
;
;  Thread Object Functions:
;   _ThreadUDGetBasicInfo()      ; Gets TID, PID, TEB, Affinity, Priority, Exit Status
;   _ThreadUDGetStartAddress()   ; Gets the start address for Thread
;
;  IO Completion Object Functions:
;   _IoCompletionObjGetInfo()    ; Gets IoCompletionStatus' Signal State
;
;  Event Object Functions:
;   _EventObjGetInfo()       ; Event Type (0 = Manual-reset, 1 = Auto-Reset), Signaled
;
;  Mutex/Mutant Object Functions:
;   _MutexObjGetInfo()       ; Signal State, Owned (by Current Thread), Abandoned
;
;  Semaphore Object Functions:
;   _SemaphoreObjGetInfo()   ; Current Count, Maximum Count
;
;  Section Object Functions:
;   _SectionObjGetInfo()     ; Gets Base Address, Attributes, and Size of Section
;   _SectionObjGetExInfo()   ; More Section Info, primarily related to PE sections
;
;  Timer Object Functions:
;   _TimerObjGetInfo()       ; Timer Object Information: time remaining, signaled
;                            ; Missing info: Type: Notification (manual-reset) vs Synchronization (auto-reset)
;
;  Mailslot Object Functions:
;   _MailSlot_Verify()       ; Verifies if the 'File' Object is a Mailslot object
;   _MailSlot_GetInfo()      ; Returns Mailslot Info: #Message, Message Size, Max Size, Read Timeout
;
;  Pipe (Anonymous AND Named) Object Functions:
;   _Pipe_GetBasicInfoUD()           ; Gets Pipe Read Mode (message/byte {0}) & WaitMode (blocking/nonblocking {0})
;
;  NamedPipe Object Functions:
;   _NamedPipe_Verify()              ; Verifies if the 'File' Object is a NamedPipe Object
;   _NamedPipe_GetInfo()             ; Gets NamedPipe Info: Type (client/server/byte/message), Buffer Sizes, Max instances
;   _NamedPipe_GetServerProcessId()  ; Gets the process ID # of the NamedPipe creator/owner
;   _NamedPipe_GetClientProcessId()  ; Gets the Process ID # of the NamedPipe client process
;
;  File Object Functions:
;   _FileGetNameUD()         ; Gets Filename for File object using 'Undocumented' query
;   _FileGetShortNameUD()    ; Gets short 8.3 Filename using Undocumented query
;   _FileGetFileInfo()       ; Gets File attributes, size, times, etc using handle (GetFileInformationByHandle)
;
;  Registry Object Functions:
;   _RegistryDeviceToLogicalPath()   ; Converts most 'Key' Object device paths to logical path
;
;  General Object Functions:
;   _ObjectGetTypeInfoUD()       ; Gets information/stats about objects of same type.
;                                ; See _NT_OS_GetSystemObjectTypes() for same info for ALL Objects
;   _ObjectGetBasicInfoUD()      ; Gets Object Attributes, Access Mask, Handle Count, etc
;   _ObjectGetNameUD()           ; Gets the object name (if available), in NT native path format
;   _ObjectGetNameUD_Threaded()  ; Special threaded version of _ObjectGetNameUD()
;
;  Object-Index Map Functions:
;   _NTObjBuildTypesIndexMap()   ; Builds Index-Object Type Map, stores in $g_NTObjIndexMap
;   _NTObjMapLookupIndex()       ; Looks up the index# of a given Object Type (e.g. "File")
;   _NTObjMapLookupIndices()     ; Same as above, except looks up for an array of Type strings
;
;  Object Collection Functions:
;   _NT_OS_GetSystemObjectTypes() ; Gets an array of O/S NT Object Types and stats regarding them
;   _NTObjGetHandlesUD()          ; Gets an array of object Handles based on given filters
;   _NTObjGetHandlesInfoEx()      ; Gets an array of Object Handles and Info based on filters

Querying time issues: Note that any call to query handles (_NTObjGetHandlesUD, _NTObjGetHandlesInfoEx) relies on a call to NtQuerySystemInformation, which gathers information on EVERY handle held by the system and it's processes.  This can take a few seconds! Be patient.  (Also, _NTObjBuildTypesIndexMap calls it indirectly)

IMPORTANT: Be a little careful with looking for 'File' objects on Vista and Win7.. on XP there's already some safeguards which unfortunately prevent detecting certain objects. Newer versions of the O/S don't seem to have problems with threaded probing of File objects, but there may be some cases..

 

The Console output is still a bit noisy, but its good for analyzing where there's problems in reading handles, or analyzing "File" handles which can cause major problems, especially in the case of NamedPipes.

Some example UDFs are included:

• NTSystemObjectsList: displays a list of System Object Types
• NTKernelObjectsCollectExample: A collection query at its simplest (see below for this example)
• NTKernelObjectsSelfExamine: creates a number of different Objects before listing everything
• NTKernelObjectsInspect: Inspect Kernel Objects with Filtering options from a GUI

  This GUI needs work! Notice that with the ArrayDisplay function, there is a 'Run User Func' option which will display any extra info retrieved for the object (see ExInfo column).

• NTKernelObjectsSpam: Creates a crapload of Kernel Objects. This is mostly useless, but its here to demonstrate how NTKernelObjectsInspect now is able to report correct handle values beyond 65,536

 

NTKernelObjectsCollectExample

In this example I query only 2 processes for handles, and use exclusion criteria to remove "File" and "EtwRegistration" from the resultant list.

; ===========================================================================================================
; <NTKernelObjectsCollectExample.au3>
;
; Pretty barebones example of NTKernelObjectsInfo, showing the ease with which objects can be collected
; Uses multipe query types, multiple processes, and multiple Object Types with exclusion rules
;
; Author: Ascend4nt
; ===========================================================================================================
#include "NTKernelObjectsInfo.au3"
#include <Array.au3>

; -= FLAGS to Tweak Object Querying =-

; Force Win2000/XP Attribute skipping (must appear AFTER #include):
;$g_NTKO_bNamedPipeProtect = True

; Alternatively set own:
;$g_NTKO_sFileAttribSkipList = "0x0012019F|"

; Additionally, can force BadMask Skipping to OFF (not recommended):
;$g_NTKO_bSkipBadMasks = False


; Other queries available, although less often used:
; $NTOBJ_QUERYBY_PID (example: @AuotItPID), $NTOBJ_QUERYBY_OBJTYPE (ex: 28), and $NTOBJ_QUERYBY_HANDLE (actual object handle)

$aRet = _NTObjGetHandlesInfoEx($NTOBJ_QUERYBY_PROCESSNAME, "firefox.exe|autoit3.exe", _
        $NTOBJ_QUERYBY_OBJTYPENAME + $NTOBJ_QUERY_EXCLUDE, "File|EtwRegistration")

ConsoleWrite("Errors: " & @error & ", @extended = " & @extended & @CRLF)
_ArrayDisplay($aRet, "_NTObjGetHandlesInfoEx")

Thanks for testing this out!

Change History:

2014-10-01:

Fixed: _NTObjGetHandlesUD() was failing to return an error code if $FilterBy was 0, resulting in empty arras

Improved: _NTObjGetHandlesUD() now tracks & calculates internally what handle values are beyond 65,536

Added: NTKernelObjectSpam (simple test of creating tons of Kernel Objects)

2014-09-26:

Added: Multiple processes and/or types allowed in filters, plus exclusion flag

Changed: NTKernelObjectsInspect example now allows multiple filters and/or processes

         Also, Object totals are displayed where possible (elevated rights necessary for some processes)

Added: $NTOBJ_QUERYBY_PROCESSNAME Filter -> Must be used with processes as names

Improved: DLL Handles used for DLLCalls()

       Note that the biggest time-sink is still the API call NtQuerySystemInformation

       which grabs ALL system handles in one fell swoop (used inside _NTObjGetHandlesUD)

Added: Internal functions which rely on DLLHandles for speed improvements

Added: $g_NTKO_bNamedPipeProtect which can be toggled to prevent certain attributes

       from being scanned when "File" types are found. This is toggled ON for XP/2003 and under,

       OFF for Vista+

Added: $g_NTKO_sFileAttribSkipList -> assign additional Skip-Attributes with this if desired

Added: 'BadMasks' check in collection function which flags certain Attributes which have

       previously caused Thread timeouts. Use $g_NTKO_bSkipBadMasks to toggle this OFF (ON by default)

Changed: HKCU detection relies on a different technique than _Security__LookupAccountName() to get SID

Added: Smaller simpler example (NTKernelObjectsCollectExample)

Misc: Other misc. changes, additions, fixes which shouldn't affect UDF usage

NTKernelObjects.zip  ~prev Downloads: 55

Edited October 2, 2014 by Ascend4nt

[trancexx]

Excellent! Thanks for sharing mister.

...I did something similar few weeks ago when that thread about hotkeys was actual, to see if there would be any sign of hotkeys on this level. However, nothing as complete as your code. I wonder if your motivation was similar? That would be mighty cool.

Thanks again.

[Danyfirex]

Insteresting. thanks for sharing  Ascend4nt. I've learned a lot from your codes.

Saludos

[Ascend4nt]

trancexx, Actually I've had parts of the code laying around for years now.  I was intending to release something like this last year when I first showed Decipher a >function on getting Object names using a thread.  I've done a bit of work in C++ with multithreading using Semaphores and Mutexes, which gave me the need to query those objects too. Why MS never 'officially' exposed these non-mutating query API's (NtQueryMutant, NtQuerySemaphore, etc), I'll never know.. they are pretty important tools for debugging.

Additionally, I've also seen a few random people asking about handles, files, and other system objects that made me think it might be worth something to someone to release this.

As an unexpected result of this, I've also been able to see why certain processes are eating thousands of handles, sometimes for no good reason (khalmnpr.exe - you dirty little slut).

Ah, but back to that HotKeys thread.. that actually did get me digging, but instead of finding an answer, I went and released that '>Atom Tables' UDF a month or so ago, as some hotkey strings are occasionally stored there.

Man, I dunno about you.. but with this project, I'm pretty much done with the Undocumented Windows world. haha

With the Processes and Threads, Atom Tables, multi-CPU Usage and other UDF's I've released, I think I've had my fill.

[JScript]

All your codes are very interesting, somehow ends up completing one another!

Thanks for sharing another pearl.

JS

[Ascend4nt]

Insteresting. thanks for sharing  Ascend4nt. I've learned a lot from your codes.

Glad to hear it!

Btw, I'd like to hear if the Registry keys, File paths, and NamedPipes/Mailslots etc are being detected properly. (excluding those reporting negative values in the 'ExInfo' column)

Also, hmm.. it appears when running from Scite, querying one of the 'AutoIt3Wrapper' processes fails for certain File types, probably NamedPipes.. the thread querying these doesn't even terminate properly.  I may have to force protection for that specific attribute even on Windows 7

[Danyfirex]

This is my output running as admin in windows 7 Home Basic x86

Handles Info.txt

Typemapping.txt

Saludos

[Ascend4nt]

JScript: thanks for the compliment

DanyfireX, much appreciated! Are you running a Spanish (or otherwise) language version of the O/S?

[Danyfirex]

Spanish language.

Saludos

[step887]

Nice work

I recently been trying to work my way through calling process handles and I was trying to get the path name.

here is output of the self inspect (win 7 64 bit, ran with admin rights and has 64 bit)

Once I get a bit more time, I am going to dig through your code and see what I can learn.

 

 

autoit.xlsx

Edited September 22, 2014 by step887

[MikahS]

Very cool, thank you for sharing Ascend4nt

Edited September 22, 2014 by MikahS

[Ascend4nt]

here is output of the self inspect (win 7 64 bit, ran with admin rights and has 64 bit)

step887, thanks for the output. I'm curious why HKEY_CURENT_USER isn't being detected for you properly.  The line that reads "HKEY_USERSS-1-5-21...Software" should be converted, so I'm assuming the call to _Security__LookupAccountName() isn't returning the proper SID for you.  Just out of curiosity, try running this and see if the SID matches the one used in that line:

#include <Security.au3>
ConsoleWrite("SID = " & _Security__LookupAccountName(@UserName)[0] & @CRLF)

But anyway, I'm releasing a new version of the UDF shortly which uses a different method of getting the SID, so we'll see if that one works for you

[step887]

#include <Security.au3>
ConsoleWrite("SID = " & _Security__LookupAccountName(@UserName)[0] & @CRLF)

SID = S-1-5-21-3849832183-2284427975-4186142399-1001

[Ascend4nt]

SID = S-1-5-21.xxx

That's very odd.  Well, give the new UDF a try and see what the output is!

 

Update:

2014-09-26:

Added: Multiple processes and/or types allowed in filters, plus exclusion flag

Changed: NTKernelObjectsInspect example now allows multiple filters and/or processes

         Also, Object totals are displayed where possible (elevated rights necessary for some processes)

Added: $NTOBJ_QUERYBY_PROCESSNAME Filter -> Must be used with processes as names

Improved: DLL Handles used for DLLCalls()

       Note that the biggest time-sink is still the API call NtQuerySystemInformation

       which grabs ALL system handles in one fell swoop (used inside _NTObjGetHandlesUD)

Added: Internal functions which rely on DLLHandles for speed improvements

Added: $g_NTKO_bNamedPipeProtect which can be toggled to prevent certain attributes

       from being scanned when "File" types are found. This is toggled ON for XP/2003 and under,

       OFF for Vista+

Added: $g_NTKO_sFileAttribSkipList -> assign additional Skip-Attributes with this if desired

Added: 'BadMasks' check in collection function which flags certain Attributes which have

       previously caused Thread timeouts. Use $g_NTKO_bSkipBadMasks to toggle this OFF (ON by default)

Changed: HKCU detection relies on a different technique than _Security__LookupAccountName() to get SID

Added: Smaller simpler example (NTKernelObjectsCollectExample)

Misc: Other misc. changes, additions, fixes which shouldn't affect UDF usage

[step887]

Here is the output of NTKernelObjectsSelfExamine.au3

[Ascend4nt]

Here is the output of NTKernelObjectsSelfExamine.au3

I hope that was a mistake? lol

[Ascend4nt]

Update: I was thankfully wrong about the 64K limit on system object handles.  There's actually a really high limit - 16,777,216 - and its not actually O/S-wide, but Process-specific (which gives quite a lot of legroom).  The unfortunate problem that creates is that 16+million is more than what would fit in a 16-bit variable.  The NtQuerySystemInformation API call which is used to gather information on handles returns an array of SYSTEM_HANDLE_INFORMATION structures, and those structures limit handle values to 16-bits.  This amount is okay when a Process's handles are never in excess of 16-bits, but for values above this (65,536+), the upper bits are lost, and the handle value then indexes other handles, which can create a mess of confusion. Fortunately, this shouldn't be a problem with most processes, as anything more than a few thousand handles for one process is highly unlikely (and probably a sign of a buggy program).

Another important thing to note is that each handle # is process-specific. So even if there are 7,000 handles for 100 processes, the API call will still return the correct results, as each handle # is unique only to the process to which it belongs. This also means that handle # 4 in one process is something completely different in another process.

Also important to note: handle values (or handle-table indexes) are each offset by 4, so approximately 64K/4, or 16K handles can be consumed by a process before it surpasses 16-bits. The example below shows this.  Use NTKernelObjectsInspect to compare handle values to see where the 'wraparound' happens just after 0xFFFC.

For information on system handles, check out Mark Russinovich's excellent blogs, specifically "Pushing the Limits of Windows: Handles"

expandcollapse popup

#include <WinAPI.au3>
#include <Array.au3>

; Handle values (indexes into handle tables) are offset 4 bytes from each other, so aroune 16K handles
; would be the max representable in 16-bits. (There are already handles at this point, so this will
; indeed cause 17+ bits to be used)
Local $aHandles = _HandleGenerate(65536 / 4)
;MsgBox(0, "Handles Generated", "Generated handles = " & UBound($aHandles))
_ArrayDisplay($aHandles, "Handles Generated")

Func _HandleGenerate($iMax)
    ; Limit of 4999999 is much less than the real max (16 million+)
    If $iMax < 0 Or $iMax > 4999999 Then Return SetError(1, 0, 0)

    Local $hEvent, $aHandles[$iMax]

    $hEvent = _WinAPI_CreateEvent(0, True, True, "_EVENTOBJ_")
    If @error Or $hEvent = 0 Then Return SetError(-1, @error, 0)
    ConsoleWrite("Event Handle for '_EVENTOBJ' = " & $hEvent& @CRLF)

    $aHandles[0] = $hEvent

    For $i = 1 To $iMax - 1
        ; DUPLICATE_SAME_ACCESS (2)
        $aHandles[$i] = _WinAPI_DuplicateHandle(Ptr(-1), $hEvent, Ptr(-1), 0, 0, 2)
        If @error Or $aHandles[$i] = 0 Then
            SetError(-1, @error, 0)
            ExitLoop
        EndIf
    Next
    If @error Then
        If $aHandles[$i] = 0 Then
            ReDim $aHandles[$i - 1]
        EndIf
        Return SetError(2, @extended, $aHandles)
    EndIf
    Return $aHandles
EndFunc

[step887]

I hope that was a mistake? lol

it was uploaded the file but forgot to attach

autoit.xlsx

[Ascend4nt]

step887, much appreciated. It looks to be working as it should.  I'd recommend deleting or obfuscating the S-1-5-21-xxx SIDs, as I'm not sure if that info can be used for malicious purposes

[Ascend4nt]

Never one to be satisfied with edge cases, I took the initative to properly fool-proof this beast against processes with 16K+ handles. Yup, now the truncated bits of handles are properly restored using simple wraparound detection. Enjoy!

Update 2014-10-01:

Fixed: _NTObjGetHandlesUD() was failing to return an error code if $FilterBy was 0, resulting in empty arras
Improved: _NTObjGetHandlesUD() now tracks & calculates internally what handle values are beyond 65,536
Added: NTKernelObjectSpam (simple test of creating tons of Kernel Objects)