MFT-Access - Reading & parsing the Master File Table on NTFS Filesystems

[KaFu]

HiHo Community,

for some time now I'm following on the topic of reading & parsing the Master File Table on NTFS Filesystems.

Though I believe I'm on the right track, it's quiet complex and time consuming. So I wanted to share what I have and invite those who are interested to contribute in creating an UDF in the long-run.

Besides the code I also attached an excellent reading I found on the topic containing all relevant information.

expandcollapse popup

;===============================================================================
;
; Description:      MFT-Access, UDF under development
;                   Reading & parsing the Master File Table on NTFS Filesystems
; Version:          v0.0.0.3
; Last modified:    2009-May-02
; URL:              http://www.autoitscript.com/forum/index.php?showtopic=94269
; Author(s):        KaFu (http://www.funk.eu), trancexx
;
;===============================================================================
;
#RequireAdmin
#include <winapi.au3>

Global  $sDriveLetter = "c", $nBytes, $MFT_Offset, $bytes_to_read, $tBuffer, $hdd_handle, $hFile

ConsoleWrite("Analyzing Drive " & $sDriveLetter & ":, Filesystem detected: " & DriveGetFileSystem($sDriveLetter & ":") & @crlf & @crlf)

if DriveGetFileSystem($sDriveLetter & ":") <> "NTFS" then exit

; Read Boot-Sector
; http://www.autoitscript.com/forum/index.php?showtopic=92529&view=findpost&p=665010
; trancexx
Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)
Global $bRaw = FileRead($hComIn, 84); 84 bytes is enough
ConsoleWrite(_HexEncode($bRaw) & @CRLF)
FileClose($hComIn)
Global $tRaw = DllStructCreate("byte [" & BinaryLen($bRaw) & "]")
DllStructSetData($tRaw, 1, $bRaw)
Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _
        "char SystemName[8];" & _
        "ushort BytesPerSector;" & _
        "ubyte SectorsPerCluster;" & _
        "ushort ReservedSectors;" & _
        "ubyte[3];" & _
        "ushort;" & _
        "ubyte MediaDescriptor;" & _
        "ushort;" & _
        "ushort SectorsPerTrack;" & _
        "ushort NumberOfHeads;" & _
        "dword HiddenSectors;" & _
        "dword;" & _
        "dword;" & _
        "int64 TotalSectors;" & _
        "int64 LogicalClusterNumberforthefileMFT;" & _
        "int64 LogicalClusterNumberforthefileMFTMirr;" & _
        "dword ClustersPerFileRecordSegment;" & _
        "dword ClustersPerIndexBlock;" & _
        "int64 NTFSVolumeSerialNumber;" & _
        "dword Checksum", _
        DllStructGetPtr($tRaw))


ConsoleWrite("Jump:  " & DllStructGetData($tBootSectorSections, "Jump") & @CRLF)
ConsoleWrite("SystemName:  " & DllStructGetData($tBootSectorSections, "SystemName") & @CRLF)
ConsoleWrite("BytesPerSector:  " & DllStructGetData($tBootSectorSections, "BytesPerSector") & @CRLF)
ConsoleWrite("SectorsPerCluster:  " & DllStructGetData($tBootSectorSections, "SectorsPerCluster") & @CRLF)
ConsoleWrite("ReservedSectors:  " & DllStructGetData($tBootSectorSections, "ReservedSectors") & @CRLF)
;ConsoleWrite("6:  " & DllStructGetData($tBootSectorSections, 6) & @CRLF)
;ConsoleWrite("7:  " & DllStructGetData($tBootSectorSections, 7) & @CRLF)
ConsoleWrite("MediaDescriptor:  " & DllStructGetData($tBootSectorSections, "MediaDescriptor") & @CRLF)
;ConsoleWrite("9:  " & DllStructGetData($tBootSectorSections, 9) & @CRLF)
ConsoleWrite("SectorsPerTrack:  " & DllStructGetData($tBootSectorSections, "SectorsPerTrack") & @CRLF)
ConsoleWrite("NumberOfHeads:  " & DllStructGetData($tBootSectorSections, "NumberOfHeads") & @CRLF)
ConsoleWrite("HiddenSectors:  " & DllStructGetData($tBootSectorSections, "HiddenSectors") & @CRLF)
;ConsoleWrite("13:  " & DllStructGetData($tBootSectorSections, 13) & @CRLF)
;ConsoleWrite("14:  " & DllStructGetData($tBootSectorSections, 14) & @CRLF)
ConsoleWrite("TotalSectors:  " & DllStructGetData($tBootSectorSections, "TotalSectors") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFT:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFTMirr:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFTMirr") & @CRLF)
ConsoleWrite("ClustersPerFileRecordSegment:  " & DllStructGetData($tBootSectorSections, "ClustersPerFileRecordSegment") & @CRLF)
ConsoleWrite("ClustersPerIndexBlock:  " & DllStructGetData($tBootSectorSections, "ClustersPerIndexBlock") & @CRLF)
ConsoleWrite("VolumeSerialNumber:  " & Ptr(DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber")) & @CRLF)
ConsoleWrite("NTFSVolumeSerialNumber:  " & DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber") & @CRLF)
ConsoleWrite("Checksum:  " & DllStructGetData($tBootSectorSections, "Checksum") & @CRLF)
ConsoleWrite(@CRLF)



; =============================================
; Seek & Read MFT
; =============================================

$MFT_Offset = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") * DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT")
ConsoleWrite("$MFT_Offset: " & $MFT_Offset & @CRLF & @CRLF)

$MFT_Record_Size = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") / 4
; ClustersPerFileRecordSegment = 246
; ConsoleWrite("0x" & hex(246,2)) = 0xF6 >>> 0xF6 = 1/4 Cluster

$tBuffer = DllStructCreate("byte[" & $MFT_Record_Size & "]")
$hdd_handle = "\\.\" & $sDriveLetter & ":"
ConsoleWrite($hdd_handle & @crlf)
$hFile = _WinAPI_CreateFile($hdd_handle, 2, 6, 6)
;_stprintf(szDriveCreate, _T(\\\\.\\%c), cDrive);
; handle = CreateFile("\\.\PhysicalDrive" & "0", GENERIC_READ + GENERIC_WRITE, FILE_SHARE_READ + FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0)

ConsoleWrite("Harddisk Handle: " & $hFile & @crlf)

; =============================================
; first 16 records of MFT contain special information to describe the master file table itself
; http://www.ntfs.com/ntfs-mft.htm
; =============================================
for $i = 0 to 15
    ; first 16 records of MFT contain special information to describe the master file table itself
    ; http://www.ntfs.com/ntfs-mft.htm
    _WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)
    ;ConsoleWrite("_WinAPI_GetLastError(): " & _WinAPI_GetLastError() & @crlf)
    ;ConsoleWrite("_WinAPI_SetFilePointer: " & @error & @crlf & @crlf)
    _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)
    ConsoleWrite("MFT Table Record "& $i & @crlf)
    ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF)
next

; =============================================
; File & Dir information starts at record 23++; A_Dir or A_File
; http://www.scribd.com/doc/2187280/NTFS-Documentation
; =============================================
for $i = 24 to 34
    _WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)
    ;ConsoleWrite("_WinAPI_GetLastError(): " & _WinAPI_GetLastError() & @crlf)
    ;ConsoleWrite("_WinAPI_SetFilePointer: " & @error & @crlf & @crlf)
    _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)
    ConsoleWrite("MFT Table Record "& $i & @crlf)
    ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF)
next

_WinAPI_CloseHandle($hFile)

;$sText = BinaryToString(DllStructGetData($tBuffer, 1))
;ConsoleWrite('2) ' & $sText & @LF)





Func _WinAPI_SetFilePointerEx($hFile, $iPos, $iMethod = 0)
    Local $aResult

    $aResult = DllCall("kernel32.dll", "dword", "SetFilePointerEx", "hwnd", $hFile, "uint64", $iPos, "uint64*", 0, "dword", $iMethod)
    If @error Then Return SetError(1, 0, -1)
    If $aResult[0] = $__WINAPCONSTANT_INVALID_SET_FILE_POINTER Then Return SetError(2, 0, -1)
    Return $aResult[0]
EndFunc   ;==>_WinAPI_SetFilePointer





Func _HexEncode($bInput)
    Local $tInput = DllStructCreate("byte[" & BinaryLen($bInput) & "]")
    DllStructSetData($tInput, 1, $bInput)
    Local $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", 0, _
            "dword*", 0)

    If @error Or Not $a_iCall[0] Then
        Return SetError(1, 0, "")
    EndIf

    Local $iSize = $a_iCall[5]
    Local $tOut = DllStructCreate("char[" & $iSize & "]")

    $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", DllStructGetPtr($tOut), _
            "dword*", $iSize)

    If @error Or Not $a_iCall[0] Then
        Return SetError(2, 0, "")
    EndIf

    Return SetError(0, 0, DllStructGetData($tOut, 1))

EndFunc  ;==>_HexEncode

Happy Coding and with best regards ...

Edit: Added #RequireAdmin on trancexx remark.

Edit: Replaced attachement with link to the NTFS Documentation

Edited January 31, 2010 by KaFu

[ptrex]

@Kafu

Not that I can use this right now.

Could be a start of a MFT defrag analyser.

Great script !!

Regards

ptrex

[Zedna]

@KaFu

Nice script. Great example of raw disk/file reading!

[trancexx]

I didn't mention (when writing my part) that #RequireAdmin is... required - for Vista and similar.

[JFX]

Nice Scipt.

Anyone get it working with Autoit 3.3.6.0?

Guess this line no longer works correctly

Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)

[ProgAndy]

Raw-Reading mode was removed in 3.3.4.0 or so. You have to use the _WinAPI_... functions now. (_WinAPI_CreateFle, _WinAPI_ReadFile, _WinAPI_CloseHandle).

[Digisoul]

Nice Scipt.

Anyone get it working with Autoit 3.3.6.0?

Guess this line no longer works correctly

Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)

This will work for you

Global $tBuffer=DllStructCreate("byte[512]"),$nBytes
Global $hComIn = _WinAPI_CreateFile("\\.\" & $sDriveLetter & ":",2,2,7)
If $hComIn = 0 then exit
$read = _WinAPI_ReadFile($hComIn, DllStructGetPtr($tBuffer), 512, $nBytes)
If $read = 0 then exit
Global $bRaw = DllStructGetData($tBuffer,1)
ConsoleWrite(_HexEncode($bRaw)& @CRLF)
_WinAPI_CloseHandle($hComIn)


Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _
        "char SystemName[8];" & _
        "ushort BytesPerSector;" & _
        "ubyte SectorsPerCluster;" & _
        "ushort ReservedSectors;" & _
        "ubyte[3];" & _
        "ushort;" & _
        "ubyte MediaDescriptor;" & _
        "ushort;" & _
        "ushort SectorsPerTrack;" & _
        "ushort NumberOfHeads;" & _
        "dword HiddenSectors;" & _
        "dword;" & _
        "dword;" & _
        "int64 TotalSectors;" & _
        "int64 LogicalClusterNumberforthefileMFT;" & _
        "int64 LogicalClusterNumberforthefileMFTMirr;" & _
        "dword ClustersPerFileRecordSegment;" & _
        "dword ClustersPerIndexBlock;" & _
        "int64 NTFSVolumeSerialNumber;" & _
        "dword Checksum", _
    DllStructGetPtr($tBuffer))

[asdf8]

Hi!

Maybe anyone else knows how to read from the MFT list of all files on disk?

[trancexx]

Hi!

Maybe anyone else knows how to read from the MFT list of all files on disk?

You missed Digisoul's post or what?

[asdf8]

You missed Digisoul's post or what?

I do not understand how to get out of this list of files and folders available on the volume

[KaFu]

I do not understand how to get out of this list of files and folders available on the volume

That's the part still missing ... the script can point you to the location on an NTFS volume where the MFT is located, but I didn't succeeded in writing a proper parser for that. Look at the documentation mentioned above, it's quiet complex and what discouraged me most time consuming .

[joakim]

I expanded on the code and can now reassemble the complete MFT, regardless of runs (fragments) in either direction. The below code will exit after the MFT is exported. However, it should not be a problem to use it on any of the files in the MFT and use the real name when exporting the file.

expandcollapse popup

#RequireAdmin
#include <winapi.au3>
#include <Array.au3>
$outputMFT = @ScriptDir & '\MFTTest1.bin'
$testfile = _WinAPI_CreateFile($outputMFT,3,6,7)
Global Const $HX_REF="0123456789ABCDEF"
Global $RUN_Cluster[1], $RUN_Sectors[1]
Global  $sDriveLetter = "c", $nBytes, $MFT_Offset, $bytes_to_read, $tBuffer, $hdd_handle, $hFile, $MFTFull
ConsoleWrite("Analyzing Drive " & $sDriveLetter & ":, Filesystem detected: " & DriveGetFileSystem($sDriveLetter & ":") & @crlf & @crlf)
if DriveGetFileSystem($sDriveLetter & ":") <> "NTFS" then exit

Global $tBuffer=DllStructCreate("byte[512]"),$nBytes
Global $hComIn = _WinAPI_CreateFile("\\.\" & $sDriveLetter & ":",2,2,7)
If $hComIn = 0 then exit
$read = _WinAPI_ReadFile($hComIn, DllStructGetPtr($tBuffer), 512, $nBytes)
If $read = 0 then exit
Global $bRaw = DllStructGetData($tBuffer,1)
;ConsoleWrite(_HexEncode($bRaw)& @CRLF)
_WinAPI_CloseHandle($hComIn)

Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _
        "char SystemName[8];" & _
        "ushort BytesPerSector;" & _
        "ubyte SectorsPerCluster;" & _
        "ushort ReservedSectors;" & _
        "ubyte[3];" & _
        "ushort;" & _
        "ubyte MediaDescriptor;" & _
        "ushort;" & _
        "ushort SectorsPerTrack;" & _
        "ushort NumberOfHeads;" & _
        "dword HiddenSectors;" & _
        "dword;" & _
        "dword;" & _
        "int64 TotalSectors;" & _
        "int64 LogicalClusterNumberforthefileMFT;" & _
        "int64 LogicalClusterNumberforthefileMFTMirr;" & _
        "dword ClustersPerFileRecordSegment;" & _
        "dword ClustersPerIndexBlock;" & _
        "int64 NTFSVolumeSerialNumber;" & _
        "dword Checksum", _
    DllStructGetPtr($tBuffer))

ConsoleWrite("Jump:  " & DllStructGetData($tBootSectorSections, "Jump") & @CRLF)
ConsoleWrite("SystemName:  " & DllStructGetData($tBootSectorSections, "SystemName") & @CRLF)
ConsoleWrite("BytesPerSector:  " & DllStructGetData($tBootSectorSections, "BytesPerSector") & @CRLF)
ConsoleWrite("SectorsPerCluster:  " & DllStructGetData($tBootSectorSections, "SectorsPerCluster") & @CRLF)
ConsoleWrite("ReservedSectors:  " & DllStructGetData($tBootSectorSections, "ReservedSectors") & @CRLF)
;ConsoleWrite("6:  " & DllStructGetData($tBootSectorSections, 6) & @CRLF)
;ConsoleWrite("7:  " & DllStructGetData($tBootSectorSections, 7) & @CRLF)
ConsoleWrite("MediaDescriptor:  " & DllStructGetData($tBootSectorSections, "MediaDescriptor") & @CRLF)
;ConsoleWrite("9:  " & DllStructGetData($tBootSectorSections, 9) & @CRLF)
ConsoleWrite("SectorsPerTrack:  " & DllStructGetData($tBootSectorSections, "SectorsPerTrack") & @CRLF)
ConsoleWrite("NumberOfHeads:  " & DllStructGetData($tBootSectorSections, "NumberOfHeads") & @CRLF)
ConsoleWrite("HiddenSectors:  " & DllStructGetData($tBootSectorSections, "HiddenSectors") & @CRLF)
;ConsoleWrite("13:  " & DllStructGetData($tBootSectorSections, 13) & @CRLF)
;ConsoleWrite("14:  " & DllStructGetData($tBootSectorSections, 14) & @CRLF)
ConsoleWrite("TotalSectors:  " & DllStructGetData($tBootSectorSections, "TotalSectors") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFT:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFTMirr:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFTMirr") & @CRLF)
ConsoleWrite("ClustersPerFileRecordSegment:  " & DllStructGetData($tBootSectorSections, "ClustersPerFileRecordSegment") & @CRLF)
ConsoleWrite("ClustersPerIndexBlock:  " & DllStructGetData($tBootSectorSections, "ClustersPerIndexBlock") & @CRLF)
ConsoleWrite("VolumeSerialNumber:  " & Ptr(DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber")) & @CRLF)
ConsoleWrite("NTFSVolumeSerialNumber:  " & DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber") & @CRLF)
ConsoleWrite("Checksum:  " & DllStructGetData($tBootSectorSections, "Checksum") & @CRLF)
ConsoleWrite(@CRLF)
Global $SectorsPerCluster = DllStructGetData($tBootSectorSections, "SectorsPerCluster")
; =============================================
; Seek & Read MFT
; =============================================
$MFT_Offset = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") * DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT")
ConsoleWrite("$MFT_Offset: " & $MFT_Offset & @CRLF & @CRLF)
$MFT_Record_Size = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") / 4
; ClustersPerFileRecordSegment = 246
; ConsoleWrite("0x" & hex(246,2)) = 0xF6 >>> 0xF6 = 1/4 Cluster
$tBuffer = DllStructCreate("byte[" & $MFT_Record_Size & "]")
$hdd_handle = "\\.\" & $sDriveLetter & ":"
ConsoleWrite($hdd_handle & @crlf)
$hFile = _WinAPI_CreateFile($hdd_handle, 2, 6, 6)
;_stprintf(szDriveCreate, _T(\\\\.\\%c), cDrive);
; handle = CreateFile("\\.\PhysicalDrive" & "0", GENERIC_READ + GENERIC_WRITE, FILE_SHARE_READ + FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0)
ConsoleWrite("Harddisk Handle: " & $hFile & @crlf)
;------------- 2*....
$SI_Sig = '10'; hex
$FILE_NAME = '30'
$DATA = '80'
$BITMAP = 'B0'
$MFTHeaderSize = 56*2 ;Hardcoded for XP and later, but will fail on 2k
;$SI_Offset = $MFTHeaderSize+(1*2)+1
;$Resident = 1
for $i = 0 to 1
    If $i = 1 Then ExitLoop
    _WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)
    _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)
    ConsoleWrite("MFT Table Record "& $i & @crlf)
;    ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF)
    $MFTEntry = DllStructGetData($tBuffer, 1)
    $SI_Offset = (Dec(StringMid($MFTEntry,43,2))*2)+3
    ConsoleWrite("$SI_Offset = " & $SI_Offset & @crlf)
    ConsoleWrite("(Dec(StringMid($MFTEntry,43,2))*2)+3 = " & (Dec(StringMid($MFTEntry,43,2))*2)+3 & @crlf)
    $SI_Size = StringMid($MFTEntry,$SI_Offset+8,8)
    ConsoleWrite("$SI_Size = " & $SI_Size & @crlf)
    $SI_Size = StringMid($SI_Size,7,2) & StringMid($SI_Size,5,2) & StringMid($SI_Size,3,2) & StringMid($SI_Size,1,2)
    $SI_Size = Dec($SI_Size)
    ConsoleWrite("$SI_Size = " & $SI_Size & @crlf)
    $FN_Offset = $SI_Offset + ($SI_Size*2)
    ConsoleWrite("$FN_Offset = " & $FN_Offset & @crlf)
    $FN_Size = StringMid($MFTEntry,$FN_Offset+8,8)
    ConsoleWrite("$FN_Size = " & $FN_Size & @crlf)
    $FN_Size = StringMid($FN_Size,7,2) & StringMid($FN_Size,5,2) & StringMid($FN_Size,3,2) & StringMid($FN_Size,1,2)
    $FN_Size = Dec($FN_Size)
    ConsoleWrite("$FN_Size = " & $FN_Size & @crlf)
    $DATA_Offset = $FN_Offset + ($FN_Size*2)
    ConsoleWrite("$DATA_Offset = " & $DATA_Offset & @crlf)
    $DATA_Size = StringMid($MFTEntry,$DATA_Offset+8,8)
    ConsoleWrite("$DATA_Size = " & $DATA_Size & @crlf)
    $DATA_Size = StringMid($DATA_Size,7,2) & StringMid($DATA_Size,5,2) & StringMid($DATA_Size,3,2) & StringMid($DATA_Size,1,2)
    $DATA_Size = Dec($DATA_Size)
    ConsoleWrite("$DATA_Size = " & $DATA_Size & @crlf)
    $DATA_VCNs = StringMid($MFTEntry,$DATA_Offset+48,16)
;   $DATA_VCNs = StringMid($MFTEntry,$DATA_Offset+48,8)
    ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)
    $DATA_VCNs = StringMid($DATA_VCNs,15,2) & StringMid($DATA_VCNs,13,2) & StringMid($DATA_VCNs,11,2) & StringMid($DATA_VCNs,9,2) & StringMid($DATA_VCNs,7,2) & StringMid($DATA_VCNs,5,2) & StringMid($DATA_VCNs,3,2) & StringMid($DATA_VCNs,1,2)
;   $DATA_VCNs = StringMid($DATA_VCNs,7,2) & StringMid($DATA_VCNs,5,2) & StringMid($DATA_VCNs,3,2) & StringMid($DATA_VCNs,1,2)
    ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)
    $DATA_VCNs = _HexToDec($DATA_VCNs)
;   $DATA_VCNs = Dec($DATA_VCNs)
    ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)
    $NonResidentFlag = StringMid($MFTEntry,$DATA_Offset+16,2)
    ConsoleWrite("$NonResidentFlag = " & $NonResidentFlag & @crlf)
    If $NonResidentFlag = '01' Then
        $RunListOffset = StringMid($MFTEntry,$DATA_Offset+20,4)
        ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
        $RunListOffset = StringMid($RunListOffset,3,2) & StringMid($RunListOffset,1,2)
        $RunListOffset = Dec($RunListOffset)
        ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
        $RunListID = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2),2)
        ConsoleWrite("$RunListID = " & $RunListID & @crlf)
        $RunListSectorsLenght = Dec(StringMid($RunListID,2,1))
        ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)
        $RunListClusterLenght = Dec(StringMid($RunListID,1,1))
        ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)
        $RunListSectors = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2,$RunListSectorsLenght*2)
        ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
        $r = 1
        $entry = ''
        For $u = 0 To $RunListSectorsLenght-1
            $mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)
            $entry &= $mod
        Next
        $RunListSectors = Dec($entry)
        _ArrayInsert($RUN_Sectors,1,$RunListSectors)
        ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
        $RunListCluster = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)
        ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
        $entry = ''
        For $u = 0 To $RunListClusterLenght-1
            $mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)
            $entry &= $mod
        Next
        $RunListCluster = Dec($entry)
        _ArrayInsert($RUN_Cluster,1,$RunListCluster)
        ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
        If $RunListSectors = $DATA_VCNs+1 Then
            ConsoleWrite("No more data runs." & @crlf)
        Else
            ConsoleWrite("More data runs exist." & @crlf)
            $NewRunOffsetBase = $DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght+$RunListClusterLenght)*2
            _GetAllRuns($MFTEntry,$DATA_Offset,$DATA_Size,$DATA_VCNs,$NewRunOffsetBase)
        EndIf
        _ReassembleDataRuns($DATA_VCNs)
    EndIf
    $BITMAP_Offset = $DATA_Offset + ($DATA_Size*2)
    ConsoleWrite("$BITMAP_Offset = " & $BITMAP_Offset & @crlf)
    $BITMAP_Size = StringMid($MFTEntry,$BITMAP_Offset+8,8)
    ConsoleWrite("$BITMAP_Size = " & $BITMAP_Size & @crlf)
    $BITMAP_Size = StringMid($BITMAP_Size,7,2) & StringMid($BITMAP_Size,5,2) & StringMid($BITMAP_Size,3,2) & StringMid($BITMAP_Size,1,2)
    $BITMAP_Size = Dec($BITMAP_Size)
    ConsoleWrite("$BITMAP_Size = " & $BITMAP_Size & @crlf)
Next
;_ArrayDisplay($RUN_Sectors,"Sectors per RUN")
;_ArrayDisplay($RUN_Cluster,"Cluster number for RUN")
_WinAPI_CloseHandle($hFile)
Exit

Func _WinAPI_SetFilePointerEx($hFile, $iPos, $iMethod = 0)
    Local $aResult

    $aResult = DllCall("kernel32.dll", "dword", "SetFilePointerEx", "hwnd", $hFile, "uint64", $iPos, "uint64*", 0, "dword", $iMethod)
    If @error Then Return SetError(1, 0, -1)
    If $aResult[0] = $__WINAPCONSTANT_INVALID_SET_FILE_POINTER Then Return SetError(2, 0, -1)
    Return $aResult[0]
EndFunc   ;==>_WinAPI_SetFilePointer

Func _HexEncode($bInput)
    Local $tInput = DllStructCreate("byte[" & BinaryLen($bInput) & "]")
    DllStructSetData($tInput, 1, $bInput)
    Local $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", 0, _
            "dword*", 0)

    If @error Or Not $a_iCall[0] Then
        Return SetError(1, 0, "")
    EndIf

    Local $iSize = $a_iCall[5]
    Local $tOut = DllStructCreate("char[" & $iSize & "]")

    $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", DllStructGetPtr($tOut), _
            "dword*", $iSize)

    If @error Or Not $a_iCall[0] Then
        Return SetError(2, 0, "")
    EndIf

    Return SetError(0, 0, DllStructGetData($tOut, 1))

EndFunc  ;==>_HexEncode

Func _GetAllRuns($MFTEntry,$DATA_Offset,$DATA_Size,$DATA_VCNs,$RunListOffset)
Local $RunListClusterNext
_ArrayDelete($RUN_Cluster,1)
_ArrayDelete($RUN_Sectors,1)
$RunListOffset = StringMid($MFTEntry,$DATA_Offset+20,4)
ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
$RunListOffset = StringMid($RunListOffset,3,2) & StringMid($RunListOffset,1,2)
$RunListOffset = Dec($RunListOffset)
ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
$RunListID = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2),2)
ConsoleWrite("$RunListID = " & $RunListID & @crlf)
$RunListSectorsLenght = Dec(StringMid($RunListID,2,1))
ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)
$RunListClusterLenght = Dec(StringMid($RunListID,1,1))
ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)
$RunListSectors = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2,$RunListSectorsLenght*2)
ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
$entry = ''
For $u = 0 To $RunListSectorsLenght-1
    $mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)
    $entry &= $mod
Next
$RunListSectors = Dec($entry)
ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
_ArrayInsert($RUN_Sectors,1,$RunListSectors)
$RunListCluster = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)
ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
$entry = ''
For $u = 0 To $RunListClusterLenght-1
    $mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)
    $entry &= $mod
Next
$RunListCluster = Dec($entry)
ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
_ArrayInsert($RUN_Cluster,1,$RunListCluster)
If $RunListSectors = $DATA_VCNs+1 Then
    ConsoleWrite("No more data runs." & @crlf)
    Return
EndIf
$NewRunOffsetBase = $DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght+$RunListClusterLenght)*2
ConsoleWrite("$NewRunOffsetBase = " & $NewRunOffsetBase & @crlf)
$r = 1
$RunListSectorsTotal = $RunListSectors
While $RunListSectors < $DATA_VCNs+1
    $r += 1
    ConsoleWrite("$r = " & $r & @crlf)
    $RunListID = StringMid($MFTEntry,$NewRunOffsetBase,2)
    ConsoleWrite("$RunListID = " & $RunListID & @crlf)
    $RunListSectorsLenght = Dec(StringMid($RunListID,2,1))
    ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)
    $RunListClusterLenght = Dec(StringMid($RunListID,1,1))
    ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)
    $RunListSectors = StringMid($MFTEntry,$NewRunOffsetBase+2,$RunListSectorsLenght*2)
    ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
    $entry = ''
    For $u = 0 To $RunListSectorsLenght-1
        $mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)
        $entry &= $mod
    Next
    $RunListSectors = Dec($entry)
    ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
    _ArrayInsert($RUN_Sectors,$r,$RunListSectors)
    $RunListCluster = StringMid($MFTEntry,$NewRunOffsetBase+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)
    ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
    $NegativeMove = 0
;Here we must read to check positive/negative relative move
    If Dec(StringMid($RunListCluster,5,1)) > 7 Then
        $NegativeMove = 1
    EndIf
    ConsoleWrite("$NegativeMove = " & $NegativeMove & @crlf)
    $entry = ''
    For $u = 0 To $RunListClusterLenght-1
        $mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)
    $entry &= $mod
    Next
    $RunListCluster = Dec($entry)
    ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
    If $NegativeMove = 1 Then
        $p2 = ''
        For $hexnum = 1 To $RunListClusterLenght*2 ;My stupid xor substitute
            $p1 = 'F'
            $p2 &= $p1
        Next
        $FFXor = _HexToDec($p2) - $RunListCluster
        $RunListClusterTmp = $FFXor + 1
        $RunListClusterNext = $RUN_Cluster[$r-1] - $RunListClusterTmp
    ElseIf $NegativeMove = 0 Then
        $RunListClusterNext = $RUN_Cluster[$r-1] + $RunListCluster
    EndIf
    ConsoleWrite("$RunListClusterNext = " & $RunListClusterNext & @crlf)
    _ArrayInsert($RUN_Cluster,$r,$RunListClusterNext)
    $RunListSectorsTotal += $RunListSectors
    If $RunListSectorsTotal = $DATA_VCNs+1 Then
        ConsoleWrite("No more data runs." & @crlf)
        Return
    EndIf
    $NewRunOffsetBase = $NewRunOffsetBase+2+($RunListSectorsLenght+$RunListClusterLenght)*2
    ConsoleWrite("$NewRunOffsetBase = " & $NewRunOffsetBase & @crlf)
WEnd
ConsoleWrite("DataRuns: Something must have gone wrong. Should not be here..." & @crlf)
Return
EndFunc

Func _ReassembleDataRuns($DATA_VCNs)
_WinAPI_SetFilePointer($testfile, ($DATA_VCNs+1)*$SectorsPerCluster*512) ;$DATA_VCNs
_WinAPI_SetEndOfFile($testfile)
_WinAPI_FlushFileBuffers($testfile)
_WinAPI_SetFilePointer($testfile, 0,$FILE_BEGIN)
$TargetOff = 0
For $runs = 1 To Ubound($RUN_Cluster)-1
    ConsoleWrite("Run " & $runs & " reassembling.." & @crlf)
    $tBuffer = DllStructCreate("byte[" & ($RUN_Sectors[$runs]*$SectorsPerCluster*512) & "]")
    _WinAPI_SetFilePointerEx($hFile, $RUN_Cluster[$runs]*$SectorsPerCluster*512,$FILE_BEGIN)
    _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), ($RUN_Sectors[$runs]*$SectorsPerCluster*512), $nBytes)
    $MFTFragment1 = DllStructGetData($tBuffer, 1)
    ConsoleWrite("Size of buffer = " & DllStructGetSize($tBuffer) & @crlf)
    _WinAPI_WriteFile($testfile, DllStructGetPtr($tBuffer), DllStructGetSize($tBuffer), $nBytes)
    _WinAPI_FlushFileBuffers($testfile)
    $TargetOff += $RUN_Sectors[$runs]*$SectorsPerCluster*512
    _WinAPI_SetFilePointer($testfile, $TargetOff,$FILE_BEGIN)
Next
_WinAPI_CloseHandle($testfile)
Return

EndFunc
; #INDEX# ================================================================
; Title .........: Hex
; Description ...: This module contains numeric conversions between hexadecimal and
;                  decimal numbers overriding the AutoIt limits / capabilities
; Author ........: jennico (jennicoattminusonlinedotde)
; =======================================================================

; #FUNCTION# ==============================================================
; Function Name..: _HexToDec ( "expression" )
; Description ...: Returns decimal expression of a hexadecimal string.
; Parameters ....: expression   - String representation of a hexadecimal expression to be converted to decimal.
; Return values .: Success      - Returns decimal expression of a hexadecimal string.
;                  Failure      - Returns "" (blank string) and sets @error to 1 if string is not hexadecimal type.
; Author ........: jennico (jennicoattminusonlinedotde)
; Remarks .......: working input format: "FFFF" or "0xFFFF" (string format), do NOT pass 0xFFFF without quotation marks (number format).
;                  current AutoIt Dec() limitation: 0x7FFFFFFF (2147483647).
; Related .......: Hex(), Dec(), _DecToHex()
; =======================================================================
Func _HexToDec($hx_hex)
    If StringLeft($hx_hex, 2) = "0x" Then $hx_hex = StringMid($hx_hex, 3)
    If StringIsXDigit($hx_hex) = 0 Then
        SetError(1)
        Return ""
    EndIf
    Local $ret="", $hx_count=0, $hx_array = StringSplit($hx_hex, ""), $Ii, $hx_tmp
    For $Ii = $hx_array[0] To 1 Step -1
        $hx_tmp = StringInStr($HX_REF, $hx_array[$Ii]) - 1
        $ret += $hx_tmp * 16 ^ $hx_count
        $hx_count += 1
    Next
    Return $ret
EndFunc  ;==>_HexToDec()

; #FUNCTION# ==============================================================
; Function Name..: _DecToHex ( expression [, length] )
; Description ...: Returns a string representation of an integer converted to hexadecimal.
; Parameters ....: expression   - The integer to be converted to hexadecimal.
;                  length       - [optional] Number of characters to be returned (no limit).
;                                 If no length specified, leading zeros will be stripped from result.
; Return values .: Success      - Returns a string of length characters representing a hexadecimal expression, zero-padded if necessary.
;                  Failure      - Returns "" (blank string) and sets @error to 1 if expression is not an integer.
; Author ........: jennico (jennicoattminusonlinedotde)
; Remarks .......: Output format "FFFF".
;                  The function will also set @error to 1 if requested length is not sufficient - the returned string will be left truncated.
;                  Be free to modify the function to be working with binary type input - I did not try it though.
;                  current AutoIt Hex() limitation: 0xFFFFFFFF (4294967295).
; Related .......: Hex(), Dec(), _HexToDec()
; =======================================================================
Func _DecToHex($hx_dec, $hx_length = 21)
    If IsInt($hx_dec) = 0 Then
        SetError(1)
        Return ""
    EndIf
    Local $ret = "", $Ii, $hx_tmp, $hx_max
    If $hx_dec < 4294967296 Then
        If $hx_length < 9 Then Return Hex($hx_dec, $hx_length)
        If $hx_length = 21 Then
            $ret = Hex($hx_dec)
            While StringLeft($ret, 1) = "0"
                $ret = StringMid($ret, 2)
            WEnd
            Return $ret
        EndIf
    EndIf
    For $Ii = $hx_length - 1 To 0 Step -1
        $hx_max = 16 ^ $Ii - 1
        If $ret = "" And $hx_length = 21 And $hx_max > $hx_dec Then ContinueLoop
        $hx_tmp = Int($hx_dec/($hx_max+1))
        If $ret = "" And $hx_length = 21 And $Ii > 0 And $hx_tmp = 0 Then ContinueLoop
        $ret &= StringMid($HX_REF, $hx_tmp+1, 1)
        $hx_dec -= $hx_tmp * ($hx_max + 1)
    Next
    $ret=String($ret)
    If $hx_length < 21 And StringLen($ret) < $hx_length Then SetError(1)
    Return $ret
EndFunc  ;==>_DecToHex()

I'm working on a MFT parser as well, a sort of a "decode and log to csv" sort of stuff.

Joakim

[KaFu]

Joakim, the result looks most excellent (on my system I had to add Global Const $__WINAPCONSTANT_INVALID_SET_FILE_POINTER = -1 to the top). Looking forward to a working parser .

[joakim]

Contrary to what I rpeviously insinuated, the code will currently only work for exporting $MFT (and maybe a few others) as the "sequence" of the attributes are hardcoded. The parser in its alpha state is almost done and will be uploaded in short time. With that code I will guess it is also possible to recover deleted files (unless their sectors are overwritten).

[joakim]

I completely forgot to repost here that I created the project at; http://code.google.com/p/mft2csv/downloads/list

One of the important things missing is the filename to mftnumber resolving. Apart from that it is possible to recover deleted files, as long as the sectors are not overwritten and the MFT entry (with the runs information) is also there.

[KaFu]

Now, this is not a starting point anymore but a full framework of utilities for full-blown NTFS data extraction!

Great work Joakim, posting your update is really appreciated !

[joakim]

The newest sources attached.

NTFS_211211.zip

[llewxam]

Love it, thanks!

Ian

[joakim]

One of the important things missing is the filename to mftnumber resolving.

A slightly incorrect statement. It should say that the filename path to mtf record number is not yet resolved. Filenames are of course solved, but doing a a raw extraction based on that, may lead to several files with the same name without knowing the path to each of them. Also just updated mft2csv with some fixes regarding integrity checks of records..

mft2csv_v1.4.zip

[joakim]

The calculation of the runs are redone with several fixes and I'm confident that badly fragmented files are now also correctly extracted. My worst $MFT with 121 runs (including many negative moves) are now correct. However compressed and/or sparse files are not yet supported, but should not be too hard to solve I think.