7z.dll Extract File to Memory?

[Biatu]

I found some examples scripts awhile back on reading 7z files using the native 7z.dll, but I couldnt manage to create a function to extract a file. As an alternative I decided to use 7z.exe to dump all files to STDOUT, but I couldn't get a way to identify file boundaries. STDERR spit out 7z.exe ouput, and STDOUT for the binary, the data output was too fast for autoit...I could not set a variable to the output "extracting ...xxx.ext" from STDERR, then catch data from STDOUT. Way out of sync.

So i chose another method which was to have 7z dump a list of files filter out that list with RegExp then extract each file to STDOUT. This method works but is slow.

Im not looking for a complete solution here, but im attempting to use 7z.dll with memoryDLL, so i can read, and extract 7z files. I already know about the other 7-zip UDF. And unless I can get a 1:1 Compression Ratio and/or performance as compared to 7z.dll then it's not an option.

If anyone can help me it would be greatly appreciated.

Thanks guys 

[computergroove]

STDERR spit out 7z.exe ouput, and STDOUT for the binary, the data output was too fast for autoit..

 

I'm only half understanding what you are looking for here. Are you saying you just want the filelist from a zipped file? You are trying to read the file list and then only extract 1 file from that list without having to unzip the entire contents of the zipped file? For the above did you try to open a command prompt and then run the command? It probably wont be too fast for autoit. then you can send a sleep command and then exit after you received the data.

I would like to see what code you have tried and what you want to accomplish with it. I almost didn't answer because you mentioned 1:1 ratios which I have no experience in.

[Biatu]

I am attempting to extract files of a specific type from a 7z archive using the native 7z.dll. i can list files but cannot extract them.

As for the 7z.exe attempts, when using the -so switch 7z.exe will output data to STDOUT and output status messages to stderr. The issue is that the output of stdout is not in sync with stderr. Which does not make sense, so i figured that autoit was simply failing to keep up.

[computergroove]

Files of a specific type ie *.txt or *.jpg? When you list them can you read them into autoit?

[Jos]

I am attempting to extract files of a specific type from a 7z archive using the native 7z.dll. i can list files but cannot extract them.

As for the 7z.exe attempts, when using the -so switch 7z.exe will output data to STDOUT and output status messages to stderr. The issue is that the output of stdout is not in sync with stderr. Which does not make sense, so i figured that autoit was simply failing to keep up.

Why would you want to use STDOUT at all?

Are you trying to get Binary files directly into Memory via STDOUT?

Edited August 10, 2014 by Jos

[Biatu]

Yes, eliminating the need for an HDD, no RamD either. And yes inf files to be specific. I can list the contents and filter out inf files using the dll just fine, i will post the current funcs shortly.

 

Edit: added 7z.dll modified example from an old thread.

 

expandcollapse popup

#include <WinApi.au3>
#Include <7zConst.au3>
#Include <Array.au3>
#Include-Once

;Global Const $sFilePath=
;Implement MemoryMode
Global $_h7z_Dll
Global $_o7z_IInArchive,$_o7z_IStream,$_t7z_IInStream,$_o7z_IInStream

_7zInit()
_7zOpen(@ScriptDir&"\Test.7z")
;_7zSearch(".inf")
;_7zGetPaths()
_7zExtract("matchver\FORCED\5x86\D-Link\DGE-660\DGE660T.INF")
_7zClose()
;_7zUnInit()

Func _7zGetPaths()
    Local $aList=_7zListToArray()
    Local $sResults
    Local $aResults[0][2]
    Local $iBounds=UBound($aList,1)-1
    For $iIndex=0 To $iBounds
        $bTest=$aList[$iIndex][1]="True"
        If $bTest Then
            $sResults&=$iIndex&"|"&$aList[$iIndex][0]&@CRLF
        EndIf
    Next
    If StringRight($sResults,2)=@CRLF Then $sResults=StringTrimRight($sResults,2)
    __ArrayAdd($aResults,$sResults)
    Return $aResults
EndFunc

;"Extract hresult(uint;uint;int;ptr)

; STDMETHOD(Extract)(const UInt32* indices,UInt32 numItems,Int32 testMode,IArchiveExtractCallback *extractCallback) x; \

Func _7zExtract($vFilePath)
    Local $bIsPath,$bIsMulti
    If IsArray($vFilePath) Then
        ;HandelArray
        Return 1
    EndIf
    If IsString($vFilePath) Then
        Local $aList=_7zListToArray()
        $_i7z_FileIndex=_ArraySearch($aList,$vFilePath)
        Local $hCallback
        ConsoleWrite($_o7z_IInArchive.Extract($_i7z_FileIndex,1,False,$hCallback)&@CRLF)
        ConsoleWrite(VarGetType($hCallback)&"|"&$hCallback&"|"&Binary($hCallback)&@CRLF)
    EndIf
EndFunc

Func _7zFileExtractMem()
;$_o7z_IStream= _WinAPI_SHCreateStreamOnFile($sFilePath)
EndFunc

Func _7zSearch($sQuery)
    Local $aList=_7zListToArray()
    Local $sResults
    Local $aResults[0][2]
    $aSearch=_ArrayFindAll($aList,$sQuery,0,0,0,1,0)
    $iBounds=UBound($aSearch,1)-1
    For $iIndex=1 To $iBounds
        If $iIndex=$iBounds Then
            $sResults&=$aSearch[$iIndex]&"|"&$aList[$aSearch[$iIndex]][0]
        Else
            $sResults&=$aSearch[$iIndex]&"|"&$aList[$aSearch[$iIndex]][0]&@CRLF
        EndIf
    Next
    __ArrayAdd($aResults,$sResults)
    Return $aResults
EndFunc

Func _7zInit()
    If $_h7z_Dll="" Then
        $_h7z_Dll=DllOpen(@ScriptDir&"\7z.dll")
        If $_h7z_Dll=-1 Then
            Return 1
        EndIf
    EndIf
    ; 1. Create IInArchive object:
    $_o7z_IInArchive= _7z_CreateObject($_s7z_CLSID_Format7z,$_s7z_IID_IInArchive,$_tag7z_IInArchive)
    If Not IsObj($_o7z_IInArchive) Then
        $_o7z_IInArchive=""
    EndIf
EndFunc

Func _7zOpen($sFilePath)
    ; 2. Create IStream on file:
    $_o7z_IStream= _WinAPI_SHCreateStreamOnFile($sFilePath)
    If Not IsObj($_o7z_IInArchive) Then Exit 2
    ; 2a. Create IInStream on file (wrapping IStream inside methods):
    $_t7z_IInStream=""
    $_o7z_IInStream=ObjectFromTag("IInStream_",$_tag7z_IInStream,$_t7z_IInStream,Default,$_s7z_IID_IInStream)
    If Not IsObj($_o7z_IInArchive) Then Exit 6
    ; 3. Load the archive:
    $_o7z_IInArchive.Open($_o7z_IInStream,1024,0)
    ;Return $_o7z_IInStream
EndFunc


Func _7zClose()
    $_o7z_IInArchive.Close()
EndFunc

Func _7zGetItemCount()
    ; 4. Do something with it. For example I will check how many compressed items are inside:
    Local $iCount
    $_o7z_IInArchive.GetNumberOfItems($iCount)
    Return $iCount
EndFunc

Func _7zGetPropertyCount()
    Local $iCount
    $_o7z_IInArchive.GetNumberOfArchiveProperties($iCount)
    Return $iCount
EndFunc

Func _7zListToArray()
    $iMaxCount=_7zGetItemCount()
    Local $iTempNumOfProperties
    Local $vResult
    Local $vResults=""
    Local $aResults[0][4]
    For $b=0 To $iMaxCount-1
        $vResult=""
        $iMaxProps=UBound($_a7z_PropID,1)-1
        $vResult=""
        $_o7z_IInArchive.GetProperty($b,3,$vResult)
        $vResults&=$vResult&"|"
        $vResult=""
        $_o7z_IInArchive.GetProperty($b,6,$vResult)
        $vResults&=$vResult&"|"
        $vResult=""
        $_o7z_IInArchive.GetProperty($b,7,$vResult)
        $vResults&=$vResult&"|"
        $vResult=""
        $_o7z_IInArchive.GetProperty($b,8,$vResult)
        If $b=$iMaxCount-1 Then
            $vResults&=$vResult
        Else
            $vResults&=$vResult&@CRLF
        EndIf
        ;onsoleWrite($vResult&@CRLF)
        ;_ArrayDisplay($aResults)
    Next
    __ArrayAdd($aResults,$vResults)
    Return $aResults
EndFunc

;<<<<<< The End >>>>>>>;
;======================;

Func __ArrayAdd(ByRef $avArray,$vValue,$iStart=0,$sDelim_Item="|",$sDelim_Row=@CRLF,$hDataType=0)
    If $iStart=Default Then $iStart=0
    If $sDelim_Item=Default Then $sDelim_Item="|"
    If $sDelim_Row=Default Then $sDelim_Row=@CRLF
    If $hDataType=Default Then $hDataType=0
    If Not IsArray($avArray) Then Return SetError(1,0,-1)
    Local $iDim_1=UBound($avArray,$UBOUND_ROWS)
    Switch UBound($avArray,$UBOUND_DIMENSIONS)
        Case 1
            If IsArray($vValue) Then
                If UBound($vValue,$UBOUND_DIMENSIONS)<>1 Then Return SetError(5,0,-1)
                $hDataType=0
            Else
                Local $aTmp=StringSplit($vValue,$sDelim_Item,$STR_NOCOUNT+$STR_ENTIRESPLIT)
                If UBound($aTmp,$UBOUND_ROWS)=1 Then
                    $aTmp[0]=$vValue
                    $hDataType=0
                EndIf
                $vValue=$aTmp
            EndIf
            Local $iAdd=UBound($vValue,$UBOUND_ROWS)
            ReDim $avArray[$iDim_1+$iAdd]
            For $i=0 To $iAdd-1
                If IsFunc($hDataType) Then
                    $avArray[$iDim_1+$i]=$hDataType($vValue[$i])
                Else
                    $avArray[$iDim_1+$i]=$vValue[$i]
                EndIf
            Next
            Return $iDim_1+$iAdd-1
        Case 2
            Local $iDim_2=UBound($avArray,$UBOUND_COLUMNS)
            If $iStart<0 Or $iStart>$iDim_2-1 Then Return SetError(4,0,-1)
            Local $iValDim_1,$iValDim_2
            If IsArray($vValue) Then
                If UBound($vValue,$UBOUND_DIMENSIONS)<>2 Then Return SetError(5,0,-1)
                $iValDim_1=UBound($vValue,$UBOUND_ROWS)
                $iValDim_2=UBound($vValue,$UBOUND_COLUMNS)
                $hDataType=0
            Else
                ; Convert string to 2D array
                Local $aSplit_1=StringSplit($vValue,$sDelim_Row,$STR_NOCOUNT+$STR_ENTIRESPLIT)
                $iValDim_1=UBound($aSplit_1,$UBOUND_ROWS)
                StringReplace($aSplit_1[0],$sDelim_Item,"")
                $iValDim_2=@extended+1
                Local $aTmp[$iValDim_1][$iValDim_2],$aSplit_2
                For $i=0 To $iValDim_1-1
                    $aSplit_2=StringSplit($aSplit_1[$i],$sDelim_Item,$STR_NOCOUNT+$STR_ENTIRESPLIT)
                    ;_ArrayDisplay($aSplit_2)
                    If UBound($aSplit_2,1)=1 Then ContinueLoop
                    For $j=0 To $iValDim_2-1
                        $aTmp[$i][$j]=$aSplit_2[$j]
                    Next
                Next
                $vValue=$aTmp
            EndIf
            ; Check if too many columns to fit
            If UBound($vValue,$UBOUND_COLUMNS)+$iStart>UBound($avArray,$UBOUND_COLUMNS) Then Return SetError(3,0,-1)
            ReDim $avArray[$iDim_1+$iValDim_1][$iDim_2]
            For $iWriteTo_Index=0 To $iValDim_1-1
                For $j=0 To $iDim_2-1
                    If $j<$iStart Then
                        $avArray[$iWriteTo_Index+$iDim_1][$j]=""
                    ElseIf $j-$iStart>$iValDim_2-1 Then
                        $avArray[$iWriteTo_Index+$iDim_1][$j]=""
                    Else
                        If IsFunc($hDataType) Then
                            $avArray[$iWriteTo_Index+$iDim_1][$j]=$hDataType($vValue[$iWriteTo_Index][$j-$iStart])
                        Else
                            $avArray[$iWriteTo_Index+$iDim_1][$j]=$vValue[$iWriteTo_Index][$j-$iStart]
                        EndIf
                    EndIf
                Next
            Next
        Case Else
            Return SetError(2,0,-1)
    EndSwitch
    Return UBound($avArray,$UBOUND_ROWS)-1
EndFunc   ;==>_ArrayAdd


; IInStream methods
Func IInStream_QueryInterface($pSelf,$pRIID,$pObj)
    Local $tStruct=DllStructCreate("ptr",$pObj)
    ConsoleWrite("@IInStream_QueryInterface "& _WinAPI_StringFromGUID($pRIID)&@CRLF&@CRLF)
    Switch _WinAPI_StringFromGUID($pRIID)
        Case $_s7z_IID_IInStream
        Case Else
            ConsoleWrite("@IInStream_QueryInterface -> ERR"&@CRLF)
            Return $E_NOINTERFACE
    EndSwitch
    DllStructSetData($tStruct,1,$pSelf)
    Return $S_OK
EndFunc

Func IInStream_AddRef($pSelf)
    Return 1
EndFunc

Func IInStream_Release($pSelf)
    Return 0
EndFunc

Func IInStream_Read($pSelf,$pData,$iSize,$iProcessedSize)
    Return $_o7z_IStream.Read($pData,$iSize,$iProcessedSize)
EndFunc

Func IInStream_Seek($pSelf,$iOffset,$iSeekOrigin,$iNewPosition)
    Return $_o7z_IStream.Seek($iOffset,$iSeekOrigin,$iNewPosition)
EndFunc

Func _7z_CreateObject($sCLSID,$sIID,$sTag)
    Local $tCLSID= _WinAPI_GUIDFromString($sCLSID)
    Local $tIID= _WinAPI_GUIDFromString($sIID)
    Local $aCall=DllCall($_h7z_Dll,"long","CreateObject","struct*",$tCLSID,"struct*",$tIID,"ptr*",0)
    If @error Or $aCall[0] Then Return SetError(1,0,0)
    Return ObjCreateInterface($aCall[3],$sIID,$sTag)
EndFunc

Func _WinAPI_SHCreateStreamOnFile($sFile)
    Local $iGrfMode=0x00000002
    Local $aCall=DllCall("shlwapi.dll","long","SHCreateStreamOnFileW","wstr",$sFile,"uint",$iGrfMode,"ptr*",0)
    If @error Or $aCall[0] Then Return SetError(1,0,0)
    Return ObjCreateInterface($aCall[3],$_s7z_IID_IStream,$_tag7z_IStream)
EndFunc

Func ObjectFromTag($sFunctionPrefix,$_tag7z_Interface, ByRef $tInterface,$bIsUnknown=Default,$sIID="{00000000-0000-0000-C000-000000000046}") ; last param is IID_IUnknown by default
    If $bIsUnknown=Default Then $bIsUnknown=True
    Local $sInterface=$_tag7z_Interface ; copy interface description
    Local $_tag7z_IUnknown="QueryInterface hresult(ptr;ptr*);"& _
            "AddRef dword();"& _
            "Release dword();"
    ; Adding IUnknown methods
    If $bIsUnknown Then $_tag7z_Interface=$_tag7z_IUnknown&$_tag7z_Interface
    ; Below line is really simple even though it looks super complex. It's just written weird to fit in one line,not to steal your attention
    Local $aMethods=StringSplit(StringTrimRight(StringReplace(StringRegExpReplace(StringRegExpReplace($_tag7z_Interface,"\w+\*","ptr"),"\h*(\w+)\h*(\w+\*?)\h*(\((.*?)\))\h*(;|;*\z)","$1\|$2;$4"&@LF),";"&@LF,@LF),1),@LF,3)
    Local $iUbound=UBound($aMethods)
    Local $sMethod,$aSplit,$sNamePart,$aTagPart,$sTagPart,$sRet,$sParams,$hCallback
    ; Allocation
    $tInterface=DllStructCreate("int RefCount;int Size;ptr Object;ptr Methods["&$iUbound&"];int_ptr Callbacks["&$iUbound&"];ulong_ptr Slots[16]") ; 16 pointer sized elements more to create space for possible private props
    If @error Then Return SetError(1,0,0)
    For $i=0 To $iUbound-1
        $aSplit=StringSplit($aMethods[$i],"|",2)
        If UBound($aSplit)<>2 Then ReDim $aSplit[2]
        $sNamePart=$aSplit[0]
        ; Replace COM types by matching dllcallback types
        $sTagPart=StringReplace(StringReplace(StringReplace(StringReplace($aSplit[1],"object","idispatch"),"hresult","long"),"bstr","ptr"),"variant","ptr")
        $sMethod=$sFunctionPrefix&$sNamePart
        $aTagPart=StringSplit($sTagPart,";",2)
        $sRet=$aTagPart[0]
        $sParams=StringReplace($sTagPart,$sRet,"",1)
        $sParams="ptr"&$sParams
        $hCallback=DllCallbackRegister($sMethod,$sRet,$sParams)
        DllStructSetData($tInterface,"Methods",DllCallbackGetPtr($hCallback),$i+1) ; save callback pointer
        DllStructSetData($tInterface,"Callbacks",$hCallback,$i+1) ; save callback handle
    Next
    DllStructSetData($tInterface,"RefCount",1) ; initial ref count is 1
    DllStructSetData($tInterface,"Size",$iUbound) ; number of interface methods
    DllStructSetData($tInterface,"Object",DllStructGetPtr($tInterface,"Methods")) ; Interface method pointers
    Return ObjCreateInterface(DllStructGetPtr($tInterface,"Object"),$sIID,$sInterface,$bIsUnknown) ; pointer that's wrapped into object
EndFunc

Func DeleteObjectFromTag(ByRef $tInterface)
    For $i=1 To DllStructGetData($tInterface,"Size")
        DllCallbackFree(DllStructGetData($tInterface,"Callbacks",$i))
    Next
    $tInterface=0
EndFunc

Edited August 10, 2014 by Biatu

[computergroove]

This would have been easier to read with the comments in the original 7z.au3.

Looks like 7zlisttoarray would work to get the list and start identifying whether there were inf files. Does 7zExtract work to extract only inf files?

[Biatu]

Not at all. I was looking into the client7z.cpp for reference, but couldnt really find a good way to implement a call back function. I tried using a null pointer, but there didnt seem to be a way to get any data without a callback, let alone init the acutal extraction.

[Bert]

I've never heard of needing to do such a thing. If you get the ability to do this, what will you use it for? Trying to educate myself here....

[Biatu]

I am working on a device driver managment script im calling Infinity.DriverManager.
It's function is to download/update DriverPacks in the 7z format, read all INFs, and

build a database containing all HWIDs, and device specific information.

After the initialization stage/first run, the script will:

-If running in WinPE, it will detect devices, extract drivers, then load them using drvload/DriverImportPE.

-If running on a normal system...detect devices, extract drivers, install/update drivers.

Along with console paramaters to import/export drivers from online/offline systems, import/export preloaded

configurations, virtual disk images containing drivers to load on boot for a familiar system in WinPE mode.

[mLipok]

I've never heard of needing to do such a thing. If you get the ability to do this, what will you use it for? Trying to educate myself here....

Certain data are packaged in a database. 

I want to download them from the database, unpack in the background, ie. Without saving to disk. 

The stored data (ZIP archive) file is RTF and XML or XAdES. 

I personally need to do to print these RTF files.

[Bert]

I am working on a device driver managment script im calling Infinity.DriverManager.

It's function is to download/update DriverPacks in the 7z format, read all INFs, and

build a database containing all HWIDs, and device specific information.

After the initialization stage/first run, the script will:

-If running in WinPE, it will detect devices, extract drivers, then load them using drvload/DriverImportPE.

-If running on a normal system...detect devices, extract drivers, install/update drivers.

 

Along with console paramaters to import/export drivers from online/offline systems, import/export preloaded

configurations, virtual disk images containing drivers to load on boot for a familiar system in WinPE mode.

Interesting! I've worked with Windows PE but not in the way you are doing. I will be interested to see the final result of this. Thanks for the information. Edited August 11, 2014 by MBALZESHARI

[Biatu]

The goal is to modularize it, and add persistence. Im hoping to do this with autoit. A small mobile version of windows. A have functional alpha called Infinity7PE. 500Mb image designed for servicing PCs on the go.

The DriverManager will be a part of the Infinity.SystemManager.

Right now im working on creating a batch file for grub to automatically create a ramdrive to specified amount of free ram, then dump NTFS FileSystem to the size of RamD. WinPE boots up with low level RamD. The purpose is configuration before OS boot. And since I cant seem to find a good way to make/resize an NTFS partition from grub, im going to use kexec-loader so we have the power of linux to config WinPE.

[Biatu]

Using an example from another thread, I found a solution.