Compact Files functionality

Comments, questions, bug reports, etc.
Post Reply
BiatuAutMiahn
Posts: 7
Joined: Sun Nov 29, 2015 3:57 am

Compact Files functionality

Post by BiatuAutMiahn »

Hello wimlib,
I am attempting to reproduce the method you use to compact files from win32_apply.c but im having some trouble.

I follow your process, and each call returns "The operation completed successfully" however, the files do not get compressed.
Now, the thing is that, i ported the funcs to AutoIt3 Script.

Any ideas?

here's my WIP:

Edit: A Syntax Highlighted Version is here: http://infinitycommunicationsgateway.ne ... tTest.html

Code: Select all

#RequireAdmin
#include <Array.au3>
#include <WinAPIDiag.au3>
#include <WinAPIFiles.au3>

Global Const $tagWOF_EXTERNAL_INFO="uint WOFEI_Version;uint WOFEI_Provider"
Global Const $tagFILE_PROVIDER_EXTERNAL_INFO="uint FPEI_Version;uint FPEI_CompressionFormat"
Global Const $tagIOSTATUSBLOCK = "ptr Status;ptr Information"
Global Const $tagOBJECTATTRIBUTES = "ulong Length;handle RootDirectory;ptr ObjectName;ulong Attributes;ptr SecurityDescriptor;ptr SecurityQualityOfService"
Global Const $tagUNICODESTRING = "ushort Length;ushort MaximumLength;ptr Buffer"
Global Const $FILE_OPEN=0x00000001
Global Const $STATUS_PENDING=0x00000103
Global Const $STATUS_SUCCESS=0x00000000
Global Const $STATUS_ACCESS_DENIED=0xC0000022
Global Const $OBJ_CASE_INSENSITIVE = 0x00000040
Global Const $FILE_SHARE_VALID_FLAGS=0x00000007
Global Const $FILE_OPEN_REPARSE_POINT=0x00200000
Global Const $FSCTL_SET_EXTERNAL_BACKING=0x9030C
Global Const $FILE_OPEN_FOR_BACKUP_INTENT=0x00004000
Global Const $STATUS_INVALID_DEVICE_REQUEST=0xC0000010
Global Const $STATUS_COMPRESSION_NOT_BENEFICIAL=0xC000046F
Global Const $FILE_PROVIDER_COMPRESSION_FORMAT_XPRESS4K=0
Global Const $FILE_PROVIDER_COMPRESSION_FORMAT_LZX=1
Global Const $FILE_PROVIDER_COMPRESSION_FORMAT_XPRESS8K=2
Global Const $FILE_PROVIDER_COMPRESSION_FORMAT_XPRESS16K=3
Global Const $FILE_PROVIDER_CURRENT_VERSION=1
Global Const $WOF_CURRENT_VERSION=1
Global Const $WOF_PROVIDER_FILE=2

Global $hDll_NTDLL=DllOpen("ntdll.dll")
MsgBox(64,"",FileGetSize("C:\Test.dat")&@CRLF)
ConsoleWrite(_WinAPI_SetSystemCompression("C:\Test.dat",$FILE_PROVIDER_COMPRESSION_FORMAT_LZX)&"|"&@Error&"|"&@Extended&@CRLF)
MsgBox(64,"",FileGetSize("C:\Test.dat")&@CRLF)

; Try to attach an instance of the Windows Overlay Filesystem filter driver
; to the specified drive (such as C:)
Func DriveAttachWOF($sDrive); win32_try_to_attach_wof
    Local $iRet,$hDll_FltLib
    $hDll_FltLib=DllOpen("FltLib.dll")
    If @error Then Return SetError(1,0,0)
    $aRet=DllCall($hDll_FltLib,"int","FilterAttach","WSTR","wof","wstr",$sDrive,"ptr",0,"int",0,"ptr",0)
    If @error Or Not $aRet[0] Then
        $aRet=DllCall($hDll_FltLib,"int","FilterAttach","WSTR","wofadk","wstr",$sDrive,"ptr",0,"int",0,"ptr",0)
        If @error Or Not $aRet[0] Then Return SetError(@error, @extended, 0)
    EndIf
    DllClose($hDll_FltLib)
    Return SetError(0,0,$aRet[0])
EndFunc

Func PathGetDrive($sPath)
    $sPath=_WinAPI_GetFullPathName($sPath)
    If @error Or $sPath="" Then Return SetError(1,0,0)
    $aTest=StringRegExp($sPath,"^([A-Za-z]\:).*$",1)
    If @error Then Return SetError(1,0,0)
    Return SetError(0,0,"\\.\"&StringLower($aTest[0]))
EndFunc

Func _WinAPI_SetSystemCompression($sFilePath,$iFormat=0)
    Local $isWin10=0,$iCompatFlag=0
    If @OSVersion="WIN_10" Then $isWin10=1
        ;$sFilePath="\\.\"&_WinAPI_GetFullPathName($sFilePath) (TODO)

    ;For compatibility with the Windows bootloader (TODO)
    ;incompatible = match_pattern_list(dentry->d_full_path,&bootloader_patterns);
    If $iCompatFlag And ($isWin10 Or $iFormat<>0) Then
        If $isWin10 Then;Win10 only supports XPRESS4K format
            $iFormat=0
        Else;Anything Else cannot handle boot file compression, die
            Return SetError(1,1,0)
        EndIf
    EndIf
    $tName=DllStructCreate("wchar[260]")
    $tUName= DllStructCreate($tagUNICODESTRING)
    $tOA=DllStructCreate($tagOBJECTATTRIBUTES)
    DllStructSetData($tName,1,$sFilePath)
    $aRet=DllCall($hDll_NTDLL,"none","RtlInitUnicodeString","ptr",DllStructGetPtr($tUName),"ptr",DllStructGetPtr($tName))
    If @error Or Not $aRet[0] Then Return SetError(@Error,@Extended,0)
    ;MsgBox(64,"RtlInitUnicodeString",_WinAPI_GetLastErrorMessage())
    DllStructSetData($tOA,"Length",DllStructGetSize($tOA))
    DllStructSetData($tOA,"RootDirectory",0)
    DllStructSetData($tOA,"ObjectName",DllStructGetPtr($tUName))
    DllStructSetData($tOA,"Attributes",$OBJ_CASE_INSENSITIVE)
    DllStructSetData($tOA,"SecurityDescriptor",0)
    DllStructSetData($tOA,"SecurityQualityOfService",0)
    $tIOSB=DllStructCreate($tagIOSTATUSBLOCK)
    $tFile=DllStructCreate("HWND")
    $pFile=DllStructGetPtr($tFile)
    $aRet=DllCall($hDll_NTDLL,"handle","NtCreateFile", _
                             "handle*",$pFile, _
                               "ulong",BitOR($GENERIC_READ,$GENERIC_WRITE,20), _
                                 "ptr",DllStructGetPtr($tOA), _
                                 "ptr",DllStructGetPtr($tIOSB), _
                                 "ptr",0, _
                               "ulong",0, _
                               "ulong",$FILE_SHARE_VALID_FLAGS, _
                               "ulong",$FILE_OPEN, _
                               "ulong",BitOR($FILE_OPEN_FOR_BACKUP_INTENT,$FILE_OPEN_REPARSE_POINT), _
                                 "ptr",0, _
                               "ulong",0)
    If @error Or Not $aRet[0] Then Return SetError(@Error,@Extended,0)
    ;MsgBox(64,"NtCreateFile",_WinAPI_GetLastErrorMessage())
    $tInputBuffer=DllStructCreate("STRUCT;"&$tagWOF_EXTERNAL_INFO&";ENDSTRUCT;STRUCT;"&$tagFILE_PROVIDER_EXTERNAL_INFO&";ENDSTRUCT")
    DllStructSetData($tInputBuffer,"WOFEI_Version",$WOF_CURRENT_VERSION)
    DllStructSetData($tInputBuffer,"WOFEI_Provider",$WOF_PROVIDER_FILE)
    DllStructSetData($tInputBuffer,"FPEI_Version",$FILE_PROVIDER_CURRENT_VERSION)
    DllStructSetData($tInputBuffer,"FPEI_CompressionFormat",$iFormat)
    $pInputBuffer=DllStructGetPtr($tInputBuffer)
    $iInputBuffer=DllStructGetSize($tInputBuffer)
    Local $iTried=0
    Do
        $iRet=_WinAPI_NtFsControlFile(DllStructGetData($tFile,1),$FSCTL_SET_EXTERNAL_BACKING,$pInputBuffer,$iInputBuffer)
        If @error Then Return SetError(@Error,@Extended,0)
        ;MsgBox(64,"_WinAPI_NtFsControlFile",_WinAPI_GetLastErrorMessage())
        If $iRet=$STATUS_INVALID_DEVICE_REQUEST And Not $iTried Then
            DriveAttachWOF(PathGetDrive($sFilePath))
            $iTried=1
            ContinueLoop
        EndIf
        $iTried=1
    Until $iTried

    ;_ArrayDisplay($aRet,@error)

    Return 0;SetError(0,0,$iRet[0])
EndFunc
;typedef uint32_t u32;

;~ 	/* We intentionally use NtFsControlFile() rather than DeviceIoControl()
;~ 	 * here because the "compressing this object would not save space"
;~ 	 * status code does not map to a valid Win32 error code on older
;~ 	 * versions of Windows (before Windows 10?).  This can be a problem if
;~ 	 * the WOFADK driver is being used rather than the regular WOF, since
;~ 	 * WOFADK can be used on older versions of Windows.  */
;~ 	status = winnt_fsctl(h, $FSCTL_SET_EXTERNAL_BACKING,
;~ 			     &in, sizeof(in), NULL, 0, NULL);

; Synchronously execute a filesystem control method.  This is a wrapper around
; NtFsControlFile() that handles STATUS_PENDING.  Note that SYNCHRONIZE
; permission is, in general, required on the handle.
Func _WinAPI_NtFsControlFile($hFile,$iFsControlCode,$pInputBuffer,$iInputBuffer,$pOutputBuffer=0,$iOutputAvail=0)
    $tIOSB=DllStructCreate($tagIOSTATUSBLOCK)
    $pIOSB=DllStructGetPtr($tIOSB)
    $aRet=DllCall($hDll_NTDLL,"int","NtFsControlFile", _
    "HANDLE",$hFile, _
    "ptr",0, _
    "ptr",0, _
    "ptr",0, _
    "ptr",$pIOSB, _
    "uint",$iFsControlCode, _
    "ptr",$pInputBuffer, _
    "uint",$iInputBuffer, _
    "ptr",$pOutputBuffer, _
    "uint",$iOutputAvail)
    If @Error Or Not $aRet[0] Then Return SetError(@Error,@Extended,0)
    ;~     MsgBox(64,"NtFsControlFile",_WinAPI_GetLastErrorMessage())

    If $aRet[0]=$STATUS_PENDING Then;Rare
        $iRet=_WinAPI_WaitForSingleObject($hFile)
        $iError=_WinAPI_GetLastError()
        If Not $iError Then SetError(1,$iError,0)
        Return SetError(1,$iRet,0)
    EndIf
    Return SetError(0,0,$aRet[0])
EndFunc
JFX
Posts: 40
Joined: Tue Aug 18, 2015 3:59 pm

Re: Compact Files functionality

Post by JFX »

Why no simply use deviceiocontrol() instead of Native API?

BTW: Use GetCompressedFileSize() to get the compressed size.
synchronicity
Site Admin
Posts: 474
Joined: Sun Aug 02, 2015 10:31 pm

Re: Compact Files functionality

Post by synchronicity »

I don't have time to debug your script, but yes assuming you want the new "system compression", not the traditional NTFS compression, FSCTL_SET_EXTERNAL_BACKING is the right API to use. Also as JFX noted using the native API (ntdll) is not required; however, there is a small reason (noted in the code comment) why NtFsControlFile() may be preferred to DeviceIoControl() in this particular case.
BiatuAutMiahn
Posts: 7
Joined: Sun Nov 29, 2015 3:57 am

Re: Compact Files functionality

Post by BiatuAutMiahn »

The idea was to port directly over. Honestly it was my first attempt at C -> AutoIt :D

Dannyfirex on the AutoIt forums modified it slightly to get it to work, so I guess I was close.

Thx for ur feedback! Now I can create a daemon for compressing system files...esp for WinPE ;)
Post Reply