The CompressionExclusionList option

[Skyblue]

Hi synchronicity,

The CompressionExclusionList option supported by imagex and dism does not seem to processed by wimlib. Here is a quick test :

Code: Select all

md E:\test

echo 12345123451234512345123451234512345123451234512345 > E:\test\text1.txt

echo 12345123451234512345123451234512345123451234512345 > E:\test\text2.txt

copy E:\test\text1.txt + E:\test\text2.txt E:\test\text3.txt

echo ver > E:\test\batch1.bat

echo ipconfig > E:\test\batch2.bat

type config.ini
[CompressionExclusionList]
*.txt

wimcapture.cmd E:\test sample.wim --config=config.ini
Scanning "E:\test"
230 bytes scanned (5 files, 1 directories)
Using LZX compression with 1 thread
Archiving file data: 177 bytes of 177 bytes (100%) done

Opening sample.wim with 7-Zip, I can see that the packed size of text3.txt is 58 bytes. It should be 107 bytes.

Could wimlib support the CompressionExclusionList option in the configuration files? Thanks.

[Skyblue]

Hi synchronicity,

Did you have the opportunity to check the issue? Thanks.

[synchronicity]

I just haven't been convinced that compression exclusions are actually needed, and they're a bit tricky to implement (e.g. since file contents are deduplicated, and files can have multiple hard links). Can you elaborate on why you need support for compression exclusions? Is it just for performance reasons, and if so, are you more concerned about compression performance or about decompression performance?

[Skyblue]

Hi synchronicity,

It's about compression performance. Skipping some files during a wimlib backup session could improve the performance in my modest opinion. A real life example: At work, I have a Sharepoint server hosting a SQL database. The first thing I do before doing a backup with wimlib is to defragment the .mdf and .ldf files but not much drive space gain after the defrag operation. Naturally, the files of the SQL database are growing with time. An example of compression exclusion scheme :

Code: Select all

[CompressionExclusionList]
\WINDOWS\SYSMSI\SSEE\MSSQL.2005\MSSQL\Data\*.mdf
\WINDOWS\SYSMSI\SSEE\MSSQL.2005\MSSQL\Data\*.ldf

All the files having the .mdf and .ldf extensions are skipped during the wimlib backup process. Since those files are not compressed and copied directly to the .wim archive, a performance gain should be observed at this point - reduced backup time \ network traffic and maybe less CPU usage.

Another typical usage of compression exclusion is to skip the .jpg and .mp4 files during the backup operations of employees :

Code: Select all

[CompressionExclusionList]
\Users\Jack\Documents\*.jpg
\Users\Jack\Documents\*.mp4

It's often not possible to know in advance the exact location of those files as the user could place them in any subfolder located under \Users\Employee\Documents. No meaning to recompress the .jpg and .mp4 files.