Transparent Compression

Comments, questions, bug reports, etc.
Post Reply
jaxyspace
Posts: 2
Joined: Sun Mar 04, 2018 9:09 pm

Transparent Compression

Post by jaxyspace »

Is there any way to integrate wimlib LZX compression into Windows explorer? To use in a similar way as "compact /exe:lzx" but taking advantage on the superior compression ratio of wimlib? Or even better, use LZMS with transparent compression

I'm aware of a paid software called ZIPmagic who claims they do this using wimlib, but I don't know how they reach this.
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Transparent Compression

Post by synchronicity »

'compact /exe' compresses files using the "System Compression" mechanism. Microsoft's driver which handles these files only supports XPRESS and LZX (max chunk size 32768), and if I remember correctly it's impossible for third-party software to create such files on Windows without going through Microsoft's compression code (or, of course, circumventing the filesystem and writing to the raw disk...). But yes, these are the same compression formats supported by wimlib. I actually reuse wimlib's XPRESS and LZX decompression code for the ntfs-3g-system-compression plugin for NTFS-3G, which allows system-compressed files to be read on Linux.

The paid software you referred to, on the other hand, uses the "WIMBoot" feature, where files are "externally backed" by a WIM archive. You can create such files with wimlib-imagex using 'wimapply --wimboot'. Similarly, Microsoft's driver which handles these files only supports XPRESS and LZX (max chunk size 32768). But of course, you could still capture the WIM using the best wimlib can do with these settings, e.g. --compress=lzx:100.
jaxyspace
Posts: 2
Joined: Sun Mar 04, 2018 9:09 pm

Re: Transparent Compression

Post by jaxyspace »

Let's see if I understood
This is how you suggest me to do?
g.png
g.png (159.95 KiB) Viewed 9958 times
But there's no way to apply something like this in a transparent way because it relies on Windows System Compression feature, meaning that this could only be possible if someone create a custom driver that can read those WIM on the fly, right?
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Transparent Compression

Post by synchronicity »

Read my post again. You could use either system compression or WIMBoot, but neither works with LZMS compression (or solid-mode compression), and wimlib's compressors can only be used for the latter currently.
zipmagic
Posts: 61
Joined: Thu Aug 06, 2015 7:09 am

Re: Transparent Compression

Post by zipmagic »

jaxyspace wrote: Sun Mar 04, 2018 9:31 pm Is there any way to integrate wimlib LZX compression into Windows explorer? To use in a similar way as "compact /exe:lzx" but taking advantage on the superior compression ratio of wimlib? Or even better, use LZMS with transparent compression

I'm aware of a paid software called ZIPmagic who claims they do this using wimlib, but I don't know how they reach this.
I am not sure if your post is confused or not...

If you're asking whether ZIPmagic is able to transparently compress a disk, the answer is yes.

If you're asking whether ZIPmagic is able to transparently display archives as folders in Windows Explorer, the answer is again yes.

ZIPmagic is both a `traditional` transparent file archiver, and a transparent disk compressor.

wimlib is only involved in the transparent disk compression side, of course.

ZIPmagic relies on its own 7-Zip based file compression stack for the `traditional` archiving features, plus WinZip compatible ZIPX support (including JPEG compression) through a commercial third party library (BricolSoft). But these libraries don't actually help with the transparency of the file archiving stack, of course.

ZIPmagic has developed its own proprietary Windows File Explorer shell namespace extension over the years, which it uses to achieve the transparency in its `traditional` file archiving stack - where archives appear like folders, are browsed as such with lightning speed, with full support for copy/paste, drag/drop, double-click to extract/edit/compress, etc.

You can look up information about shell namespace extensions online, but only ZIPmagic has a high performance, scalable, and stable NSE - this is the result of years of R&D investment in this area. Faster by an order of magnitude than even Windows's own NSE, this was no minor feat. You can also find multiple third party source codes/binaries, but all of them are plagued by some problem or the other which makes the resultant NSE's utterly unusable in real world scenarios.
Post Reply