wimlib experimental ARM bug

[zipmagic]

So we've got a truly strange issue with ARM Windows!

Of course, as any good issue; it doesn't reproduce on a clean installation of Windows 11 25H2 - but it does reproduce with my 4 year trusty Windows 11 (originally 22H2) VM.

Fortunately, it *does* reproduce with wimcapture, out-of-the-box. A totally random number of files fail capture like so:

Code: Select all

[ERROR] Can't open "\??\e:\Windows\System32\settings.dat:WofCompressedData" for reading (status=c0000022): {Access Denied}
A process has requested access to an object, but has not been granted those access rights
ERROR: Exiting with error code 47:
       Failed to open a file.

(most of the files were in WinSxS in my own tests with my own DLL client, but this is the first file that tripped the error with your command line binary)

The error message is always:
A process has requested access to an object, but has not been granted those access rights
ERROR: Exiting with error code 47:
Failed to open a file.

And the error source is always the WofCompressedData alternate data stream.

Now, I can just run Notepad on this file:

Code: Select all

C:\wim>notepad e:\windows\system32\settings.dat

And the contents show just fine. There is no access denied error!

The file is NOT WoF compressed either!

Code: Select all

C:\wim>compact e:\windows\system32\settings.dat

 Listing e:\windows\system32\
 New files added to this directory will not be compressed.

     8192 :      8192 = 1.0 to 1   settings.dat

Of 1 files within 1 directories
0 are compressed and 1 are not compressed.
8,192 total bytes of data are stored in 8,192 bytes.
The compression ratio is 1.0 to 1.

Here's the security settings of the file, showing that it is perfectly normally accessible:

Code: Select all

C:\wim>icacls e:\windows\system32\settings.dat
e:\windows\system32\settings.dat NT SERVICE\TrustedInstaller:(F)
                                 BUILTIN\Administrators:(RX)
                                 NT AUTHORITY\SYSTEM:(RX)
                                 BUILTIN\Users:(RX)
                                 APPLICATION PACKAGE AUTHORITY\ALL APPLICATION PACKAGES:(RX)
                                 APPLICATION PACKAGE AUTHORITY\ALL RESTRICTED APPLICATION PACKAGES:(RX)

Successfully processed 1 files; Failed processing 0 files

AND, just to make things a little more interesting - if I pre-compress the entire disk with WoF compression BEFORE attempting a capture; everything DOES work fine.

What do you think is going on here?

Has Windows PE/RE introduced some kind of strange limitation for WoF alternate data streams at some point?

Is there a way for wimlib to NOT request manual access to THOSE STREAMS ONLY, letting the WoF provider serve the necessary data where applicable?

This would also provide in better space savings thanks to integrated data deduplication - no?

[synchronicity]

wimlib uses NtQueryInformationFile with FileStreamInformation to list each file's named data streams. Would you be able to confirm that Windows is indeed listing WofCompressedData as one of the named streams, despite the file not being WOF compressed (as you say)?

[zipmagic]

So I have a test VM where this issue happens, and I have been using that same VM as a way to validate the changes we have been making to WIMLIB in order to ensure it doesn't fail on (intermittent) disk read errors/extractions/applications of WIMs with damaged capture blobs.

Do you have any code (or a simpler way like a command line test) for me to run that check on this affected VM, both inside Windows and inside the Windows PE/RE recovery environment that is part of that VM from which I am able to access that odd Windows installation "externally" (without having to use a boot USB or similar)?

Or should I just ask Claude to spit out some code for me using the APIs you have mentioned so as to list the named data streams of given files?

Note that I pressed Claude to directly fix this issue as well, but it doesn't seem to have worked. The error types logged simply changed from (and I'm just paraphrasing here) "error reading WofCompressedData" to "corrupted file during compaction" - while the total list of files affected from capture to capture haven't really been affected at all (sometimes many, sometimes few).

My hunch is that when a VM bluescreens during multi-threaded WoF compression, something happens to the internal file descriptors at some end, leaving the system in this inconsistent state. This is not a terminal dead-end, as just recompressing the entire PC with WOF compression (using fewer threads to ensure it succeeds) fixes the issue; and even stock WIMLIB can then capture this affected VM successfully.

So this partially-damaged VM is primarily a great test case to check whether WIMLIB has been hardened enough to handle read errors at capture time (and extract damaged WIMs at apply time).

The major unknowns at this time are:

- Whether the vibe coded read-error fixes in WIMLIB cover all cases of read errors (yes, it was a major hassle to get it to even cover all the errors we encountered in this particular VM - some errors were from day one [at least seemingly] unrelated to WofCompressedData [although those errors also disappear when the entire disk has been recompressed with WoF successfully])

- Whether this issue is actually a broader, legitimate issue that would highlight an edge case WIMLIB is currently missing (that is to say, whether this issue may surface in cases OTHER THAN when a PC blue-screens during WoF compression due to overzealous multi-threading)

Last but not least, I consider it scandalous that modern PC hardware and/or Windows has gotten unstable to the point that compressing a disk using WoF, using multiple threads, can blue screen the system. These are fast hardware (virtual or not) running on bleeding edge processors and SSD storage.

Only anecdotally related here, but I cannot help mention that during just ordinary file compression (but aggressively multi-threaded so as to use all available CPU cores) using good 'ole 7-Zip FRIED three (3) Dell XPS 9730 laptops powered by the i9-13900H CPU. I'm pretty sure that is some kind of hardware issue or the other as Dell eventually discontinued that product line. Just doing file compression several times a week in user mode (and the multi-threaded part wouldn't even exceed 40 minutes per session, with maybe five sessions per week maximum) absolutely destroyed the CPU and/or the motherboard on these devices. Sigh.