Best practises for applying an image to existing volume

Comments, questions, bug reports, etc.
supercilious
Posts: 19
Joined: Thu Sep 03, 2015 9:24 pm

Re: Best practises for applying an image to existing volume

Post by supercilious »

When I said kernel error, I meant a bug in the kernel or NTFS driver. Sorry about the confusion, I should have been clearer.

As for the list of files that you attached, they seem exclusively related to the contents of the WindowsApps folder. There is perhaps some new attribute, flag or ACL entry that is not normally exposed via the normal APIs?
supercilious
Posts: 19
Joined: Thu Sep 03, 2015 9:24 pm

Re: Best practises for applying an image to existing volume

Post by supercilious »

Ok, I had a hunch that there was some "extra" metadata on those files where wimlib was getting an access denied error... to confirm it, I captured the Windows 10 installation as an image exactly as before, but rather than try to apply that very same WIM onto the image from which it was captured, I first formatted the volume, then applied the image (with no extra options). Now, on top of this newly wimapply'd image, I again wimapply'd the very same image, this time with --compact=lzx... and as expected it applied perfectly cleanly.

So, it seems like wimlib can capture a Windows image, and apply it correctly, but there is some additional metadata associated with those folders that wimlib doens't capture, and it is this that prevents the FILE_WRITE_DATA request from being granted. Fortunately, it seems that Windows does not mind this metadata being missing, so wimapply'd images work fine, so long as they are applied on a clean drive.
supercilious
Posts: 19
Joined: Thu Sep 03, 2015 9:24 pm

Re: Best practises for applying an image to existing volume

Post by supercilious »

I forgot to add these images to my last post.
Attachments
Apply over clean applied image.png
Apply over clean applied image.png (73.8 KiB) Viewed 16902 times
Clean formatted apply.png
Clean formatted apply.png (70.92 KiB) Viewed 16902 times
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Best practises for applying an image to existing volume

Post by synchronicity »

I found the hidden metadata that causes the problem. In the file's security descriptor, there is a new ACE type in the file's SACL:

Type 0x14
Flags 0x03
Size 0x0018
Payload a9011f00010200000000001300020000

According to this link: https://social.msdn.microsoft.com/Forum ... windowssdk,
the ACE type was recently added and has the following name:

#define SYSTEM_PROCESS_TRUST_LABEL_ACE_TYPE (0x14)

(I don't know the structure of the ACE payload yet.)

Using NTFS-3g on Linux, I manually removed this ACE from the SACL of one of the files. After that, on Windows wimlib-imagex no longer encountered the access denied error on that file.

However, on Windows, the ACE seems non-removable, even by calling NtSetSecurityObject() with BACKUP_SECURITY_INFORMATION on a handle opened with backup semantics by a process with the restore operator privilege.

I don't yet know the full purpose of the ACE but I'm guessing it's some sort of bullshit DRM thing.
supercilious
Posts: 19
Joined: Thu Sep 03, 2015 9:24 pm

Re: Best practises for applying an image to existing volume

Post by supercilious »

It may not be easy (or even possible) to remove that ACL entry, but it is possible to take ownership of the folder and delete it entirely. You can then re-create it without the offending ACL. I just tested that image in a Windows 10 VM, I can easily take (recursive) ownership of the entire WindowsApps folder and delete it in its entirety without windows objecting to it.
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Best practises for applying an image to existing volume

Post by synchronicity »

Yes, you can just delete the files and re-create them again. The ACL entry does not prevent the file from being deleted. It's little difficult if the problematic file is actually a directory (since you'd have to delete its contents recursively) but it can be done.

I have no idea why Microsoft thinks this kind of thing is even remotely a good idea.
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Best practises for applying an image to existing volume

Post by synchronicity »

I believe this will be harder to work around than I initially thought.

The real problem is extracting data to nondirectory files. Obviously, to extract any data at all to a nondirectory file, wimlib needs write access to it.

Normally, if there is an existing nondirectory file at a location at which wimlib needs to extract a nondirectory file, wimlib will just delete it and re-create it. This removes any undesired metadata.

Unfortunately, this doesn't appear to prevent Windows from denying access to the new file, if it is created in a subdirectory of WindowsApps. What I expect is happening is the PROCESS_TRUST_LABEL_ACE is being inherited from the parent directory to any new files in it.

So it may be the case that the only workaround that doesn't involve reformatting the volume or a using a non-broken operating system is to remove the WindowsApps directory *entirely* before trying to extract anything at all to it.
supercilious
Posts: 19
Joined: Thu Sep 03, 2015 9:24 pm

Re: Best practises for applying an image to existing volume

Post by supercilious »

Is there any way to be sure that the lack of PROCESS_TRUST_LABEL_ACE entries on the files in WindowsApps does not cause any problems? I do not really use any of the "modern" apps in windows 8.1 or 10 so I cannot be sure that Windows doesn't mind the ACL entries being messed up, but it would be important to verify this.

I can confirm that manually taking ownership and deleting WindowsApps does work and allows the image to be applied successfully, and apparently works OK. I have no purchases on the Windows store, so there might be additional verification of paid apps and such.
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Best practises for applying an image to existing volume

Post by synchronicity »

I don't know. It might be necessary to test different types of apps, including paid apps, as you've suggested.

It certainly would be quite hostile to users, on Microsoft's part, if your apps would inevitably break after a simple backup and restore.

Regardless, I'm not planning to update wimlib to automatically delete entire directories. Instead there will need to be a warning message explaining the problem and possible workarounds such as reformatting the filesystem.
supercilious
Posts: 19
Joined: Thu Sep 03, 2015 9:24 pm

Re: Best practises for applying an image to existing volume

Post by supercilious »

Can wimlib successfully capture and apply the PROCESS_TRUST_LABEL_ACE ACL entry under Linux using ntfs-3g?
Post Reply