Kernel Extended Attribute Support on NTFS

Comments, questions, bug reports, etc.
zipmagic
Posts: 61
Joined: Thu Aug 06, 2015 7:09 am

Kernel Extended Attribute Support on NTFS

Post by zipmagic »

You can check for existing kernel extended attributes on files using:

fsutil file queryea <filename>

Some issues earlier reported on this forum:

viewtopic.php?t=579

Might be due to these kernel extended attributes not being preserved by wimlib.

Is it possible to add support for these attributes?

Lacking such support in wimlib, I suppose tool vendors relying on wimlib could always manually back them up and restore them later?
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Kernel Extended Attribute Support on NTFS

Post by synchronicity »

This is already supported since wimlib v1.13.0, released in 2018:

Code: Select all

Version 1.13.0:
        On Windows, wimlib now supports capturing and applying extended
        attributes (EAs).  It is compatible with DISM with the /EA option,
        available since Windows 10 version 1607.  wimlib's EA support is on by
        default and works on older versions of Windows too.
If it does not work properly on some files, please report the specific issue with details.
zipmagic
Posts: 61
Joined: Thu Aug 06, 2015 7:09 am

Re: Kernel Extended Attribute Support on NTFS

Post by zipmagic »

The emphasis is on *kernel* extended attributes. wimlib does indeed handle non-kernel extended attributes properly since the version you mentioned.

Apparently these are preceded with the string $Kernel in the attribute name:

https://learn.microsoft.com/en-us/windo ... ptions-s14
"...Extended attributes with prefix “$Kernel.” in name will be skipped because only user mode extended attributes are captured..."
If that still doesn't help, I'd be happy to obtain an exact file path for you; but you may need to install an Xbox game for that.
zipmagic
Posts: 61
Joined: Thu Aug 06, 2015 7:09 am

Re: Kernel Extended Attribute Support on NTFS

Post by zipmagic »

Here's a marginally related post:

https://superuser.com/questions/396692/ ... -windows-8

I believe these kernel extended attributes are why we've seen lots of oddball issues with so-called modern Windows apps - they're all full of them.

Even the latest Notepad itself won't run the first (couple) times you launch it after restoring an image with WIMLIB. Just trying to launch it again ultimately takes care of the issue, but it's a nuisance at best (and does affect literally every so-called modern app).

Now that we actually have apps relying on these attributes as a way of piracy detection/prevention - a much more harmful side effect of missing the kernel extended attributes - I suppose we'll just have to figure this out one way or the other?
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Kernel Extended Attribute Support on NTFS

Post by synchronicity »

According to https://learn.microsoft.com/en-us/windo ... attributes, these new "kernel extended attributes" cannot be set from user mode.

So, it sounds like there is nothing that user mode tools can do about them; Microsoft specifically intended for them to not be backed up and restored.
zipmagic
Posts: 61
Joined: Thu Aug 06, 2015 7:09 am

Re: Kernel Extended Attribute Support on NTFS

Post by zipmagic »

I do wonder how their Xbox app then writes/reads these attributes. Are they using a driver on the system? These are "modern" apps too, so that they can go so low-level while we can't is nothing short of astounding.

Would we basically need to write, and extended-verification code sign an intermediary driver just to be able to backup and restore these KEAs?
zipmagic
Posts: 61
Joined: Thu Aug 06, 2015 7:09 am

Re: Kernel Extended Attribute Support on NTFS

Post by zipmagic »

We're considering building a driver to work around this issue.
We intend for the driver to be open sourced, although it will need to be signed with an EV certificate of course.
Would you support this effort by talking to the driver implementor(s) so they can provide interfaces for wimlib in the easiest way for you to integrate?
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Kernel Extended Attribute Support on NTFS

Post by synchronicity »

It's hard to say without knowing more information.

Do you know if these EAs are *gettable*? If they are gettable, just not settable, then the driver would only need to provide a single fsctl. It would allow setting these EAs.

But I would like to understand *why* Microsoft did not make these EAs settable. Based on https://learn.microsoft.com/en-us/windo ... attributes they are just a performance optimization. Are you sure that not setting these EAs is even the cause of the problem viewtopic.php?p=1416 to begin with?
zipmagic
Posts: 61
Joined: Thu Aug 06, 2015 7:09 am

Re: Kernel Extended Attribute Support on NTFS

Post by zipmagic »

Great news, I've got the first build of the driver saving KEA, a command line tool, and a DLL which all show how to consume it:
setea_poc.7z
(3.51 MiB) Downloaded 62 times
This should solve all remaining WIMLIB file-based imaging issues for all Windows systems that block level tools don't experience.

Would you be able to integrate this with WIMLIB?

Does WIMLIB currently store KEAs in the WIMs it creates as well?
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Kernel Extended Attribute Support on NTFS

Post by synchronicity »

Interesting. Do I understand correctly from your proof-of-concept that getting these EAs already works as-is with NtQueryEaFile(); it's just setting them that doesn't work as-is (with NtSetEaFile()), so your driver adds an ioctl for that purpose? In that case wimlib would be saving these EAs in WIM files already. To set them, set_xattrs() in src/win32_apply.c would need to be modified to use the new ioctl.

I don't plan to implement this myself, for several reasons. E.g. this would require a third-party driver, my questions above about the usefulness and appropriateness of restoring these EAs haven't yet been answered, and to the extent that I find time for wimlib at all these days there are other issues to address. But you're welcome to explore making the changes you need (if you in fact need them).
Post Reply