Kernel Extended Attribute Support on NTFS
-
- Site Admin
- Posts: 474
- Joined: Sun Aug 02, 2015 10:31 pm
Re: Kernel Extended Attribute Support on NTFS
By the way, please note that the directions for building wimlib on Windows changed slightly in v1.14.0, and things should be a bit easier than they used to be. Refer to the "Building from source" section of README.WINDOWS.md.
Re: Kernel Extended Attribute Support on NTFS
It's great to hear the build is now simpler.
According to the MS docs I linked to above:
I completely understand your reluctance to do so, at least the time part (if not so much the necessity part - we have by now established, both by communications with MS and our exactly-a-decade by now real world use, that KEAs are absolutely necessary for a 100% accurate imaging operation on Windows).
Rest assured we'll give this a shot and let you know what we find. At that point we may also discuss making the driver itself open source.
According to the MS docs I linked to above:
So the most logical place of insertion would be in your own code.Querying the EAs on a file from user mode returns both normal and Kernel EAs.
I completely understand your reluctance to do so, at least the time part (if not so much the necessity part - we have by now established, both by communications with MS and our exactly-a-decade by now real world use, that KEAs are absolutely necessary for a 100% accurate imaging operation on Windows).
Rest assured we'll give this a shot and let you know what we find. At that point we may also discuss making the driver itself open source.
-
- Posts: 13
- Joined: Sat Dec 16, 2023 10:39 pm
Re: Kernel Extended Attribute Support on NTFS
Kernel EA's *ARE* something that wimlib absolutely should support.
It is good to see this issue being discussed and that there's a sample solution that might work for wimlib, but everything would need to be open source here including the driver for it to be practically workable for many (and I assume to even be something that could be confidently integrated with wimlib).
I ran into this when testing my own software and discovered much as it seems you both have that while kernel EA's can be read from user mode, they can't be set unless you have a driver in kernel mode doing it. That's where I gave up on kernetl EA's. As it seems the wimlib dev did too.
As a sidenote, to aid in your testing, you both may be interested in my free GUI Windows tool that visually can find all EA's and can exclude types you don't want (like non-$Kernel.*) etc. It also can find/report on reparse points and ADS. It's "NTFS Hidden Data Finder" - scroll to the end at https://ssesetup.com/download.html
It is good to see this issue being discussed and that there's a sample solution that might work for wimlib, but everything would need to be open source here including the driver for it to be practically workable for many (and I assume to even be something that could be confidently integrated with wimlib).
I ran into this when testing my own software and discovered much as it seems you both have that while kernel EA's can be read from user mode, they can't be set unless you have a driver in kernel mode doing it. That's where I gave up on kernetl EA's. As it seems the wimlib dev did too.
As a sidenote, to aid in your testing, you both may be interested in my free GUI Windows tool that visually can find all EA's and can exclude types you don't want (like non-$Kernel.*) etc. It also can find/report on reparse points and ADS. It's "NTFS Hidden Data Finder" - scroll to the end at https://ssesetup.com/download.html
Re: Kernel Extended Attribute Support on NTFS
Thanks for the support.
Changing wimlib was easy enough, we're trying to get the driver to load successfully in our Windows RE environment so we may finalize testing.
Right now our log is full of errors related to (K)EA assignment, so it stands to reason that our changes to wimlib were successful.
Changing wimlib was easy enough, we're trying to get the driver to load successfully in our Windows RE environment so we may finalize testing.
Right now our log is full of errors related to (K)EA assignment, so it stands to reason that our changes to wimlib were successful.
Re: Kernel Extended Attribute Support on NTFS
Force loading the driver in test signing mode does not seem to have helped, the log is still full of the same errors.
We'll keep digging.
We'll keep digging.
Re: Kernel Extended Attribute Support on NTFS
Looks like the driver's own test case is failing, so either it is an issue with the WinRE setup, or a bona fide bug in the driver.
We probably are all set on the wimlib side of things, and waiting for the developer to respond.
We probably are all set on the wimlib side of things, and waiting for the developer to respond.
Re: Kernel Extended Attribute Support on NTFS
Actually the issue looks like it reproduces even on a regular Windows installation (with secure boot disabled and test signing enabled), so it is a bona file bug in the driver which can hopefully be fixed in short order.
Re: Kernel Extended Attribute Support on NTFS
Success! The driver has been fixed and it works everywhere ![Smile :)](./images/smilies/icon_e_smile.gif)
Now we're moving to our actual test cases, having verified that KEAs are being retained accurately.
![Smile :)](./images/smilies/icon_e_smile.gif)
Now we're moving to our actual test cases, having verified that KEAs are being retained accurately.
-
- Posts: 13
- Joined: Sat Dec 16, 2023 10:39 pm
Re: Kernel Extended Attribute Support on NTFS
Great, once you do so, I would humbly ask that you see about making that driver open source so that it can benefit wimlib across-the-board and perhaps be integrated by wimlib dev. I think many (myself included) would have concern with implementing a closed-source kernel-level driver with wimlib.
Re: Kernel Extended Attribute Support on NTFS
With a public apology from reboot.pro for their systemic and wanton abuse of our ZIPmagic brand, most specifically I recall one guy called jaclaz or something; we'd be willing to do that.chrislong2 wrote: ↑Fri Feb 02, 2024 11:48 pmGreat, once you do so, I would humbly ask that you see about making that driver open source so that it can benefit wimlib across-the-board and perhaps be integrated by wimlib dev. I think many (myself included) would have concern with implementing a closed-source kernel-level driver with wimlib.