Is it possiable to capture FAT/FAT32 filesystem with file attributes?

[Jobcrazy]

Thanks for synchronicity's work.

I've wrote a program with wimlib and run it on linux, to backup and restore Windows OS. It can capture and apply ntfs fs with file attributes( such as: hidden, read-only, etc )

Is there a way to capture and apply FAT/FAT32 filesystem with these file attributes with wimlib on Linux?

[synchronicity]

No.

[techryda]

While I'm usually imaging/deploying EXT4 and NTFS filesystems, many system repair / recovery / BIOS flashing partitions are created as VFAT/FAT32.

I was wondering if there was a technical reason for not supporting file attributes (Archive, Readonly, System & Hidden) on VFAT? It would be nice to be able to use (without attribute data loss) wimlib for this dated but flexible and widespread filesystem.

I realize that I can always just tar/xz a VFAT filesystem but I think it would be nice to be able to use one tool to backup and deploy all of my popular filesystems (and VFAT/FAT32 would come in 3rd on my list of popular filesystems). I'd also miss out on the single file instance per archive feature of WIM files.

Thanks again for your work on wimlib!

-techryda

[synchronicity]

On Linux this is possible with the FAT_IOCTL_GET_ATTRIBUTES and FAT_IOCTL_SET_ATTRIBUTES ioctls, assuming that you've mounted your FAT filesystem using the filesystem driver included with the Linux kernel.

I implemented this and pushed it to the branch "linux_fat_support" in the git repository (git://wimlib.net/wimlib). The new code also knows that FAT filesystems don't support symbolic links, hard links, and case sensitive filenames and will handle the extraction accordingly.

I'm not sure this is really worth adding, but feel free to try it out.

[techryda]

Thanks, I'll try it out!

Related question (I don't have a wim handy to test this): Does wimlib store info regarding the source filesystem? ...and does wiminfo display this info, if stored? This isn't a feature request, as I can just put this info in the Description field if I need it.

Thanks again.

-TechRyda

[synchronicity]

If you're creating a WIM image from the filesystem of a Windows operating system, then wimlib will automatically populate XML information such as the Windows version details. This is new in the 1.9.0 betas. Otherwise, no XML information is automatically populated other than possibly the image name (wimlib-imagex defaults it to the source directory) and generic elements such as FILECOUNT.

Users can also define custom XML elements when creating a WIM image.

However, 'wiminfo' only knows about predefined elements when it formats its default output.

[chungy]

I think it'd be nice if this was merged into the master branch. There's no really good way to backup FAT with metadata from Linux[1], except for partclone... which has issues all on its own like no deduplication for multiple images and basically being just a direct block copy of the volume minus the blocks not used by the file system (making it inconvenient/difficult to browse the backup and extract files, and making it difficult to restore onto a smaller volume than the original).

As a bit of a fun test, I have Windows 95 installed in QEMU, and I used the linux_fat_support branch to backup the file system. I made a new hard disk image, half the size, applied it, all the attributes are preserved wonderfully. Just two ms-sys passes to install the compatible MBR and PBR, and the operating system booted on the new disk, and appropriately showed the smaller size. Granted that backing up Windows 9x using WIM might not exactly be a usual use case, I still find it rather nice to be able to do this.

On an aside, it looks like Windows 9x isn't really all that picky about attributes. I tested the apply with the normal version of wimlib, and Windows 95 still fully boots even though none of the hidden or system attributes were preserved. Still, I would find it worthy, especially for keeping the hidden file attribute.

[1] If Linux exposed the attributes via xattrs instead of an ioctl, the problem would automatically resolve itself as tools like tar and zip would need no modification... though wimlib would still have no support. Sadly the kernel devs decided against the xattr approach...

[synchronicity]

Good to see you found the code useful. However, there are a few reasons why I am a little hesitant to merge it into the master branch.

* The proposed code would live alongside the "generic" UNIX capture/apply code, but really would be specific not only to Linux but to a specific filesystem; this makes the code harder to maintain.
* When scanning any directory tree on Linux, every file would potentially have to be opened an extra time to execute the FAT_GET_ATTRIBUTES ioctl, even if no files are on a FAT filesystem (since you don't know when you might get to a mountpoint that is FAT; I suppose you could look at /proc/mtab, but that adds another layer of complexity).
* Attributes are not the only special metadata used by FAT; another is short filenames. Although the FAT driver in the Linux kernel provides an ioctl to list the short filenames in a directory, it provides no ioctl to set the short name of a file. This means that lossless backup and restore of short names is not possible when working through the FAT driver in the Linux kernel.

Personally I feel the lack of short name support would be the largest problem. Though short filenames are increasingly outdated, FAT filesystems are exactly the place where you would expect to find situations in which they are still used.

NTFS also suffered from similar problems (with short names as well as various other types of metadata); the solution for that was to use libntfs-3g directly rather than rely on a mounted filesystem. I am not aware of a similar library for FAT, however.

[chungy]

Thanks for the insight. I hadn't even thought of the short names issue. I went back and compared the Windows 95 status between the original install and the re-applied image, mounting them both with "-t msdos" so only the short names are visible in the VFS. Indeed, a few of the short names differed from the original install. It didn't seem to affect the operation of a plain W95 install, but I do recall issues back in the 90s related to short names changing between backups and restores, such as a program that decides to store a path to "C:\PROGRA~1" instead of "C:\Program Files" and then following a restore, that same directory becomes "C:\PROGRA~2"... poof, things broke.

Off-hand I'm only aware of TFFS for a FAT library, but it looks like it was abandoned shortly after being started over 7 years ago (and evidenced by still being on Google Code). I do agree that this would be the sensible approach (and would make it consistent with capturing NTFS).

Anyhow, thanks again for the feedback, I totally understand. If it's not too much to ask, keeping the current branch around would be convenient at least for this unusual use-case, unless/until a saner approach goes into master.