Not honoring directory hard link message

[Skyblue]

Hello,

Archiving a directory holding the rsync snapshot of the operating system with wimlib 1.14 is reporting a lot of "Not honoring directory hard link" messages. Is there a way to fix this issue?

[root@testpc26 ~]# wimcapture --unix-data /mnt/backup temp4.wim --compress=fast --threads=1024
Scanning "/mnt/backup"
[WARNING] Not honoring directory hard link of "/data5/lib"
[WARNING] Not honoring directory hard link of "/data5/etc/init.d"
[WARNING] Not honoring directory hard link of "/data5/etc/yum/pluginconf.d"
[WARNING] Not honoring directory hard link of "/data5/etc/yum/protected.d"
[WARNING] Not honoring directory hard link of "/data5/etc/yum/vars"
[WARNING] Not honoring directory hard link of "/data5/etc/ssl/certs"
[WARNING] Not honoring directory hard link of "/data5/etc/rc5.d"
[WARNING] Not honoring directory hard link of "/data5/etc/X11/fontpath.d/liberation-sans-fonts"
[WARNING] Not honoring directory hard link of "/data5/etc/X11/fontpath.d/liberation-mono-fonts"
[WARNING] Not honoring directory hard link of "/data5/etc/rc1.d"
[WARNING] Not honoring directory hard link of "/data5/etc/rc4.d"
[WARNING] Not honoring directory hard link of "/data5/etc/.java/.systemPrefs"
[WARNING] Not honoring directory hard link of "/data5/etc/swid/swidtags.d/oracle.com"
[WARNING] Not honoring directory hard link of "/data5/etc/rc0.d"
[WARNING] Not honoring directory hard link of "/data5/etc/rc6.d"
[WARNING] Not honoring directory hard link of "/data5/etc/xdg/systemd/user"
[WARNING] Not honoring directory hard link of "/data5/etc/rc2.d"
[WARNING] Not honoring directory hard link of "/data5/etc/rc3.d"
[WARNING] Not honoring directory hard link of "/data5/sbin"
[WARNING] Not honoring directory hard link of "/data5/usr/java/latest"
[WARNING] Not honoring directory hard link of "/data5/usr/java/default"
[WARNING] Not honoring directory hard link of "/data5/usr/java/jre1.8.0_211-amd64/man/ja"
[WARNING] Not honoring directory hard link of "/data5/usr/share/alsa/init"
[WARNING] Not honoring directory hard link of "/data5/usr/share/alsa/ucm2/conf.d/sof-skl_hda_card"
[WARNING] Not honoring directory hard link of "/data5/usr/share/groff/current"
[WARNING] Not honoring directory hard link of "/data5/usr/share/doc/vim-common/docs"
[WARNING] Not honoring directory hard link of "/data5/usr/share/libquvi-scripts/0.9"
[WARNING] Not honoring directory hard link of "/data5/usr/share/gawk"
[WARNING] Not honoring directory hard link of "/data5/usr/lib/firmware/nvidia/ga103/gsp"
.
.
.
[WARNING] Not honoring directory hard link of "/data5/usr/lib/firmware/nvidia/tu104/gsp"
[WARNING] Not honoring directory hard link of "/data5/usr/lib/kbd/keymaps/legacy/ppc"
[WARNING] Not honoring directory hard link of "/data5/usr/lib/debug/lib"
[WARNING] Not honoring directory hard link of "/data5/usr/lib/debug/sbin"
[WARNING] Not honoring directory hard link of "/data5/usr/lib/debug/usr/.dwz"
[WARNING] Not honoring directory hard link of "/data5/usr/lib/debug/lib64"
[WARNING] Not honoring directory hard link of "/data5/usr/lib/debug/bin"
[WARNING] Not honoring directory hard link of "/data5/usr/tmp"
[WARNING] Not honoring directory hard link of "/data5/usr/lib64/firefox/dictionaries"
[WARNING] Not honoring directory hard link of "/data5/usr/lib64/firefox/browser/defaults/preferences"
[WARNING] Not honoring directory hard link of "/data5/usr/libexec/gawk"
[WARNING] Not honoring directory hard link of "/data5/lib64"
[WARNING] Not honoring directory hard link of "/data5/bin"
[WARNING] Not honoring directory hard link of "/data5/var/mail"
[WARNING] Not honoring directory hard link of "/data5/var/run"

In the list above, /var, /usr, /llib64 etc. are the traditional directories of a Linux system.

Operating system : Oracle Linux 8.10

[synchronicity]

This indicates that the filesystem on that system does not provide unique inode numbers for directories, making it seem like directory hard links are present. What filesystem type are you are using? Presumably it's not something normal like ext4, but rather something more exotic?

[Skyblue]

Hi synchronicity,

The file system is the traditional ext4 :

Code: Select all

[root@testpc26 ~]# mount | grep nvme0n1
/dev/nvme0n1p3 on / type ext4 (rw,relatime)
/dev/nvme0n1p2 on /boot type ext4 (rw,relatime)
/dev/nvme0n1p4 on /home type ext4 (rw,relatime)
/dev/nvme0n1p1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro)
/dev/nvme0n1p6 on /mnt/backup type ext4 (rw,relatime)

[synchronicity]

Are there any bind mounts in /mnt/backup?

[Skyblue]

Hi synchronicity,

No any bind mounts. The well known configuration :

Code: Select all

mount /dev/nvme0n1p6 /mnt/backup

[synchronicity]

I'm puzzled, then! Either the filesystem is reporting duplicate inode numbers (despite being ext4), or something is really going wrong in wimlib. Could you try applying this patch which adds an additional message for debugging?

[Skyblue]

Hello synchronicity,

A quick correction, the operating system is Oracle Linux 8.10

Here are the results.

Creating the test environment :

Code: Select all

[root@testpc26 ~]# rsync -aAXHv / --delete --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} /mnt/backup/archive0


[root@testpc26 ~]# echo test > test1.txt
[root@testpc26 ~]# rsync -aAXHv --delete --link-dest /mnt/backup/archive0 --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} / /mnt/backup/archive1
sending incremental file list
created directory /mnt/backup/archive1
root/
root/test1.txt
.
.

sent 13,699,669 bytes  received 23,444 bytes  2,111,248.15 bytes/sec
total size is 9,295,471,084  speedup is 677.36
[root@testpc26 ~]# echo test2 > test2.txt
[root@testpc26 ~]# rsync -aAXHv --delete --link-dest /mnt/backup/archive1 --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} / /mnt/backup/archive2
sending incremental file list
created directory /mnt/backup/archive2
root/
root/test2.txt
root/.cache/tracker/meta.db
.
.

sent 13,700,148 bytes  received 23,453 bytes  2,111,323.23 bytes/sec
total size is 9,295,471,533  speedup is 677.33

Code: Select all

[root@testpc26 ~]# ls /mnt/backup/
archive0  archive1  archive2  lost+found

Capturing the directory with the recompiled wimlib including the patched source file :

Code: Select all

[root@testpc26 ~]# wimcapture --unix-data /mnt/backup/ test.wim --compress=fast --threads=512
Scanning "/mnt/backup/"
[WARNING] Not honoring directory hard link of "/archive0/etc/ssl/certs"
[WARNING] name=certs ino=263330 devno=66310 noshare=0
[WARNING] Not honoring directory hard link of "/archive0/etc/rc6.d"
[WARNING] name=rc6.d ino=262194 devno=66310 noshare=0
[WARNING] Not honoring directory hard link of "/archive0/etc/rc2.d"
[WARNING] name=rc2.d ino=262190 devno=66310 noshare=0
[WARNING] Not honoring directory hard link of "/archive0/etc/rc0.d"
[WARNING] name=rc0.d ino=262188 devno=66310 noshare=0
[WARNING] Not honoring directory hard link of "/archive0/etc/alternatives/jre_11"
[WARNING] name=jre_11 ino=262377 devno=66310 noshare=0
[WARNING] Not honoring directory hard link of "/archive0/etc/alternatives/jre_openjdk"
[WARNING] name=jre_openjdk ino=262379 devno=66310 noshare=0
[WARNING] Not honoring directory hard link of "/archive0/etc/alternatives/jre_11_openjdk"
[WARNING] name=jre_11_openjdk ino=262378 devno=66310 noshare=0
.
.
.
long list

[Skyblue]

Attached is the complete log file.

[synchronicity]

Thanks! I might know what's causing this. Are all these files symlinks that are hard-linked to each other? If you run 'find /mnt/backup -inum 263330', for example, I think that 4 results will be found, and they will all be symlinks with the same target. Is that the case? If so, that means that your filesystem contains hard-linked symlinks, i.e. symlinks that share the same inode.

Unfortunately the WIM file format treats symlinks and hard links as mutually exclusive, so this case cannot actually be represented in WIM. So what wimlib does is "un-hard-link" the symlinks and print warnings. That actually seems like the correct behavior, though the warning message should be made clearer. It maybe should be a single message at the end like "Storing N hard-linked symlinks as separate inodes, since WIM does not support hard-linked symlinks".

[Skyblue]

Hello synchronicity,

All the files in /mnt/backup are hard-linked. The snapshot technique :

https://digitalis.io/blog/linux/increme ... ard-links/

Output of the find command :

Code: Select all

[root@testpc26 ~]# find /mnt/backup -inum 263330
/mnt/backup/archive0/etc/ssl/certs
/mnt/backup/archive1/etc/ssl/certs
/mnt/backup/archive2/etc/ssl/certs

3 lines.

It looks like that the hard-linked are sharing the same inode number.

The message "Storing N hard-linked symlinks as separate inodes," is a good idea.

I tested the backup after removing all the subdirectories of /mnt/backup :

Code: Select all

[root@testpc26 ~]# df -h | grep "/mnt/backup"
/dev/nvme0n1p6   10G  9,0G  521M   95% /mnt/backup

[root@testpc26 ~]# rm -rf /mnt/backup/archive2
[root@testpc26 ~]# rm -rf /mnt/backup/archive1
[root@testpc26 ~]# rm -rf /mnt/backup/archive0

[root@testpc26 ~]# wimapply --unix-data test.wim /mnt/backup/
Applying image 1 ("backup") from "/root/test.wim" to directory "/mnt/backup/"
Creating files: 70915 of 70915 (100%) done
Extracting file data: 8561 MiB of 8561 MiB (100%) done
Applying metadata to files: 65885 of 65885 (100%) done
Done applying WIM image.

[root@testpc26 ~]# df -h | grep "/mnt/backup"
/dev/nvme0n1p6   10G  9,0G  485M   95% /mnt/backup

wimlib restored the files without any issues.