backup live systems

Comments, questions, bug reports, etc.
bliblubli
Posts: 88
Joined: Thu Dec 31, 2015 10:45 am

Re: backup live systems

Post by bliblubli »

I also checked the difference in file count. Windows explorer reports 463 000 files, Wimlib scanned 441475. I have a lot of Hardlinks and junctions on the drive. It can't be due to the junctions as they are for folders containing 90 000 files, so the difference would be much higher. So is it due to X hardlinks being reported as 1 file?

Edit: it seems I have my answer. When extracting the wim, I get about the same file number as on the original. I think it would make sense to report numbers comparable to the one explorer does to avoid misunderstanding. You could put the actual number wimlib processes between brackets or the other way round, but I think many people will be disturbed when 2 program report different values. Specifically when it's about data safety.

I know have to test if it's bootable when extracting back :) A really great tool you do!
synchronicity
Site Admin
Posts: 474
Joined: Sun Aug 02, 2015 10:31 pm

Re: backup live systems

Post by synchronicity »

Yes, when reporting scan progress wimlib counts a file just once, no matter how many names (hard links) it happens to have. This is technically the correct way.

I'm not sure what's so special about Explorer. The reality is that different programs count files in different ways as there are corner cases in which there exist legitimate differences of opinion or accidental differences. In addition to the hard link question, there are various others:

* Should the root directory be counted?
* Should symbolic links be counted as files? What about "directory" symbolic links and junctions on NTFS?
* Are other types of NTFS reparse points "files"? What if the reparse point is set on a directory --- then should it be counted as a directory?
* Should NTFS named streams be counted as separate files?
* Should files with the "hidden" attribute be counted?
* What about directories with ACLs that forbid access? On Windows, some backup programs like wimlib can access all files regardless of ACL when running with sufficient privileges, but that is not true of file browsers like Explorer, which sometimes are unable to see that some files even exist.
* What about files that are only accessible via paths > MAX_PATH in length on Windows? Again, some file archiving programs such as wimlib can access such files, but that is not true of many programs written for Windows, including Explorer.

So even if I attempted to replicate Explorer exactly, the numbers still wouldn't always match because sometimes Explorer is wrong. And other programs would have different counts.

I don't think you can use file counts for anything other than a rough guide, unless taken in a context in which things are more precisely defined.
bliblubli
Posts: 88
Joined: Thu Dec 31, 2015 10:45 am

Re: backup live systems

Post by bliblubli »

Yep, sorry. I forgot that even in 2015, explorer still doesn't use an API for long path names that was created in 2002. 13 years after, microsoft still doesn't support it's own API :D
Ok, you are right. Then I'll try to do a bit-comparison of the 2 folders (original and extracted one)
Vortex
Posts: 3
Joined: Sat Jan 02, 2016 9:09 am

Re: backup live systems

Post by Vortex »

Hi synchronicity,

Thanks for the new release. It works fine on XP. By the way, there are still a lot of XP installations used by companies. I will continue to test the new VSS feature.
bliblubli
Posts: 88
Joined: Thu Dec 31, 2015 10:45 am

Re: backup live systems

Post by bliblubli »

Have you some good way to test the extraction of a wim archive, while protecting the system? I just tried to extract the live system I captured in a directory and now I have crazy file that can't be deleted.
files extracted and not deletable
files extracted and not deletable
wtf.PNG (5.64 KiB) Viewed 14825 times
synchronicity
Site Admin
Posts: 474
Joined: Sun Aug 02, 2015 10:31 pm

Re: backup live systems

Post by synchronicity »

By design, wimlib can extract files that may be difficult to delete (e.g. restrictive ACLs, strange names, or long paths).

Are these correct files which existed originally and appear in the WIM directory listing?
bliblubli
Posts: 88
Joined: Thu Dec 31, 2015 10:45 am

Re: backup live systems

Post by bliblubli »

Yes, they were in the original drive. Just discovered them with this. It's a copy protection software. So my games should work when I restore it.
synchronicity
Site Admin
Posts: 474
Joined: Sun Aug 02, 2015 10:31 pm

Re: backup live systems

Post by synchronicity »

@bliblubli

After thinking about it, I did decide to make some tweaks the the way wimlib-imagex displays progress information.

The number of files reported now counts all hard links. Although not technically correct, this makes the counts consistent with the counts that wimlib saves in the FILECOUNT and DIRCOUNT elements of the XML document. I think it's also what the majority of users will expect --- even though as I described previously, it's not always realistic to expect all programs to come up with *exactly* the same numbers (there can still be issues with reparse points, named streams, etc.).

I also tweaked the write progress message to not include the word "uncompressed" if data is being written compressed.

The changes are in 1.9.0-BETA6.

Thanks for the feedback!
bliblubli
Posts: 88
Joined: Thu Dec 31, 2015 10:45 am

Re: backup live systems

Post by bliblubli »

@synchronicity

Nice. Those little tweaks can really help people check that everything went well :)
Post Reply