Page 1 of 1

Folders and security descriptors without recursive directory iteration

Posted: Thu May 03, 2018 12:02 pm
by hastbacr
Hi,

I plan to use this excellent library to store and compress files and folders for my backup software. My program has a list of files and folder to capture. I don't want wimlib to iterate recursively thru directories as it does by default, when wimlib_update_image is used with a folder as source.

The problem is that, if only files are captured the attributes and security descriptors of the parent folder is not captured to the wim file.

If a directory is captured the attributes and security descriptors are captured for the directory, but then wimlib will capture all sub files and directories, so I can't choose what files to capture to the directory.

Does anybody know how to capture selected files and folder with the folders attributes and security descriptors preserved?

Re: Folders and security descriptors without recursive directory iteration

Posted: Fri May 04, 2018 2:28 am
by synchronicity
Since you're already using the API, you can control which files are included by providing WIMLIB_ADD_FLAG_TEST_FILE_EXCLUSION in the add_flags, then making your progress function handle WIMLIB_PROGRESS_MSG_TEST_FILE_EXCLUSION. Set will_exclude=true to all files and directories you don't want.

An alternative, less flexible way is to provide a capture configuration file to wimlib_add_image() or the wimlib_add_command. Though, it's really designed more about exclusions than inclusions, so you'd have to first exclude everything, then except (include) just the files you want:

Code: Select all

[ExclusionList]
* 

[ExclusionException]
/file1
/file2
/subdir/file3
/subdir/file4
But, that use case doesn't actually work as expected because "/subdir" is still technically excluded, so the files in it are never seen, and end up excluded too. I think I should fix it so that excepting a path also excepts its ancestor directories (but not their other contents)...

Re: Folders and security descriptors without recursive directory iteration

Posted: Wed May 16, 2018 10:18 am
by hastbacr
Thanx for your answer,

I have been looking in to WIMLIB_ADD_FLAG_TEST_FILE_EXCLUSION and WIMLIB_PROGRESS_MSG_TEST_FILE_EXCLUSION. And yes, that would be possible to exclude files that is not on my file list. But it would also generate quite an overhead to search the list for every file and check if it needs to be excluded. That is if the files are many (like a full hard drive). It's doable with a clever search function that sorts by subdirectories, but it won't ever be really fast.

I'm looking forward for a WIMLIB_ADD_FLAG_DONT_SCAN_SUBDIRECTORIES flag :D That would help a lot.

Re: Folders and security descriptors without recursive directory iteration

Posted: Fri May 18, 2018 6:57 am
by synchronicity
Well, something still has to actually check whether each file is excluded or included. wimlib currently just checks every ExclusionList and ExclusionException pattern for each file, which has worked until now as people normally use small lists. It's on my (increasingly long) TODO list to make it more scalable.

Re: Folders and security descriptors without recursive directory iteration

Posted: Sat Jul 21, 2018 5:48 pm
by synchronicity
Regarding my earlier comment:
But, that use case doesn't actually work as expected because "/subdir" is still technically excluded, so the files in it are never seen, and end up excluded too. I think I should fix it so that excepting a path also excepts its ancestor directories (but not their other contents)...
This now works in wimlib-1.13.0-BETA3.