Folders and security descriptors without recursive directory iteration

Comments, questions, bug reports, etc.
Post Reply
hastbacr
Posts: 2
Joined: Thu May 03, 2018 8:12 am

Folders and security descriptors without recursive directory iteration

Post by hastbacr »

Hi,

I plan to use this excellent library to store and compress files and folders for my backup software. My program has a list of files and folder to capture. I don't want wimlib to iterate recursively thru directories as it does by default, when wimlib_update_image is used with a folder as source.

The problem is that, if only files are captured the attributes and security descriptors of the parent folder is not captured to the wim file.

If a directory is captured the attributes and security descriptors are captured for the directory, but then wimlib will capture all sub files and directories, so I can't choose what files to capture to the directory.

Does anybody know how to capture selected files and folder with the folders attributes and security descriptors preserved?
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Folders and security descriptors without recursive directory iteration

Post by synchronicity »

Since you're already using the API, you can control which files are included by providing WIMLIB_ADD_FLAG_TEST_FILE_EXCLUSION in the add_flags, then making your progress function handle WIMLIB_PROGRESS_MSG_TEST_FILE_EXCLUSION. Set will_exclude=true to all files and directories you don't want.

An alternative, less flexible way is to provide a capture configuration file to wimlib_add_image() or the wimlib_add_command. Though, it's really designed more about exclusions than inclusions, so you'd have to first exclude everything, then except (include) just the files you want:

Code: Select all

[ExclusionList]
* 

[ExclusionException]
/file1
/file2
/subdir/file3
/subdir/file4
But, that use case doesn't actually work as expected because "/subdir" is still technically excluded, so the files in it are never seen, and end up excluded too. I think I should fix it so that excepting a path also excepts its ancestor directories (but not their other contents)...
hastbacr
Posts: 2
Joined: Thu May 03, 2018 8:12 am

Re: Folders and security descriptors without recursive directory iteration

Post by hastbacr »

Thanx for your answer,

I have been looking in to WIMLIB_ADD_FLAG_TEST_FILE_EXCLUSION and WIMLIB_PROGRESS_MSG_TEST_FILE_EXCLUSION. And yes, that would be possible to exclude files that is not on my file list. But it would also generate quite an overhead to search the list for every file and check if it needs to be excluded. That is if the files are many (like a full hard drive). It's doable with a clever search function that sorts by subdirectories, but it won't ever be really fast.

I'm looking forward for a WIMLIB_ADD_FLAG_DONT_SCAN_SUBDIRECTORIES flag :D That would help a lot.
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Folders and security descriptors without recursive directory iteration

Post by synchronicity »

Well, something still has to actually check whether each file is excluded or included. wimlib currently just checks every ExclusionList and ExclusionException pattern for each file, which has worked until now as people normally use small lists. It's on my (increasingly long) TODO list to make it more scalable.
synchronicity
Site Admin
Posts: 472
Joined: Sun Aug 02, 2015 10:31 pm

Re: Folders and security descriptors without recursive directory iteration

Post by synchronicity »

Regarding my earlier comment:
But, that use case doesn't actually work as expected because "/subdir" is still technically excluded, so the files in it are never seen, and end up excluded too. I think I should fix it so that excepting a path also excepts its ancestor directories (but not their other contents)...
This now works in wimlib-1.13.0-BETA3.
Post Reply