wimlib
4 years agoUpdate wimmount documentation
Eric Biggers [Sat, 30 May 2015 20:48:19 +0000 (15:48 -0500)]
Update wimmount documentation

4 years agoReparse point updates
Eric Biggers [Sat, 30 May 2015 20:48:18 +0000 (15:48 -0500)]
Reparse point updates

- Rename 'struct reparse_data' => 'struct link_reparse_point'
- Rename parse_reparse_data() => parse_link_reparse_data()
- Rename make_reparse_data() => make_link_reparse_point()
- Rename rp_unknown_1 => unknown_0x54 and move out of reparse point union
- Rename rp_unknown_2 => rp_reserved and store/restore
- Rename not_rpfixed => rp_flags and use 0x0001 as rpfix flag
- Use 'struct reparse_buffer_disk' in more places and not raw byte array
- Add new helper function: complete_reparse_point()
- Eliminate need for various assertions in reparse.c
- Improve handling of NT namespace paths in wim_inode_readlink()
- More closely follow POSIX readlink semantics in wim_inode_readlink()
- Set reparse tag and file attributes in wim_inode_readlink()
- Handle reparse point fixup in wim_inode_readlink() and support fixing
  up link targets in mounted WIM image
- Handle slashes between root and root-relative portion consistently
  between UNIX and Windows, and allow lossless handling of trailing
  slashes
- Use 0 as RP_NOT_FIXED in win32_apply.c
- Improve comments

4 years agoNew helper macro: ALIGN()
Eric Biggers [Sat, 30 May 2015 20:48:16 +0000 (15:48 -0500)]
New helper macro: ALIGN()

4 years agoUse explicit casting in ctype macros
Eric Biggers [Sat, 30 May 2015 20:48:15 +0000 (15:48 -0500)]
Use explicit casting in ctype macros

4 years agosha1.c: annotate load which may be unaligned
Eric Biggers [Sat, 30 May 2015 20:48:14 +0000 (15:48 -0500)]
sha1.c: annotate load which may be unaligned

4 years agoenlarge_blob_table(): remove unnecessary hlist_del()
Eric Biggers [Sat, 30 May 2015 20:48:13 +0000 (15:48 -0500)]
enlarge_blob_table(): remove unnecessary hlist_del()

4 years agoNew helper: wim_reshdr_to_desc_and_blob()
Eric Biggers [Sat, 30 May 2015 20:48:12 +0000 (15:48 -0500)]
New helper: wim_reshdr_to_desc_and_blob()

4 years agowrite.c: do not remove empty blobs (not possible anymore)
Eric Biggers [Sat, 30 May 2015 20:48:11 +0000 (15:48 -0500)]
write.c: do not remove empty blobs (not possible anymore)

4 years agoresource reading cleanups
Eric Biggers [Sat, 30 May 2015 20:48:09 +0000 (15:48 -0500)]
resource reading cleanups

4 years agoAdd GUID helper functions
Eric Biggers [Sat, 30 May 2015 20:48:08 +0000 (15:48 -0500)]
Add GUID helper functions

4 years agoClean up pipable WIM extraction
Eric Biggers [Sat, 30 May 2015 20:48:07 +0000 (15:48 -0500)]
Clean up pipable WIM extraction

4 years agowrite.c: handle empty buffers in write_wim_resource_from_buffer()
Eric Biggers [Sat, 30 May 2015 20:48:06 +0000 (15:48 -0500)]
write.c: handle empty buffers in write_wim_resource_from_buffer()

4 years agoDisallow empty blobs from being read
Eric Biggers [Sat, 30 May 2015 20:48:05 +0000 (15:48 -0500)]
Disallow empty blobs from being read

4 years agoinode.c: do not create blob from zero-sized data buffer
Eric Biggers [Sat, 30 May 2015 20:48:04 +0000 (15:48 -0500)]
inode.c: do not create blob from zero-sized data buffer

Moving forwards towards eliminating zero-size blobs: make
inode_add_stream_with_data() and inode_replace_stream_data() use a NULL
blob pointer for zero-length buffers.  Currently this can happen if a
Windows reparse point has no reparse data or if an empty named data
stream is created in a mounted WIM image.

4 years agoRequire size_in_wim == uncompressed_size for uncompressed resources
Eric Biggers [Sat, 30 May 2015 20:48:03 +0000 (15:48 -0500)]
Require size_in_wim == uncompressed_size for uncompressed resources

It required quite a bit of extra code to handle this case, which really
should just be considered invalid.

4 years agoUse little endian types for 'struct wim_header_disk'
Eric Biggers [Sat, 30 May 2015 20:48:02 +0000 (15:48 -0500)]
Use little endian types for 'struct wim_header_disk'

4 years agoFix tagged item parsing infinite loop when 32-bit and len=0xFFFFFFF8
Eric Biggers [Sat, 30 May 2015 20:48:00 +0000 (15:48 -0500)]
Fix tagged item parsing infinite loop when 32-bit and len=0xFFFFFFF8

4 years agoutil.c: Simplify strdup and wcsdup
Eric Biggers [Sat, 30 May 2015 20:47:59 +0000 (15:47 -0500)]
util.c: Simplify strdup and wcsdup

4 years agoUse "EXIT" instead of "exit" in bash trap statements
Eric Biggers [Sat, 30 May 2015 20:47:58 +0000 (15:47 -0500)]
Use "EXIT" instead of "exit" in bash trap statements

The lower case version does not work in a Turkish locale.  The upper case
version is what the bash docs actually mention and is what bash actually
checks against using strcasecmp().

4 years agov1.8.1 v1.8.1
Eric Biggers [Sun, 17 May 2015 22:30:45 +0000 (17:30 -0500)]
v1.8.1

4 years agowrite.c: Move and fix check for defaulting to solid compression
Eric Biggers [Sun, 17 May 2015 21:47:47 +0000 (16:47 -0500)]
write.c: Move and fix check for defaulting to solid compression

This check should be made higher up, not in write_file_data_blobs().

In addition, it must be checked that WIMLIB_WRITE_FLAG_PIPABLE is not set.

4 years agowimlib-imagex: Adjust wording of extract progress message
Eric Biggers [Sun, 17 May 2015 21:17:08 +0000 (16:17 -0500)]
wimlib-imagex: Adjust wording of extract progress message

4 years agoDisallow image metadata in solid resources
Eric Biggers [Sat, 16 May 2015 15:23:00 +0000 (10:23 -0500)]
Disallow image metadata in solid resources

4 years agowin32_apply: Improved warning messages for WIMBoot extraction
Eric Biggers [Tue, 12 May 2015 04:09:05 +0000 (23:09 -0500)]
win32_apply: Improved warning messages for WIMBoot extraction

4 years agowin32_apply: Stricter checks of resource format for WOF external backing
Eric Biggers [Tue, 12 May 2015 04:07:33 +0000 (23:07 -0500)]
win32_apply: Stricter checks of resource format for WOF external backing

- Only allow resources from WIM files with original file format
- Only allow XPRESS 4K/8K/16K/32K and LZX 32K resources

4 years agowin32_apply: Support multiple source WIM files for WIMBoot extraction
Eric Biggers [Tue, 12 May 2015 01:41:06 +0000 (20:41 -0500)]
win32_apply: Support multiple source WIM files for WIMBoot extraction

4 years agowiminfo: fix string definitions for Windows
Eric Biggers [Sat, 9 May 2015 01:30:55 +0000 (20:30 -0500)]
wiminfo: fix string definitions for Windows

4 years agoencoding.c: Upgrade upcase table and rewrite decompression code
Eric Biggers [Thu, 7 May 2015 02:38:30 +0000 (21:38 -0500)]
encoding.c: Upgrade upcase table and rewrite decompression code

4 years agoAlways use abbreviated integer types in library code
Eric Biggers [Thu, 7 May 2015 01:26:21 +0000 (20:26 -0500)]
Always use abbreviated integer types in library code

4 years agowin32_common.c: Missed place where winnt_error() can be used
Eric Biggers [Thu, 7 May 2015 01:25:17 +0000 (20:25 -0500)]
win32_common.c: Missed place where winnt_error() can be used

4 years agowimlib-imagex-apply.1: abbreviate off-topic docs about booting Windows
Eric Biggers [Tue, 5 May 2015 05:04:30 +0000 (00:04 -0500)]
wimlib-imagex-apply.1: abbreviate off-topic docs about booting Windows

4 years agowiminfo: consolidate boolean flags into single line
Eric Biggers [Tue, 5 May 2015 03:03:53 +0000 (22:03 -0500)]
wiminfo: consolidate boolean flags into single line

4 years agowimlib_wim_info: update docs for flag bits
Eric Biggers [Tue, 5 May 2015 03:19:54 +0000 (22:19 -0500)]
wimlib_wim_info: update docs for flag bits

4 years agowimlib-imagex-capture.1: doc improvements
Eric Biggers [Tue, 5 May 2015 04:42:13 +0000 (23:42 -0500)]
wimlib-imagex-capture.1: doc improvements

4 years agoSwap all slashes in UNIX <=> Windows symlink translation
Eric Biggers [Tue, 5 May 2015 04:12:32 +0000 (23:12 -0500)]
Swap all slashes in UNIX <=> Windows symlink translation

If we swap both slash types when translating each way, then the
translation is lossless.

4 years agoImproved error reporting if loading capture configuration file fails
Eric Biggers [Sun, 3 May 2015 21:35:11 +0000 (16:35 -0500)]
Improved error reporting if loading capture configuration file fails

4 years ago<sys/types.h> is sometimes required before <attr/xattr.h>
Eric Biggers [Sun, 3 May 2015 20:01:58 +0000 (15:01 -0500)]
<sys/types.h> is sometimes required before <attr/xattr.h>

4 years ago<sys/types.h> is required before <sys/sysctl.h> on FreeBSD
Eric Biggers [Sun, 3 May 2015 19:53:10 +0000 (14:53 -0500)]
<sys/types.h> is required before <sys/sysctl.h> on FreeBSD

4 years agoheader.h: fix outdated comment
Eric Biggers [Sun, 3 May 2015 01:55:55 +0000 (20:55 -0500)]
header.h: fix outdated comment

4 years agowrite_wim_header(): remove unnecessary condition
Eric Biggers [Sun, 3 May 2015 01:19:34 +0000 (20:19 -0500)]
write_wim_header(): remove unnecessary condition

4 years agoMove CPU and memory information to util
Eric Biggers [Sat, 2 May 2015 22:54:04 +0000 (17:54 -0500)]
Move CPU and memory information to util

4 years agoMake win32.h includable on non-Windows
Eric Biggers [Sat, 2 May 2015 22:47:07 +0000 (17:47 -0500)]
Make win32.h includable on non-Windows

4 years agowimlib_export_image(): error if src_wim == dest_wim
Eric Biggers [Sat, 2 May 2015 22:40:06 +0000 (17:40 -0500)]
wimlib_export_image(): error if src_wim == dest_wim

4 years agowrite: various cleanups
Eric Biggers [Sat, 2 May 2015 21:54:38 +0000 (16:54 -0500)]
write: various cleanups

- When writing or overwriting WIM file, fill in a new variable
  wim->out_hdr instead of overwriting wim->hdr which was subject to bugs
- When writing WIM file, create new header from scratch instead of
  copying old one
- Remove redundant private flags
- Remove debug statements (they are rarely used, outdated, and now
  missing from most parts of the library)

4 years agomount_image: ensure we always begin with empty file descriptor array
Eric Biggers [Sat, 2 May 2015 22:25:29 +0000 (17:25 -0500)]
mount_image: ensure we always begin with empty file descriptor array

4 years agoxml.c: Fix writing XML data for one image only
Eric Biggers [Sat, 2 May 2015 20:07:50 +0000 (15:07 -0500)]
xml.c: Fix writing XML data for one image only

4 years agowimlib_print_header(): fix printing of magic number
Eric Biggers [Sat, 2 May 2015 20:52:43 +0000 (15:52 -0500)]
wimlib_print_header(): fix printing of magic number

4 years agosha1.h: remove unused inline functions
Eric Biggers [Sat, 2 May 2015 20:40:58 +0000 (15:40 -0500)]
sha1.h: remove unused inline functions

4 years agowim.c: adjust wimlib_compression_type casting
Eric Biggers [Sat, 2 May 2015 19:57:56 +0000 (14:57 -0500)]
wim.c: adjust wimlib_compression_type casting

4 years agoWin32 capture: sort streams by starting LCN
Eric Biggers [Sat, 2 May 2015 18:39:17 +0000 (13:39 -0500)]
Win32 capture: sort streams by starting LCN

4 years agoNTFS-3g capture: use reference-counted NTFS volumes
Eric Biggers [Sat, 2 May 2015 16:35:59 +0000 (11:35 -0500)]
NTFS-3g capture: use reference-counted NTFS volumes

This avoids the hack of storing the NTFS volume in
'struct wim_image_metadata'.

4 years agoNTFS-3g capture: sort attributes by starting LCN
Eric Biggers [Sat, 2 May 2015 15:41:47 +0000 (10:41 -0500)]
NTFS-3g capture: sort attributes by starting LCN

4 years agoNTFS-3g capture: open inodes by inode number
Eric Biggers [Sat, 2 May 2015 15:11:43 +0000 (10:11 -0500)]
NTFS-3g capture: open inodes by inode number

4 years agowim_inode: Remove i_list, leaving only i_hlist
Eric Biggers [Sat, 2 May 2015 02:51:40 +0000 (21:51 -0500)]
wim_inode: Remove i_list, leaving only i_hlist

4 years agoVarious cleanups
Eric Biggers [Fri, 1 May 2015 02:26:32 +0000 (21:26 -0500)]
Various cleanups

4 years agoNew helper: inode_get_unnamed_data_stream()
Eric Biggers [Fri, 1 May 2015 03:53:05 +0000 (22:53 -0500)]
New helper: inode_get_unnamed_data_stream()

4 years agomount_image.c: set st_blocks to compressed size
Eric Biggers [Fri, 1 May 2015 03:45:49 +0000 (22:45 -0500)]
mount_image.c: set st_blocks to compressed size

4 years agoxml.c: Simplify image statistics calculation
Eric Biggers [Fri, 1 May 2015 03:15:05 +0000 (22:15 -0500)]
xml.c: Simplify image statistics calculation

4 years agoUse -D_POSIX_THREAD_SAFE_FUNCTIONS for MinGW build
Eric Biggers [Fri, 1 May 2015 00:38:21 +0000 (19:38 -0500)]
Use -D_POSIX_THREAD_SAFE_FUNCTIONS for MinGW build

The latest MinGW only exposes gmtime_r() if this is defined.

4 years agowimlib_global_init(): check for conflicting case sensitivity mode
Eric Biggers [Tue, 28 Apr 2015 01:54:54 +0000 (20:54 -0500)]
wimlib_global_init(): check for conflicting case sensitivity mode

4 years agoClean up setting of compression type
Eric Biggers [Sat, 25 Apr 2015 20:00:07 +0000 (15:00 -0500)]
Clean up setting of compression type

- Use the enum instead of an int
- Consolidate the per-compression-type information into an array

4 years agowimlib.h: various documentation cleanups and improvements
Eric Biggers [Sat, 25 Apr 2015 00:22:48 +0000 (19:22 -0500)]
wimlib.h: various documentation cleanups and improvements

4 years agontfs-3g_apply.c: use WIMLIB_ERR_SET_REPARSE_DATA
Eric Biggers [Sat, 25 Apr 2015 16:51:03 +0000 (11:51 -0500)]
ntfs-3g_apply.c: use WIMLIB_ERR_SET_REPARSE_DATA

4 years agowimlib_create_new_wim(): check for NULL wim_ret
Eric Biggers [Sat, 25 Apr 2015 15:44:53 +0000 (10:44 -0500)]
wimlib_create_new_wim(): check for NULL wim_ret

4 years agowimlib_get_xml_data(): should use NO_FILENAME error
Eric Biggers [Sat, 25 Apr 2015 06:13:08 +0000 (01:13 -0500)]
wimlib_get_xml_data(): should use NO_FILENAME error

4 years agoSerialize wimlib_global_init() and wimlib_global_cleanup()
Eric Biggers [Sat, 25 Apr 2015 05:09:35 +0000 (00:09 -0500)]
Serialize wimlib_global_init() and wimlib_global_cleanup()

4 years agowimlib_iterate_lookup_table(): include unhashed blobs
Eric Biggers [Sat, 25 Apr 2015 04:56:39 +0000 (23:56 -0500)]
wimlib_iterate_lookup_table(): include unhashed blobs

4 years agoRemove unnecessary argument to hlist iteration macros
Eric Biggers [Sat, 11 Apr 2015 14:38:58 +0000 (09:38 -0500)]
Remove unnecessary argument to hlist iteration macros

4 years agoRemove --enable-verify-compression option
Eric Biggers [Sat, 11 Apr 2015 13:55:27 +0000 (08:55 -0500)]
Remove --enable-verify-compression option

It is easy to do manual verification with 'wimverify', and this was
already broken since "destructive" compression was introduced.

4 years agov1.8.1-BETA
Eric Biggers [Sat, 4 Apr 2015 04:52:24 +0000 (23:52 -0500)]
v1.8.1-BETA

4 years agowin32_capture.c: fix capture of unnamed stream only
Eric Biggers [Sat, 4 Apr 2015 04:52:08 +0000 (23:52 -0500)]
win32_capture.c: fix capture of unnamed stream only

4 years agoinode/blob cleanups
Eric Biggers [Thu, 2 Apr 2015 02:46:38 +0000 (21:46 -0500)]
inode/blob cleanups

4 years agoImprove helper functions for setting blob locations
Eric Biggers [Thu, 2 Apr 2015 04:43:54 +0000 (23:43 -0500)]
Improve helper functions for setting blob locations

4 years agoOptimize layout of 'struct blob_descriptor'
Eric Biggers [Thu, 2 Apr 2015 02:13:13 +0000 (21:13 -0500)]
Optimize layout of 'struct blob_descriptor'

4 years agoTrim down 'flags' member of 'struct blob_descriptor'
Eric Biggers [Thu, 2 Apr 2015 01:43:36 +0000 (20:43 -0500)]
Trim down 'flags' member of 'struct blob_descriptor'

Most "blob" flags were actually only meaningful for WIM resources.
Resource flags are actually saved in the 'flags' member of 'struct
wim_resource_descriptor', and we don't need to duplicate them in 'struct
blob_descriptor'.

4 years agowin32_apply.c: simplify inode_longest_named_data_stream_spec()
Eric Biggers [Sun, 29 Mar 2015 23:32:11 +0000 (18:32 -0500)]
win32_apply.c: simplify inode_longest_named_data_stream_spec()

4 years agolzx_decompress.c: add missing validation in RLE length decoding
Eric Biggers [Fri, 27 Mar 2015 05:16:58 +0000 (00:16 -0500)]
lzx_decompress.c: add missing validation in RLE length decoding

Bad input found by afl-fuzz:

1f 20 00 42 00 00 33 03 45 43 45 00 20 00 00 0e

4 years agontfs-3g_capture.c: fix capture of empty named data stream
Eric Biggers [Fri, 27 Mar 2015 02:00:48 +0000 (21:00 -0500)]
ntfs-3g_capture.c: fix capture of empty named data stream

4 years agotest-imagex-ntfs: fix empty xattr creation
Eric Biggers [Fri, 27 Mar 2015 01:54:53 +0000 (20:54 -0500)]
test-imagex-ntfs: fix empty xattr creation

4 years agotest-imagex-ntfs: remove unusable DOS name tests
Eric Biggers [Fri, 27 Mar 2015 01:54:22 +0000 (20:54 -0500)]
test-imagex-ntfs: remove unusable DOS name tests

4 years agotests: adjust calling of wimlib-imagex
Eric Biggers [Fri, 27 Mar 2015 00:40:05 +0000 (19:40 -0500)]
tests: adjust calling of wimlib-imagex

4 years agowimlib_iterate_dir_tree(): checksum unhashed blobs
Eric Biggers [Thu, 26 Mar 2015 04:32:29 +0000 (23:32 -0500)]
wimlib_iterate_dir_tree(): checksum unhashed blobs

4 years agoinode.c: fix initialization of i_streams
Eric Biggers [Thu, 26 Mar 2015 04:07:55 +0000 (23:07 -0500)]
inode.c: fix initialization of i_streams

4 years agodetect attr/xattr.h for tree-cmp.c
Eric Biggers [Thu, 26 Mar 2015 04:07:20 +0000 (23:07 -0500)]
detect attr/xattr.h for tree-cmp.c

4 years agontfs-3g_apply.c: avoid warning about sacl_offset unused
Eric Biggers [Thu, 26 Mar 2015 00:58:44 +0000 (19:58 -0500)]
ntfs-3g_apply.c: avoid warning about sacl_offset unused

4 years agoAdjust handling of blob reference counts
Eric Biggers [Wed, 25 Mar 2015 01:08:13 +0000 (20:08 -0500)]
Adjust handling of blob reference counts

4 years agomount_image.c: rename create_dentry() to create_file()
Eric Biggers [Wed, 25 Mar 2015 02:23:42 +0000 (21:23 -0500)]
mount_image.c: rename create_dentry() to create_file()

4 years agoLimit exposure of dentry and inode creation
Eric Biggers [Wed, 25 Mar 2015 01:46:01 +0000 (20:46 -0500)]
Limit exposure of dentry and inode creation

- Don't allow creating d_inode=NULL dentries outside of dentry.c
- Don't allow creating i_nlink=0 inodes

4 years agoblob_table.c: fix NTFS location clone error path
Eric Biggers [Tue, 24 Mar 2015 01:47:38 +0000 (20:47 -0500)]
blob_table.c: fix NTFS location clone error path

4 years agoextract.c: file_name_valid(): fix bad memdup
Eric Biggers [Tue, 24 Mar 2015 00:31:15 +0000 (19:31 -0500)]
extract.c: file_name_valid(): fix bad memdup

4 years agoStream and blob updates
Eric Biggers [Tue, 17 Mar 2015 03:17:15 +0000 (22:17 -0500)]
Stream and blob updates

- Rename "lookup table entry" to "blob descriptor"
- Rename "lookup table" to "blob table"
- Use single array for all an inode's streams
- Explicitly annotate each stream with its type
- Account for fact that EFSRPC raw data includes all data streams
- Other cleanups

4 years agoAdd additional reparse point tests to win32-test-imagex-capture_and_apply.bat
Eric Biggers [Sun, 22 Mar 2015 20:44:31 +0000 (15:44 -0500)]
Add additional reparse point tests to win32-test-imagex-capture_and_apply.bat

4 years agowin32_apply.c: Don't externally back very large files
Eric Biggers [Sun, 15 Mar 2015 01:47:39 +0000 (20:47 -0500)]
win32_apply.c: Don't externally back very large files

4 years agowin32_apply.c: try to clear attributes on existing directories
Eric Biggers [Sun, 15 Mar 2015 01:25:53 +0000 (20:25 -0500)]
win32_apply.c: try to clear attributes on existing directories

4 years agowin32_apply.c: handle target directory more like other directories
Eric Biggers [Sun, 15 Mar 2015 01:22:22 +0000 (20:22 -0500)]
win32_apply.c: handle target directory more like other directories

If we don't send the target directory through the same code path as the
other directories, we can run into several inconsistencies such as empty
named data streams not being created and compression attributes not being
set or cleared.

We actually *can* send the target directory through create_directory() as
long as we don't ask for DELETE access (in case it's the current working
directory) or set its short name (since that doesn't make sense anyway).

4 years agowin32-test-imagex-capture_and_apply improvements
Eric Biggers [Sun, 15 Mar 2015 01:11:13 +0000 (20:11 -0500)]
win32-test-imagex-capture_and_apply improvements

- Make test failure message always get printed if anything goes wrong
- Fix bad file comparison
- Apply the WIM image twice to test the existing files case
- Add test for a readonly directory with a named data stream
- Add test for root directory with empty named data stream

4 years agov1.8.0 v1.8.0
Eric Biggers [Wed, 25 Feb 2015 04:55:41 +0000 (22:55 -0600)]
v1.8.0

4 years agoLZMS compression: a few minor cleanups
Eric Biggers [Wed, 25 Feb 2015 04:44:13 +0000 (22:44 -0600)]
LZMS compression: a few minor cleanups

4 years agoLZMS compression: use position instead of pointer for delta hash
Eric Biggers [Tue, 24 Feb 2015 01:23:44 +0000 (19:23 -0600)]
LZMS compression: use position instead of pointer for delta hash

This gives predictable results regardless of how the buffer is aligned.

4 years agolzms_decompress.c: Make bit decoding function names consistent with compressor
Eric Biggers [Tue, 24 Feb 2015 00:08:17 +0000 (18:08 -0600)]
lzms_decompress.c: Make bit decoding function names consistent with compressor