Eric Biggers [Sun, 15 Jan 2017 01:00:13 +0000 (17:00 -0800)]
Don't generate GUID in wimlib_create_new_wim()
It's not necessary to generate a GUID in wimlib_create_new_wim() because
one is generated later by wimlib_write(), and nothing seems to assume
that a WIMStruct not yet backed by a file has a valid GUID. This saves
a call to get_random_bytes(). Also remove some unnecessary
initializations to 0.
Eric Biggers [Sat, 14 Jan 2017 08:56:39 +0000 (00:56 -0800)]
lzx_compress: fix corruption with long literal run
The last round of updates to the LZX compressor made it start being able
to use larger blocks, up to ~100KB. Unfortunately it was overlooked
that this allows literal runs > 65535 bytes while in one place the
length of a literal run was still being stored in a u16. Therefore, on
incompressible input data this could be wrapped around, causing
incorrect compression. Fix this by enlarging the variable.
Eric Biggers [Fri, 13 Jan 2017 04:04:18 +0000 (20:04 -0800)]
wimappend: support --delta-from option
--delta-from was previously only supported by wimcapture. But it is
trivial to make wimappend support it too, and it could be useful.
Eric Biggers [Sun, 8 Jan 2017 06:34:32 +0000 (22:34 -0800)]
hc_matchfinder: use well-defined initialization of best_matchptr
The initial value of best_matchptr is not truly used, but since we do
always compute 'in_next - best_matchptr', assign an initial value which
avoids undefined behavior.
Eric Biggers [Sun, 8 Jan 2017 00:05:21 +0000 (16:05 -0800)]
unix_apply.c: use #ifdef HAVE_XATTR_SUPPORT, not #if HAVE_XATTR_SUPPORT
Eric Biggers [Sat, 7 Jan 2017 23:08:09 +0000 (15:08 -0800)]
extract.c: recommend --unix-data mode when appropriate
Eric Biggers [Sat, 7 Jan 2017 22:35:07 +0000 (14:35 -0800)]
tagged_items: rename inode_set_tagged_data() to inode_set_tagged_item()
(for consistency with inode_get_tagged_item())
Eric Biggers [Sat, 7 Jan 2017 22:35:06 +0000 (14:35 -0800)]
wimdir: make documention correctly reflect that IMAGE is optional
Eric Biggers [Sat, 7 Jan 2017 22:35:06 +0000 (14:35 -0800)]
wimmount.1: mention long forms
Eric Biggers [Sat, 7 Jan 2017 22:35:06 +0000 (14:35 -0800)]
wimlib-imagex: fix recommendation to read PDF documentation
Recommend wimFOO.pdf instead of wimlib-imagex-FOO.pdf. (Also `man
wimFOO` instead of `man wimlib-imagex-FOO'.)
Eric Biggers [Mon, 2 Jan 2017 00:19:25 +0000 (18:19 -0600)]
v1.11.0-BETA5
Eric Biggers [Sun, 1 Jan 2017 23:44:19 +0000 (17:44 -0600)]
wlfuzz: sometimes generate security.capability xattrs
Eric Biggers [Sun, 1 Jan 2017 23:44:19 +0000 (17:44 -0600)]
unix_apply.c: fix applying security xattrs
Eric Biggers [Sun, 1 Jan 2017 23:44:18 +0000 (17:44 -0600)]
Documentation updates, mainly to the man pages
- Use short command names like wimFOO instead of wimlib-imagex FOO
- Lots of other small improvements
Eric Biggers [Wed, 28 Dec 2016 04:36:28 +0000 (22:36 -0600)]
v1.11.0-BETA4
Eric Biggers [Tue, 27 Dec 2016 23:24:56 +0000 (17:24 -0600)]
wlfuzz: generate and compare extended attributes
Eric Biggers [Tue, 27 Dec 2016 23:24:56 +0000 (17:24 -0600)]
wlfuzz: generate and compare standard UNIX file permissions and special files
Eric Biggers [Tue, 27 Dec 2016 23:24:56 +0000 (17:24 -0600)]
wlfuzz: generate and compare symlinks
Eric Biggers [Tue, 27 Dec 2016 23:24:56 +0000 (17:24 -0600)]
wlfuzz: generate and compare object IDs
Eric Biggers [Tue, 27 Dec 2016 23:24:56 +0000 (17:24 -0600)]
wlfuzz: don't generate hard linked reparse points
Eric Biggers [Tue, 27 Dec 2016 23:24:55 +0000 (17:24 -0600)]
wlfuzz: don't generate . and .. filenames
Eric Biggers [Tue, 27 Dec 2016 23:24:55 +0000 (17:24 -0600)]
wlfuzz: compare generated image after just writing it out
Eric Biggers [Tue, 27 Dec 2016 23:24:55 +0000 (17:24 -0600)]
Document xattr support
Eric Biggers [Tue, 27 Dec 2016 23:24:55 +0000 (17:24 -0600)]
unix_apply.c: support applying extended attributes
Eric Biggers [Tue, 27 Dec 2016 23:24:55 +0000 (17:24 -0600)]
unix_capture.c: support capturing extended attributes
Eric Biggers [Tue, 27 Dec 2016 23:24:55 +0000 (17:24 -0600)]
Add new error codes for xattr extraction errors
Eric Biggers [Tue, 27 Dec 2016 23:24:55 +0000 (17:24 -0600)]
Add basic infrastructure for storing xattr items
Define a new tagged metadata item to hold a list of names and values of
Linux-style extended attributes, and prepare for supporting
capture/apply of extended attributes.
I considered making the xattrs a stream instead, referenced from the
tagged item which would just hold a hash. This would have allowed
xattrs to be deduplicated between files. However, I ultimately decided
against this because WIMGAPI and older versions of wimlib would discard
the streams on optimize/export, and extraction would be much more
complicated because xattr streams could come up for extraction before
other streams --- which would be especially problematic for symlinks.
Eric Biggers [Tue, 27 Dec 2016 23:24:55 +0000 (17:24 -0600)]
tagged_items updates
- Expose tagged_item functions in new header tagged_items.h
- Make object_id functions inline functions in object_id.h
- Make inode_get_tagged_item() return stored length, not aligned length
- Add a new function inode_set_tagged_data() which removes existing
items before setting the new one, and use it for inode_set_object_id()
- Make inode_add_tagged_item() append item rather than prepend
- Keep items 8-byte aligned in memory
Eric Biggers [Tue, 27 Dec 2016 03:18:39 +0000 (21:18 -0600)]
.gitignore: ignore files generated by running tests
Eric Biggers [Tue, 27 Dec 2016 02:27:29 +0000 (20:27 -0600)]
Improve random number generation
wimlib used rand() to generate random numbers, e.g. for GUIDs. This was
neither cryptographically secure nor thread-safe. Use getrandom(),
/dev/urandom, or RtlGenRandom() instead.
Eric Biggers [Sun, 18 Dec 2016 09:19:26 +0000 (01:19 -0800)]
Constify some data
Eric Biggers [Sat, 17 Dec 2016 03:47:44 +0000 (19:47 -0800)]
join.c: clean up verify_swm_set()
UBSAN complained when the parts_to_swms array had 0 length. Clean this
up by sorting the parts first, making the verification simpler. Also
don't bother checking compression_type and chunk_size anymore; checking
guid should be sufficient, and it doesn't really matter if the
compression formats are different since now everything will be written
out correctly anyway.
Eric Biggers [Sat, 17 Dec 2016 03:47:44 +0000 (19:47 -0800)]
util.c: check for calloc multiplication overflow
Eric Biggers [Sat, 17 Dec 2016 03:47:44 +0000 (19:47 -0800)]
lcpit_matchfinder.c: update link to libdivsufsort
Eric Biggers [Thu, 15 Dec 2016 04:49:55 +0000 (20:49 -0800)]
wlfuzz: don't generate reserved short names with extension
Eric Biggers [Thu, 15 Dec 2016 04:49:55 +0000 (20:49 -0800)]
wlfuzz: updates for sparse files
Eric Biggers [Thu, 15 Dec 2016 04:49:55 +0000 (20:49 -0800)]
win32-test-imagex-capture_and_apply: sparse file tests
Eric Biggers [Thu, 15 Dec 2016 04:49:55 +0000 (20:49 -0800)]
Extract sparse files as sparse
When extracting a stream belonging to an inode with
FILE_ATTRIBUTE_SPARSE_FILE set, before writing any data, mark the
extracted stream as sparse if needed and skip preallocating space.
Then, skip writing zero regions. This makes it so that sparse files are
still sparse after extraction.
Eric Biggers [Thu, 15 Dec 2016 04:49:55 +0000 (20:49 -0800)]
unix_apply.c: preallocate space for regular files
Eric Biggers [Thu, 15 Dec 2016 04:49:55 +0000 (20:49 -0800)]
unix_capture.c: mark regular files as sparse when appropriate
Eric Biggers [Thu, 15 Dec 2016 04:49:55 +0000 (20:49 -0800)]
resource: pass blob and offset to consume_chunk
This makes it so that users don't need to keep track of the current blob
and offset themselves.
Eric Biggers [Sat, 3 Dec 2016 06:48:23 +0000 (22:48 -0800)]
Improved handling of Data Deduplication reparse points
Eric Biggers [Mon, 28 Nov 2016 03:38:50 +0000 (19:38 -0800)]
v1.11.0-BETA1
Eric Biggers [Mon, 28 Nov 2016 03:19:41 +0000 (19:19 -0800)]
examples: C++ and Windows compatibility
Eric Biggers [Sun, 27 Nov 2016 23:37:21 +0000 (15:37 -0800)]
Include import library and header in Windows releases
Eric Biggers [Sun, 27 Nov 2016 23:24:38 +0000 (15:24 -0800)]
wimlib.h: define bool for old MSVC versions
It's also unnecessary to define bool in C++ mode.
Eric Biggers [Sun, 27 Nov 2016 23:24:37 +0000 (15:24 -0800)]
wimlib.h: include <stdint.h> instead of <inttypes.h>
We only need stdint.h, and old (VS pre-2013) MSVC versions don't have
inttypes.h.
Eric Biggers [Sun, 27 Nov 2016 23:24:37 +0000 (15:24 -0800)]
wimlib.h: test _WIN32 instead of __WIN32__
MSVC defines _WIN32 but not __WIN32__. MinGW defines both.
Eric Biggers [Sun, 27 Nov 2016 23:24:36 +0000 (15:24 -0800)]
wimlib.h: define timespec for MSVC compatibility
Eric Biggers [Sun, 27 Nov 2016 23:24:36 +0000 (15:24 -0800)]
wlfuzz: note Windows restrictions on short names
Eric Biggers [Sun, 27 Nov 2016 23:24:35 +0000 (15:24 -0800)]
wlfuzz: allow short name associated with long name ending with dot or space
Eric Biggers [Sun, 27 Nov 2016 23:24:34 +0000 (15:24 -0800)]
ntfs-3g_apply.c: add message about another known libntfs-3g bug
Eric Biggers [Thu, 17 Nov 2016 07:03:04 +0000 (23:03 -0800)]
README.WINDOWS: document building with release script
Eric Biggers [Thu, 17 Nov 2016 05:32:13 +0000 (21:32 -0800)]
integrity.c: add casts when converting endianness in-place
This fixes some 'sparse' warnings. No change in behavior.
Eric Biggers [Thu, 17 Nov 2016 05:32:12 +0000 (21:32 -0800)]
registry.c: be explicit with le16 => bool conversion
This fixes some 'sparse' warnings. No change in behavior.
Eric Biggers [Thu, 17 Nov 2016 05:32:11 +0000 (21:32 -0800)]
compiler.h: define STATIC_ASSERT() to no-op when checking with sparse
The expression used in STATIC_ASSERT() was not compatible with 'sparse'.
Eric Biggers [Thu, 17 Nov 2016 05:32:10 +0000 (21:32 -0800)]
extract.c: don't use ifdef within macro argument
This was nonstandard and 'sparse' complained about it.
Eric Biggers [Thu, 17 Nov 2016 05:32:10 +0000 (21:32 -0800)]
Add script to run 'sparse' static analyzer
Eric Biggers [Tue, 15 Nov 2016 01:45:11 +0000 (17:45 -0800)]
Makefile.am: include nasm_lt.sh in release tarball
Configuring with --enable-ssse3-sha1 did not work from a release tarball
because build-aux/nasm_lt.sh was missing. Add it.
Eric Biggers [Sat, 8 Oct 2016 02:59:14 +0000 (19:59 -0700)]
mkwinpeimg: use case insensitive mode when updating boot.wim
It was reported that some Windows PE images have a system directory
called "windows" rather than "Windows". Use case insensitive mode to
ensure added files go to the right place.
Eric Biggers [Thu, 15 Sep 2016 03:46:10 +0000 (20:46 -0700)]
util.c: for aligned_malloc, store original pointer directly
Eric Biggers [Thu, 15 Sep 2016 03:32:11 +0000 (20:32 -0700)]
wim.c: langinfo.h no longer needs to be included
Eric Biggers [Sat, 20 Aug 2016 03:13:27 +0000 (20:13 -0700)]
v1.10.0
Eric Biggers [Fri, 19 Aug 2016 07:25:44 +0000 (00:25 -0700)]
registry: avoid negating INT32_MIN
Eric Biggers [Tue, 16 Aug 2016 06:55:52 +0000 (23:55 -0700)]
wimexport: allow exporting to archive directly on block device
Eric Biggers [Tue, 16 Aug 2016 06:55:51 +0000 (23:55 -0700)]
wimapply.1: fix documentation for valid Windows filenames
* "must end with space or period" is not a restriction for NT APIs
* control characters are forbidden
Eric Biggers [Tue, 16 Aug 2016 06:55:50 +0000 (23:55 -0700)]
win32_apply.c: detect case-sensitive filename support
Eric Biggers [Fri, 29 Jul 2016 21:43:57 +0000 (14:43 -0700)]
ntfs-3g_apply.c: open $Secure
Eric Biggers [Wed, 27 Jul 2016 16:30:01 +0000 (09:30 -0700)]
ntfs-3g_capture.c: update $Secure handling
Eric Biggers [Wed, 27 Jul 2016 00:10:05 +0000 (17:10 -0700)]
libattr is no longer needed
wimlib only uses the extended attributes interface on Linux, where it
appears it is now safe to assume the functions are present in libc (see:
http://lists.nongnu.org/archive/html/acl-devel/2012-04/msg00001.html).
Note: the setfattr program from the "attr" package is still required to
run the NTFS-3G test script.
Eric Biggers [Tue, 26 Jul 2016 16:16:43 +0000 (09:16 -0700)]
Sync AVL tree code from project
Eric Biggers [Tue, 26 Jul 2016 02:57:20 +0000 (19:57 -0700)]
avl_tree: remove now-unused support for "unlinked" marker
Eric Biggers [Tue, 26 Jul 2016 02:14:42 +0000 (19:14 -0700)]
Unify case-sensitive and case-insensitive filename indices
Eric Biggers [Tue, 26 Jul 2016 02:14:21 +0000 (19:14 -0700)]
avl_tree: add avl_tree_prev_in_order()
Eric Biggers [Mon, 25 Jul 2016 02:01:09 +0000 (19:01 -0700)]
Add program to report NTFS volume fragmentation
Eric Biggers [Mon, 25 Jul 2016 00:22:09 +0000 (17:22 -0700)]
ntfs-3g_apply.c: note version in which ntfs_attr_pwrite() was fixed
Eric Biggers [Sun, 24 Jul 2016 21:13:17 +0000 (14:13 -0700)]
lzx_compress: make lzx_cost_for_probability() more robust
Eric Biggers [Tue, 19 Jul 2016 16:00:17 +0000 (09:00 -0700)]
lzms_common.c: support x32 ABI in find_next_opcode_sse4_2()
Eric Biggers [Tue, 19 Jul 2016 16:00:17 +0000 (09:00 -0700)]
avl_tree: replace 'AVL_INLINE' with 'forceinline'
Eric Biggers [Tue, 19 Jul 2016 16:00:16 +0000 (09:00 -0700)]
README.WINDOWS: describe 64-bit build by default
Eric Biggers [Tue, 19 Jul 2016 16:00:15 +0000 (09:00 -0700)]
README.WINDOWS: update build instructions for v1.10.0
Eric Biggers [Sat, 9 Jul 2016 17:13:23 +0000 (12:13 -0500)]
configure.ac: Do not check for <sys/param.h>
This header is conditionally included by <ntfs-3g/endians.h>. It defines
too much stuff on certain platforms, e.g. an ALIGN() macro on FreeBSD,
and it appears redundant with other methods of determining the
endianness.
Eric Biggers [Sat, 9 Jul 2016 17:12:14 +0000 (12:12 -0500)]
ntfs-3g_capture.c: include <ntfs-3g/compat.h> to get ENODATA definition
Some platforms, e.g. FreeBSD, do not define ENODATA. On such platforms,
libntfs-3g uses ENOENT instead, and <ntfs-3g/compat.h> defines ENODATA as
ENOENT.
Eric Biggers [Sat, 9 Jul 2016 15:01:27 +0000 (10:01 -0500)]
Update NEWS
Eric Biggers [Sat, 9 Jul 2016 15:01:26 +0000 (10:01 -0500)]
Stop force-inlining everything marked 'inline'
Instead, replace 'inline' with 'forceinline' in selected places.
Eric Biggers [Sat, 9 Jul 2016 15:01:25 +0000 (10:01 -0500)]
bitops: rename bit scan functions
Our bit scan functions use 0-based indices and do not allow zero inputs.
Rename them to 'bsr' and 'bsf' to match the x86 instructions and avoid
confusion with another common convention for 'fls' and 'ffs'.
Eric Biggers [Sat, 9 Jul 2016 15:01:24 +0000 (10:01 -0500)]
xpress_decompress: tweak comment about end-of-block symbol
Eric Biggers [Sat, 9 Jul 2016 15:01:23 +0000 (10:01 -0500)]
decompress_common: move temp space for building decode table to heap
Eric Biggers [Sat, 9 Jul 2016 15:01:22 +0000 (10:01 -0500)]
xpress_decompress: allocate decode table on heap instead of stack
Eric Biggers [Sat, 9 Jul 2016 15:01:22 +0000 (10:01 -0500)]
decompress_common: introduce fast path for lz_copy()
Eric Biggers [Sat, 9 Jul 2016 15:01:21 +0000 (10:01 -0500)]
decompress_common: switch to subtables for Huffman decoding
Eric Biggers [Sat, 9 Jul 2016 15:01:20 +0000 (10:01 -0500)]
lzx_common: add constant for maximum number of extra offset bits
Eric Biggers [Sat, 9 Jul 2016 15:01:19 +0000 (10:01 -0500)]
lzx_common: add constants for minimum aligned offset slot
Eric Biggers [Sat, 9 Jul 2016 15:01:18 +0000 (10:01 -0500)]
lzx_decompress: optimize use of extra offset bits array
Eric Biggers [Sat, 9 Jul 2016 15:01:17 +0000 (10:01 -0500)]
lzx_common: make lzx_offset_slot_base hold unadjusted offsets
Eric Biggers [Sat, 9 Jul 2016 15:01:16 +0000 (10:01 -0500)]
lzx_decompress: decompressor cleanup
Eric Biggers [Sat, 2 Jul 2016 14:05:19 +0000 (09:05 -0500)]
v1.10.0-BETA
Eric Biggers [Sat, 2 Jul 2016 03:50:30 +0000 (22:50 -0500)]
Add program to run compression benchmarks
Eric Biggers [Sat, 2 Jul 2016 03:31:17 +0000 (22:31 -0500)]
LZX compressor improvements
- Improved block splitting algorithm
- Consider gap matches
- Improved default cost initialization
- Other improvements and cleanups
Eric Biggers [Wed, 22 Jun 2016 01:02:00 +0000 (20:02 -0500)]
Introduce ilog2_ceil()