wimlib
3 years agoAdd support for unsafe in-place compaction of WIM files
Eric Biggers [Sun, 4 Oct 2015 18:20:19 +0000 (13:20 -0500)]
Add support for unsafe in-place compaction of WIM files

This commit adds support for compacting WIM files by passing the flag
WIMLIB_WRITE_FLAG_UNSAFE_COMPACT to wimlib_overwrite().

3 years agoWrite raw-copy resources before other resources
Eric Biggers [Sun, 4 Oct 2015 18:23:42 +0000 (13:23 -0500)]
Write raw-copy resources before other resources

As another preparation for adding compaction support, always write "raw
copy" resources before other resources.  In the non-compaction case, any
order can be used.

3 years agoUpdates to prepare for in-place compaction support
Eric Biggers [Sun, 4 Oct 2015 18:15:59 +0000 (13:15 -0500)]
Updates to prepare for in-place compaction support

Update some names and comments to clarify they mean overwrites via
append.  No changes in behavior.

3 years agowrite.c: cleanups and bug fixes
Eric Biggers [Sun, 4 Oct 2015 18:08:34 +0000 (13:08 -0500)]
write.c: cleanups and bug fixes

- Sort blobs correctly if a single on-disk WIM has multiple WIMStructs
- Prevent non-solid resources from being incorrectly reused solid WIMs
- Rename num_bytes_to_compress => num_nonraw_bytes
- Make may_*_filter_blobs() functions return bool
- Improved some comments

3 years agox86_cpu_features.c: include "config.h"
Eric Biggers [Sun, 4 Oct 2015 01:53:56 +0000 (20:53 -0500)]
x86_cpu_features.c: include "config.h"

3 years agoMark system compression support non-experimental
Eric Biggers [Sat, 3 Oct 2015 23:47:20 +0000 (18:47 -0500)]
Mark system compression support non-experimental

3 years agoMinor tweaks to recently changed code
Eric Biggers [Sat, 3 Oct 2015 22:27:04 +0000 (17:27 -0500)]
Minor tweaks to recently changed code

3 years agowimlib-imagex: don't do "metadata not found" warning if applying from stdin
Eric Biggers [Sat, 3 Oct 2015 20:46:35 +0000 (15:46 -0500)]
wimlib-imagex: don't do "metadata not found" warning if applying from stdin

3 years agoUpdate NEWS
Eric Biggers [Sat, 3 Oct 2015 17:44:04 +0000 (12:44 -0500)]
Update NEWS

3 years agoAdd WIMLIB_ERR_WIM_IS_INCOMPLETE
Eric Biggers [Sat, 3 Oct 2015 19:56:32 +0000 (14:56 -0500)]
Add WIMLIB_ERR_WIM_IS_INCOMPLETE

Usually WIMLIB_ERR_XML was returned in this situation, but a separate
error code is more appropriate.

3 years agoRemove WIMLIB_ERR_LIBXML_UTF16_HANDLER_NOT_AVAILABLE
Eric Biggers [Sat, 3 Oct 2015 19:34:22 +0000 (14:34 -0500)]
Remove WIMLIB_ERR_LIBXML_UTF16_HANDLER_NOT_AVAILABLE

3 years agoAdd wimlib_get_image_property() and wimlib_set_image_property()
Eric Biggers [Sat, 3 Oct 2015 17:42:35 +0000 (12:42 -0500)]
Add wimlib_get_image_property() and wimlib_set_image_property()

3 years agoxml.c: refactor to use document tree
Eric Biggers [Sat, 3 Oct 2015 16:59:49 +0000 (11:59 -0500)]
xml.c: refactor to use document tree

3 years agoUpdate COMPRESSION RATIO section of README
Eric Biggers [Thu, 24 Sep 2015 06:33:19 +0000 (01:33 -0500)]
Update COMPRESSION RATIO section of README

3 years agobt_matchfinder: add BT_MATCHFINDER_REQUIRED_NBYTES
Eric Biggers [Fri, 25 Sep 2015 03:46:48 +0000 (22:46 -0500)]
bt_matchfinder: add BT_MATCHFINDER_REQUIRED_NBYTES

3 years agolcpit_matchfinder: prefetch multiple steps ahead
Eric Biggers [Mon, 21 Sep 2015 00:04:10 +0000 (19:04 -0500)]
lcpit_matchfinder: prefetch multiple steps ahead

3 years agolzx_compress: update lzx_update_costs() with codeword limits
Eric Biggers [Sun, 20 Sep 2015 21:01:02 +0000 (16:01 -0500)]
lzx_compress: update lzx_update_costs() with codeword limits

3 years agolzx_compress.c: adjust parameters for slightly stronger compression
Eric Biggers [Sat, 19 Sep 2015 21:29:38 +0000 (16:29 -0500)]
lzx_compress.c: adjust parameters for slightly stronger compression

3 years agoxpress_compress.c: adjust parameters for slightly stronger compression
Eric Biggers [Sat, 19 Sep 2015 21:29:33 +0000 (16:29 -0500)]
xpress_compress.c: adjust parameters for slightly stronger compression

3 years agolzms_decompress() optimization
Eric Biggers [Sat, 19 Sep 2015 18:56:18 +0000 (13:56 -0500)]
lzms_decompress() optimization

The four "pending" variables can be replaced with a single
'prev_item_type' variable, with adjusted logic.  The new way is a little
faster.

3 years agolzx_common: rename to lzx_preprocess/lzx_postprocess
Eric Biggers [Sat, 19 Sep 2015 20:06:37 +0000 (15:06 -0500)]
lzx_common: rename to lzx_preprocess/lzx_postprocess

3 years agolzx_common: get rid of unnecessary functions
Eric Biggers [Sat, 19 Sep 2015 19:53:43 +0000 (14:53 -0500)]
lzx_common: get rid of unnecessary functions

3 years agobt_matchfinder: use 4-byte hashing for trees
Eric Biggers [Sat, 19 Sep 2015 18:56:14 +0000 (13:56 -0500)]
bt_matchfinder: use 4-byte hashing for trees

3 years agobt_matchfinder: make callers do max_len check
Eric Biggers [Sat, 19 Sep 2015 18:56:14 +0000 (13:56 -0500)]
bt_matchfinder: make callers do max_len check

3 years agolzx_compress.c: optimize aligned cost consideration and enable by default
Eric Biggers [Sat, 19 Sep 2015 18:56:13 +0000 (13:56 -0500)]
lzx_compress.c: optimize aligned cost consideration and enable by default

3 years agoxpress_compress.c: rename a few variables
Eric Biggers [Sat, 19 Sep 2015 18:56:12 +0000 (13:56 -0500)]
xpress_compress.c: rename a few variables

3 years agolz_hash.h: remove unused code
Eric Biggers [Sat, 19 Sep 2015 18:56:11 +0000 (13:56 -0500)]
lz_hash.h: remove unused code

3 years agoRename 'pos_t' to 'mf_pos_t'
Eric Biggers [Sat, 19 Sep 2015 18:56:10 +0000 (13:56 -0500)]
Rename 'pos_t' to 'mf_pos_t'

3 years agoRemove LZX_ASSERT() (now almost unused)
Eric Biggers [Sat, 19 Sep 2015 18:56:10 +0000 (13:56 -0500)]
Remove LZX_ASSERT() (now almost unused)

3 years agobt_matchfinder optimizations
Eric Biggers [Sat, 19 Sep 2015 18:56:09 +0000 (13:56 -0500)]
bt_matchfinder optimizations

3 years agoAllow hc_matchfinder and bt_matchfinder to be "templated"
Eric Biggers [Sat, 19 Sep 2015 18:56:08 +0000 (13:56 -0500)]
Allow hc_matchfinder and bt_matchfinder to be "templated"

3 years agolzx_compress.c: optimize bit output
Eric Biggers [Sat, 19 Sep 2015 18:56:03 +0000 (13:56 -0500)]
lzx_compress.c: optimize bit output

3 years agolzx_compress.c: optimize output of items
Eric Biggers [Sat, 19 Sep 2015 18:56:02 +0000 (13:56 -0500)]
lzx_compress.c: optimize output of items

3 years agolzx_compress.c: optimization for lzx_compute_precode_items()
Eric Biggers [Sat, 19 Sep 2015 18:56:01 +0000 (13:56 -0500)]
lzx_compress.c: optimization for lzx_compute_precode_items()

3 years agohc_matchfinder optimizations
Eric Biggers [Sat, 19 Sep 2015 18:56:01 +0000 (13:56 -0500)]
hc_matchfinder optimizations

3 years agoGet rid of matchfinder_common.h and manual memsets
Eric Biggers [Sat, 19 Sep 2015 18:55:59 +0000 (13:55 -0500)]
Get rid of matchfinder_common.h and manual memsets

3 years agoSplit prefetch() into prefetchr() and prefetchw()
Eric Biggers [Sat, 19 Sep 2015 18:56:28 +0000 (13:56 -0500)]
Split prefetch() into prefetchr() and prefetchw()

3 years agosystem compression: force bootloader-accessed files to uncompressed if image is not...
Eric Biggers [Fri, 25 Sep 2015 00:57:06 +0000 (19:57 -0500)]
system compression: force bootloader-accessed files to uncompressed if image is not Windows 10

3 years agosystem compression: add several more exclusion patterns
Eric Biggers [Fri, 25 Sep 2015 00:57:01 +0000 (19:57 -0500)]
system compression: add several more exclusion patterns

3 years agoUpdate NEWS
Eric Biggers [Thu, 24 Sep 2015 04:05:50 +0000 (23:05 -0500)]
Update NEWS

3 years agosystem compression: force XPRESS4K on files accessed by Windows bootloader
Eric Biggers [Thu, 24 Sep 2015 03:30:34 +0000 (22:30 -0500)]
system compression: force XPRESS4K on files accessed by Windows bootloader

3 years agoWIMBoot / system compression: try WOFADK in addition to WOF
Eric Biggers [Thu, 24 Sep 2015 02:56:03 +0000 (21:56 -0500)]
WIMBoot / system compression: try WOFADK in addition to WOF

3 years agosystem compression: try to attach WOF if compression fails
Eric Biggers [Thu, 24 Sep 2015 02:54:32 +0000 (21:54 -0500)]
system compression: try to attach WOF if compression fails

3 years agosystem compression: fix for WOFADK on older Windows
Eric Biggers [Thu, 24 Sep 2015 02:01:23 +0000 (21:01 -0500)]
system compression: fix for WOFADK on older Windows

3 years agov1.8.3-BETA
Eric Biggers [Sat, 19 Sep 2015 18:21:18 +0000 (13:21 -0500)]
v1.8.3-BETA

3 years agoAlways specify AT_UNNAMED when opening unnamed stream with libntfs-3g
Eric Biggers [Sat, 19 Sep 2015 16:31:06 +0000 (11:31 -0500)]
Always specify AT_UNNAMED when opening unnamed stream with libntfs-3g

Using another empty string causes problems.

3 years agowimlib-imagex: improve error message
Eric Biggers [Sat, 19 Sep 2015 02:28:09 +0000 (21:28 -0500)]
wimlib-imagex: improve error message

Improved message if the user specifies the non-first part of a split WIM.

3 years agowimlib-imagex: add --ref option to wimdir
Eric Biggers [Wed, 16 Sep 2015 06:15:05 +0000 (01:15 -0500)]
wimlib-imagex: add --ref option to wimdir

3 years agoReplace BUILD_BUG_ON() with STATIC_ASSERT()
Eric Biggers [Sun, 13 Sep 2015 16:36:22 +0000 (11:36 -0500)]
Replace BUILD_BUG_ON() with STATIC_ASSERT()

4 years agoUpdate version-info
Eric Biggers [Thu, 27 Aug 2015 00:01:14 +0000 (19:01 -0500)]
Update version-info

Should have been done for v1.8.2.  Doesn't matter much since the only API
change was the addition of a few flags...

4 years agov1.8.2 v1.8.2
Eric Biggers [Sat, 22 Aug 2015 17:06:15 +0000 (12:06 -0500)]
v1.8.2

4 years agoUpdate NEWS
Eric Biggers [Sat, 22 Aug 2015 17:06:13 +0000 (12:06 -0500)]
Update NEWS

4 years agox86_cpu_features.c: xgetbv fix for old binutils
Eric Biggers [Sat, 22 Aug 2015 16:47:15 +0000 (11:47 -0500)]
x86_cpu_features.c: xgetbv fix for old binutils

4 years agox86_cpu_features.c: cpuid fix for x86_32 PIC with old GCC versions
Eric Biggers [Sat, 22 Aug 2015 16:27:12 +0000 (11:27 -0500)]
x86_cpu_features.c: cpuid fix for x86_32 PIC with old GCC versions

4 years agolzms_common.c: tweak opcode branching in lzms_x86_filter()
Eric Biggers [Sat, 22 Aug 2015 14:08:06 +0000 (09:08 -0500)]
lzms_common.c: tweak opcode branching in lzms_x86_filter()

4 years agoMention wofadk.sys in "compact" documentation
Eric Biggers [Tue, 18 Aug 2015 23:13:25 +0000 (18:13 -0500)]
Mention wofadk.sys in "compact" documentation

4 years agowin32_common.c: don't short-circuit when acquiring privileges
Eric Biggers [Tue, 18 Aug 2015 23:02:39 +0000 (18:02 -0500)]
win32_common.c: don't short-circuit when acquiring privileges

4 years agowin32_common.c: include SeManageVolumePrivilege in apply privileges
Eric Biggers [Tue, 18 Aug 2015 23:00:51 +0000 (18:00 -0500)]
win32_common.c: include SeManageVolumePrivilege in apply privileges

Based on a user report, this may be required for
FSCTL_SET_EXTERNAL_BACKING when using wofadk.sys.

4 years agolzms_common.c: tweak organization of x86 translation logic and improve comments
Eric Biggers [Mon, 17 Aug 2015 02:17:51 +0000 (21:17 -0500)]
lzms_common.c: tweak organization of x86 translation logic and improve comments

4 years agoLZMS: more bit decoding/encoding optimizations
Eric Biggers [Sun, 16 Aug 2015 17:00:48 +0000 (12:00 -0500)]
LZMS: more bit decoding/encoding optimizations

4 years agoUse 'restrict' on pointer arguments to all compress() and decompress() functions
Eric Biggers [Sun, 16 Aug 2015 19:09:09 +0000 (14:09 -0500)]
Use 'restrict' on pointer arguments to all compress() and decompress() functions

4 years agontfs-3g_capture.c: workaround for duplicate streams
Eric Biggers [Fri, 14 Aug 2015 03:53:11 +0000 (22:53 -0500)]
ntfs-3g_capture.c: workaround for duplicate streams

4 years agoUpdate references to specific Windows versions
Eric Biggers [Sat, 8 Aug 2015 20:05:59 +0000 (15:05 -0500)]
Update references to specific Windows versions

- Say "Vista" and later instead of listing specific versions
- Prefer "ISO" to "DVD"

4 years agontfs-3g_apply.c: add hint about known libntfs-3g bug
Eric Biggers [Sat, 8 Aug 2015 19:52:07 +0000 (14:52 -0500)]
ntfs-3g_apply.c: add hint about known libntfs-3g bug

4 years agoUpdate NEWS
Eric Biggers [Sat, 8 Aug 2015 19:08:36 +0000 (14:08 -0500)]
Update NEWS

4 years agoconfigure.ac: Don't use AC_CHECK_LIB() when PKG_CHECK_MODULES() suffices
Eric Biggers [Sat, 8 Aug 2015 06:19:07 +0000 (01:19 -0500)]
configure.ac: Don't use AC_CHECK_LIB() when PKG_CHECK_MODULES() suffices

4 years agoAvoid having to check for NTFS-3g 2013.1.13 or later at configure time
Eric Biggers [Sat, 8 Aug 2015 06:18:14 +0000 (01:18 -0500)]
Avoid having to check for NTFS-3g 2013.1.13 or later at configure time

4 years agotools/windeps: upgrade to winpthreads 4.0.4
Eric Biggers [Sat, 8 Aug 2015 05:53:38 +0000 (00:53 -0500)]
tools/windeps: upgrade to winpthreads 4.0.4

4 years agoLZMS: share 'struct lzms_probabilities' between compressor and decompressor
Eric Biggers [Sat, 8 Aug 2015 04:45:18 +0000 (23:45 -0500)]
LZMS: share 'struct lzms_probabilities' between compressor and decompressor

4 years agolzms_decompress.c: optimize lzms_decompress() to hold some variables on stack
Eric Biggers [Fri, 7 Aug 2015 05:13:45 +0000 (00:13 -0500)]
lzms_decompress.c: optimize lzms_decompress() to hold some variables on stack

4 years agolzms_decompress.c: move range normalization upwards
Eric Biggers [Fri, 7 Aug 2015 04:35:43 +0000 (23:35 -0500)]
lzms_decompress.c: move range normalization upwards

4 years agolzms_decompress.c: inline decoding of all bits and symbols
Eric Biggers [Fri, 7 Aug 2015 04:10:37 +0000 (23:10 -0500)]
lzms_decompress.c: inline decoding of all bits and symbols

(including lengths and rep bits)

4 years agohc_matchfinder.h: fix comment
Eric Biggers [Fri, 7 Aug 2015 02:55:49 +0000 (21:55 -0500)]
hc_matchfinder.h: fix comment

4 years agolzms_decompress.c: pack states together in struct lzms_decompressor
Eric Biggers [Fri, 7 Aug 2015 02:55:48 +0000 (21:55 -0500)]
lzms_decompress.c: pack states together in struct lzms_decompressor

4 years agolzms_decompress.c: inline lzms_range_decode_bit() into lzms_decode_bit()
Eric Biggers [Fri, 7 Aug 2015 02:55:47 +0000 (21:55 -0500)]
lzms_decompress.c: inline lzms_range_decode_bit() into lzms_decode_bit()

4 years agolzms_decompress.c: read a whole word at a time for backwards bitstream
Eric Biggers [Fri, 7 Aug 2015 02:55:46 +0000 (21:55 -0500)]
lzms_decompress.c: read a whole word at a time for backwards bitstream

4 years agolzms_decompress.c: move first queue update into rep case
Eric Biggers [Fri, 7 Aug 2015 02:55:45 +0000 (21:55 -0500)]
lzms_decompress.c: move first queue update into rep case

4 years agolzms_decompress.c: make 'lens' and 'decode_table' share memory
Eric Biggers [Fri, 7 Aug 2015 02:55:44 +0000 (21:55 -0500)]
lzms_decompress.c: make 'lens' and 'decode_table' share memory

4 years agolzms_common.c: SSE4.2-optimized lzms_x86_filter()
Eric Biggers [Fri, 7 Aug 2015 02:55:43 +0000 (21:55 -0500)]
lzms_common.c: SSE4.2-optimized lzms_x86_filter()

4 years agoAdd support for runtime x86 CPU feature detection
Eric Biggers [Fri, 7 Aug 2015 02:55:42 +0000 (21:55 -0500)]
Add support for runtime x86 CPU feature detection

4 years agolzms_common.c: optimize opcode branching in lzms_x86_filter()
Eric Biggers [Fri, 7 Aug 2015 02:55:41 +0000 (21:55 -0500)]
lzms_common.c: optimize opcode branching in lzms_x86_filter()

4 years agolzms_common.c: Change lzms_x86_filter to maintain pointer instead of index
Eric Biggers [Fri, 7 Aug 2015 02:55:40 +0000 (21:55 -0500)]
lzms_common.c: Change lzms_x86_filter to maintain pointer instead of index

4 years agolzms_common.c: Split lzms_x86_filter into subroutines
Eric Biggers [Fri, 7 Aug 2015 02:55:37 +0000 (21:55 -0500)]
lzms_common.c: Split lzms_x86_filter into subroutines

4 years agoReplace SourceForge references with wimlib.net
Eric Biggers [Mon, 3 Aug 2015 02:21:06 +0000 (21:21 -0500)]
Replace SourceForge references with wimlib.net

4 years agowimlib-imagex.1: correct license statement
Eric Biggers [Sat, 25 Jul 2015 16:19:10 +0000 (11:19 -0500)]
wimlib-imagex.1: correct license statement

4 years agoadd System Compression support
Eric Biggers [Wed, 22 Jul 2015 01:28:59 +0000 (20:28 -0500)]
add System Compression support

4 years agodecompress_common.h: make bitstream_read_bytes() fill buffer itself
Eric Biggers [Mon, 20 Jul 2015 05:02:07 +0000 (00:02 -0500)]
decompress_common.h: make bitstream_read_bytes() fill buffer itself

4 years agolzx_decompress.c: switch to out* pointer style
Eric Biggers [Mon, 20 Jul 2015 04:59:48 +0000 (23:59 -0500)]
lzx_decompress.c: switch to out* pointer style

4 years agoxpress_decompress.c: consolidate XPRESS decompressor into one function
Eric Biggers [Mon, 20 Jul 2015 04:36:19 +0000 (23:36 -0500)]
xpress_decompress.c: consolidate XPRESS decompressor into one function

4 years agoxpress_decompress.c: make 'lens' and 'decode_table' share memory
Eric Biggers [Mon, 20 Jul 2015 04:35:31 +0000 (23:35 -0500)]
xpress_decompress.c: make 'lens' and 'decode_table' share memory

4 years agomake_huffman_decode_table(): Note that 'lens' may alias 'decode_table'
Eric Biggers [Mon, 20 Jul 2015 04:30:08 +0000 (23:30 -0500)]
make_huffman_decode_table(): Note that 'lens' may alias 'decode_table'

4 years agoAvoid branch in 'num_bits == 0' branch when peeking bits
Eric Biggers [Mon, 20 Jul 2015 04:13:53 +0000 (23:13 -0500)]
Avoid branch in 'num_bits == 0' branch when peeking bits

4 years agoMake read_huffsym() return unsigned int
Eric Biggers [Mon, 20 Jul 2015 03:18:10 +0000 (22:18 -0500)]
Make read_huffsym() return unsigned int

4 years agontfs-3g_apply.c: only call sd_fixup() if ntfs_set_ntfs_acl() fails
Eric Biggers [Mon, 13 Jul 2015 00:49:05 +0000 (19:49 -0500)]
ntfs-3g_apply.c: only call sd_fixup() if ntfs_set_ntfs_acl() fails

4 years agontfs-3g_capture.c: remove unused path_len argument
Eric Biggers [Sat, 18 Jul 2015 01:26:11 +0000 (20:26 -0500)]
ntfs-3g_capture.c: remove unused path_len argument

4 years agoverify: update progress per-chunk instead of per-blob
Eric Biggers [Thu, 2 Jul 2015 23:59:59 +0000 (18:59 -0500)]
verify: update progress per-chunk instead of per-blob

4 years agoUse list_replace() instead of list_transfer()
Eric Biggers [Thu, 2 Jul 2015 23:44:48 +0000 (18:44 -0500)]
Use list_replace() instead of list_transfer()

4 years agoinode_table: remove unused num_entries member
Eric Biggers [Thu, 2 Jul 2015 23:16:40 +0000 (18:16 -0500)]
inode_table: remove unused num_entries member

4 years agoblob_table.c: use power-of-2 sized hash table for faster hashing
Eric Biggers [Tue, 30 Jun 2015 01:46:51 +0000 (20:46 -0500)]
blob_table.c: use power-of-2 sized hash table for faster hashing

SHA-1 is a cryptographic hash, so we do not need to worry about
the distribution.

4 years agodentry.c: Use simpler check that handles both deep and cyclic directory structures
Eric Biggers [Tue, 30 Jun 2015 01:46:48 +0000 (20:46 -0500)]
dentry.c: Use simpler check that handles both deep and cyclic directory structures