wimlib
6 years agowrite_stream_list(): Write streams uncompressed when advantageous
Eric Biggers [Wed, 25 Dec 2013 15:32:42 +0000 (09:32 -0600)]
write_stream_list():  Write streams uncompressed when advantageous

6 years agoread_compressed_resource(): Fix use of decompressor
Eric Biggers [Wed, 25 Dec 2013 14:40:59 +0000 (08:40 -0600)]
read_compressed_resource(): Fix use of decompressor

6 years agoAdd deprecated compression/decompression functions
Eric Biggers [Wed, 25 Dec 2013 14:34:36 +0000 (08:34 -0600)]
Add deprecated compression/decompression functions

Don't want to increase the shared library version number as there are no
other incompatibilities.

6 years agoMake --disable-multithreaded-compression work again
Eric Biggers [Wed, 25 Dec 2013 07:12:14 +0000 (01:12 -0600)]
Make --disable-multithreaded-compression work again

6 years agoRemove --enable-more-debug and --disable-custom-memory-allocator options
Eric Biggers [Wed, 25 Dec 2013 07:03:56 +0000 (01:03 -0600)]
Remove --enable-more-debug and --disable-custom-memory-allocator options

The former was of little use due to the lack of DEBUG2() statements, and
the latter barely saved any space and wasn't worth having an option for.

6 years agoRemove obsolete ChangeLog
Eric Biggers [Wed, 25 Dec 2013 06:46:22 +0000 (00:46 -0600)]
Remove obsolete ChangeLog

6 years agoComment fixes
Eric Biggers [Wed, 25 Dec 2013 06:12:59 +0000 (00:12 -0600)]
Comment fixes

6 years agoxpress_decompress(): Fix comment
Eric Biggers [Wed, 25 Dec 2013 05:50:39 +0000 (23:50 -0600)]
xpress_decompress(): Fix comment

6 years agoNew compression/decompression API
Eric Biggers [Wed, 25 Dec 2013 04:53:18 +0000 (22:53 -0600)]
New compression/decompression API

To avoid the proliferation of functions for compressing and decompressing
in different formats, allow all the compression algorithms to be accessed
using a single API:

Compression:
- wimlib_create_compressor()
- wimlib_compress()
- wimlib_free_compressor()
- wimlib_set_default_compressor_params()

Decompression:
- wimlib_create_decompressor()
- wimlib_decompress()
- wimlib_free_decompressor()
- wimlib_set_default_decompressor_params()

This also makes it easier to allocate larger blocks of memory or do other
initializations in any decompressor or compressor implementation.

This commit adds a skeleton for the LZMS compressor but it doesn't do
anything yet.

6 years agowimlib_overwrite(): Don't create multiple packs per WIM
Eric Biggers [Tue, 24 Dec 2013 18:00:12 +0000 (12:00 -0600)]
wimlib_overwrite(): Don't create multiple packs per WIM

For compatibility with WIMGAPI, don't allow updating a WIM to contain
multiple stream packs; instead compress new streams independently.

This could change if it turns out there is a way to cause WIMGAPI to
recognize multiple packs.

6 years agoFix progress for exclusion of absolute symlinks
Eric Biggers [Tue, 24 Dec 2013 16:58:21 +0000 (10:58 -0600)]
Fix progress for exclusion of absolute symlinks

6 years agoScan progress: Add # of files, directories, and bytes scanned
Eric Biggers [Tue, 24 Dec 2013 16:36:53 +0000 (10:36 -0600)]
Scan progress:  Add # of files, directories, and bytes scanned

6 years agomkwinpeimg: Fix --start-script when script not in current directory
Eric Biggers [Tue, 24 Dec 2013 07:24:43 +0000 (01:24 -0600)]
mkwinpeimg: Fix --start-script when script not in current directory

6 years agowrite_stream_list(): Do progress on raw resource copies
Eric Biggers [Tue, 24 Dec 2013 07:16:37 +0000 (01:16 -0600)]
write_stream_list(): Do progress on raw resource copies

6 years agolookup_table_insert(): Grow table when capacity reached
Eric Biggers [Tue, 24 Dec 2013 06:58:06 +0000 (00:58 -0600)]
lookup_table_insert(): Grow table when capacity reached

This avoids slow hash list searches when the lookup table was initially
allocated very small due to opening a WIM containing few streams, but
more entries were added later (e.g. with wimlib_reference_resources()).

6 years agowrite.c: Document write_stream_list()
Eric Biggers [Tue, 24 Dec 2013 05:44:42 +0000 (23:44 -0600)]
write.c: Document write_stream_list()

6 years agowimlib.h: Document pack streams write flags
Eric Biggers [Tue, 24 Dec 2013 05:44:31 +0000 (23:44 -0600)]
wimlib.h:  Document pack streams write flags

6 years agoMake sequential extraction the default
Eric Biggers [Tue, 24 Dec 2013 02:18:24 +0000 (20:18 -0600)]
Make sequential extraction the default

This is needed to avoid bad default performance extracting packed WIMs, plus
wimlib-imagex has been always using sequential extraction anyway.

6 years agowimlib_overwrite()): Change WIM version when appropriate
Eric Biggers [Tue, 24 Dec 2013 02:03:44 +0000 (20:03 -0600)]
wimlib_overwrite()): Change WIM version when appropriate

6 years agowimlib_overwrite(): Don't force full rebuild of packed WIM
Eric Biggers [Tue, 24 Dec 2013 01:39:14 +0000 (19:39 -0600)]
wimlib_overwrite(): Don't force full rebuild of packed WIM

Version 3584 WIMs may contain both packed and unpacked streams, no there's
no need to force a full rebuild if WIMLIB_WRITE_FLAG_NO_PACK_STREAMS is
required.

At the same time, a full rebuild is in general required if the default
compression type or chunk size has been changed.

6 years agowimoptimize: Add --no-pack-streams option
Eric Biggers [Tue, 24 Dec 2013 01:38:46 +0000 (19:38 -0600)]
wimoptimize: Add --no-pack-streams option

6 years agoread_lookup_table(): Ignore PACKED_STREAMS flag if default WIM version
Eric Biggers [Tue, 24 Dec 2013 01:37:37 +0000 (19:37 -0600)]
read_lookup_table(): Ignore PACKED_STREAMS flag if default WIM version

6 years agowimlib-imagex: Allow specifying LZMS compression
Eric Biggers [Tue, 24 Dec 2013 01:30:56 +0000 (19:30 -0600)]
wimlib-imagex: Allow specifying LZMS compression

6 years agoDo not wait full streams for WIMLIB_PROGRESS_MSG_WRITE_STREAMS
Eric Biggers [Tue, 24 Dec 2013 00:33:25 +0000 (18:33 -0600)]
Do not wait full streams for WIMLIB_PROGRESS_MSG_WRITE_STREAMS

6 years agoread_stream_list(): Fix calculation of next stream
Eric Biggers [Tue, 24 Dec 2013 00:11:41 +0000 (18:11 -0600)]
read_stream_list(): Fix calculation of next stream

6 years agowimoptimize: Add --pack-streams option
Eric Biggers [Tue, 24 Dec 2013 00:09:02 +0000 (18:09 -0600)]
wimoptimize:  Add --pack-streams option

6 years agoAdd new files for chunk compression
Eric Biggers [Mon, 23 Dec 2013 20:33:28 +0000 (14:33 -0600)]
Add new files for chunk compression

6 years agoread_compressed_wim_resource(): Fix reading chunk table
Eric Biggers [Mon, 23 Dec 2013 20:07:25 +0000 (14:07 -0600)]
read_compressed_wim_resource(): Fix reading chunk table

6 years agoCleanup
Eric Biggers [Mon, 23 Dec 2013 19:31:42 +0000 (13:31 -0600)]
Cleanup

6 years agoRewrite of write_stream_list(), and writing packed resources
Eric Biggers [Mon, 23 Dec 2013 08:14:37 +0000 (02:14 -0600)]
Rewrite of write_stream_list(), and writing packed resources

6 years agolzms-decompress.c: Fix comments
Eric Biggers [Sat, 21 Dec 2013 00:51:52 +0000 (18:51 -0600)]
lzms-decompress.c:  Fix comments

6 years agocreate_temporary_file(): Fix use-after-free
Eric Biggers [Sat, 21 Dec 2013 00:31:35 +0000 (18:31 -0600)]
create_temporary_file(): Fix use-after-free

6 years agoread_lookup_table(): Cleanup
Eric Biggers [Sat, 21 Dec 2013 00:21:30 +0000 (18:21 -0600)]
read_lookup_table(): Cleanup

6 years agoFix can_raw_copy()
Eric Biggers [Fri, 20 Dec 2013 23:32:41 +0000 (17:32 -0600)]
Fix can_raw_copy()

6 years agoFix RAW_CHUNKS reads
Eric Biggers [Fri, 20 Dec 2013 23:13:42 +0000 (17:13 -0600)]
Fix RAW_CHUNKS reads

6 years agoinit_output_bitstream(): Set bytes_remaining correctly
Eric Biggers [Fri, 20 Dec 2013 23:06:31 +0000 (17:06 -0600)]
init_output_bitstream(): Set bytes_remaining correctly

6 years agoCleanup and add more comments
Eric Biggers [Fri, 20 Dec 2013 22:59:15 +0000 (16:59 -0600)]
Cleanup and add more comments

6 years agoUse read_stream_list() for extraction
Eric Biggers [Fri, 20 Dec 2013 20:21:03 +0000 (14:21 -0600)]
Use read_stream_list() for extraction

6 years agoAdd read_stream_list()
Eric Biggers [Fri, 20 Dec 2013 07:26:04 +0000 (01:26 -0600)]
Add read_stream_list()

6 years agoAdd independent LZMS decompressor
Eric Biggers [Thu, 19 Dec 2013 16:24:38 +0000 (10:24 -0600)]
Add independent LZMS decompressor

6 years agoAdd kind-of-working LZMS decompression using cabinet.dll API
Eric Biggers [Sat, 14 Dec 2013 21:34:34 +0000 (15:34 -0600)]
Add kind-of-working LZMS decompression using cabinet.dll API

6 years agoTweak reading of concat runs
Eric Biggers [Sat, 14 Dec 2013 21:34:10 +0000 (15:34 -0600)]
Tweak reading of concat runs

6 years agoWin32: compile
Eric Biggers [Sat, 14 Dec 2013 18:34:08 +0000 (12:34 -0600)]
Win32: compile

6 years agoextract_streams_from_pipe(): Fix
Eric Biggers [Sat, 14 Dec 2013 17:20:50 +0000 (11:20 -0600)]
extract_streams_from_pipe(): Fix

6 years agoupdate; add lzms_decompress() stub
Eric Biggers [Sat, 14 Dec 2013 17:18:32 +0000 (11:18 -0600)]
update; add lzms_decompress() stub

6 years agoRead concat runs
Eric Biggers [Sat, 14 Dec 2013 07:41:05 +0000 (01:41 -0600)]
Read concat runs

6 years agoSplit wim_resource_spec from wim_lookup_table_entry
Eric Biggers [Sat, 14 Dec 2013 05:06:07 +0000 (23:06 -0600)]
Split wim_resource_spec from wim_lookup_table_entry

6 years agolzx_prepare_block_fast(): Increase max_offset
Eric Biggers [Fri, 13 Dec 2013 20:02:37 +0000 (14:02 -0600)]
lzx_prepare_block_fast(): Increase max_offset

6 years agoLZX decompress: Fix comment & docs
Eric Biggers [Fri, 13 Dec 2013 19:46:17 +0000 (13:46 -0600)]
LZX decompress: Fix comment & docs

6 years agoUse STACK_MAX
Eric Biggers [Fri, 13 Dec 2013 19:27:41 +0000 (13:27 -0600)]
Use STACK_MAX

6 years agowim.c: Cleanup
Eric Biggers [Fri, 13 Dec 2013 18:45:59 +0000 (12:45 -0600)]
wim.c: Cleanup

6 years agoverify_swm_set(): Check chunk size
Eric Biggers [Fri, 13 Dec 2013 18:43:37 +0000 (12:43 -0600)]
verify_swm_set(): Check chunk size

6 years agowrite_stream_list(): Cleanup
Eric Biggers [Fri, 13 Dec 2013 18:18:24 +0000 (12:18 -0600)]
write_stream_list(): Cleanup

6 years agoFix reading > 16 bits from bitstream
Eric Biggers [Fri, 13 Dec 2013 17:15:19 +0000 (11:15 -0600)]
Fix reading > 16 bits from bitstream

6 years agobitstream_put_bits(): Mask high-order bits
Eric Biggers [Fri, 13 Dec 2013 16:29:35 +0000 (10:29 -0600)]
bitstream_put_bits(): Mask high-order bits

6 years agocmp_nodes_by_freq(): Correctly handle large frequencies
Eric Biggers [Fri, 13 Dec 2013 16:20:43 +0000 (10:20 -0600)]
cmp_nodes_by_freq(): Correctly handle large frequencies

6 years agowim_recalculate_refcnts(): Check for error
Eric Biggers [Fri, 13 Dec 2013 16:11:28 +0000 (10:11 -0600)]
wim_recalculate_refcnts(): Check for error

6 years agofile_io.c: Cleanup
Eric Biggers [Fri, 13 Dec 2013 16:07:43 +0000 (10:07 -0600)]
file_io.c: Cleanup

6 years agoresource.c: Fix check for invalid chunk size
Eric Biggers [Fri, 13 Dec 2013 15:59:43 +0000 (09:59 -0600)]
resource.c:  Fix check for invalid chunk size

6 years agoresource.c: Fix reading pipable resources
Eric Biggers [Fri, 13 Dec 2013 09:27:48 +0000 (03:27 -0600)]
resource.c:  Fix reading pipable resources

6 years agopaths.c: Cleanup
Eric Biggers [Fri, 13 Dec 2013 09:20:20 +0000 (03:20 -0600)]
paths.c:  Cleanup

6 years agoresource.c: Cleanup and refactor
Eric Biggers [Fri, 13 Dec 2013 08:15:27 +0000 (02:15 -0600)]
resource.c:  Cleanup and refactor

6 years agoFix error/warning conditional
Eric Biggers [Thu, 12 Dec 2013 19:59:06 +0000 (13:59 -0600)]
Fix error/warning conditional

6 years agoVariable LZX window sizes
Eric Biggers [Thu, 12 Dec 2013 19:56:37 +0000 (13:56 -0600)]
Variable LZX window sizes

6 years agoVariable chunk size support (currently XPRESS only)
Eric Biggers [Mon, 9 Dec 2013 06:26:22 +0000 (00:26 -0600)]
Variable chunk size support (currently XPRESS only)

6 years agoClean up other compression/decompression code
Eric Biggers [Mon, 9 Dec 2013 03:06:57 +0000 (21:06 -0600)]
Clean up other compression/decompression code

6 years agowimcapture: Document --compress-slow v1.5.3
Eric Biggers [Sun, 8 Dec 2013 09:24:22 +0000 (03:24 -0600)]
wimcapture: Document --compress-slow

6 years agowimoptimize: make --recompress-slow imply --compress=maximum
Eric Biggers [Sun, 8 Dec 2013 09:17:10 +0000 (03:17 -0600)]
wimoptimize: make --recompress-slow imply --compress=maximum

6 years agoMakefile.am: Fix EXTRA_DIST
Eric Biggers [Sun, 8 Dec 2013 09:11:07 +0000 (03:11 -0600)]
Makefile.am:  Fix EXTRA_DIST

6 years agoAllow changing WIM compression type
Eric Biggers [Sun, 8 Dec 2013 09:10:25 +0000 (03:10 -0600)]
Allow changing WIM compression type

6 years agowimoptimize: Add --recompress-slow, imply --recompress
Eric Biggers [Sun, 8 Dec 2013 08:26:55 +0000 (02:26 -0600)]
wimoptimize:  Add --recompress-slow, imply --recompress

6 years agoFix wimoptimize help
Eric Biggers [Sun, 8 Dec 2013 08:19:51 +0000 (02:19 -0600)]
Fix wimoptimize help

6 years agolzx-compress.c: Cleanup
Eric Biggers [Sun, 8 Dec 2013 08:14:55 +0000 (02:14 -0600)]
lzx-compress.c:  Cleanup

6 years agoAdd --compress-slow back to wimoptimize help
Eric Biggers [Sun, 8 Dec 2013 07:05:23 +0000 (01:05 -0600)]
Add --compress-slow back to wimoptimize help

6 years agoRemove unused files from programs/
Eric Biggers [Sun, 8 Dec 2013 06:52:32 +0000 (00:52 -0600)]
Remove unused files from programs/

wimapply.c was redundant with examples/applywim.c and was prone to be
confused with the 'wimapply' alias for 'wimlib-imagex apply'.

install.cmd was just an example script for a specific usage that wasn't
explained anywhere.

6 years agoRemove unused LZX compression code
Eric Biggers [Sun, 8 Dec 2013 06:48:33 +0000 (00:48 -0600)]
Remove unused LZX compression code

6 years agoUpdate clang-scan-build.sh
Eric Biggers [Sun, 8 Dec 2013 06:43:10 +0000 (00:43 -0600)]
Update clang-scan-build.sh

6 years agoUpdate version (v1.5.3)
Eric Biggers [Sun, 8 Dec 2013 06:36:47 +0000 (00:36 -0600)]
Update version (v1.5.3)

6 years agoMerge LZX compressor updates
Eric Biggers [Sun, 8 Dec 2013 06:07:37 +0000 (00:07 -0600)]
Merge LZX compressor updates

- Use a suffix array-based match-finder (construction code borrowed from
  libdivsufsort).  Ideal for this case because compression in WIM chunks
  does not use a sliding window.
- Disable block splitting by default (no efficient and effective
  algorithm is available yet)

6 years agoAdd wimlib_get_xml_data() API
Eric Biggers [Sat, 7 Dec 2013 01:04:19 +0000 (19:04 -0600)]
Add wimlib_get_xml_data() API

6 years agoxml.c: Fix writing <WINDOWS> element
Eric Biggers [Sat, 7 Dec 2013 00:40:57 +0000 (18:40 -0600)]
xml.c:  Fix writing <WINDOWS> element

A positive return value of xmlTextWriterEndElement would be incorrectly
passed up the call chain, causing an unexpected error when writing the
XML data.

6 years agowimlib_mount_image(): Correctly handle unnamed stream in ADS entries
Eric Biggers [Mon, 18 Nov 2013 03:35:13 +0000 (21:35 -0600)]
wimlib_mount_image():  Correctly handle unnamed stream in ADS entries

6 years agoFix wimlib_lzx_compress()
Eric Biggers [Mon, 18 Nov 2013 01:48:39 +0000 (19:48 -0600)]
Fix wimlib_lzx_compress()

6 years agostruct wimlib_lzx_params: Don't use anonymous union v1.5.2
Eric Biggers [Mon, 18 Nov 2013 00:13:34 +0000 (18:13 -0600)]
struct wimlib_lzx_params: Don't use anonymous union

6 years agoUpdate version info and NEWS in preparation for v1.5.2
Eric Biggers [Sun, 17 Nov 2013 23:18:30 +0000 (17:18 -0600)]
Update version info and NEWS in preparation for v1.5.2

6 years agoComment fixes / cleanups
Eric Biggers [Sun, 17 Nov 2013 23:09:02 +0000 (17:09 -0600)]
Comment fixes / cleanups

6 years agoMerge experimental LZX compressor
Eric Biggers [Sun, 17 Nov 2013 22:12:49 +0000 (16:12 -0600)]
Merge experimental LZX compressor

6 years agoImprove rename behavior on Windows
Eric Biggers [Thu, 14 Nov 2013 01:41:11 +0000 (19:41 -0600)]
Improve rename behavior on Windows

Attempt to handle the case where another process has the destination file
open by renaming and deleting the destination file.

6 years agoimagex.c: Fix string literal used in oom path only
Eric Biggers [Thu, 14 Nov 2013 00:09:27 +0000 (18:09 -0600)]
imagex.c:  Fix string literal used in oom path only

6 years agoAdd notes about possible new WIM format and LZMS compression
Eric Biggers [Wed, 13 Nov 2013 08:05:59 +0000 (02:05 -0600)]
Add notes about possible new WIM format and LZMS compression

6 years agoUpdate NEWS
Eric Biggers [Wed, 13 Nov 2013 02:59:30 +0000 (20:59 -0600)]
Update NEWS

6 years agoAdd ADS entry workaround
Eric Biggers [Wed, 13 Nov 2013 02:48:43 +0000 (20:48 -0600)]
Add ADS entry workaround

Some versions of Microsoft's WIM software (the boot driver(s) in WinPE
3.0, for example) contain a bug where they assume the first alternate
data stream (ADS) entry of a dentry with a nonzero ADS count specifies
the unnamed stream, even if it has a name and the unnamed stream is
already specified in the hash field of the dentry itself.

wimlib now works around this behavior by carefully emulating the behavior
of (most versions of) ImageX/WIMGAPI, which move the unnamed stream
reference into the alternate stream entries whenever there are named data
streams, even though there is already a field in the dentry itself for
the unnamed stream reference, which then goes to waste.

6 years agowrite.c: Fix two minor bugs in error paths
Eric Biggers [Sun, 3 Nov 2013 05:01:23 +0000 (00:01 -0500)]
write.c:  Fix two minor bugs in error paths

- Ensure that overwrite_wim_inplace() restores the original header if it
  is unable to complete successfully.
- If overwrite_wim_inplace() fails after having set
  WIM_HDR_FLAG_WRITE_IN_PROGRESS in the physical file's header, attempt
  to clear it.

6 years agontfs_3g_apply.c: Remove unused variable
Eric Biggers [Fri, 1 Nov 2013 05:28:09 +0000 (00:28 -0500)]
ntfs_3g_apply.c:  Remove unused variable

6 years ago{lzx,xpress}_decode_match(): Fix undefined pointer evalution
Eric Biggers [Fri, 1 Nov 2013 05:19:32 +0000 (00:19 -0500)]
{lzx,xpress}_decode_match():  Fix undefined pointer evalution

According to C99 6.5.6.8, calculating a pointer that points before the
allocated object has undefined behavior.  However, this was done by
lzx_decode_match() and xpress_decode_match() if they received a match
pointing before the end of the window (invalid match offset).  This
commit fixes this check to be fully defined.

This change likely has no visible effect in actual builds, e.g. with gcc.

6 years agonew_wim_struct(): Don't dereference NULL in out-of-memory case
Eric Biggers [Fri, 1 Nov 2013 04:51:47 +0000 (23:51 -0500)]
new_wim_struct():  Don't dereference NULL in out-of-memory case

7 years agolzx-decompress.c: Fix alignment bug
Eric Biggers [Sat, 12 Oct 2013 00:11:41 +0000 (19:11 -0500)]
lzx-decompress.c:  Fix alignment bug

The pretree decode table was not declared to be 16 byte aligned, as
expected by make_huffman_decode_table().

This bug had no effect if the compiler aligned this table on a 16 byte
boundary anyway, but if not it caused a segmentation fault on x86 platforms
where the SSE2 instructions were available, since in that case stores
requiring 16 byte alignment are used to fill in table entries.

7 years agoarchlinux/PKGBUILD: Fix pkgdesc
Eric Biggers [Fri, 11 Oct 2013 19:46:01 +0000 (14:46 -0500)]
archlinux/PKGBUILD:  Fix pkgdesc

7 years agomkwinpeimg: Quote $0 v1.5.1
Eric Biggers [Fri, 11 Oct 2013 16:06:49 +0000 (11:06 -0500)]
mkwinpeimg:  Quote $0

7 years agowimcapture: Allow specifying `--delta-from' multiple times
Eric Biggers [Fri, 11 Oct 2013 15:38:45 +0000 (10:38 -0500)]
wimcapture:  Allow specifying `--delta-from' multiple times