From: Eric Biggers Date: Tue, 14 Jan 2014 19:38:54 +0000 (-0600) Subject: Adjust documentation for alternate chunk sizes X-Git-Tag: v1.6.1~50 X-Git-Url: https://wimlib.net/git/?p=wimlib;a=commitdiff_plain;h=5ab0691733e0724ba5cd73bd4f9c0963cfe1d276 Adjust documentation for alternate chunk sizes --- diff --git a/doc/imagex-capture.1.in b/doc/imagex-capture.1.in index 41054667..6d995f5b 100644 --- a/doc/imagex-capture.1.in +++ b/doc/imagex-capture.1.in @@ -202,18 +202,23 @@ compression ratio. This currently only has an effect for LZX ("maximum", the default) and LZMS ("recovery") compression. .TP \fB--chunk-size\fR=\fISIZE\fR -Set the WIM compression chunk size to \fISIZE\fR. Using this option is not -recommended because WIM chunk sizes other than the default of 32768 have varying -levels of compatibility with Microsoft's software (depending on the compression -type, chunk size, and software version). In addition, wimlib versions before -1.6.0 do not support alternate chunk sizes. But if you decide to use this -option regardless, you can choose a chunk size that is a power of 2 greater than -or equal to 2^15 (32768) up to a maximum determined by the compression format. -Larger chunks mean larger LZ77 dictionaries and better compression ratios on -sufficiently large files, but slower random access. For LZX ("maximum") -compression, the maximum allowed chunk size is 2^21 (2097152), and for XPRESS -("fast") and LZMS ("recovery") compression, the maximum allowed chunk size is -2^26 (67108644). +Set the WIM compression chunk size to \fISIZE\fR. Larger chunks mean larger +LZ77 dictionaries and better compression ratios on sufficiently large files, but +slower random access. \fBUsing this option is generally not recommended because +of the compatibility limitations detailed in the next paragraph.\fR But if you +decide to use this option regardless, you may choose a chunk size that is a +power of 2 greater than or equal to 2^15 (32768) up to a maximum determined by +the compression format. For LZX ("maximum") compression, the maximum allowed +chunk size is 2^21 (2097152), and for XPRESS ("fast") and LZMS ("recovery") +compression, the maximum allowed chunk size is 2^26 (67108864). +.IP "" +For XPRESS and LZX compression, Microsoft's implementation (as of Windows 8) +does not appear to support alternate chunk sizes; although it will still open +such files, it will crash, extract the data incorrectly, or report that the data +is invalid. For LZMS compression, Micrososft's implementation (as of Windows 8) +appears to only support chunk sizes that are powers of 2 between 2^15 (32768) +and 2^20 (1048576) inclusively. In addition, wimlib versions before 1.6.0 do +not support alternate chunk sizes. .TP \fB--pack-streams\fR, \fB--solid\fR Create a "solid" archive that compresses multiple unique streams ("files") diff --git a/include/wimlib.h b/include/wimlib.h index a690f725..67b38379 100644 --- a/include/wimlib.h +++ b/include/wimlib.h @@ -3383,23 +3383,21 @@ wimlib_set_image_descripton(WIMStruct *wim, int image, * Set the compression chunk size of a WIM to use in subsequent calls to * wimlib_write() or wimlib_overwrite(). * - * A compression chunk size will result in a greater compression ratio, but the - * speed of random access to the WIM will be reduced, and the effect of an - * increased compression chunk size is limited by the size of each file being - * compressed. + * For compatibility reasons, using this function is not generally recommended. + * See the documentation for the @c --chunk-size option of wimlib-imagex + * capture for more information. * - * WARNING: Microsoft's software is seemingly incompatible with LZX chunk - * sizes other than 32768. Chunk sizes other than 32768 (for any format) are - * also incompatible with wimlib v1.5.3 and earlier. + * A larger compression chunk size will likely result in a better compression + * ratio, but the speed of random access to the WIM will be reduced. + * Furthermore, the effect of a larger compression chunk size is limited by the + * size of each stream ("file") being compressed. * * @param wim * ::WIMStruct for a WIM. * @param chunk_size * The chunk size (in bytes) to set. The valid chunk sizes are dependent - * on the compression format. The XPRESS and LZMS compression formats - * support chunk sizes that are powers of 2 with exponents between 15 and - * 26 inclusively, whereas the LZX compression format supports chunk sizes - * that are powers of 2 with exponents between 15 and 21 inclusively. As a + * on the compression format; see the documentation for the @c --chunk-size + * option of wimlib-imagex capture for more information. As a * special case, if @p chunk_size is specified as 0, the chunk size is set * to the default for the currently selected output compression type. * @@ -3416,7 +3414,8 @@ wimlib_set_output_chunk_size(WIMStruct *wim, uint32_t chunk_size); * @ingroup G_writing_and_overwriting_wims * * Similar to wimlib_set_output_chunk_size(), but set the chunk size for writing - * packed streams. + * packed streams. For compatibility reasons, using this function is not + * generally recommended. */ extern int wimlib_set_output_pack_chunk_size(WIMStruct *wim, uint32_t chunk_size); diff --git a/src/wim.c b/src/wim.c index 11e2f904..b36da84f 100644 --- a/src/wim.c +++ b/src/wim.c @@ -130,30 +130,14 @@ wim_chunk_size_valid(u32 chunk_size, int ctype) * 25 33554432 * 26 67108864 */ + + /* See the documentation for the --chunk-size option of `wimlib-imagex + * capture' for information about allowed chunk sizes. */ switch (ctype) { case WIMLIB_COMPRESSION_TYPE_LZX: - /* For LZX compression, the chunk size corresponds to the LZX - * window size, which according the LZX specification can be any - * power of 2 between 2^15 and 2^21, inclusively. All these are - * supported by wimlib; however, unfortunately only 2^15 is - * supported by WIMGAPI[1] so this value is used by default. - * - * [1] WIMGAPI (Windows 7) attempts to decompress LZX chunk - * sizes > 2^15 but seems to have bug(s) that cause it to fail - * or crash. (I tried several tweaks to the LZX data but none - * resulted in successful decompression.) WIMGAPI (Windows 8) - * appears to refuse to open WIMs with chunk size > 2^15 - * entirely. */ return order >= 15 && order <= 21; case WIMLIB_COMPRESSION_TYPE_XPRESS: - /* WIMGAPI (Windows 7, Windows 8) doesn't seem to support XPRESS - * chunk size below 32768 bytes, but larger power-of-two sizes, - * up ta 67108864 bytes, appear to work. (Note, however, that - * the offsets of XPRESS matches are still limited to 65535 - * bytes even when a much larger chunk size is used!) */ - return order >= 15 && order <= 26; - case WIMLIB_COMPRESSION_TYPE_LZMS: return order >= 15 && order <= 26; } @@ -175,10 +159,10 @@ wim_default_chunk_size(int ctype) } /* - * Calls a function on images in the WIM. If @image is WIMLIB_ALL_IMAGES, @visitor - * is called on the WIM once for each image, with each image selected as the - * current image in turn. If @image is a certain image, @visitor is called on - * the WIM only once, with that image selected. + * Calls a function on images in the WIM. If @image is WIMLIB_ALL_IMAGES, + * @visitor is called on the WIM once for each image, with each image selected + * as the current image in turn. If @image is a certain image, @visitor is + * called on the WIM only once, with that image selected. */ int for_image(WIMStruct *wim, int image, int (*visitor)(WIMStruct *))