-.TH WIMLIB-IMAGEX "1" "August 2013" "@IMAGEX_PROGNAME@ @VERSION@" "User Commands"
+.TH WIMLIB-IMAGEX "1" "January 2014" "@IMAGEX_PROGNAME@ @VERSION@" "User Commands"
.SH NAME
@IMAGEX_PROGNAME@-capture, @IMAGEX_PROGNAME@-append \- Create or append a WIM image
.SH SYNOPSIS
\fISOURCE\fR specifies the location of the files to create the new WIM image
from. If \fISOURCE\fR is a directory, the WIM image is captured from that
directory (see \fBDIRECTORY CAPTURE (UNIX)\fR or \fBDIRECTORY CAPTURE
-(WINDOWS)\fR. Alternatively, if the \fB--source-list\fR option is specified,
+(WINDOWS)\fR). Alternatively, if the \fB--source-list\fR option is specified,
\fISOURCE\fR is interpreted as a file that itself provides a list of
files and directories to include in the new WIM image. Still
alternatively, only on UNIX-like systems, if \fISOURCE\fR is a
regular file or block device, it is interpreted as an NTFS volume from
which a WIM image is to be captured using libntfs-3g (see \fBNTFS VOLUME CAPTURE
-(UNIX)\fR.
+(UNIX)\fR).
.PP
\fIIMAGE_NAME\fR and \fIIMAGE_DESCRIPTION\fR specify the name and description to
give the new WIM image. If \fIIMAGE_NAME\fR is not specified, it defaults to
UNIX file owners, groups, and modes. (Exception: see the \fB--unix-data\fR
option.) As a result, file permissions will not be stored, and files that are
neither regular files, directories, nor symbolic links, such as device files and
-FIFOs, cannot be captured.
+FIFOs, cannot be captured and will be excluded by default.
.IP \[bu]
Extended attributes. This mainly includes extensions to the traditional UNIX
security model, such as SELinux security labels, POSIX ACLs, and capabilities
labels.
.IP \[bu]
-Files that are neither regular files, directories, nor symbolic links, such as
-device files and FIFOs. These will be excluded by default.
+Linux file attributes, as can be changed using the \fBchattr\fR (1) utility.
.PP
-Notes: hard links and symbolic links are supported by the WIM format and
-\fIare\fR stored. (Symbolic links are turned into "native" Windows symbolic
-links via reparse points; this process is reversible, e.g. automatically by
-\fB@IMAGEX_PROGNAME@ apply\fR.) Timestamps are stored with 100 nanosecond
-granularity and include last modification time (mtime) and last access time
-(atime), but not last status change time (ctime).
+Notes: Timestamps are stored with 100 nanosecond granularity and include last
+modification time (mtime) and last access time (atime), but not last status
+change time (ctime). Hard links and symbolic links are supported by the WIM
+format and \fIare\fR stored. Symbolic links are turned into "native" Windows
+symbolic links, or "reparse points"; this process is fully reversible, e.g.
+automatically by \fB@IMAGEX_PROGNAME@ apply\fR, unless the symbolic link target
+contains backslash characters.
+.PP
+Pedantic note: A limitation of the WIM format prevents the unusual case where a
+single symbolic link file itself has multiple names (hard links); in this
+unlikely case, each symbolic link is stored as an independent file.
.SH NTFS VOLUME CAPTURE (UNIX)
This section documents how \fB@IMAGEX_PROGNAME@\fR captures files directly from
-an NTFS volume image on UNIX-like systems. See \fBDIRECTORY CAPTURE
-(WINDOWS)\fR for the corresponding documentation for Windows.
+an NTFS volume image on UNIX-like systems.
.PP
On UNIX-like systems, a special image capture mode is entered when \fISOURCE\fR
is a regular file or block device. In this mode, \fISOURCE\fR is assumed to be
-an NTFS volume or volume image, and wimlib will capture a WIM image containing a
-full contents of the NTFS volume, including NTFS-specific data. This is done
-using libntfs-3g.
+an NTFS volume or volume image, and \fB@IMAGEX_PROGNAME@\fR will capture a WIM
+image containing the full contents of the NTFS volume, including NTFS-specific
+data. This is done using libntfs-3g.
.PP
Please note that the NTFS volume capture mode is \fInot\fR entered if
\fISOURCE\fR is a directory, even if an NTFS filesystem is mounted on
On Windows, \fB@IMAGEX_PROGNAME@ capture\fR and \fB@IMAGEX_PROGNAME@ append\fR
try to archive as much data and metadata as possible, including:
.IP \[bu] 4
-All data streams of all files, unless running on a version of Windows prior to
-Vista, in which case named data streams (if supported by the source filesystem)
-will not be captured.
+All data streams of all files.
.IP \[bu]
Reparse points, including symbolic links, junction points, and other reparse
points, if supported by the source filesystem. (Note: see \fB--rpfix\fR and
apply\fR (on Windows) extracts the captured WIM image, it will extract all of
the above information, at least to the extent supported by the destination
filesystem. One exception is that since encrypted files are stored as
-unencrypted, their data will not be available if restored on a Windows system
+encrypted, their data will not be available if restored on a Windows system
that does not have the decryption key.
+.PP
+Pedantic note: since Windows is not fully compatible with its own filesystem
+(NTFS), on Windows wimlib cannot archive certain files that may exist on a valid
+NTFS filesystem but are inaccessible to the Windows API, for example two files
+with names differing only in case in the same directory, or a file whose name
+contains certain characters considered invalid by Windows. If you run into
+problems archiving such files consider using the \fBNTFS VOLUME CAPTURE
+(UNIX)\fR mode from Linux.
.SH OPTIONS
.TP 6
\fB--boot\fR
\fB--compress\fR=\fITYPE\fR
Specifies the compression type for the new WIM file. This flag is only valid
for \fB@IMAGEX_PROGNAME@ capture\fR, since the compression mode for
-\fB@IMAGEX_PROGNAME@ append\fR must be the same as that of the existing WIM (and
-is automatically set as such).
-\fITYPE\fR may be "none", "fast", or "maximum". By default, it is "maximum".
-This default behavior is different from Microsoft's ImageX, where the default is
-"fast". \fB@IMAGEX_PROGNAME@ capture\fR instead gives you the best compression
-ratio by default and makes up for the slightly slower compression by being
-faster than Microsoft's software in the first place and using multiple CPUs when
-available.
+\fB@IMAGEX_PROGNAME@ append\fR must be the same as that of the existing
+WIM (and is automatically set as such). \fITYPE\fR may be "none",
+"fast", or "maximum". As of wimlib v1.5.3, the default is LZX compression, but
+in a special mode that is somewhere in between "fast" and "maximum" in terms of
+speed and compression ratio. Use \fB--compress\fR=\fImaximum\fR to explicitly
+request a better compression ratio at the cost of more time spent compressing.
.IP ""
You may also specify the actual names of the compression algorithms, "XPRESS"
and "LZX", instead of "fast" and "maximum", respectively.
+.IP ""
+As of wimlib v1.6.0, a third compression type, "recovery" or "LZMS", is also
+available. Its use is generally not recommended because other than wimlib
+itself, as of Windows 8 it is only compatible with WIMGAPI and Windows Setup
+(not even ImageX or Dism). However, LZMS is the compression algorithm used in
+packed resources created if the \fB--pack-streams\fR option is specified.
+.TP
+\fB--compress-slow\fR
+Spend even more time compressing the data to achieve a very slightly better
+compression ratio. This currently only has an effect for LZX ("maximum", the
+default) and LZMS ("recovery") compression.
+.TP
+\fB--chunk-size\fR=\fISIZE\fR
+Set the WIM compression chunk size to \fISIZE\fR. Larger chunks mean larger
+LZ77 dictionaries and better compression ratios on sufficiently large files, but
+slower random access. \fBUsing this option is generally not recommended because
+of the compatibility limitations detailed in the next paragraph.\fR But if you
+decide to use this option regardless, you may choose a chunk size that is a
+power of 2 greater than or equal to 2^15 (32768) up to a maximum determined by
+the compression format. For LZX ("maximum") compression, the maximum allowed
+chunk size is 2^21 (2097152), and for XPRESS ("fast") and LZMS ("recovery")
+compression, the maximum allowed chunk size is 2^26 (67108864).
+.IP ""
+For XPRESS and LZX compression, Microsoft's implementation (as of Windows 8)
+does not appear to support alternate chunk sizes; although it will still open
+such files, it will crash, extract the data incorrectly, or report that the data
+is invalid. For LZMS compression, Microsoft's implementation (as of Windows 8)
+appears to only support chunk sizes that are powers of 2 between 2^15 (32768)
+and 2^20 (1048576) inclusively. In addition, wimlib versions before 1.6.0 do
+not support alternate chunk sizes.
+.TP
+\fB--pack-streams\fR, \fB--solid\fR
+Create a "solid" archive that compresses multiple unique streams ("files")
+together, rather than each unique stream ("file") independently. This can
+result in a significantly better compression ratio, but this format greatly
+decreases the performance of random access to the data, as may occur on a WIM
+mounted with \fB@IMAGEX_PROGNAME@ mount\fR. Also, WIMs created using this
+option use a different version number in their header and as of Windows 8 are
+only compatible with Windows Setup and WIMGAPI, not even ImageX and Dism.
+.IP ""
+Packed resources use a compression type and chunk size that is independent of
+the WIM's "default compression type" and "default chunk size" (which may be
+adjusted by the \fB--compress\fR and \fB--chunk-size\fR options, respectively).
+For compatibility reasons, \fB@IMAGEX_PROGNAME@ capture\fR currently has no
+option to change the compression type used in packed resources; however, the
+\fB--pack-chunk-size\fR option may be used to set the chunk size.
+.TP
+\fB--pack-chunk-size\fR=\fISIZE\fR, \fB--solid-chunk-size\fR=\fISIZE\fR
+Like \fB--chunk-size\fR, but set the chunk size used in packed resources. The
+compression format is LZMS, so the chunk size can be any power of 2 between 2^15
+and 2^26, inclusively. WIMGAPI (Windows 8) appears to be compatible with all
+these sizes, despite not being compatible with sizes greater than 2^20 in
+non-packed resources. The default is currently 2^25 (33554432). Note:
+currently, the LZMS compression algorithm uses about 15 times the chunk size in
+memory per thread, which is about 500 MB per thread for the default pack chunk
+size of 2^25 or 1 GB per thread if you change it to 2^26 (67108864).
.TP
\fB--threads\fR=\fINUM_THREADS\fR
Number of threads to use for compressing data. Default: autodetect (number of
Specify a string to use in the <FLAGS> element of the XML data for the new
image.
.TP
-\fB--verbose\fR
-Print the names of files and directories as they are captured.
-.TP
\fB--dereference\fR
(UNIX-like systems only) Follow symbolic links and archive the files they point
to, rather than archiving the links themselves.
output (\fIWIMFILE\fR specified as "-"), and also for \fB@IMAGEX_PROGNAME@
append\fR, except when appending to a WIM that is already pipable.
.TP
-\fB--as-update-of\fR=[\fIWIMFILE\fR]:\fIIMAGE\fR
-Declares that the image being captured from \fISOURCE\fR is mostly the same as
+\fB--update-of\fR=[\fIWIMFILE\fR:]\fIIMAGE\fR
+Declares that the image being captured or appended from \fISOURCE\fR is mostly the same as
the existing image \fIIMAGE\fR in \fIWIMFILE\fR, but captured at a later point
in time, possibly with some modifications in the intervening time. This is
designed to be used in incremental backups of the same filesystem or directory
tree. \fIIMAGE\fR can be a 1-based index or name of an existing image in
\fIWIMFILE\fR. It can also be a negative integer to index backwards into the
-images (e.g. -1 means the last existing image).
+images (e.g. -1 means the last existing image in \fIWIMFILE\fR).
.IP ""
When this option is provided, the capture or append of the new image will be
optimized by not reading files that, based on metadata such as timestamps,
appear not to have been modified since they were archived in the existing
\fIIMAGE\fR. Barring manipulation of timestamps, this option only affects
-performance and does not change the resulting WIM file.
+performance and does not change the resulting WIM image.
.IP ""
As shown, the full syntax for the argument to this option is to specify the WIM
-file, a colon, and the image; for example, "--as-update-of mywim.wim:1".
-However, the WIM file may be omitted, in which case it will default to the WIM
-file being appended to for append operations, or the WIM file from which a delta
-is being taken (with \fB--as-delta-from\fR, if specified) for capture operations.
+file, a colon, and the image; for example, "--update-of mywim.wim:1". However,
+the WIM file and colon may be omitted, in which case the WIM file will default
+to the WIM file being appended to for append operations, or the WIM file from
+which a delta is being taken (only if \fB--delta-from\fR is specified exactly
+once) for capture operations.
.TP
-\fB--as-delta-from\fR=\fIWIMFILE\fR
+\fB--delta-from\fR=\fIWIMFILE\fR
For \fB@IMAGEX_PROGNAME@ capture\fR only: capture the new WIM as a "delta" from
\fIWIMFILE\fR. Any streams that would ordinarily need to be archived in the new
WIM are omitted if they are already present in the \fIWIMFILE\fR on which the
delta is being based. The new WIM will still contain a full copy of the image
metadata, but this is typically only a small fraction of a WIM's total size.
.IP ""
+This option can be specified multiple times, in which case the resulting delta
+WIM will only contain streams not present in any of the specified base
+\fIWIMFILE\fRs.
+.IP ""
To operate on the resulting delta WIM using other commands such as
\fB@IMAGEX_PROGNAME@ apply\fR, you must specify the delta WIM as the WIM file to
-operate on, but also reference the base WIM using the \fB--ref\fR option.
-Beware to retain the proper functioning of the delta WIM, you can only add, not
-delete, files and images to the base WIM following the capture of a delta from
-it.
+operate on, but also reference the base WIM(s) using the \fB--ref\fR option.
+Beware: to retain the proper functioning of the delta WIM, you can only add, not
+delete, files and images to the base WIM(s) following the capture of a delta
+from it.
.IP ""
-\fB--as-delta-from\fR may be combined with \fB--as-update-of\fR to increase the
+\fB--delta-from\fR may be combined with \fB--update-of\fR to increase the
speed of capturing a delta WIM.
.IP ""
As an example, consider the following backup and restore sequence:
(some days later, create second backup as delta from first)
$ wimcapture /some/directory bkup-2013-08-20.dwm \\
- --as-update-of=winbkup.wim:-1 --as-delta-from=winbkup.wim
+ --update-of bkup-base.wim:-1 --delta-from bkup-base.wim
(restoring the second backup)
using \fB@IMAGEX_PROGNAME@ append\fR. Delta WIMs should be used only if it's
desired to base the backups or images on a separate, large file that is rarely
modified.
+.IP ""
+Note: unlike "pipable" WIMs (created with the \fB--pipable\fR option), "delta"
+WIMs (created with the \fB--delta-from\fR option) are compatible with
+Microsoft's software. You can use the /ref option of imagex.exe to reference
+the base WIM(s), similar to above.
+.IP ""
+Additional note: \fB@IMAGEX_PROGNAME@\fR is generalized enough that you can in
+fact combine \fB--pipable\fR and \fB--delta-from\fR to create pipable delta
+WIMs. In such cases, the base WIM(s) must be captured as pipable as well as the
+delta WIM, and when applying an image, the base WIM(s) must be sent over the
+pipe after the delta WIM.
.SH NOTES
\fB@IMAGEX_PROGNAME@ append\fR does not support appending an image to a split WIM.
.PP
Pipable WIMs are incompatible with Microsoft's software. Pipable WIMs are
created only if \fIWIMFILE\fR was specified as "-" (standard output) or if
the \fB--pipable\fR flag was specified.
+.IP \[bu]
+WIMs captured with a non-default chunk size (with the \fB--chunk-size\fR option)
+or as solid archives (with the \fB--pack-streams\fR option) or with LZMS
+compression (with \fB--compress\fR=LZMS or \fB--compress\fR=recovery) have
+varying levels of compatibility with Microsoft's software. The best
+compatibility is achieved with WIMGAPI itself (not ImageX or Dism) on Windows 8
+or later.
.SH EXAMPLES
First example: Create a new WIM 'mywim.wim' with "maximum" (LZX) compression
that will contain a captured image of the directory tree 'somedir'. Note that
-\fB@IMAGEX_PROGNAME@\fR uses "maximum" (LZX) compression by default, so
-\fB--compress\fR does \fInot\fR need to be specified; furthermore, the image
-name need not be specified and will default to 'somedir':
+the image name need not be specified and will default to 'somedir':
.RS
.PP
-@IMAGEX_PROGNAME@ capture somedir mywim.wim
+@IMAGEX_PROGNAME@ capture somedir mywim.wim --compress=maximum
.RE
.PP
or, if the \fBwimcapture\fR hard link or batch file has been installed, the
abbreviated form can be used:
.RS
.PP
-wimcapture somedir mywim.wim
+wimcapture somedir mywim.wim --compress=maximum
.RE
.PP
The remaining examples will use the long form, however. Next, append the image
git://wimlib.net / wimlib / blobdiff