Source list mode
authorEric Biggers <ebiggers3@gmail.com>
Wed, 6 Mar 2013 07:52:16 +0000 (01:52 -0600)
committerEric Biggers <ebiggers3@gmail.com>
Wed, 6 Mar 2013 07:52:16 +0000 (01:52 -0600)
doc/imagex-capture.1.in
programs/imagex.c
src/add_image.c
src/join.c
src/wimlib.h

index e97d32d..f17f0e6 100644 (file)
@@ -23,8 +23,11 @@ in the entire WIM, regardless of how many images the file appears in.
 
 \fISOURCE\fR specifies the location of the files to create the new WIM image
 from.  If \fISOURCE\fR is a directory, the WIM image is captured from that
-directory.  If \fISOURCE\fR is a regular file or block device, it is interpreted
-as a NTFS volume from which a WIM image is to be captured.
+directory.  Alternatively, \fISOURCE\fR is a regular file or block device, it is
+interpreted as a NTFS volume from which a WIM image is to be captured.  Still
+alternatively, if the \fB--source-list\fR option is given, \fISOURCE\fR is
+interpreted as a file that itself provides a list of files and directories to
+include in the new WIM image.
 
 \fIIMAGE_NAME\fR and \fIDESCRIPTION\fR specify the name and description of the
 new image.  If \fIIMAGE_NAME\fR is not given, it is taken to be the same as the
@@ -96,6 +99,13 @@ File attribute flags.
 All names of all files, including names in the Win32 namespace, DOS namespace,
 Win32+DOS namespace, and POSIX namespace.  This includes hard links.
 
+.SH SOURCE LIST MODE
+
+Yet another capture mode is entered when the \fB--source-list\fR option is
+given.  It is essentially an extension of the \fBNORMAL MODE\fR that allows
+multiple files or directories to be incorporated into a WIM image in a single
+command.  See the documentation for \fB--source-list\fR below.
+
 .SH OPTIONS
 .TP 6
 \fB--boot\fR
@@ -196,12 +206,57 @@ directories.  This is done by adding a special alternate data stream to each
 directory entry that contains this information.  Please note that this flag is
 for convenience only, in case you want to use \fBimagex\fR to archive files on
 UNIX.  Microsoft's software will not understand this special information.
+.TP
+\fB--source-list\fR
+\fBimagex capture\fR and \fBimagex append\fR, as of wimlib 1.2.7, support a new
+option to create a WIM image from multiple files or directories.  When
+\fB--source-list\fR is specified, the \fISOURCE\fR argument specifies the name
+of a text file, each line of which is either 1 or 2 whitespace separated
+filenames.  The first filename, the source, specifies the path to a file or
+directory to capture into the WIM image.  It may be either absolute or relative
+to the current working directory.  The second filename, if provided, is the
+target and specifies the path  in the WIM image that this file or directory will
+be saved as.  Leading and trailing slashes are ignored.  "/" indicates that
+the directory is to become the root of the WIM image.  If not specified, the
+target string defaults to the same as the source string.
+
+An example is as follows:
+
+.RS
+.RS
+.PP
+.nf
+# Make the WIM image from the 'winpe' directory
+winpe  /
+
+# Send the 'overlay' directory to '/overlay' in the WIM image
+overlay        /overlay
+
+# Overlay a separate directory directly on the root of the WIM image.
+# This is only legal if there are no conflicting files.
+/data/stuff    /
+.RE
+
+Subdirectories in the WIM are created as needed.  Multiple source directories
+may share the same target, which implies an overlay; however, an error is issued
+if the same file appears in different overlays.
+
+Filenames containing whitespace may be quoted with either single quotes or
+double quotes.  Quotes may not be escaped.
+
+Empty lines, and lines beginning with '#' followed by optional whitespace, are
+ignored.
+
+As a special case, if \fISOURCE\fR is "-" the source list is read from standard
+input rather than an external file.
+
+The NTFS capture mode cannot be used with \fB--source-list\fR.
 
 .SH NOTES
 
 \fBimage append\fR does not support appending an image to a split WIM.
 
-The two different capture modes only specify the data that is captured and don't
+The different capture modes only specify the data that is captured and don't
 specify a special WIM format.  A WIM file can contain images captured using
 different modes.  However, all images in a WIM must have the same compression
 type, and \fBimagex\fR always enforces this.
@@ -228,6 +283,12 @@ image with reading each file only one time, although this mode would have some
 limitations--- for example, a stream might be compressed only to be thrown away
 as a duplicate once it's been checksummed.
 
+\fISOURCE\fR may be a symbolic link to a directory rather than a directory
+itself.  However, additional symbolic links in subdirectories, or in additional
+source directories not destined for the WIM image root with
+(\fB--source-list\fR), are not dereferenced unless \fB--dereference\fR is
+specified.
+
 .SH EXAMPLES
 Create a new WIM 'mywim.wim' from the directory 'somedir', using LZX compression and
 including an integrity table:
index 97caae9..0b9af00 100644 (file)
@@ -26,6 +26,7 @@
 
 #include "wimlib.h"
 
+#include <ctype.h>
 #include <errno.h>
 #include <getopt.h>
 #include <glob.h>
 #include <sys/stat.h>
 #include <unistd.h>
 
+#ifdef HAVE_ALLOCA_H
+#include <alloca.h>
+#endif
+
 #define ARRAY_LEN(array) (sizeof(array) / sizeof(array[0]))
 
 #define for_opt(c, opts) while ((c = getopt_long_only(argc, (char**)argv, "", \
@@ -67,7 +72,8 @@ static const char *usage_strings[] = {
 "imagex append (DIRECTORY | NTFS_VOLUME) WIMFILE [IMAGE_NAME]\n"
 "                     [DESCRIPTION] [--boot] [--check] [--flags EDITION_ID]\n"
 "                     [--verbose] [--dereference] [--config=FILE]\n"
-"                     [--threads=NUM_THREADS] [--rebuild] [--unix-data]\n",
+"                     [--threads=NUM_THREADS] [--rebuild] [--unix-data]\n"
+"                     [--source-list]\n",
 [APPLY] =
 "imagex apply WIMFILE [IMAGE_NUM | IMAGE_NAME | all]\n"
 "                    (DIRECTORY | NTFS_VOLUME) [--check] [--hardlink]\n"
@@ -76,7 +82,8 @@ static const char *usage_strings[] = {
 "imagex capture (DIRECTORY | NTFS_VOLUME) WIMFILE [IMAGE_NAME]\n"
 "                      [DESCRIPTION] [--boot] [--check] [--compress=TYPE]\n"
 "                      [--flags EDITION_ID] [--verbose] [--dereference]\n"
-"                      [--config=FILE] [--threads=NUM_THREADS] [--unix-data]\n",
+"                      [--config=FILE] [--threads=NUM_THREADS] [--unix-data]\n"
+"                      [--source-list]\n",
 [DELETE] =
 "imagex delete WIMFILE (IMAGE_NUM | IMAGE_NAME | all) [--check] [--soft]\n",
 [DIR] =
@@ -134,6 +141,7 @@ static const struct option capture_or_append_options[] = {
        {"threads",     required_argument, NULL, 't'},
        {"rebuild",     no_argument,       NULL, 'R'},
        {"unix-data",   no_argument,       NULL, 'U'},
+       {"source-list", no_argument,       NULL, 'S'},
        {NULL, 0, NULL, 0},
 };
 static const struct option delete_options[] = {
@@ -295,23 +303,177 @@ static const char *default_capture_config =
 "*.cab\n"
 "\\WINDOWS\\inf\\*.pnf\n";
 
+static char *stdin_get_contents(size_t *len_ret)
+{
+       char *buf = NULL;
+       size_t newlen = 1024;
+       size_t pos = 0;
+       size_t inc = 1024;
+       for (;;) {
+               char *p = realloc(buf, newlen);
+               size_t bytes_read, bytes_to_read;
+               if (!p) {
+                       imagex_error("out of memory while reading stdin");
+                       break;
+               }
+               buf = p;
+               bytes_to_read = newlen - pos;
+               bytes_read = fread(&buf[pos], 1, bytes_to_read, stdin);
+               pos += bytes_read;
+               if (bytes_read != bytes_to_read) {
+                       if (feof(stdin)) {
+                               *len_ret = pos;
+                               return buf;
+                       } else {
+                               imagex_error_with_errno("error reading stdin");
+                               break;
+                       }
+               }
+               newlen += inc;
+               inc *= 3;
+               inc /= 2;
+       }
+       free(buf);
+       return NULL;
+}
+
+enum {
+       PARSE_FILENAME_SUCCESS = 0,
+       PARSE_FILENAME_FAILURE = 1,
+       PARSE_FILENAME_NONE = 2,
+};
+
+static int parse_filename(char **fn_ret, char **line_p, size_t *len_p)
+{
+       size_t len = *len_p;
+       char *line = *line_p;
+       char *fn;
+       int ret;
+       char quote_char;
+
+       /* Skip leading whitespace */
+       for (;;) {
+               if (len == 0)
+                       return PARSE_FILENAME_NONE;
+               if (!isspace(*line) && *line != '\0')
+                       break;
+               line++;
+               len--;
+       }
+       quote_char = *line;
+       if (quote_char == '"' || quote_char == '\'') {
+               /* Quoted filename */
+               line++;
+               len--;
+               fn = line;
+               line = memchr(line, quote_char, len);
+               if (line) {
+                       *line = '\0';
+                       len -= line - fn;
+                       ret = PARSE_FILENAME_SUCCESS;
+               } else {
+                       imagex_error("Missing closing quote: %s", fn - 1);
+                       return PARSE_FILENAME_FAILURE;
+               }
+       } else {
+               /* Unquoted filename.  Go until whitespace.  Line is terminated
+                * by '\0', so no need to check 'len'. */
+               fn = line;
+               do {
+                       line++;
+                       len--;
+               } while (!isspace(*line) && *line != '\0');
+               *line = '\0';
+               ret = PARSE_FILENAME_SUCCESS;
+       }
+       *len_p = len;
+       *line_p = line;
+       *fn_ret = fn;
+       return ret;
+}
+
+static bool
+parse_source_list_line(char *line, size_t len,
+                      struct wimlib_capture_source *source)
+{
+       int ret;
+       ret = parse_filename(&source->fs_source_path, &line, &len);
+       if (ret != PARSE_FILENAME_SUCCESS)
+               return false;
+       ret = parse_filename(&source->wim_target_path, &line, &len);
+       if (ret == PARSE_FILENAME_NONE)
+               source->wim_target_path = source->fs_source_path;
+       return ret != PARSE_FILENAME_FAILURE;
+}
+
+static bool is_comment_line(const char *line, size_t len)
+{
+       for (;;) {
+               if (*line == '#')
+                       return true;
+               if (!isspace(*line) && *line != '\0')
+                       return false;
+               ++line;
+               --len;
+               if (len == 0)
+                       return true;
+       }
+}
+
+static struct wimlib_capture_source *
+parse_source_list(char *source_list_contents, size_t source_list_nbytes,
+                 size_t *nsources_ret)
+{
+       size_t nlines;
+       char *p;
+       struct wimlib_capture_source *sources;
+       size_t i, j;
+
+       nlines = 0;
+       for (i = 0; i < source_list_nbytes; i++)
+               if (source_list_contents[i] == '\n')
+                       nlines++;
+       sources = calloc(nlines, sizeof(*sources));
+       if (!sources) {
+               imagex_error("out of memory");
+               return NULL;
+       }
+       p = source_list_contents;
+       j = 0;
+       for (i = 0; i < nlines; i++) {
+               char *endp = strchr(p, '\n');
+               size_t len = endp - p + 1;
+               *endp = '\0';
+               if (!is_comment_line(p, len)) {
+                       if (!parse_source_list_line(p, len, &sources[j++])) {
+                               free(sources);
+                               return NULL;
+                       }
+               }
+               p = endp + 1;
+
+       }
+       *nsources_ret = j;
+       return sources;
+}
+
 static char *file_get_contents(const char *filename, size_t *len_ret)
 {
        struct stat stbuf;
-       char *buf;
+       char *buf = NULL;
        size_t len;
        FILE *fp;
 
        if (stat(filename, &stbuf) != 0) {
                imagex_error_with_errno("Failed to stat the file `%s'", filename);
-               return NULL;
+               goto out;
        }
        len = stbuf.st_size;
 
        fp = fopen(filename, "rb");
        if (!fp) {
                imagex_error_with_errno("Failed to open the file `%s'", filename);
-               return NULL;
+               goto out;
        }
 
        buf = malloc(len);
@@ -326,12 +488,14 @@ static char *file_get_contents(const char *filename, size_t *len_ret)
                goto out_free_buf;
        }
        *len_ret = len;
-       return buf;
+       goto out_fclose;
 out_free_buf:
        free(buf);
+       buf = NULL;
 out_fclose:
        fclose(fp);
-       return NULL;
+out:
+       return buf;
 }
 
 static int file_writable(const char *path)
@@ -384,7 +548,13 @@ static int imagex_progress_func(enum wimlib_progress_msg msg,
                        putchar('\n');
                break;
        case WIMLIB_PROGRESS_MSG_SCAN_BEGIN:
-               printf("Scanning `%s'...\n", info->scan.source);
+               printf("Scanning `%s'", info->scan.source);
+               if (*info->scan.wim_target_path) {
+                       printf(" (loading as WIM path: `/%s')...\n",
+                              info->scan.wim_target_path);
+               } else {
+                       printf(" (loading as root of WIM image)...\n");
+               }
                break;
        case WIMLIB_PROGRESS_MSG_SCAN_DENTRY:
                if (info->scan.excluded)
@@ -560,7 +730,7 @@ static unsigned parse_num_threads(const char *optarg)
 
 
 /* Extract one image, or all images, from a WIM file into a directory. */
-static int imagex_apply(int argc, const char **argv)
+static int imagex_apply(int argc, char **argv)
 {
        int c;
        int open_flags = WIMLIB_OPEN_FLAG_SPLIT_OK;
@@ -673,28 +843,38 @@ out:
        return ret;
 }
 
-static int imagex_capture_or_append(int argc, const char **argv)
+static int imagex_capture_or_append(int argc, char **argv)
 {
        int c;
        int open_flags = 0;
        int add_image_flags = 0;
        int write_flags = 0;
        int compression_type = WIMLIB_COMPRESSION_TYPE_XPRESS;
-       const char *source;
        const char *wimfile;
        const char *name;
        const char *desc;
        const char *flags_element = NULL;
-       const char *config_file = NULL;
-       char *config_str = NULL;
-       size_t config_len = 0;
        WIMStruct *w = NULL;
        int ret;
        int cur_image;
-       char *default_name;
        int cmd = strcmp(argv[0], "append") ? CAPTURE : APPEND;
        unsigned num_threads = 0;
 
+       char *source;
+       size_t source_name_len;
+       char *source_copy;
+
+       const char *config_file = NULL;
+       char *config_str = NULL;
+       size_t config_len = 0;
+
+       bool source_list = false;
+       size_t source_list_nbytes;
+       char *source_list_contents = NULL;
+       bool capture_sources_malloced = false;
+       struct wimlib_capture_source *capture_sources;
+       size_t num_sources;
+
        for_opt(c, capture_or_append_options) {
                switch (c) {
                case 'b':
@@ -732,6 +912,9 @@ static int imagex_capture_or_append(int argc, const char **argv)
                case 'U':
                        add_image_flags |= WIMLIB_ADD_IMAGE_FLAG_UNIX_DATA;
                        break;
+               case 'S':
+                       source_list = true;
+                       break;
                default:
                        usage(cmd);
                        return -1;
@@ -739,24 +922,56 @@ static int imagex_capture_or_append(int argc, const char **argv)
        }
        argc -= optind;
        argv += optind;
+
        if (argc < 2 || argc > 4) {
                usage(cmd);
                return -1;
        }
+
        source = argv[0];
        wimfile = argv[1];
 
-       char source_copy[strlen(source) + 1];
-       memcpy(source_copy, source, strlen(source) + 1);
-       default_name = basename(source_copy);
-
-       name = (argc >= 3) ? argv[2] : default_name;
+       if (argc >= 3) {
+               name = argv[2];
+       } else {
+               source_name_len = strlen(source);
+               source_copy = alloca(source_name_len + 1);
+               name = basename(strcpy(source_copy, source));
+       }
        desc = (argc >= 4) ? argv[3] : NULL;
 
+       if (source_list) {
+               if (source[0] == '-' && source[1] == '\0') {
+                       source_list_contents = stdin_get_contents(&source_list_nbytes);
+               } else {
+                       source_list_contents = file_get_contents(source,
+                                                                &source_list_nbytes);
+               }
+               if (!source_list_contents)
+                       return -1;
+
+               capture_sources = parse_source_list(source_list_contents,
+                                                   source_list_nbytes,
+                                                   &num_sources);
+               if (!capture_sources) {
+                       ret = -1;
+                       goto out;
+               }
+               capture_sources_malloced = true;
+       } else {
+               capture_sources = alloca(sizeof(struct wimlib_capture_source));
+               capture_sources[0].fs_source_path = source;
+               capture_sources[0].wim_target_path = NULL;
+               capture_sources[0].reserved = 0;
+               num_sources = 1;
+       }
+
        if (config_file) {
                config_str = file_get_contents(config_file, &config_len);
-               if (!config_str)
-                       return -1;
+               if (!config_str) {
+                       ret = -1;
+                       goto out;
+               }
        }
 
        if (cmd == APPEND)
@@ -767,28 +982,32 @@ static int imagex_capture_or_append(int argc, const char **argv)
        if (ret != 0)
                goto out;
 
-       struct stat stbuf;
-
-       ret = stat(source, &stbuf);
-       if (ret == 0) {
-               if (S_ISBLK(stbuf.st_mode) || S_ISREG(stbuf.st_mode)) {
-                       printf("Capturing WIM image from NTFS filesystem on `%s'\n",
-                              source);
-                       add_image_flags |= WIMLIB_ADD_IMAGE_FLAG_NTFS;
-               }
-       } else {
-               if (errno != ENOENT) {
-                       imagex_error_with_errno("Failed to stat `%s'", source);
-                       ret = -1;
-                       goto out;
+       if (!source_list) {
+               struct stat stbuf;
+               ret = stat(source, &stbuf);
+               if (ret == 0) {
+                       if (S_ISBLK(stbuf.st_mode) || S_ISREG(stbuf.st_mode)) {
+                               printf("Capturing WIM image from NTFS filesystem on `%s'\n",
+                                      source);
+                               add_image_flags |= WIMLIB_ADD_IMAGE_FLAG_NTFS;
+                       }
+               } else {
+                       if (errno != ENOENT) {
+                               imagex_error_with_errno("Failed to stat `%s'", source);
+                               ret = -1;
+                               goto out;
+                       }
                }
        }
 
-       ret = wimlib_add_image(w, source, name,
-                              (config_str ? config_str : default_capture_config),
-                              (config_str ? config_len : strlen(default_capture_config)),
-                              add_image_flags, imagex_progress_func);
-
+       ret = wimlib_add_image_multisource(w, capture_sources,
+                                          num_sources, name,
+                                          (config_str ? config_str :
+                                               default_capture_config),
+                                          (config_str ? config_len :
+                                               strlen(default_capture_config)),
+                                          add_image_flags,
+                                          imagex_progress_func);
        if (ret != 0)
                goto out;
        cur_image = wimlib_get_num_images(w);
@@ -816,11 +1035,14 @@ static int imagex_capture_or_append(int argc, const char **argv)
 out:
        wimlib_free(w);
        free(config_str);
+       free(source_list_contents);
+       if (capture_sources_malloced)
+               free(capture_sources);
        return ret;
 }
 
 /* Remove image(s) from a WIM. */
-static int imagex_delete(int argc, const char **argv)
+static int imagex_delete(int argc, char **argv)
 {
        int c;
        int open_flags = 0;
@@ -893,7 +1115,7 @@ out:
 }
 
 /* Print the files contained in an image(s) in a WIM file. */
-static int imagex_dir(int argc, const char **argv)
+static int imagex_dir(int argc, char **argv)
 {
        const char *wimfile;
        WIMStruct *w;
@@ -945,7 +1167,7 @@ out:
 
 /* Exports one, or all, images from a WIM file to a new WIM file or an existing
  * WIM file. */
-static int imagex_export(int argc, const char **argv)
+static int imagex_export(int argc, char **argv)
 {
        int c;
        int open_flags = 0;
@@ -1109,7 +1331,7 @@ out:
 
 /* Prints information about a WIM file; also can mark an image as bootable,
  * change the name of an image, or change the description of an image. */
-static int imagex_info(int argc, const char **argv)
+static int imagex_info(int argc, char **argv)
 {
        int c;
        bool boot         = false;
@@ -1389,7 +1611,7 @@ out:
 }
 
 /* Join split WIMs into one part WIM */
-static int imagex_join(int argc, const char **argv)
+static int imagex_join(int argc, char **argv)
 {
        int c;
        int swm_open_flags = WIMLIB_OPEN_FLAG_SPLIT_OK;
@@ -1415,15 +1637,16 @@ static int imagex_join(int argc, const char **argv)
                goto err;
        }
        output_path = argv[0];
-       return wimlib_join(++argv, --argc, output_path, swm_open_flags,
-                          wim_write_flags, imagex_progress_func);
+       return wimlib_join((const char **)++argv, --argc, output_path,
+                          swm_open_flags, wim_write_flags,
+                          imagex_progress_func);
 err:
        usage(JOIN);
        return -1;
 }
 
 /* Mounts an image using a FUSE mount. */
-static int imagex_mount_rw_or_ro(int argc, const char **argv)
+static int imagex_mount_rw_or_ro(int argc, char **argv)
 {
        int c;
        int mount_flags = 0;
@@ -1545,7 +1768,7 @@ mount_usage:
        return -1;
 }
 
-static int imagex_optimize(int argc, const char **argv)
+static int imagex_optimize(int argc, char **argv)
 {
        int c;
        int open_flags = 0;
@@ -1616,7 +1839,7 @@ static int imagex_optimize(int argc, const char **argv)
 }
 
 /* Split a WIM into a spanned set */
-static int imagex_split(int argc, const char **argv)
+static int imagex_split(int argc, char **argv)
 {
        int c;
        int open_flags = WIMLIB_OPEN_FLAG_SPLIT_OK;
@@ -1659,7 +1882,7 @@ static int imagex_split(int argc, const char **argv)
 }
 
 /* Unmounts an image. */
-static int imagex_unmount(int argc, const char **argv)
+static int imagex_unmount(int argc, char **argv)
 {
        int c;
        int unmount_flags = 0;
@@ -1697,7 +1920,7 @@ static int imagex_unmount(int argc, const char **argv)
 
 struct imagex_command {
        const char *name;
-       int (*func)(int , const char **);
+       int (*func)(int , char **);
        int cmd;
 };
 
@@ -1735,7 +1958,7 @@ static void version()
 }
 
 
-static void help_or_version(int argc, const char **argv)
+static void help_or_version(int argc, char **argv)
 {
        int i;
        const char *p;
@@ -1795,7 +2018,7 @@ static void usage_all()
 }
 
 
-int main(int argc, const char **argv)
+int main(int argc, char **argv)
 {
        const struct imagex_command *cmd;
        int ret;
index 603f4da..0d1ffb2 100644 (file)
 #include "xml.h"
 #include <string.h>
 #include <fnmatch.h>
+#include <stdlib.h>
 #include <ctype.h>
 #include <sys/stat.h>
 #include <dirent.h>
 #include <errno.h>
 #include <unistd.h>
 
-/** Private flag: Used to mark that we currently adding the root directory of
- * the WIM image. */
-#define WIMLIB_ADD_IMAGE_FLAG_ROOT 0x80000000
+#define WIMLIB_ADD_IMAGE_FLAG_ROOT     0x80000000
+#define WIMLIB_ADD_IMAGE_FLAG_SOURCE    0x40000000
 
 /*
  * Adds the dentry tree and security data for a new image to the image metadata
@@ -136,7 +136,6 @@ static int build_dentry_tree(struct wim_dentry **root_ret,
        int ret = 0;
        int (*stat_fn)(const char *restrict, struct stat *restrict);
        struct wim_dentry *root;
-       const char *filename;
        struct wim_inode *inode;
 
        if (exclude_path(root_disk_path, config, true)) {
@@ -199,12 +198,7 @@ static int build_dentry_tree(struct wim_dentry **root_ret,
                return WIMLIB_ERR_SPECIAL_FILE;
        }
 
-       if (add_image_flags & WIMLIB_ADD_IMAGE_FLAG_ROOT)
-               filename = "";
-       else
-               filename = path_basename(root_disk_path);
-
-       root = new_dentry_with_timeless_inode(filename);
+       root = new_dentry_with_timeless_inode(path_basename(root_disk_path));
        if (!root) {
                if (errno == EILSEQ)
                        return WIMLIB_ERR_INVALID_UTF8_STRING;
@@ -240,7 +234,7 @@ static int build_dentry_tree(struct wim_dentry **root_ret,
                if (ret)
                        goto out;
        }
-       add_image_flags &= ~WIMLIB_ADD_IMAGE_FLAG_ROOT;
+       add_image_flags &= ~(WIMLIB_ADD_IMAGE_FLAG_ROOT | WIMLIB_ADD_IMAGE_FLAG_SOURCE);
        if (S_ISREG(root_stbuf.st_mode)) { /* Archiving a regular file */
 
                struct wim_lookup_table_entry *lte;
@@ -615,10 +609,240 @@ bool exclude_path(const char *path, const struct capture_config *config,
 
 }
 
-WIMLIBAPI int wimlib_add_image(WIMStruct *w, const char *source,
-                              const char *name, const char *config_str,
-                              size_t config_len, int add_image_flags,
-                              wimlib_progress_func_t progress_func)
+/* Strip leading and trailing forward slashes from a string.  Modifies it in
+ * place and returns the stripped string. */
+static const char *canonicalize_target_path(char *target_path)
+{
+       char *p;
+       if (target_path == NULL)
+               target_path = "";
+       for (;;) {
+               if (*target_path == '\0')
+                       return target_path;
+               else if (*target_path == '/')
+                       target_path++;
+               else
+                       break;
+       }
+
+       p = target_path + strlen(target_path) - 1;
+       while (*p == '/')
+               *p-- = '\0';
+       return target_path;
+}
+
+/* Strip leading and trailing slashes from the target paths */
+static void canonicalize_targets(struct wimlib_capture_source *sources,
+                                size_t num_sources)
+{
+       while (num_sources--) {
+               DEBUG("Canonicalizing { source: \"%s\", target=\"%s\"}",
+                     sources->fs_source_path,
+                     sources->wim_target_path);
+               sources->wim_target_path =
+                       (char*)canonicalize_target_path(sources->wim_target_path);
+               DEBUG("\"%s\"", sources->wim_target_path);
+               sources++;
+       }
+}
+
+static int capture_source_cmp(const void *p1, const void *p2)
+{
+       const struct wimlib_capture_source *s1, *s2;
+
+       s1 = p1;
+       s2 = p2;
+       return strcmp(s1->wim_target_path, s2->wim_target_path);
+}
+
+/* Sorts the capture sources lexicographically by target path.  This occurs
+ * after leading and trailing forward slashes are stripped.
+ *
+ * One purpose of this is to make sure that target paths that are inside other
+ * target paths are extracted after the containing target paths. */
+static void sort_sources(struct wimlib_capture_source *sources,
+                        size_t num_sources)
+{
+       qsort(sources, num_sources, sizeof(sources[0]), capture_source_cmp);
+}
+
+static int check_sorted_sources(struct wimlib_capture_source *sources,
+                               size_t num_sources, int add_image_flags)
+{
+       if (add_image_flags & WIMLIB_ADD_IMAGE_FLAG_NTFS) {
+               if (num_sources != 1) {
+                       ERROR("Must specify exactly 1 capture source "
+                             "(the NTFS volume) in NTFS mode!");
+                       return WIMLIB_ERR_INVALID_PARAM;
+               }
+               if (sources[0].wim_target_path[0] != '\0') {
+                       ERROR("In NTFS capture mode the target path inside "
+                             "the image must be the root directory!");
+                       return WIMLIB_ERR_INVALID_PARAM;
+               }
+       } else if (num_sources != 0) {
+               /* This code is disabled because the current code
+                * unconditionally attempts to do overlays.  So, duplicate
+                * target paths are OK. */
+       #if 0
+               if (num_sources > 1 && sources[0].wim_target_path[0] == '\0') {
+                       ERROR("Cannot specify root target when using multiple "
+                             "capture sources!");
+                       return WIMLIB_ERR_INVALID_PARAM;
+               }
+               for (size_t i = 0; i < num_sources - 1; i++) {
+                       size_t len = strlen(sources[i].wim_target_path);
+                       size_t j = i + 1;
+                       const char *target1 = sources[i].wim_target_path;
+                       do {
+                               const char *target2 = sources[j].wim_target_path;
+                               DEBUG("target1=%s, target2=%s",
+                                     target1,target2);
+                               if (strncmp(target1, target2, len) ||
+                                   target2[len] > '/')
+                                       break;
+                               if (target2[len] == '/') {
+                                       ERROR("Invalid target `%s': is a prefix of `%s'",
+                                             target1, target2);
+                                       return WIMLIB_ERR_INVALID_PARAM;
+                               }
+                               if (target2[len] == '\0') {
+                                       ERROR("Invalid target `%s': is a duplicate of `%s'",
+                                             target1, target2);
+                                       return WIMLIB_ERR_INVALID_PARAM;
+                               }
+                       } while (++j != num_sources);
+               }
+       #endif
+       }
+       return 0;
+
+}
+
+/* Creates a new directory to place in the WIM image.  This is to create parent
+ * directories that are not part of any target as needed.  */
+static struct wim_dentry *
+new_filler_directory(const char *name)
+{
+       struct wim_dentry *dentry;
+       DEBUG("Creating filler directory \"%s\"", name);
+       dentry = new_dentry_with_inode(name);
+       if (dentry) {
+               dentry->d_inode->i_ino = 0;
+               dentry->d_inode->i_resolved = 1;
+               dentry->d_inode->i_attributes = FILE_ATTRIBUTE_DIRECTORY;
+       }
+       return dentry;
+}
+
+/* Transfers the children of @branch to @target.  It is an error if @target is
+ * not a directory or if both @branch and @target contain a child dentry with
+ * the same name. */
+static int do_overlay(struct wim_dentry *target,
+                     struct wim_dentry *branch)
+{
+       struct rb_root *rb_root;
+
+       if (!dentry_is_directory(target)) {
+               ERROR("Cannot overlay directory `%s' over non-directory",
+                     branch->file_name_utf8);
+               /* XXX Use a different error code */
+               return WIMLIB_ERR_INVALID_DENTRY;
+       }
+
+       rb_root = &branch->d_inode->i_children;
+       while (rb_root->rb_node) { /* While @branch has children... */
+               struct wim_dentry *child;
+
+               child = container_of(rb_root->rb_node, struct wim_dentry, rb_node);
+               unlink_dentry(child);
+               if (!dentry_add_child(target, child)) {
+                       dentry_add_child(branch, child);
+                       ERROR("Overlay error: file `%s' already exists as child of `%s'",
+                             child->file_name_utf8, target->file_name_utf8);
+                       return WIMLIB_ERR_INVALID_DENTRY;
+               }
+       }
+       return 0;
+
+}
+
+/* Attach or overlay a branch onto the WIM image.
+ *
+ * @root_p:
+ *     Pointer to the root of the WIM image, or pointer to NULL if it has not
+ *     been created yet.
+ * @branch
+ *     Branch to add.
+ * @target_path:
+ *     Path in the WIM image to add the branch.
+ */
+static int attach_branch(struct wim_dentry **root_p,
+                        struct wim_dentry *branch,
+                        char *target_path)
+{
+       char *slash;
+       struct wim_dentry *dentry, *parent, *target;
+
+       if (*target_path == '\0') {
+               /* Target: root directory */
+               if (*root_p) {
+                       /* Overlay on existing root */
+                       return do_overlay(*root_p, branch);
+               } else  {
+                       /* Set as root */
+                       *root_p = branch;
+                       return 0;
+               }
+       }
+
+       /* Adding a non-root branch.  Create root if it hasn't been created
+        * already. */
+       if (!*root_p) {
+               *root_p = new_filler_directory("");
+               if (!*root_p)
+                       return WIMLIB_ERR_NOMEM;
+       }
+
+       /* Walk the path to the branch, creating filler directories as needed.
+        * */
+       parent = *root_p;
+       while ((slash = strchr(target_path, '/'))) {
+               *slash = '\0';
+               dentry = get_dentry_child_with_name(parent, target_path);
+               if (!dentry) {
+                       dentry = new_filler_directory(target_path);
+                       if (!dentry)
+                               return WIMLIB_ERR_NOMEM;
+                       dentry_add_child(parent, dentry);
+               }
+               parent = dentry;
+               target_path = slash;
+               do {
+                       ++target_path;
+                       wimlib_assert(*target_path != '\0');
+               } while (*target_path == '/');
+       }
+
+       /* If the target path already existed, overlay the branch onto it.
+        * Otherwise, set the branch as the target path. */
+       target = get_dentry_child_with_name(parent, branch->file_name_utf8);
+       if (target) {
+               return do_overlay(target, branch);
+       } else {
+               dentry_add_child(parent, branch);
+               return 0;
+       }
+}
+
+WIMLIBAPI int wimlib_add_image_multisource(WIMStruct *w,
+                                          struct wimlib_capture_source *sources,
+                                          size_t num_sources,
+                                          const char *name,
+                                          const char *config_str,
+                                          size_t config_len,
+                                          int add_image_flags,
+                                          wimlib_progress_func_t progress_func)
 {
        int (*capture_tree)(struct wim_dentry **, const char *,
                            struct wim_lookup_table *,
@@ -626,7 +850,6 @@ WIMLIBAPI int wimlib_add_image(WIMStruct *w, const char *source,
                            const struct capture_config *,
                            int, wimlib_progress_func_t, void *);
        void *extra_arg;
-
        struct wim_dentry *root_dentry = NULL;
        struct wim_security_data *sd;
        struct capture_config config;
@@ -656,16 +879,10 @@ WIMLIBAPI int wimlib_add_image(WIMStruct *w, const char *source,
                extra_arg = NULL;
        }
 
-       DEBUG("Adding dentry tree from directory or NTFS volume `%s'.", source);
-
        if (!name || !*name) {
                ERROR("Must specify a non-empty string for the image name");
                return WIMLIB_ERR_INVALID_PARAM;
        }
-       if (!source || !*source) {
-               ERROR("Must specify the name of a directory or NTFS volume");
-               return WIMLIB_ERR_INVALID_PARAM;
-       }
 
        if (w->hdr.total_parts != 1) {
                ERROR("Cannot add an image to a split WIM");
@@ -678,15 +895,12 @@ WIMLIBAPI int wimlib_add_image(WIMStruct *w, const char *source,
                return WIMLIB_ERR_IMAGE_NAME_COLLISION;
        }
 
-       DEBUG("Initializing capture configuration");
        if (!config_str) {
                DEBUG("Using default capture configuration");
                config_str = default_config;
                config_len = strlen(default_config);
        }
-       ret = init_capture_config(config_str, config_len, source, &config);
-       if (ret != 0)
-               return ret;
+       memset(&config, 0, sizeof(struct capture_config));
 
        DEBUG("Allocating security data");
 
@@ -698,27 +912,73 @@ WIMLIBAPI int wimlib_add_image(WIMStruct *w, const char *source,
        sd->total_length = 8;
        sd->refcnt = 1;
 
-       if (progress_func) {
-               union wimlib_progress_info progress;
-               progress.scan.source = source;
-               progress_func(WIMLIB_PROGRESS_MSG_SCAN_BEGIN, &progress);
+       DEBUG("Using %zu capture sources", num_sources);
+       canonicalize_targets(sources, num_sources);
+       sort_sources(sources, num_sources);
+       ret = check_sorted_sources(sources, num_sources, add_image_flags);
+       if (ret) {
+               ret = WIMLIB_ERR_INVALID_PARAM;
+               goto out_free_security_data;
        }
 
        DEBUG("Building dentry tree.");
-       ret = (*capture_tree)(&root_dentry, source, w->lookup_table, sd,
-                             &config, add_image_flags | WIMLIB_ADD_IMAGE_FLAG_ROOT,
-                             progress_func, extra_arg);
-       destroy_capture_config(&config);
-
-       if (ret != 0) {
-               ERROR("Failed to build dentry tree for `%s'", source);
-               goto out_free_security_data;
-       }
+       if (num_sources == 0) {
+               root_dentry = new_filler_directory("");
+               if (!root_dentry)
+                       goto out_free_security_data;
+       } else {
+               size_t i = 0;
+               struct wim_dentry *branch;
+               int flags;
+               do {
+                       DEBUG("Building dentry tree for source %zu of %zu "
+                             "(\"%s\" => \"%s\")", i + 1, num_sources,
+                             sources[i].fs_source_path,
+                             sources[i].wim_target_path);
+                       union wimlib_progress_info progress;
+                       if (progress_func) {
+                               memset(&progress, 0, sizeof(progress));
+                               progress.scan.source = sources[i].fs_source_path;
+                               progress.scan.wim_target_path = sources[i].wim_target_path;
+                               progress_func(WIMLIB_PROGRESS_MSG_SCAN_BEGIN, &progress);
+                       }
+                       ret = init_capture_config(config_str, config_len,
+                                                 sources[i].fs_source_path,
+                                                 &config);
+                       if (ret)
+                               goto out_free_dentry_tree;
+                       flags = add_image_flags | WIMLIB_ADD_IMAGE_FLAG_SOURCE;
+                       if (!*sources[i].wim_target_path)
+                               flags |= WIMLIB_ADD_IMAGE_FLAG_ROOT;
+                       ret = (*capture_tree)(&branch, sources[i].fs_source_path,
+                                             w->lookup_table, sd,
+                                             &config,
+                                             flags,
+                                             progress_func, extra_arg);
+                       if (ret) {
+                               ERROR("Failed to build dentry tree for `%s'",
+                                     sources[i].fs_source_path);
+                               goto out_free_dentry_tree;
+                       }
+                       if (branch) {
+                               ret = set_dentry_name(branch,
+                                                     path_basename(sources[i].wim_target_path));
+                               if (ret) {
+                                       free_dentry_tree(branch, w->lookup_table);
+                                       goto out_free_dentry_tree;
+                               }
 
-       if (progress_func) {
-               union wimlib_progress_info progress;
-               progress.scan.source = source;
-               progress_func(WIMLIB_PROGRESS_MSG_SCAN_END, &progress);
+                               ret = attach_branch(&root_dentry, branch,
+                                                   sources[i].wim_target_path);
+                               if (ret) {
+                                       free_dentry_tree(branch, w->lookup_table);
+                                       goto out_free_dentry_tree;
+                               }
+                       }
+                       destroy_capture_config(&config);
+                       if (progress_func)
+                               progress_func(WIMLIB_PROGRESS_MSG_SCAN_END, &progress);
+               } while (++i != num_sources);
        }
 
        DEBUG("Calculating full paths of dentries.");
@@ -745,7 +1005,8 @@ WIMLIBAPI int wimlib_add_image(WIMStruct *w, const char *source,
 
        if (add_image_flags & WIMLIB_ADD_IMAGE_FLAG_BOOT)
                wimlib_set_boot_idx(w, w->hdr.image_count);
-       return 0;
+       ret = 0;
+       goto out;
 out_destroy_imd:
        destroy_image_metadata(&w->image_metadata[w->hdr.image_count - 1],
                               w->lookup_table);
@@ -757,5 +1018,28 @@ out_free_security_data:
        free_security_data(sd);
 out_destroy_config:
        destroy_capture_config(&config);
+out:
+       return ret;
+}
+
+WIMLIBAPI int wimlib_add_image(WIMStruct *w, const char *source,
+                              const char *name, const char *config_str,
+                              size_t config_len, int add_image_flags,
+                              wimlib_progress_func_t progress_func)
+{
+       if (!source || !*source)
+               return WIMLIB_ERR_INVALID_PARAM;
+
+       char *fs_source_path = STRDUP(source);
+       int ret;
+       struct wimlib_capture_source capture_src = {
+               .fs_source_path = fs_source_path,
+               .wim_target_path = NULL,
+               .reserved = 0,
+       };
+       ret = wimlib_add_image_multisource(w, &capture_src, 1, name,
+                                          config_str, config_len,
+                                          add_image_flags, progress_func);
+       FREE(fs_source_path);
        return ret;
 }
index 8a222f8..4d05905 100644 (file)
@@ -160,7 +160,7 @@ static int cmp_swms_by_part_number(const void *swm1, const void *swm2)
 /*
  * Join a set of split WIMs into a stand-alone WIM.
  */
-WIMLIBAPI int wimlib_join(const char **swm_names, unsigned num_swms,
+WIMLIBAPI int wimlib_join(const char * const *swm_names, unsigned num_swms,
                          const char *output_path, int swm_open_flags,
                          int wim_write_flags,
                          wimlib_progress_func_t progress_func)
index 62a028d..4af3d76 100644 (file)
@@ -9,7 +9,7 @@
  */
 
 /*
- * Copyright (C) 2012 Eric Biggers
+ * Copyright (C) 2012, 2013 Eric Biggers
  *
  * This file is part of wimlib, a library for working with WIM files.
  *
@@ -414,6 +414,11 @@ union wimlib_progress_info {
                /** True iff @a cur_path is being excluded from the image
                 * capture due to the capture configuration file. */
                bool excluded;
+
+               /** Target path in the WIM.  Only valid on messages
+                * ::WIMLIB_PROGRESS_MSG_SCAN_BEGIN and
+                * ::WIMLIB_PROGRESS_MSG_SCAN_END. */
+               const char *wim_target_path;
        } scan;
 
        /** Valid on messages ::WIMLIB_PROGRESS_MSG_EXTRACT_IMAGE_BEGIN,
@@ -552,6 +557,22 @@ union wimlib_progress_info {
 typedef int (*wimlib_progress_func_t)(enum wimlib_progress_msg msg_type,
                                      const union wimlib_progress_info *info);
 
+/** An array of these structures is passed to wimlib_add_image_multisource() to
+ * specify the sources from which to create a WIM image. */
+struct wimlib_capture_source {
+       /** Absolute or relative path to a file or directory on the external
+        * filesystem to be included in the WIM image. */
+       char *fs_source_path;
+
+       /** Destination path in the WIM image.  Leading and trailing slashes are
+        * ignored.  The empty string or @c NULL means the root directory of the
+        * WIM image. */
+       char *wim_target_path;
+
+       /** Reserved; set to 0. */
+       long reserved;
+};
+
 
 /*****************************
  * WIMLIB_ADD_IMAGE_FLAG_*   *
@@ -866,6 +887,21 @@ extern int wimlib_add_image(WIMStruct *wim, const char *source,
                            size_t config_len, int add_image_flags,
                            wimlib_progress_func_t progress_func);
 
+/** This function is equivalent to wimlib_add_image() except it allows for
+ * multiple sources to be combined into a single WIM image.  This is done by
+ * specifying the @a sources and @a num_sources parameters instead of the @a
+ * source parameter.  The rest of the parameters are the same as
+ * wimlib_add_image().  See the documentation for <b>imagex capture</b> for full
+ * details on how this mode works. */
+extern int wimlib_add_image_multisource(WIMStruct *w,
+                                       struct wimlib_capture_source *sources,
+                                       size_t num_sources,
+                                       const char *name,
+                                       const char *config_str,
+                                       size_t config_len,
+                                       int add_image_flags,
+                                       wimlib_progress_func_t progress_func);
+
 /**
  * Creates a ::WIMStruct for a new WIM file.
  *
@@ -1414,7 +1450,7 @@ extern bool wimlib_image_name_in_use(const WIMStruct *wim, const char *name);
  * Note: wimlib_export_image() can provide similar functionality to
  * wimlib_join(), since it is possible to export all images from a split WIM.
  */
-extern int wimlib_join(const char **swms, unsigned num_swms,
+extern int wimlib_join(const char * const *swms, unsigned num_swms,
                       const char *output_path, int swm_open_flags,
                       int wim_write_flags,
                       wimlib_progress_func_t progress_func);