Eric Biggers [Sat, 22 Jul 2023 23:33:08 +0000 (16:33 -0700)]
Add a clang-format file
This approximates the de facto coding style of this project, which is
quite similar to the Linux kernel coding style.
This is just in case people to use it with 'git clang-format', as
requested at https://wimlib.net/forums/viewtopic.php?t=666. I'm not
planning to reformat all the existing code to make it match exactly.
Eric Biggers [Sat, 22 Jul 2023 22:01:50 +0000 (15:01 -0700)]
Fix compatibility issues with directory streams
Fix two related bugs where a WIM image metadata resource written by
wimlib could not be understood by some MS software versions:
- Never write an extra stream entry for the unnamed data stream of
directories. The concept of "unnamed data stream" isn't applicable to
directories. This fixes compatibility with DISM when the WIM image
contains a directory with named data stream(s). (I think DISM used to
be okay with this, but on 10.0.20348.681 I'm seeing this issue.)
- Store the reparse point hash in main_hash when no extra stream entries
are otherwise needed, i.e. when the file is a directory with no named
data streams. This fixes compatibility with the Windows 8 setup
wizard, at least for some specific version of Windows 8, as reported
at https://wimlib.net/forums/viewtopic.php?t=671.
Eric Biggers [Sat, 8 Jul 2023 19:12:08 +0000 (12:12 -0700)]
Improve encoding detection of wimupdate command files
Make wimupdate command files (read from stdin) and wimcapture source
list files be interpreted as UTF-8 or UTF-16LE (autodetected).
Previously these files were interpreted as the Windows codepage or
UTF-16LE on Windows (autodetected), and UTF-8 on non-Windows.
This makes these files be consistent with wimextract pathlist files and
wimcapture config files, which already had the behavior of "autodetect
UTF-8 or UTF-16LE on all platforms".
Eric Biggers [Mon, 26 Jun 2023 01:06:41 +0000 (18:06 -0700)]
Align stack when entering wimlib on 32-bit x86
This may resolve https://wimlib.net/forums/viewtopic.php?t=669. The
reported crash happened in sha1_blocks_x86_avx_bmi2() on the first
instruction that writes to the stack with vmovqda. That suggests the
stack was misaligned. If this indeed fixes the bug, then it has
actually always existed but it got exposed by new code in v1.14.0.
Eric Biggers [Thu, 27 Apr 2023 00:35:37 +0000 (17:35 -0700)]
cpu_features.c: fix unconditional execution of xgetbv
Ensure that the execution of the xgetbv instruction doesn't get moved
out from under the check for OSXSAVE support by a compiler optimization.
This fixes a crash on older CPUs, introduced in v1.14.0.
Reported at https://wimlib.net/forums/viewtopic.php?p=1527
Eric Biggers [Sun, 9 Apr 2023 18:39:36 +0000 (11:39 -0700)]
Fix wlfuzz and enable in CI
- Fix build failure on Linux by linking with the math library.
- Don't hard-code the temporary directory to E:\ on Windows.
- Relax some comparisons that don't work reliably on Windows:
- Security descriptor
- Last access time
- Sparse file attribute (clear only)
- Relax the timestamp comparisons when running on an ext4 filesystem,
since ext4 doesn't support the full timestamp range that Windows does.
Also, generate more timestamps that are close to the present date.
- Make the command-line argument give the number of seconds to run,
rather than the number of iterations. This makes it possible to run
wlfuzz for 2 minutes in GitHub Actions, like the libFuzzer jobs.
- Increase coverage by using a different random seed on each run.
- Make wlfuzz initialize wimlib with STRICT_{CAPTURE,APPLY}_PRIVILIGES
on Windows. I.e., don't allow running wlfuzz as non-Administrator.
- Print security descriptors and timestamps when they differ.
- Add GitHub Actions jobs that run wlfuzz on Linux and Windows.
Eric Biggers [Sun, 2 Apr 2023 06:29:13 +0000 (23:29 -0700)]
mount_image.c: avoid UBSAN warning in wimfs_listxattr()
When 'list == NULL && size == 0', the statement 'end = list + size'
executes 'NULL + 0'. clang's UndefinedBehaviorSanitizer complains that
this is undefined:
src/mount_image.c:1518:19: runtime error: applying zero offset to null pointer
Eric Biggers [Sun, 2 Apr 2023 06:29:13 +0000 (23:29 -0700)]
Upgrade WIM mounting support from fuse2 to fuse3
fuse2 is no longer maintained, and the various Linux distros want all
packages to move to fuse3. So, let's add fuse3 support.
Moreover, as it's been over 5 years since fuse3 was released, it doesn't
seem worth it to support both fuse2 and fuse3. So, remove support for
fuse2, and just support fuse3.
Note: as before, WIM mounting support can still be disabled by
configuring --without-fuse. That removes the dependency.
Eric Biggers [Fri, 31 Mar 2023 07:15:02 +0000 (00:15 -0700)]
make-windows-release: support cross-compiling for ARM64 in MSYS2
Cross-compiling for ARM64 is not possible with MSYS2 CLANGARM64 out of
the box, but it is possible to do by downloading and using a prebuilt
toolchain from https://github.com/mstorsjo/llvm-mingw. Make the
'make-windows-release' script support doing this.
Suggested at https://wimlib.net/forums/viewtopic.php?p=1488#p1488
Eric Biggers [Thu, 30 Mar 2023 08:16:52 +0000 (01:16 -0700)]
xml: fix wimlib_set_image_property()
The recent XML changes introduced a bug where
wimlib_set_image_property() rejected values of property_name containing
'/', '[', or ']', which are part of the "path" syntax.
Eric Biggers [Thu, 30 Mar 2023 07:58:20 +0000 (00:58 -0700)]
Remove the outdated 'INSTALL' file
The INSTALL file was an old version of the generic GNU installation
instructions that was not customized to this package at all and was not
referenced by any other documentation. It's unlikely to be useful.
Eric Biggers [Thu, 30 Mar 2023 07:34:14 +0000 (00:34 -0700)]
win32_*.c: avoid some format string warnings
These aren't real issues as all the types in question are 32-bit ints,
but fixing these will prevent warnings later if we're ever able to
enable __attribute__((format)) on Windows (hopefully due to
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64862 being fixed).
Eric Biggers [Thu, 30 Mar 2023 07:00:56 +0000 (00:00 -0700)]
compiler.h: remove _cold_attribute
This abstraction layer serves no purpose other than supporting gcc 4.3
and earlier, which is too old to bother supporting. Just use
__attribute__((cold)) directly.
Eric Biggers [Tue, 28 Mar 2023 06:14:42 +0000 (23:14 -0700)]
ci.yml: use -fno-sanitize-recover when using -fsanitize
Unlike ASAN, by default UBSAN doesn't abort the program on error but
instead just prints an error message. Use -fno-sanitize-recover to
explicitly request the abort-on-error behavior for testing purposes.
Eric Biggers [Tue, 28 Mar 2023 06:14:42 +0000 (23:14 -0700)]
encoding.c: avoid UBSAN warning in convert_string()
When 'in == NULL && in_nbytes == 0', the statement 'in_end = in +
in_nbytes' executes 'NULL + 0'. clang's UndefinedBehaviorSanitizer
complains that this is undefined:
src/encoding.c:223:31: runtime error: applying zero offset to null pointer
Eric Biggers [Mon, 27 Mar 2023 00:25:46 +0000 (17:25 -0700)]
Improve fuzz testing
- Convert fuzzing scripts from afl-fuzz to libFuzzer
- Add xml and wim fuzzers, including malloc failure injection
- Fuzz for 2 minutes as part of the GitHub Actions CI
Eric Biggers [Mon, 27 Mar 2023 00:25:46 +0000 (17:25 -0700)]
README: remove an unnecessary notice
The "copyright years may be listed using range notation" notice is
recommended by the GNU project, but most other people don't consider it
to be necessary or meaningful.
Eric Biggers [Mon, 27 Mar 2023 00:25:46 +0000 (17:25 -0700)]
Improve the make-windows-release script
- Automatically bootstrap the repository if needed
- Add --no-zip and --no-docs options
- Add --install-msys2-packages to install the needed MSYS2 packages
- Autodetect the architecture when using MSYS2
- Use 'strip' instead of ${ARCH}-w64-mingw32-strip, for compatibility
with MSYS2
- Make it pass 'shellcheck'
- Other cleanups
Eric Biggers [Mon, 27 Mar 2023 00:25:46 +0000 (17:25 -0700)]
Eliminate the dependency on libxml2
libxml2 is the only remaining third-party library that Windows builds of
wimlib need. It's a bit of a pain to have to download it, build it, and
trick libtool into linking it into the resulting DLL. It then
constitutes a significant part of the size of the resulting DLL, even
with the minimal libxml2 configuration options being used.
In reality, WIM files only use a small subset of XML containing the most
commonly used XML features. Using a full-featured XML library (that
supports "features" like External Entities that we have to remember to
disable) is a bit dangerous and not actually necessary. 7-Zip's WIM
support, for example, just uses a very minimal home-brew XML processor.
Another issue is that the libxml2 API always uses UTF-8, which causes
the conversion UTF-16LE => UTF-8 => UTF-16LE to be needed on Windows.
This isn't really an "issue", per se, but it shouldn't be necessary.
Finally, wimlib was integrating with libxml2 at a low level via the tree
API, and it overlooked some things. For example, libxml2 trees have
separate CDATA and TEXT nodes, but wimlib was only looking at TEXT, so
CDATA was ignored. It was also possible for wimlib to create a document
containing control characters, which is not valid XML so it could not be
read. These weren't very important issues, but the point is, just using
an XML library doesn't solve quite as many problems as one would hope...
Therefore, just add a simple XML 1.0 processor directly in the source
code. It handles all XML features that are used in WIM files, plus a
bit more for futureproofing. It's also faster than libxml2.
Eric Biggers [Mon, 20 Mar 2023 03:59:17 +0000 (20:59 -0700)]
Consistently use _WIN32 instead of __WIN32__
_WIN32 works with all compilers, while __WIN32__ is MinGW-specific.
This project used __WIN32__ in files that only support MinGW, and _WIN32
in other files such as the library header and example programs. One
place even used WIN32. Avoid this unnecessary complication by just
always using _WIN32.