Eric Biggers [Sun, 9 Apr 2023 18:39:36 +0000 (11:39 -0700)]
Fix wlfuzz and enable in CI
- Fix build failure on Linux by linking with the math library.
- Don't hard-code the temporary directory to E:\ on Windows.
- Relax some comparisons that don't work reliably on Windows:
- Security descriptor
- Last access time
- Sparse file attribute (clear only)
- Relax the timestamp comparisons when running on an ext4 filesystem,
since ext4 doesn't support the full timestamp range that Windows does.
Also, generate more timestamps that are close to the present date.
- Make the command-line argument give the number of seconds to run,
rather than the number of iterations. This makes it possible to run
wlfuzz for 2 minutes in GitHub Actions, like the libFuzzer jobs.
- Increase coverage by using a different random seed on each run.
- Make wlfuzz initialize wimlib with STRICT_{CAPTURE,APPLY}_PRIVILIGES
on Windows. I.e., don't allow running wlfuzz as non-Administrator.
- Print security descriptors and timestamps when they differ.
- Add GitHub Actions jobs that run wlfuzz on Linux and Windows.
Eric Biggers [Sun, 2 Apr 2023 06:29:13 +0000 (23:29 -0700)]
mount_image.c: avoid UBSAN warning in wimfs_listxattr()
When 'list == NULL && size == 0', the statement 'end = list + size'
executes 'NULL + 0'. clang's UndefinedBehaviorSanitizer complains that
this is undefined:
src/mount_image.c:1518:19: runtime error: applying zero offset to null pointer
Eric Biggers [Sun, 2 Apr 2023 06:29:13 +0000 (23:29 -0700)]
Upgrade WIM mounting support from fuse2 to fuse3
fuse2 is no longer maintained, and the various Linux distros want all
packages to move to fuse3. So, let's add fuse3 support.
Moreover, as it's been over 5 years since fuse3 was released, it doesn't
seem worth it to support both fuse2 and fuse3. So, remove support for
fuse2, and just support fuse3.
Note: as before, WIM mounting support can still be disabled by
configuring --without-fuse. That removes the dependency.
Eric Biggers [Fri, 31 Mar 2023 07:15:02 +0000 (00:15 -0700)]
make-windows-release: support cross-compiling for ARM64 in MSYS2
Cross-compiling for ARM64 is not possible with MSYS2 CLANGARM64 out of
the box, but it is possible to do by downloading and using a prebuilt
toolchain from https://github.com/mstorsjo/llvm-mingw. Make the
'make-windows-release' script support doing this.
Suggested at https://wimlib.net/forums/viewtopic.php?p=1488#p1488
Eric Biggers [Thu, 30 Mar 2023 08:16:52 +0000 (01:16 -0700)]
xml: fix wimlib_set_image_property()
The recent XML changes introduced a bug where
wimlib_set_image_property() rejected values of property_name containing
'/', '[', or ']', which are part of the "path" syntax.
Eric Biggers [Thu, 30 Mar 2023 07:58:20 +0000 (00:58 -0700)]
Remove the outdated 'INSTALL' file
The INSTALL file was an old version of the generic GNU installation
instructions that was not customized to this package at all and was not
referenced by any other documentation. It's unlikely to be useful.
Eric Biggers [Thu, 30 Mar 2023 07:34:14 +0000 (00:34 -0700)]
win32_*.c: avoid some format string warnings
These aren't real issues as all the types in question are 32-bit ints,
but fixing these will prevent warnings later if we're ever able to
enable __attribute__((format)) on Windows (hopefully due to
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64862 being fixed).
Eric Biggers [Thu, 30 Mar 2023 07:00:56 +0000 (00:00 -0700)]
compiler.h: remove _cold_attribute
This abstraction layer serves no purpose other than supporting gcc 4.3
and earlier, which is too old to bother supporting. Just use
__attribute__((cold)) directly.
Eric Biggers [Tue, 28 Mar 2023 06:14:42 +0000 (23:14 -0700)]
ci.yml: use -fno-sanitize-recover when using -fsanitize
Unlike ASAN, by default UBSAN doesn't abort the program on error but
instead just prints an error message. Use -fno-sanitize-recover to
explicitly request the abort-on-error behavior for testing purposes.
Eric Biggers [Tue, 28 Mar 2023 06:14:42 +0000 (23:14 -0700)]
encoding.c: avoid UBSAN warning in convert_string()
When 'in == NULL && in_nbytes == 0', the statement 'in_end = in +
in_nbytes' executes 'NULL + 0'. clang's UndefinedBehaviorSanitizer
complains that this is undefined:
src/encoding.c:223:31: runtime error: applying zero offset to null pointer
Eric Biggers [Mon, 27 Mar 2023 00:25:46 +0000 (17:25 -0700)]
Improve fuzz testing
- Convert fuzzing scripts from afl-fuzz to libFuzzer
- Add xml and wim fuzzers, including malloc failure injection
- Fuzz for 2 minutes as part of the GitHub Actions CI
Eric Biggers [Mon, 27 Mar 2023 00:25:46 +0000 (17:25 -0700)]
README: remove an unnecessary notice
The "copyright years may be listed using range notation" notice is
recommended by the GNU project, but most other people don't consider it
to be necessary or meaningful.
Eric Biggers [Mon, 27 Mar 2023 00:25:46 +0000 (17:25 -0700)]
Improve the make-windows-release script
- Automatically bootstrap the repository if needed
- Add --no-zip and --no-docs options
- Add --install-msys2-packages to install the needed MSYS2 packages
- Autodetect the architecture when using MSYS2
- Use 'strip' instead of ${ARCH}-w64-mingw32-strip, for compatibility
with MSYS2
- Make it pass 'shellcheck'
- Other cleanups
Eric Biggers [Mon, 27 Mar 2023 00:25:46 +0000 (17:25 -0700)]
Eliminate the dependency on libxml2
libxml2 is the only remaining third-party library that Windows builds of
wimlib need. It's a bit of a pain to have to download it, build it, and
trick libtool into linking it into the resulting DLL. It then
constitutes a significant part of the size of the resulting DLL, even
with the minimal libxml2 configuration options being used.
In reality, WIM files only use a small subset of XML containing the most
commonly used XML features. Using a full-featured XML library (that
supports "features" like External Entities that we have to remember to
disable) is a bit dangerous and not actually necessary. 7-Zip's WIM
support, for example, just uses a very minimal home-brew XML processor.
Another issue is that the libxml2 API always uses UTF-8, which causes
the conversion UTF-16LE => UTF-8 => UTF-16LE to be needed on Windows.
This isn't really an "issue", per se, but it shouldn't be necessary.
Finally, wimlib was integrating with libxml2 at a low level via the tree
API, and it overlooked some things. For example, libxml2 trees have
separate CDATA and TEXT nodes, but wimlib was only looking at TEXT, so
CDATA was ignored. It was also possible for wimlib to create a document
containing control characters, which is not valid XML so it could not be
read. These weren't very important issues, but the point is, just using
an XML library doesn't solve quite as many problems as one would hope...
Therefore, just add a simple XML 1.0 processor directly in the source
code. It handles all XML features that are used in WIM files, plus a
bit more for futureproofing. It's also faster than libxml2.
Eric Biggers [Mon, 20 Mar 2023 03:59:17 +0000 (20:59 -0700)]
Consistently use _WIN32 instead of __WIN32__
_WIN32 works with all compilers, while __WIN32__ is MinGW-specific.
This project used __WIN32__ in files that only support MinGW, and _WIN32
in other files such as the library header and example programs. One
place even used WIN32. Avoid this unnecessary complication by just
always using _WIN32.
Eric Biggers [Sat, 18 Mar 2023 07:17:54 +0000 (00:17 -0700)]
Call wimlib_global_init() when creating compressors and decompressors
All nontrivial API functions are supposed to call wimlib_global_init().
wimlib_create_compressor() and wimlib_create_decompressor() did not.
Make them do so, so that CPU feature detection can be moved to
wimlib_global_init().
git://wimlib.net / wimlib / log