Idea for a minor improvement about threads/memory usage

[Constance]

Hi there,

I've been playing around with wimlib lately (on Windows 11 x64 if that matters), and depending on whatever else I have running in the background at the time, when using solid LZMS compression, my PC may not always have enough RAM available to use all the threads that the CPU could otherwise get to working on the compression task.

When that happens, wimlib outputs the following message:

[WARNING] Couldn't create parallel chunk compressor: Ran out of memory.
Falling back to single-threaded compression.

While falling back to single-threaded does lower memory usage quite significantly and almost always ensures success, I've been thinking that the program probably could just lower the number of threads by 1 and try again, either until that lowered the required amount of memory enough for it to be able to allocate said amount, or until the number of threads gets down to 1.


Edited with more information below :

I made another attempt today and noticed there was an additional line which I didn't see last time (maybe I didn't pay enough attention, or maybe it doesn't always trigger depending on various circumstances?).

The command I ran was:

wimoptimize install.esd --recompress --compress=LZMS:100 --solid --solid-chunk-size=512M

And I got the following output:

"install.esd" original size: 2550489 KiB
[WARNING] Wanted to use 16 threads, but limiting to 7 to fit in available memory!
[WARNING] Couldn't create parallel chunk compressor: Ran out of memory.
Falling back to single-threaded compression.
Using LZMS compression with 1 thread

I had about 47.6 GB of RAM available at the time according to task manager, and it seems the program did attempt to calculate the most optimal number of threads considering memory usage, however as that failed (maybe due to a slight miscalculation?), it went down straight from 7 to 1.

So basically I'd suggest that, if running with the amount of threads it had calculated fails in that way, then try again with 1 less.
I did run the same command, with the addition of " --threads=6" just moments later and it ran with no issue.