Total Bytes Case

[Tokener]

Dear friends of the wimlib project
After adding a small file to an image, I noticed the "Total Bytes" value of this image shrunk to a smaller value,
(picture shows the image at a later time when I repeated imaging.)

Zwischenablage01_1.jpg (86.77 KiB) Viewed 17329 times

while numbers of files and folders remained correct.
Is this normal behavior when updating?

regards T.

[synchronicity]

Please explain your problem more clearly. I don't speak German, I don't know exactly which of the 5 images you are comparing to what, I don't know what software you are using to view the WIM images and what values it's showing you. Can you provide a minimal reproducer using the wimlib-imagex command line tool?

[Tokener]

Dear synchronicity
thanks for the fast response.
This case is not a real "problem" but an unexpected behavior.
I was able to reproduce the case:
Step 1 -> create an image
Step 2 -> create a delta from this created image (same source)
Step 3 -> add a file (~1kb) to this delta-image.

It is to be expected an inreased number of "Total Bytes", but it's not the case:

Step 1
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
13.07.2019 20:53:10,08
D:\AKTUELL\AUTOIT\GUI\WIMLIB\x64\WIMLIB~1.EXE capture D:\AKTUELL\AUTOIT E:\TEMP\AUTOIT.wim "AUTOIT_190713_205307" "D:/AKTUELL/AUTOIT" --include-integrity --threads=8

WIM Information:
----------------
Path: E:\TEMP\AUTOIT.wim
GUID: 0x7dea22ec3726c8cc5cc936ae52dfa230
Version: 68864
Image Count: 1
Compression: LZX
Chunk Size: 32768 bytes
Part Number: 1/1
Boot Index: 0
Size: 2515023111 bytes
Attributes: Integrity info, Relative path junction

Available Images:
-----------------
Index: 1
Name: AUTOIT_190713_205307
Description: D:/AKTUELL/AUTOIT
Directory Count: 937
File Count: 11745
Total Bytes: 3962350896
Hard Link Bytes: 0
Creation Time: Sat Jul 13 18:53:10 2019 UTC
Last Modification Time: Sat Jul 13 18:53:10 2019 UTC
WIMBoot compatible: no

Step 2
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
13.07.2019 21:00:04,55
D:\AKTUELL\AUTOIT\GUI\WIMLIB\x64\WIMLIB~1.EXE append D:\AKTUELL\AUTOIT E:\TEMP\AUTOIT_190713.dwm "AUTOIT_190713_205953" "D:/AKTUELL/AUTOIT ==delta=from:AUTOIT.wim ==update=of:AUTOIT.wim" --delta-from=E:\TEMP\AUTOIT.wim --update-of=E:\TEMP\AUTOIT.wim:AUTOIT_190713_205307 --include-integrity --threads=8 --create

WIM Information:
----------------
Path: E:\TEMP\AUTOIT_190713.dwm
GUID: 0x54d8d9ef242a40060965660eac4f799d
Version: 68864
Image Count: 1
Compression: LZX
Chunk Size: 32768 bytes
Part Number: 1/1
Boot Index: 0
Size: 613358 bytes
Attributes: Integrity info, Relative path junction

Available Images:
-----------------
Index: 1
Name: AUTOIT_190713_205953
Description: D:/AKTUELL/AUTOIT ==delta=from:AUTOIT.wim ==update=of:AUTOIT.wim
Directory Count: 937
File Count: 11745
Total Bytes: 3962350902
Hard Link Bytes: 0
Creation Time: Sat Jul 13 19:00:04 2019 UTC
Last Modification Time: Sat Jul 13 19:00:05 2019 UTC
WIMBoot compatible: no


Step 3
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
13.07.2019 21:01:13,00
D:\AKTUELL\AUTOIT\GUI\WIMLIB\x64\WIMLIB~1.EXE update --include-integrity --force --recursive --threads=8 E:\TEMP\AUTOIT_190713.dwm AUTOIT_190713_205953 <B:\Temp\update_commands.txt

WIM Information:
----------------
Path: E:\TEMP\AUTOIT_190713.dwm
GUID: 0x54d8d9ef242a40060965660eac4f799d
Version: 68864
Image Count: 1
Compression: LZX
Chunk Size: 32768 bytes
Part Number: 1/1
Boot Index: 0
Size: 1243352 bytes
Attributes: Integrity info, Relative path junction

Available Images:
-----------------
Index: 1
Name: AUTOIT_190713_205953
Description: D:/AKTUELL/AUTOIT ==delta=from:AUTOIT.wim ==update=of:AUTOIT.wim
Directory Count: 937
File Count: 11746
Total Bytes: 69431
Hard Link Bytes: 0
Creation Time: Sat Jul 13 19:00:04 2019 UTC
Last Modification Time: Sat Jul 13 19:01:13 2019 UTC
WIMBoot compatible: no

Thanks in advance T.


EDIT: I used latest wimlib -build 1.13.1

[Tokener]

Dear synchronicity

Can you provide a minimal reproducer using the wimlib-imagex command line tool?

Since there was no further answer, did I provide the requested information?

What can I do to clarify the question:
How can the "Total Bytes" value decrease after adding another file to a "Delta-WIM" ?

Has anyone else reproduced this mistake?
Is anyone interested in the matter?

Excuse me for insisting on answers.

Best regards anyway T.

[synchronicity]

Hi, looking back through some old threads:

A simple reproducer for this is:

Code: Select all

mkdir -p big small
truncate -s 10000 big/bigfile
truncate -s 1 small/smallfile
wimcapture big base.wim
wimcapture big delta.wim --delta-from=base.wim
wiminfo delta.wim | grep 'Total Bytes'
wimupdate delta.wim 1 --command='add small /'
wiminfo delta.wim | grep 'Total Bytes' 

It shows TOTALBYTES decreasing from 10000 to 1 when a file is added. What's going on is that when wimupdate is used to update an image in a delta WIM, it doesn't have access to the base WIM and therefore doesn't know the sizes of files whose data is stored in the base WIM. The sizes are defaulted to 0, which makes the updated TOTALBYTES only include file data stored in the delta WIM.

Another consequence of this issue is that files that are newly added to the delta WIM are not deduplicated with the base WIM.

To fix this, https://wimlib.net/git/?p=wimlib;a=comm ... 9a616a65ee added a --ref option to wimupdate to allow specifying the WIM(s) on which the delta WIM is based. It isn't mandatory to specify this when updating a delta WIM, but it prevents these two issues.