WimLib is crashing on big Files

[MyNameistoLong]

10 seconds ago Wimlib, this time running from a lokal drive, finishes the first time sucessfully on the Server.

Thank you so much for the hint with the Network drive.

[synchronicity]

Thanks. I am not sure why a network filesystem on Windows would be unreliable as such, but it makes sense that data would need to be paged from the DLL at the very end of the process, when the output shows "100% done". Before then it is running the same code over and over again, for several hours... then at the end it finally has to do something new to finish up.

It is unfortunate, but wimlib, like any software, can only be as reliable as the OS on which it runs.

Although apparently not related to the crash you observed, I will be updating wimlib in the next version to require less memory when exporting multiple images. Currently it retains metadata for all images in memory, when really it only needs one at a time. I would estimate the memory usage in your scenario at very roughly 2.5 GB total for solid compression plus 100-200 MB per image for metadata... so if you have dozens of images it easily may be using gigabytes of extra memory for no real benefit.

Let me know if you encounter any other problems.

I currently don't accept donations for the project.

[MyNameistoLong]

You are right.
With 1.9.1 Beta wimlib needs about 2,5-2,8 GB of RAM when i look at it.
1.9.0 uses about 11-13 in the same scenario.

The verify was successful too.

Code: Select all

C:\WIM>wimlib-imagex.exe verify D:\Win_MEGA_AIO_OPTIMUM.wim
Verifying integrity of "D:\Win_MEGA_AIO_OPTIMUM.wim": 25 GiB of 25 GiB (100%) done
Verifying metadata for image 1 of 312
Verifying metadata for image 2 of 312
Verifying metadata for image 3 of 312
Verifying metadata for image 4 of 312
Verifying metadata for image 5 of 312
Verifying metadata for image 6 of 312
Verifying metadata for image 7 of 312
Verifying metadata for image 8 of 312
Verifying metadata for image 9 of 312
Verifying metadata for image 10 of 312
Verifying metadata for image 11 of 312
Verifying metadata for image 12 of 312
Verifying metadata for image 13 of 312
Verifying metadata for image 14 of 312
Verifying metadata for image 15 of 312
Verifying metadata for image 16 of 312
Verifying metadata for image 17 of 312
Verifying metadata for image 18 of 312
Verifying metadata for image 19 of 312
Verifying metadata for image 20 of 312
Verifying metadata for image 21 of 312
Verifying metadata for image 22 of 312
Verifying metadata for image 23 of 312
Verifying metadata for image 24 of 312
Verifying metadata for image 25 of 312
Verifying metadata for image 26 of 312
Verifying metadata for image 27 of 312
Verifying metadata for image 28 of 312
Verifying metadata for image 29 of 312
Verifying metadata for image 30 of 312
Verifying metadata for image 31 of 312
Verifying metadata for image 32 of 312
Verifying metadata for image 33 of 312
Verifying metadata for image 34 of 312
Verifying metadata for image 35 of 312
Verifying metadata for image 36 of 312
Verifying metadata for image 37 of 312
Verifying metadata for image 38 of 312
Verifying metadata for image 39 of 312
Verifying metadata for image 40 of 312
Verifying metadata for image 41 of 312
Verifying metadata for image 42 of 312
Verifying metadata for image 43 of 312
[WARNING] Fixed 983 invalid hard links in WIM image
Verifying metadata for image 44 of 312
[WARNING] Fixed 1351 invalid hard links in WIM image
Verifying metadata for image 45 of 312
[WARNING] Fixed 3 invalid hard links in WIM image
Verifying metadata for image 46 of 312
[WARNING] Fixed 1354 invalid hard links in WIM image
Verifying metadata for image 47 of 312
[WARNING] Fixed 980 invalid hard links in WIM image
Verifying metadata for image 48 of 312
Verifying metadata for image 49 of 312
[WARNING] Fixed 6 invalid hard links in WIM image
Verifying metadata for image 50 of 312
[WARNING] Fixed 1306 invalid hard links in WIM image
Verifying metadata for image 51 of 312
[WARNING] Fixed 2555 invalid hard links in WIM image
Verifying metadata for image 52 of 312
[WARNING] Fixed 1296 invalid hard links in WIM image
Verifying metadata for image 53 of 312
[WARNING] Fixed 2538 invalid hard links in WIM image
Verifying metadata for image 54 of 312
[WARNING] Fixed 2 invalid hard links in WIM image
Verifying metadata for image 55 of 312
[WARNING] Fixed 8 invalid hard links in WIM image
Verifying metadata for image 56 of 312
[WARNING] Fixed 951 invalid hard links in WIM image
Verifying metadata for image 57 of 312
[WARNING] Fixed 2434 invalid hard links in WIM image
Verifying metadata for image 58 of 312
[WARNING] Fixed 948 invalid hard links in WIM image
Verifying metadata for image 59 of 312
[WARNING] Fixed 2416 invalid hard links in WIM image
Verifying metadata for image 60 of 312
Verifying metadata for image 61 of 312
[WARNING] Fixed 6 invalid hard links in WIM image
Verifying metadata for image 62 of 312
[WARNING] Fixed 1300 invalid hard links in WIM image
Verifying metadata for image 63 of 312
[WARNING] Fixed 2548 invalid hard links in WIM image
Verifying metadata for image 64 of 312
[WARNING] Fixed 1290 invalid hard links in WIM image
Verifying metadata for image 65 of 312
[WARNING] Fixed 2524 invalid hard links in WIM image
Verifying metadata for image 66 of 312
[WARNING] Fixed 2 invalid hard links in WIM image
Verifying metadata for image 67 of 312
[WARNING] Fixed 8 invalid hard links in WIM image
Verifying metadata for image 68 of 312
[WARNING] Fixed 951 invalid hard links in WIM image
Verifying metadata for image 69 of 312
[WARNING] Fixed 2426 invalid hard links in WIM image
Verifying metadata for image 70 of 312
[WARNING] Fixed 948 invalid hard links in WIM image
Verifying metadata for image 71 of 312
[WARNING] Fixed 2416 invalid hard links in WIM image
Verifying metadata for image 72 of 312
Verifying metadata for image 73 of 312
[WARNING] Fixed 4 invalid hard links in WIM image
Verifying metadata for image 74 of 312
[WARNING] Fixed 1299 invalid hard links in WIM image
Verifying metadata for image 75 of 312
[WARNING] Fixed 2543 invalid hard links in WIM image
Verifying metadata for image 76 of 312
[WARNING] Fixed 1284 invalid hard links in WIM image
Verifying metadata for image 77 of 312
[WARNING] Fixed 2519 invalid hard links in WIM image
Verifying metadata for image 78 of 312
[WARNING] Fixed 2 invalid hard links in WIM image
Verifying metadata for image 79 of 312
[WARNING] Fixed 8 invalid hard links in WIM image
Verifying metadata for image 80 of 312
[WARNING] Fixed 947 invalid hard links in WIM image
Verifying metadata for image 81 of 312
[WARNING] Fixed 2424 invalid hard links in WIM image
Verifying metadata for image 82 of 312
[WARNING] Fixed 946 invalid hard links in WIM image
Verifying metadata for image 83 of 312
[WARNING] Fixed 2412 invalid hard links in WIM image
Verifying metadata for image 84 of 312
Verifying metadata for image 85 of 312
[WARNING] Fixed 2 invalid hard links in WIM image
Verifying metadata for image 86 of 312
[WARNING] Fixed 1293 invalid hard links in WIM image
Verifying metadata for image 87 of 312
[WARNING] Fixed 2543 invalid hard links in WIM image
Verifying metadata for image 88 of 312
[WARNING] Fixed 1280 invalid hard links in WIM image
Verifying metadata for image 89 of 312
[WARNING] Fixed 2517 invalid hard links in WIM image
Verifying metadata for image 90 of 312
[WARNING] Fixed 2 invalid hard links in WIM image
Verifying metadata for image 91 of 312
[WARNING] Fixed 8 invalid hard links in WIM image
Verifying metadata for image 92 of 312
[WARNING] Fixed 943 invalid hard links in WIM image
Verifying metadata for image 93 of 312
[WARNING] Fixed 2424 invalid hard links in WIM image
Verifying metadata for image 94 of 312
[WARNING] Fixed 942 invalid hard links in WIM image
Verifying metadata for image 95 of 312
[WARNING] Fixed 2412 invalid hard links in WIM image
Verifying metadata for image 96 of 312
Verifying metadata for image 97 of 312
[WARNING] Fixed 2 invalid hard links in WIM image
Verifying metadata for image 98 of 312
Verifying metadata for image 99 of 312
[WARNING] Fixed 2 invalid hard links in WIM image
Verifying metadata for image 100 of 312
[WARNING] Fixed 1280 invalid hard links in WIM image
Verifying metadata for image 101 of 312
[WARNING] Fixed 2533 invalid hard links in WIM image
Verifying metadata for image 102 of 312
Verifying metadata for image 103 of 312
Verifying metadata for image 104 of 312
Verifying metadata for image 105 of 312
Verifying metadata for image 106 of 312
Verifying metadata for image 107 of 312
Verifying metadata for image 108 of 312
Verifying metadata for image 109 of 312
Verifying metadata for image 110 of 312
Verifying metadata for image 111 of 312
Verifying metadata for image 112 of 312
Verifying metadata for image 113 of 312
Verifying metadata for image 114 of 312
Verifying metadata for image 115 of 312
Verifying metadata for image 116 of 312
Verifying metadata for image 117 of 312
Verifying metadata for image 118 of 312
Verifying metadata for image 119 of 312
Verifying metadata for image 120 of 312
Verifying metadata for image 121 of 312
Verifying metadata for image 122 of 312
Verifying metadata for image 123 of 312
Verifying metadata for image 124 of 312
Verifying metadata for image 125 of 312
Verifying metadata for image 126 of 312
Verifying metadata for image 127 of 312
Verifying metadata for image 128 of 312
Verifying metadata for image 129 of 312
Verifying metadata for image 130 of 312
Verifying metadata for image 131 of 312
Verifying metadata for image 132 of 312
Verifying metadata for image 133 of 312
Verifying metadata for image 134 of 312
Verifying metadata for image 135 of 312
Verifying metadata for image 136 of 312
Verifying metadata for image 137 of 312
Verifying metadata for image 138 of 312
Verifying metadata for image 139 of 312
Verifying metadata for image 140 of 312
Verifying metadata for image 141 of 312
Verifying metadata for image 142 of 312
Verifying metadata for image 143 of 312
Verifying metadata for image 144 of 312
Verifying metadata for image 145 of 312
Verifying metadata for image 146 of 312
Verifying metadata for image 147 of 312
Verifying metadata for image 148 of 312
Verifying metadata for image 149 of 312
Verifying metadata for image 150 of 312
Verifying metadata for image 151 of 312
Verifying metadata for image 152 of 312
Verifying metadata for image 153 of 312
Verifying metadata for image 154 of 312
Verifying metadata for image 155 of 312
Verifying metadata for image 156 of 312
Verifying metadata for image 157 of 312
Verifying metadata for image 158 of 312
Verifying metadata for image 159 of 312
Verifying metadata for image 160 of 312
Verifying metadata for image 161 of 312
Verifying metadata for image 162 of 312
Verifying metadata for image 163 of 312
Verifying metadata for image 164 of 312
Verifying metadata for image 165 of 312
Verifying metadata for image 166 of 312
Verifying metadata for image 167 of 312
Verifying metadata for image 168 of 312
Verifying metadata for image 169 of 312
Verifying metadata for image 170 of 312
Verifying metadata for image 171 of 312
Verifying metadata for image 172 of 312
Verifying metadata for image 173 of 312
Verifying metadata for image 174 of 312
Verifying metadata for image 175 of 312
Verifying metadata for image 176 of 312
Verifying metadata for image 177 of 312
Verifying metadata for image 178 of 312
Verifying metadata for image 179 of 312
Verifying metadata for image 180 of 312
Verifying metadata for image 181 of 312
Verifying metadata for image 182 of 312
Verifying metadata for image 183 of 312
Verifying metadata for image 184 of 312
Verifying metadata for image 185 of 312
Verifying metadata for image 186 of 312
Verifying metadata for image 187 of 312
Verifying metadata for image 188 of 312
Verifying metadata for image 189 of 312
Verifying metadata for image 190 of 312
Verifying metadata for image 191 of 312
Verifying metadata for image 192 of 312
Verifying metadata for image 193 of 312
Verifying metadata for image 194 of 312
Verifying metadata for image 195 of 312
Verifying metadata for image 196 of 312
Verifying metadata for image 197 of 312
Verifying metadata for image 198 of 312
Verifying metadata for image 199 of 312
Verifying metadata for image 200 of 312
Verifying metadata for image 201 of 312
Verifying metadata for image 202 of 312
Verifying metadata for image 203 of 312
Verifying metadata for image 204 of 312
Verifying metadata for image 205 of 312
Verifying metadata for image 206 of 312
Verifying metadata for image 207 of 312
Verifying metadata for image 208 of 312
Verifying metadata for image 209 of 312
Verifying metadata for image 210 of 312
Verifying metadata for image 211 of 312
Verifying metadata for image 212 of 312
Verifying metadata for image 213 of 312
Verifying metadata for image 214 of 312
Verifying metadata for image 215 of 312
Verifying metadata for image 216 of 312
Verifying metadata for image 217 of 312
[WARNING] Fixed 1140 invalid hard links in WIM image
Verifying metadata for image 218 of 312
[WARNING] Fixed 1134 invalid hard links in WIM image
Verifying metadata for image 219 of 312
Verifying metadata for image 220 of 312
Verifying metadata for image 221 of 312
[WARNING] Fixed 194 invalid hard links in WIM image
Verifying metadata for image 222 of 312
[WARNING] Fixed 120 invalid hard links in WIM image
Verifying metadata for image 223 of 312
Verifying metadata for image 224 of 312
Verifying metadata for image 225 of 312
[WARNING] Fixed 1091 invalid hard links in WIM image
Verifying metadata for image 226 of 312
[WARNING] Fixed 1082 invalid hard links in WIM image
Verifying metadata for image 227 of 312
Verifying metadata for image 228 of 312
Verifying metadata for image 229 of 312
[WARNING] Fixed 160 invalid hard links in WIM image
Verifying metadata for image 230 of 312
[WARNING] Fixed 120 invalid hard links in WIM image
Verifying metadata for image 231 of 312
Verifying metadata for image 232 of 312
Verifying metadata for image 233 of 312
[WARNING] Fixed 1091 invalid hard links in WIM image
Verifying metadata for image 234 of 312
[WARNING] Fixed 1084 invalid hard links in WIM image
Verifying metadata for image 235 of 312
Verifying metadata for image 236 of 312
Verifying metadata for image 237 of 312
[WARNING] Fixed 162 invalid hard links in WIM image
Verifying metadata for image 238 of 312
[WARNING] Fixed 120 invalid hard links in WIM image
Verifying metadata for image 239 of 312
Verifying metadata for image 240 of 312
Verifying metadata for image 241 of 312
[WARNING] Fixed 1091 invalid hard links in WIM image
Verifying metadata for image 242 of 312
[WARNING] Fixed 1082 invalid hard links in WIM image
Verifying metadata for image 243 of 312
Verifying metadata for image 244 of 312
Verifying metadata for image 245 of 312
[WARNING] Fixed 160 invalid hard links in WIM image
Verifying metadata for image 246 of 312
[WARNING] Fixed 120 invalid hard links in WIM image
Verifying metadata for image 247 of 312
[WARNING] Fixed 44 invalid hard links in WIM image
Verifying metadata for image 248 of 312
Verifying metadata for image 249 of 312
Verifying metadata for image 250 of 312
Verifying metadata for image 251 of 312
Verifying metadata for image 252 of 312
Verifying metadata for image 253 of 312
Verifying metadata for image 254 of 312
Verifying metadata for image 255 of 312
Verifying metadata for image 256 of 312
Verifying metadata for image 257 of 312
Verifying metadata for image 258 of 312
Verifying metadata for image 259 of 312
Verifying metadata for image 260 of 312
Verifying metadata for image 261 of 312
Verifying metadata for image 262 of 312
Verifying metadata for image 263 of 312
Verifying metadata for image 264 of 312
Verifying metadata for image 265 of 312
Verifying metadata for image 266 of 312
Verifying metadata for image 267 of 312
Verifying metadata for image 268 of 312
Verifying metadata for image 269 of 312
Verifying metadata for image 270 of 312
Verifying metadata for image 271 of 312
Verifying metadata for image 272 of 312
Verifying metadata for image 273 of 312
Verifying metadata for image 274 of 312
Verifying metadata for image 275 of 312
Verifying metadata for image 276 of 312
Verifying metadata for image 277 of 312
Verifying metadata for image 278 of 312
Verifying metadata for image 279 of 312
Verifying metadata for image 280 of 312
Verifying metadata for image 281 of 312
Verifying metadata for image 282 of 312
Verifying metadata for image 283 of 312
Verifying metadata for image 284 of 312
Verifying metadata for image 285 of 312
Verifying metadata for image 286 of 312
Verifying metadata for image 287 of 312
Verifying metadata for image 288 of 312
Verifying metadata for image 289 of 312
Verifying metadata for image 290 of 312
Verifying metadata for image 291 of 312
Verifying metadata for image 292 of 312
Verifying metadata for image 293 of 312
Verifying metadata for image 294 of 312
Verifying metadata for image 295 of 312
Verifying metadata for image 296 of 312
Verifying metadata for image 297 of 312
Verifying metadata for image 298 of 312
Verifying metadata for image 299 of 312
Verifying metadata for image 300 of 312
Verifying metadata for image 301 of 312
Verifying metadata for image 302 of 312
Verifying metadata for image 303 of 312
Verifying metadata for image 304 of 312
Verifying metadata for image 305 of 312
Verifying metadata for image 306 of 312
Verifying metadata for image 307 of 312
Verifying metadata for image 308 of 312
Verifying metadata for image 309 of 312
Verifying metadata for image 310 of 312
Verifying metadata for image 311 of 312
Verifying metadata for image 312 of 312
Verifying file data: 116 GiB of 116 GiB (100%) done

"D:\Win_MEGA_AIO_OPTIMUM.wim" was successfully verified.