bzip3

[Vortex]

A better, faster and stronger spiritual successor to BZip2. Features higher compression ratios and better performance thanks to a order-0 context mixing entropy coder, a fast Burrows-Wheeler transform code making use of suffix arrays and a RLE with Lempel Ziv+Prediction pass based on LZ77-style string matching and PPM-style context modeling.

Like its ancestor, BZip3 excels at compressing text or code.

https://github.com/kspalaiologos/bzip3

[Gunther]

Thank you for the link, Erol.  Are there any advantages over 7zip?

[Vortex]

Hi Gunther,

7-Zip is an archiver and can pack multiple files while the bzip family can process only one file at one time.

[Gunther]

Erol,

7-Zip is an archiver and can pack multiple files while the bzip family can process only one file at one time.

thank you for the quick reply. I will have a look at the C sources anyway - as soon as I find time.

[jj2007]

Hi Erol,

I made a quick test using the binary found here.
bzip3    compressed a 2.7MB source to  368,764 bytes
FreeArc compressed the same source to 373,802 bytes

So far, so good. But bzip3 took 630 milliseconds, while FreeArc needed only 422 milliseconds to do the job.

[hutch--]

Had a quick play, file size reduction of win64.inc went from 786888 down to 108846 bytes.

Setting the thread count increases the speed of compression. On this old 6 core, --j 12 made it fast. Tested on an 800 meg video file.

Better compression than any option in Winrar.

[mineiro]

Some encode.su forum members have already won the hutter prize.
Following the same dilemma, speed or better compression?

You can use wrt44.zip (preprocessor) with bzip3.
Zpaq has good speed but lower compression.
paq (Matt Mahoney) series has good compression but lower speed.

http://mattmahoney.net/dc/
http://mattmahoney.net/dc/paq.html

$./paq8l -1 win64.inc
78215 bytes, Time 6.55 sec, used 38862968 bytes of memory
$./paq8l -8 win64.inc
61151 bytes, Time 44.14 sec, used 1643021320 bytes of memory

There are some versions of paq containing dictionaries that can compress more. We can create our own dictionary. Some mixing contents done to jpg,bmp,wav,... .
Jpeg used huffman in past because patents, now I see persons using arithmetic coding instead. Probably old mpeg,jpeg files can be compressed.

[jj2007]

I made more tests. With option -m9, FreeArc has much better compression and is still significantly faster.

[Vortex]

Hi Jochen,

Those versions compiled for Windows XP are outdated. You can get the latest release from here:

https://github.com/kspalaiologos/bzip3/releases

[Vortex]

The MSYS2 development system is providing a ready package for bzip3 :

Code: [Select]

# pacman -S mingw-w64-x86_64-bzip3
resolving dependencies...
looking for conflicting packages...

Packages (1) mingw-w64-x86_64-bzip3-1.2.2-1

Total Download Size:   0.16 MiB
Total Installed Size:  0.52 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages...
 mingw-w64-x86_64-bzip3-1.2...   159.6 KiB  93.8 KiB/s 00:02 [###############################] 100%
.
.
:: Processing package changes...
(1/1) installing mingw-w64-x86_64-bzip3                      [###############################] 100%
Dependencies :

Code: [Select]

# ldd /mingw64/bin/bzip3.exe
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x77970000)
        kernel32.dll => /c/Windows/system32/kernel32.dll (0x77850000)
        KERNELBASE.dll => /c/Windows/system32/KERNELBASE.dll (0x7fefd810000)
        libbzip3-0.dll => /mingw64/bin/libbzip3-0.dll (0x7fefa540000)
        msvcrt.dll => /c/Windows/system32/msvcrt.dll (0x7fefecb0000)
        libwinpthread-1.dll => /mingw64/bin/libwinpthread-1.dll (0x7fefb0a0000)

[Vortex]

Building a static binary with less dependencies :

https://github.com/kspalaiologos/bzip3/blob/master/PORTING.md

64-bit :

Code: [Select]

./configure CC=x86_64-w64-mingw32-gcc --host x86_64-w64-mingw32 --enable-static-exe

make
Dependencies :

Code: [Select]

# ldd ./bzip3.exe
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x77970000)
        kernel32.dll => /c/Windows/system32/kernel32.dll (0x77850000)
        KERNELBASE.dll => /c/Windows/system32/KERNELBASE.dll (0x7fefd810000)
        msvcrt.dll => /c/Windows/system32/msvcrt.dll (0x7fefecb0000)
32-bit :

Code: [Select]

./configure CC=i686-w64-mingw32 --host i686-w64-mingw32 --enable-static-exe

make
Dependencies :

Code: [Select]

# ldd ./bzip3.exe
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x77970000)
        ntdll.dll => /c/Windows/SysWOW64/ntdll.dll (0x77b30000)
        wow64.dll => /c/Windows/SYSTEM32/wow64.dll (0x74e80000)
        wow64win.dll => /c/Windows/SYSTEM32/wow64win.dll (0x74e20000)
        wow64cpu.dll => /c/Windows/SYSTEM32/wow64cpu.dll (0x74e10000)


[hutch--]

Apart from testing the pre-built binaries which work OK, is there a 64 bit COFF library module that can be linked into a 64 bit app ?

[jj2007]

Hi Jochen,

Those versions compiled for Windows XP are outdated. You can get the latest release from here:

https://github.com/kspalaiologos/bzip3/releases

Hi Erol,

There are a dozen archives, each with a dozen strange files. I won't download them all, but if you know which one has a functioning executable, please let me know. I found one functioning executable so far, not on that site (see above), and its performance was really disappointing, especially given the hype that I see on GitHub.

Had a quick play, file size reduction of win64.inc went from 786888 down to 108846 bytes.

136 ms for decompressing with bzip3

FreeArc 0.666 extracting archive: w64.arc
Extracted 1 file, 96,807 => 752,296 bytes. Ratio 12.8%
Extraction time: cpu 0.02 secs, real 0.02 secs. Speed 37,615 kB/s
All OK
120 ms for decompressing with FreeArc

See attachment. FreeArc is here.

[TimoVJL]

Apart from testing the pre-built binaries which work OK, is there a 64 bit COFF library module that can be linked into a 64 bit app ?

here are msvc 2022 compiled obj-files for testing.

[jj2007]

Nice, Timo

Now all we need is a bit of documentation