How difficult is it to build a shrinking/deflating routine?

[frktons]

gcc - you get what you pay for  :icon_mrgreen:

Sorry qWord, I didn't get it. Do you mean gcc, that is free, is better than vs or
the contrary?   ::)

[qWord]

nice ... you need to use the good documented linker option "-disable-long-section-names" (or for gcc "-Xlinker -disable-long-section-names") to get a standard conform PE.
:icon_rolleyes:

[jj2007]

http://wjradburn.com/software/

Merci :t

In the meantime, I found out that the part that I managed to compile (i.e. the LZMA library itself) is fully sufficient to write an archiver. Attached the first beta (Disclaimer: YOU HAVE BEEN WARNED!). It compresses around 16% better than WinZip with the non-portable maximum option.

Usage:
1. Compression: Drag a file over LzmaJJ.exe
2. Decompression: Drag a *.lzma file over LzmaJJ.exe

Limitations: 1. Only one file, 2. You need Pelles C to assemble the source (see OPT_DebugL in source code).

[MichaelW]

I have successfully compiled ARITH-N.C and the supporting code from the second edition of the Data Compression Book, using the VC Toolkit 2003 compiler with no optimization specified. The code is older than I remembered, using old-style declarations and recommending a LARGE memory model. It compiled as a Win32 console app, but with /W4 the compiler issued a bunch of warnings.

This is my preliminary result, compressing the current MASM32 windows.inc using order 3 arithmetic coding, manually timed at ~10 seconds:

Code Select Expand


Compressing \masm32\include\windows.inc to winc.z
Using Adaptive order n model with arithmetic coding
................................................................................
............
Input bytes:        977412
Output bytes:       188323
Compression ratio:  81%


Now to see if it will expand...

Looks OK, and FC /B found no differences, but the time was ~9 seconds?

With /O2 /G6 optimization compession and expansion each took ~~3 seconds.

To give you an idea of the code complexity, ARITH-N.C is 1102 lines with ~~25% comments.

[frktons]

Well Michael, considering that you have used a C compiled code, the
result is not bad at all, both in speed and compression ratio.
Probably hand written optimized assembly, with new SSE2/3  opcodes could produce
much better results on the speed side at least.
:t

[frktons]

In the meantime, I found out that the part that I managed to compile (i.e. the LZMA library itself) is fully sufficient to write an archiver. Attached the first beta (Disclaimer: YOU HAVE BEEN WARNED!). It compresses around 16% better than WinZip with the non-portable maximum option.

Usage:
1. Compression: Drag a file over LzmaJJ.exe
2. Decompression: Drag a *.lzma file over LzmaJJ.exe

Limitations: 1. Only one file, 2. You need Pelles C to assemble the source (see OPT_DebugL in source code).

It seems to work fine, Jochen :t

One thing puzzles me. Your implementatio of RtlCompress function produces
a zip file 280k long, but if I do select --> sent to --> compressed folder it
produces a 190k zip file. Shouldn't them have the same compression rate?
Isn't it the same function Windows is using?  ::)

[mineiro]

The most easy paper that I have found to understand data compression is below. It's nice because the author speaks about the logic, instead of throw mathematical hierogliphs. Good introduction.

http://www.fadden.com/techmisc/hdc/index.htm

[frktons]

The most easy paper that I have found to understand data compression is below. It's nice because the author speaks about the logic, instead of throw mathematical hierogliphs. Good introduction.

http://www.fadden.com/techmisc/hdc/index.htm

Thanks a lot mineiro, this will be certainly useful along the
way. :t

[jj2007]

It seems to work fine, Jochen :t

Thanks. One minor correction for the source posted in reply #77:
/LIBPATH:"%ProgramFiles%\PellesC\Lib"
Quotes are needed for those working with an English Windows version.

One thing puzzles me. Your implementatio of RtlCompress function produces
a zip file 280k long, but if I do select --> sent to --> compressed folder it
produces a 190k zip file. Shouldn't them have the same compression rate?
Isn't it the same function Windows is using?  ::)

That's a known issue. Microsoft uses different algos for the two functions.

[dedndave]

you can use the windows zipper by shell'ing to rundll32, i think (probably via WMI, as well)
but, it only works on files - not buffers
i suppose you could make a temp file and do it that way

i guess windows 8 has a new method - re: CreateCompressor, et al
seems kinda useless

[MichaelW]

I could not understand how the code could work correctly but have a lower expansion rate than the statistics in the book listed, but I now see that I somehow misread the statistics. For order 3 arithmetic compression and text files the listed compression/expansion rates were 1206 and 1173. Order 3 produced the highest compression/expansion rates and the highest compression ratio for text and executables, but order 2 produced a slightly higher compression ratio for graphics.