SMAC : Compression with Arithmetic Coding+Context Mixing

[Jean-Marie]

This is the (open)source of my compression engine, released under zlib license.
It uses a Statistical method (as opposed to Dictionary method like LZ77) named Arithmetic Coding, where the entire input file is converted to a (very) long decimal number between 0 & 1.
Each time new bits are read from the input, this will narrow down the interval with the effect of expanding the codified number. Most frequent symbols will narrow the interval with a smaller impact than less frequent ones, hence a gain of compression.
AC is paired off with Context Models from Order-6 to Order-1.
Suppose you have to compress an english text; statistically, the letter 'u' would appear with a global frequency (or probability) of 5%.  Now, let's suppose we know the previous character was a 'q' : the probability would now jump to 99%.
Context is simply a terminology for that previous character. In this example, we would use an Order-1 context model to gather the statistics of the following symbol. 2 previous characters would be Order-2 and so on... Obviously, using numerous models will have a bad impact on speed, and a good impact on compression.
After having calculated a probability for each model, we need to mix them up, hence the term Context Mixing. Among all different methods, I used Logistic Mixing, which is praised as one of the most efficient.
For more details, I strongly urge you to read Mark Nelson's tutorial, which was my inspiration for writing this software. Matt Mahoney's ebook is also a solid reference, and gives detail of his Logistic Mixing.
http://marknelson.us/1991/02/01/arithmetic-coding-statistical-modeling-data-compression/
http://mattmahoney.net/dc/dce.html#Section_432

Feel free to use (at your own risks), and modify : it is already optimized for speed, but may be you can find some new tricks. FYI, I have already tried replacing FPU instructions with SSE2, but surprisingly this was *much* slower. I also tried running some routines in a separate thread, using a Spinloop, and struggling against Race condition & False-sharing, with bad results either.

One last thing : I'm a die-hard TASM user, so you'll have to convert it first. Luckily, it's in MASM mode, so I guess you just have to replace the call API by this stupid invoke instruction 

[qWord]

I have already tried replacing FPU instructions with SSE2, but surprisingly this was *much* slower.

What did you replace? Sounds more like  an buggy implementation.
If the precision of 53 bit is sufficient, you should replace all basic arithmetic (+-*/,sqrt) with SSE2+, because that removes the overhead for the FPU stack. Furthermore you might replace  log2, 2^x or whole formulas with polynomial approximations.

qWord

[Jean-Marie]

I made some experiments with UpdateModel, Stretch, and also Squash. SSE2 is good for executing parallel calculations, otherwise it seems to be outperformed by FPU. Note that with FPU, you don't have the overhead of using conversion instructions(cvtdq**).
I have already used a lookup table for log₂ function. Replacing the whole Squash formula by a LUT could be done in theory, but that would need to be a huge one since p0 ranges between -16,999999999... to +15,999999999. And reducing the mantissa would hurt the compression...

Code Select Expand


;Update Weights in the logistic domain, such as:
;Wi=Wi+(Learning Rate*((1-Symbol-(Squash/2^32))*Stretch i))
fstp Temp ;store Squash value in local data (float)
pcmpeqd xmm1,xmm1
psrld xmm1,31 ;xmm1 low dword=1
sub dword ptr Temp+4,2000000H ;substract 32 from exponent field
cmp MatchPrediction,2 ;MatchPrediction=2 if no match found
subss xmm1,Symbol ;xmm1=1-Symbol
movsd xmm3,Temp ;xmm3=Squash/2^32 (float 64 bits)
cvtdq2pd xmm1,xmm1 ;xmm1=1-Symbol (float 64 bits)
subsd xmm1,xmm3 ;xmm1=(1-Symbol)-(Squash/2^32)
mulsd xmm1,@@LearningRate ;xmm1=(1-Symbol)-(Squash/2^32)*Learning Rate
cvtsd2ss xmm1,xmm1 ;xmm1=xmm0 converted to 32 bits precision
movapd xmm3,xmm1
mulss xmm1,Stretch6
addss xmm1,Weight6
movss Weight6,xmm1
movapd xmm1,xmm3
mulss xmm1,Stretch4
addss xmm1,Weight4
movss Weight4,xmm1
movapd xmm1,xmm3
mulss xmm1,Stretch3
addss xmm1,Weight3
movss Weight3,xmm1
movapd xmm1,xmm3
mulss xmm1,Stretch2
addss xmm1,Weight2
movss Weight2,xmm1
movapd xmm1,xmm3
mulss xmm1,Stretch1
addss xmm1,Weight1
movss Weight1,xmm1
je @@IF1 ;skip update of Match if model is off
movapd xmm1,xmm3
mulss xmm1,StretchM
addss xmm1,WeightM
movss WeightM,xmm1

Code Select Expand


@@Stretch:
movzx edx,word ptr [edx] ;edx=Bit history of Order-x table
@@Stretch2:
mov edx,[edi+edx*4]
movzx eax,dx ;eax=frequency of bit 0
movd xmm1,eax
shr edx,16 ;edx=frequency of bit 1
lea ecx,[eax+edx] ;ecx=n0+n1
cvtdq2ps xmm1,xmm1 ;frequency of bit 0 (float)
mulss xmm1,[ecx*4+ebx] ;xmm1=bit 0 range=Frequency*(2^32/TableCount)
movd eax,xmm1 ;eax=p0 (float)
cvtps2pd xmm2,xmm1 ;xmm2=p0 (float 64 bits)
mov edx,eax
shr eax,23 ;eax=biased exponent
sub eax,127 ;eax=exponent
and edx,7FFFFFH ;edx=mantissa
shr edx,9 ;edx=mantissa reduced to 14 bits
movd xmm1,[edx*4+esi] ;xmm1=log2(mantissa)
cvtsi2ss xmm3,eax ;xmm3=exponent (float)
addss xmm3,xmm1 ;xmm3=exp+log2(mantissa)=log2(p0)
movsd xmm1, @@TwoPower32 ;xmm1=2^32 (float 64 bits)
subsd xmm1,xmm2 ;xmm1=2^32-p0
cvtsd2ss xmm1,xmm1 ;convert xmm1 to float 32 bits
movd eax,xmm1 ;eax=2^32-p0 (float)
mov edx,eax
shr eax,23 ;eax=biased exponent
sub eax,127 ;eax=exponent
and edx,7FFFFFH ;edx=mantissa
shr edx,9 ;edx=mantissa reduced to 14 bits
movd xmm1,[edx*4+esi] ;xmm1=log2(mantissa)
cvtsi2ss xmm2,eax ;xmm2=exponent (float)
addss xmm2,xmm1 ;xmm2=exp+log2(mantissa)=log2(2^32-p0)
subss xmm3,xmm2 ;xmm3=log2(p0)-log2(2^32-p0)
movss Temp,xmm3
fld dword ptr Temp ;store result in FPU stack
ret

[jj2007]

I have translated the Tasm code to Masm syntax, but under Win7-32 it chokes in Flush_encoder (and that could obviously be a "translation error"):
        deb 4, "before sub edi, OutputBuffer", x:edi, x:OutputBuffer, x:WriteBufferEnd
        sub edi, OutputBuffer        ;last write of Output buffer to Destination file
        lea eax, Temp
        deb 4, "WriteFile in", @@TargetHandle, x:OutputBuffer, x:edi, eax
        invoke        WriteFile, @@TargetHandle, OutputBuffer, edi, eax, NULL        ; edi is nNumberOfBytesToWrite
        deb 4, "WriteFile out", $Err$()

Output:
before sub edi, OutputBuffer
x:edi           012A1026
x:OutputBuffer  112B0020
x:WriteBufferEnd        212B0020

WriteFile in
@@TargetHandle  152
x:OutputBuffer  112B0020
x:edi           EFFF1006     <<<<<<<<<<< edi is negative
eax             1245036
WriteFile out   $Err$()         Invalid access to memory location.

The official error message ("Out of Memory : file was too large to be processed") is misleading.
Testfile was \Masm32\include\masm32rt.inc, 3217 bytes.

Source attached, as plain Masm32; the *.asc file is RTF, opens e.g. in \Masm32\MasmBasic\RichMasm.exe for those who have MasmBasic installed.

EDIT: Fixed a pop reg reg translation bug, but still the same error.

[qWord]

SSE2 is good for executing parallel calculations, otherwise it seems to be outperformed by FPU

surely not! For the basic arithmetic you will commonly get at least the same speed as with the FPU.

Note that with FPU, you don't have the overhead of using conversion instructions(cvtdq**)

That is no overhead - it is required and the FPU must do the same operation for FILD[P]/FIST[P]. For your SSE2 code I see the problem that there are several conversions that could be omitted.

Replacing the whole Squash formula by a LUT could be done in theory, but that would need to be a huge one since p0 ranges between -16,999999999... to +15,999999999.

What precision is required?
I've quickly worked out the following approximation (f(x) = 2³²/(1 + 2^-x)), which divides the range -17 to 16 into 0.5 intervals. For each of them a polynomial (5th degree, mini-max approximation) is used. The |relative error| is  below 1.5*10^-9:

Code Select Expand

; xmm0 = input value
movsd xmm1,xmm0
addsd xmm0,sd_17
addsd xmm0,xmm0
cvttsd2si ecx,xmm0
imul ecx,48
lea ecx,tbl1[ecx]

movsd xmm0,REAL8 ptr [ecx+0*8]
mulsd xmm0,xmm1
addsd xmm0,REAL8 ptr [ecx+1*8]
mulsd xmm0,xmm1
addsd xmm0,REAL8 ptr [ecx+2*8]
mulsd xmm0,xmm1
addsd xmm0,REAL8 ptr [ecx+3*8]
mulsd xmm0,xmm1
addsd xmm0,REAL8 ptr [ecx+4*8]
mulsd xmm0,xmm1
addsd xmm0,REAL8 ptr [ecx+5*8]
; result in xmm0

the table:

Code Select Expand

sd_17 REAL8 17.0
align 16
; |error| <= 1.38272E-9
; N = 66
; degree = 5
; table size = 3168.
; step = 1/2
tbl1 LABEL REAL8
    REAl8 51.88733919031596483941716
    REAl8 4720.870462613202452233068
    REAl8 172884.6360901519360640412
    REAl8 3.188223395760284100183061E6
    REAl8 2.963746272042194298627888E7
    REAl8 1.112360695861580167739344E8

    REAl8 73.37093794320208307773415
    REAl8 6492.119208327092115313642
    REAl8 231301.0872467358935065903
    REAl8 4.151572985509914932599881E6
    REAl8 3.758121739160534963936367E7
    REAl8 1.374390625255227249973363E8

    REAl8 103.7444950360918885549799
    REAl8 8920.381139496640486548687
    REAl8 308957.9720557864011553151
    REAl8 5.393393990647069978685116E6
    REAl8 4.751083833688111095777013E7
    REAl8 1.691998295938006180216526E8

    REAl8 146.6815143865613352699856
    REAl8 12245.70757455918026048346
    REAl8 411978.2806894580767835712
    REAl8 6.989301161712469405214701E6
    REAl8 5.987283593461696080019291E7
    REAl8 2.075048719860747448085957E8

    REAl8 207.3682817485600260967559
    REAl8 16793.97732876753478507681
    REAl8 548338.0852446730741007231
    REAl8 9.033506521324976763685423E6
    REAl8 7.519643238579234335081641E7
    REAl8 2.534548108734618545114674E8

    REAl8 293.1216536307773869953894
    REAl8 23006.52660505454405322288
    REAl8 728381.9784360021274601921
    REAl8 1.164258358993160271096278E7
    REAl8 9.410224813898590893585648E7
    REAl8 3.082565013308120145304222E8

    REAl8 414.2539305849464936082024
    REAl8 31479.34026947846602799544
    REAl8 965457.9370602551631181308
    REAl8 1.495961451202408465767186E7
    REAl8 1.173089653982246607197822E8
    REAl8 3.732051486830684789845497E8

    REAl8 585.2783733325399634026721
    REAl8 43014.40949053542262511964
    REAl8 1.276684556244585587565104E6
    REAl8 1.915853681742603061636134E7
    REAl8 1.456361596943700486275529E8
    REAl8 4.496527615689425763894538E8

    REAl8 826.5789317166141485281632
    REAl8 58686.17363649724569071519
    REAl8 1.683853518863027986762974E6
    REAl8 2.444833048618222039159124E7
    REAl8 1.800006682851481673533409E8
    REAl8 5.389584065532640731318793E8

    REAl8 1166.701540620524424160062
    REAl8 79925.91362738045546319615
    REAl8 2.214448514969364018692124E6
    REAl8 3.107640588811522044036193E7
    REAl8 2.214027446782516076500369E8
    REAl8 6.424147749570835816767655E8

    REAl8 1645.454949384501447527862
    REAl8 108626.0494541723485621117
    REAl8 2.902719901485979087662182E6
    REAl8 3.933011677250708813703721E7
    REAl8 2.708967526917784851648481E8
    REAl8 7.611446113728958057706480E8

    REAl8 2318.018439277405107695768
    REAl8 147263.5525706455660606572
    REAl8 3.790677845570720009593682E6
    REAl8 4.953465629544336855878542E7
    REAl8 3.295392107360258782133333E8
    REAl8 8.959597617768673797197731E8

    REAl8 3260.200452275777993043329
    REAl8 199035.3156538426108962666
    REAl8 4.928732482515329218100044E6
    REAl8 6.204460080945446003032485E7
    REAl8 3.983045899138597431275463E8
    REAl8 1.047175340385311826795597E9

    REAl8 4574.786887985574618064181
    REAl8 267985.0671166693348622539
    REAl8 6.375482039357043602296603E6
    REAl8 7.722493099671766991332702E7
    REAl8 4.779565315896772268256312E8
    REAl8 1.214372485205343391588581E9

    REAl8 6398.377281369057850132229
    REAl8 359075.5502263001980307799
    REAl8 8.195776627521903805948546E6
    REAl8 9.541538074218751523172678E7
    REAl8 5.688595641485783918309568E8
    REAl8 1.396106632654373763609108E9

    REAl8 8906.868411625623359851656
    REAl8 478111.5585657789651670472
    REAl8 1.045559703676712106312419E7
    REAl8 1.168694372024877518125471E8
    REAl8 6.707151683369115095043724E8
    REAl8 1.589566334450725218731593E9

    REAl8 12315.11751038764749208423
    REAl8 631333.7374995353116777207
    REAl8 1.321140989842440560380851E7
    REAl8 1.416565384021648426046242E8
    REAl8 7.822087291184103918278085E8
    REAl8 1.790203745507358979575692E9

    REAl8 16861.04993647702652692920
    REAl8 824358.0188788822263523139
    REAl8 1.649046073334505468534142E7
    REAl8 1.695140129565972815461480E8
    REAl8 9.005654216704815700431120E8
    REAl8 1.991386813593067849564320E9

    REAl8 22754.46441713161584215514
    REAl8 1.059906537737560762124723E6
    REAl8 2.025708483621198742030132E7
    REAl8 1.996366396615991261446319E8
    REAl8 1.021041863550817982651474E9
    REAl8 2.184170452641999693025212E9

    REAl8 30052.94003938008067652104
    REAl8 1.333453789840112773980128E6
    REAl8 2.435916941937797588827382E7
    REAl8 2.304017891703084958276696E8
    REAl8 1.136439160889121268842937E9
    REAl8 2.357353101661176436114812E9

    REAl8 38398.32828116109481622022
    REAl8 1.625559490940538651663771E6
    REAl8 2.845009739625804371902311E7
    REAl8 2.590568889751042406970710E8
    REAl8 1.236826787688546966331589E9
    REAl8 2.498070046735167137716712E9

    REAl8 46508.18760778884594428123
    REAl8 1.889552700529974441505065E6
    REAl8 3.188867359407945289359876E7
    REAl8 2.814585690098661576991069E8
    REAl8 1.309822612862406588867451E9
    REAl8 2.593244476008529728633210E9

    REAl8 51286.48680556550798215726
    REAl8 2.034171855556533214405132E6
    REAl8 3.364002008981092958762780E7
    REAl8 2.920662452804249836281330E8
    REAl8 1.341956881166352482231491E9
    REAl8 2.632194212836124515045006E9

    REAl8 46458.87558548222189352842
    REAl8 1.904246973569067231921234E6
    REAl8 3.224128809169131925320706E7
    REAl8 2.845366972054214218500149E8
    REAl8 1.321689598615280224161707E9
    REAl8 2.610371726286179527153147E9

    REAl8 20975.51806748501375713108
    REAl8 1.272504738849985470093961E6
    REAl8 2.597353007120837620438940E7
    REAl8 2.534276454565097162970081E8
    REAl8 1.244445288989806200988076E9
    REAl8 2.533610123826259437033142E9

    REAl8 -41665.44784579087591401305
    REAl8 -128327.7645362013802850065
    REAl8 1.343419421779237837243741E7
    REAl8 1.972664551890367319252868E8
    REAl8 1.118589292225542095616711E9
    REAl8 2.420713940265105676429838E9

    REAl8 -159925.2357718777121206428
    REAl8 -2.482454180858725956371900E6
    REAl8 -5.327422481589664148300558E6
    REAl8 1.224372553089571028572369E8
    REAl8 9.692293754178528354979665E8
    REAl8 2.301356342160404183516524E9

    REAl8 -340704.2751791445351200297
    REAl8 -5.637361869299855401966814E6
    REAl8 -2.737675127846776945036919E7
    REAl8 4.529620595971246012769910E7
    REAl8 8.341288379899892713131339E8
    REAl8 2.206602137416587262624473E9

    REAl8 -550688.4573738412971652942
    REAl8 -8.792287556340922002923093E6
    REAl8 -4.636726080472468121215988E7
    REAl8 -1.194845141320483499575401E7
    REAl8 7.477161208745740109585162E8
    REAl8 2.154344969410214654677325E9

    REAl8 -682084.2886640759045159987
    REAl8 -1.046972974352329558032552E7
    REAl8 -5.494779866854977968977399E7
    REAl8 -3.393037845944035201999181E7
    REAl8 7.195145890567937170329878E8
    REAl8 2.139850740573633841417918E9

    REAl8 -558162.3586669414578327902
    REAl8 -9.300481840357780218319035E6
    REAl8 -5.053236320407293140768169E7
    REAl8 -2.558848389629871480683629E7
    REAl8 7.273992081693988067212635E8
    REAl8 2.142833418771404723546764E9

    REAl8 -48522.57832244910139469034
    REAl8 -5.551935872604515445945657E6
    REAl8 -3.943509185975672954158751E7
    REAl8 -9.058245938290568166120203E6
    REAl8 7.397888127157848671876201E8
    REAl8 2.146571143434234695393244E9

    REAl8 726422.4115865083010185224
    REAl8 -1.686984749507153275137732E6
    REAl8 -3.161483717689012300886614E7
    REAl8 -1.037726009165054009298133E6
    REAl8 7.439547125925731762081104E8
    REAl8 2.147446772140937142388343E9

    REAl8 1.332860634151411311832797E6
    REAl8 -74937.92759159643898995843
    REAl8 -2.982077072597132270618378E7
    REAl8 -2845.635313607631774856469
    REAl8 7.442609865909374549425550E8
    REAl8 2.147483647014656632493609E9

    REAl8 1.335383913415405996515172E6
    REAl8 71760.75132826929181392017
    REAl8 -2.981936752714387582471325E7
    REAl8 2591.855502032223925773167
    REAl8 7.442610027424416123681440E8
    REAl8 2.147483648819214358475640E9

    REAl8 732051.9171355465161504821
    REAl8 1.665853223774519926133289E6
    REAl8 -3.158354830562147849911415E7
    REAl8 1.014896776518033605177229E6
    REAl8 7.439629167360230450583053E8
    REAl8 2.147519362430839555255322E9

    REAl8 -43516.16926320761131628149
    REAl8 5.520633305127000158802576E6
    REAl8 -3.935719525108734579673865E7
    REAl8 8.961814580893349163105159E6
    REAl8 7.398481947822541111526046E8
    REAl8 2.148381601094616583203359E9

    REAl8 -555921.2122245761574794132
    REAl8 9.280871856349502145332422E6
    REAl8 -5.046390348689352598440538E7
    REAl8 2.546929239223269358817666E7
    REAl8 7.275026990367835896043112E8
    REAl8 2.152098026967556942059480E9

    REAl8 -682338.0427011109666668739
    REAl8 1.047259356078063484176302E7
    REAl8 -5.496070718958433717490771E7
    REAl8 3.395942613924233865750694E7
    REAl8 7.194819568827038098825054E8
    REAl8 2.155131196298104028886998E9

    REAl8 -552102.7944867689552141823
    REAl8 8.811748344096884160537542E6
    REAl8 -4.647425982677673214475332E7
    REAl8 1.224229716940814689065877E7
    REAl8 7.473130531668233332682037E8
    REAl8 2.140843251532804229212566E9

    REAl8 -342194.4887024370594115037
    REAl8 5.661589742560789332911280E6
    REAl8 -2.753419347450882094913362E7
    REAl8 -4.478502439885995820700179E7
    REAl8 8.332996046052223682557839E8
    REAl8 2.088902828232224080359230E9

    REAl8 -161013.7461251784840607237
    REAl8 2.502871775334219798084822E6
    REAl8 -5.480529635610485900873517E6
    REAl8 -1.218635157298784007561354E8
    REAl8 9.681549848455054270748302E8
    REAl8 1.994415271074422430609307E9

    REAl8 -42299.66082296048907950085
    REAl8 141809.1455379105287653525
    REAl8 1.331961498347535586151603E7
    REAl8 -1.967797573339104896614340E8
    REAl8 1.117556067695612863684682E9
    REAl8 1.875130359761998188226793E9

    REAl8 20679.28972340881200391904
    REAl8 -1.265467531954631347834732E6
    REAl8 2.590668279324195458102168E7
    REAl8 -2.531102602697656277104965E8
    REAl8 1.243692091343737040658066E9
    REAl8 1.762071899729681598408167E9

    REAl8 46365.72774184264862518188
    REAl8 -1.901801537210444875673743E6
    REAl8 3.221561507717393112474085E7
    REAl8 -2.844019734430491972025887E8
    REAl8 1.321336205333986142051703E9
    REAl8 1.684966258168738115434647E9

    REAl8 51296.26906479652243893437
    REAl8 -2.034453460798958838014325E6
    REAl8 3.364326199808319996811902E7
    REAl8 -2.920849017275537820841159E8
    REAl8 1.342010550377863576164611E9
    REAl8 1.662711341351493053323293E9

    REAl8 46559.93123999386323884351
    REAl8 -1.891170333934258053804151E6
    REAl8 3.190889803596906165872110E7
    REAl8 -2.815849716529626884455225E8
    REAl8 1.310217541716875540965410E9
    REAl8 1.701229356972405374298253E9

    REAl8 38460.12086416241761948303
    REAl8 -1.627645650398769250723169E6
    REAl8 2.847826463335892321484685E7
    REAl8 -2.592470128012970606378521E8
    REAl8 1.237468328576388340990004E9
    REAl8 1.796031489127965865506578E9

    REAl8 30110.28394493323552923772
    REAl8 -1.335533083132631871219997E6
    REAl8 2.438932305858293550992657E7
    REAl8 -2.306203987449487887401174E8
    REAl8 1.137231487150732082962270E9
    REAl8 1.936465685588529349550436E9

    REAl8 22802.22654190181919955177
    REAl8 -1.061757786076044164071924E6
    REAl8 2.028578268031018092051072E7
    REAl8 -1.998590461157658387397019E8
    REAl8 1.021903569309950097004185E9
    REAl8 2.109461558065995587095947E9

    REAl8 16898.57206835948319166550
    REAl8 -825906.1665794047656624242
    REAl8 1.651600817748027651538321E7
    REAl8 -1.697247799926236575789155E8
    REAl8 9.014347384432682621387512E8
    REAl8 2.302146434894433289603890E9

    REAl8 12343.57580198898116097439
    REAl8 -632579.0576846077662020517
    REAl8 1.323320550391246943614610E7
    REAl8 -1.418472523900361965095591E8
    REAl8 7.830430283356968093851157E8
    REAl8 2.503303806228949611480746E9

    REAl8 8927.973064356175892775077
    REAl8 -479087.8475410828410687272
    REAl8 1.047366041289495487078952E7
    REAl8 -1.170365269584774042270141E8
    REAl8 6.714879042000377509580123E8
    REAl8 2.703971631145601320235713E9

    REAl8 6413.797957054993430475087
    REAl8 -359827.4527423775650911503
    REAl8 8.210440342052674839141764E6
    REAl8 -9.555835596385718941175884E7
    REAl8 5.695565305924893023512178E8
    REAl8 2.897501764135767872771733E9

    REAl8 4585.942033355371218497164
    REAl8 -268556.8725734998419099484
    REAl8 6.387205317683773997301802E6
    REAl8 -7.734509864859499090204063E7
    REAl8 4.785723657594800518327158E8
    REAl8 3.079332495988220877147896E9

    REAl8 3268.214712815484995751706
    REAl8 -199466.1568980733965572464
    REAl8 4.937996550565582048580656E6
    REAl8 -6.214419335992694732549847E7
    REAl8 3.988398842101365603505676E8
    REAl8 3.246641184370984739532365E9

    REAl8 2323.748866690648900071246
    REAl8 -147585.9424742034164932004
    REAl8 3.797932371701946848766331E6
    REAl8 -4.961627315321717607565446E7
    REAl8 3.299982965658639909564438E8
    REAl8 3.397974675117164069626992E9

    REAl8 1649.538824089230525077467
    REAl8 -108866.0150823789643863096
    REAl8 2.908359667697785142307522E6
    REAl8 -3.939638702112656562451389E7
    REAl8 2.712860859716973326815121E8
    REAl8 3.532907813611478820497266E9

    REAl8 1169.605238950241866908562
    REAl8 -80103.79214136446242493789
    REAl8 2.218806973176802228917185E6
    REAl8 -3.112979954677305528516105E7
    REAl8 2.217297794048506523087861E8
    REAl8 3.651751331191003852761445E9

    REAl8 828.6401511981534130941397
    REAl8 -58817.59552867366429668745
    REAl8 1.687205104539020066107648E6
    REAl8 -2.449106532548697109630451E7
    REAl8 1.802731033217287024241470E8
    REAl8 3.755314212381145710209216E9

    REAl8 586.7398776587289522026681
    REAl8 -43111.24772569231545208833
    REAl8 1.279251015085520036800340E6
    REAl8 -1.919254413885470255252510E7
    REAl8 1.458614597137765221811009E8
    REAl8 3.844717511935121969394817E9

    REAl8 415.2893728951092056474509
    REAl8 -31550.53654240797193383960
    REAl8 967416.0181237519759476997
    REAl8 -1.498653953855606180656706E7
    REAl8 1.174940770145503320649976E8
    REAl8 3.921253105746302081620290E9

    REAl8 293.8548238435451995418708
    REAl8 -23058.77178899783276696193
    REAl8 729871.1029238171823416873
    REAl8 -1.166380474650067195995057E7
    REAl8 9.425345113327178281336295E7
    REAl8 3.986279876291964299980339E9

    REAl8 207.8872125527905035088649
    REAl8 -16832.25329239257675326877
    REAl8 549467.3277581640143523852
    REAl8 -9.050163748985590180379262E6
    REAl8 7.531928167114279138649217E7
    REAl8 4.041150086165972192611281E9

    REAl8 147.0487045383223535089716
    REAl8 -12273.70923165460640302243
    REAl8 412832.4065115245922415969
    REAl8 -7.002327291189103659302461E6
    REAl8 5.997216225499573116012021E7
    REAl8 4.087159482559950174789489E9

    REAl8 104.0042630389798698669918
    REAl8 -8940.840280572152975131376
    REAl8 309602.4906630373519650869
    REAl8 -5.403545715874895932158144E6
    REAl8 4.759078504351004841642775E7
    REAl8 4.125515636319303188802148E9

[TWell]

What is that error ?

Code Select Expand

SMAC.asm(165):  error A2070:invalid instruction operands
Microsoft (R) Macro Assembler Version 10.00.40219.01
Copyright (C) Microsoft Corporation.  All rights reserved.

Code Select Expand

DisplayHelp:
invoke WriteFile,hStdOut,offset Helpmsg,offset Waitmsg-offset Helpmsg-1,offset Dummy,NULL

[dedndave]

i rarely see that error on INVOKE lines, but you might try this...

Code Select Expand

TmpOperand = offset Waitmsg-offset Helpmsg-1
invoke WriteFile,hStdOut,offset Helpmsg,TmpOperand,offset Dummy,NULL

not really great form
if WaitMsg gets moved in the data section, the program will break

[TWell]

I compiled that file with POAsm, so there must be something i missed.

Jochen, how you manged to do that with ml.exe?

EDIT: JWasm compiled this file

[jj2007]

I compiled that file with POAsm, so there must be something i missed.

Jochen, how you manged to do that with ml.exe?

EDIT: JWasm compiled this file

I translated it to Masm syntax, see attachment to reply #3.

Astonishing that JWasm assembles your code ::)
However, it still chokes:
Please Wait...
Out of Memory : file was too large to be processed

[TWell]

Code Select Expand

JWasm v2.10, Apr 20 2013, Masm-compatible assembler.
Portions Copyright (c) 1992-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.

SMAC.asm: 1651 lines, 4 passes, 16 ms, 0 warnings, 0 errors
BTW: this was my exercise to convert it for POAsm.exe, as i'm not an assembler programmer  ;).
I could be glad if pro's correct my mistakes.

[jj2007]

TWell, good job :t
Can you compress files? With your exe, I still get the "file too large" error, now on Win XP SP3.
Same for Jean-Marie's executable...
@echo off
SMAC.exe /c win32.inc
pause >nul

[TWell]

Code Select Expand

smac /c SMAC.asm.bak
Please Wait...
Finished!
smac /d SMAC.asm.bak.sma
Please Wait...
Finished!


[jj2007]

Mysterious ::)

I use exactly the same commandline, and both in WinXP SP3 and Win7-32, I get that error message; which is misleading on Win7-32 because it's WriteFile reporting a bad address, and misleading on WinXP because the processed file is tiny.

Who else has tested it? Attached a diagnostic version, which yields for me
- bad address in WriteFile on Win7-32
- not enough memory for BitHistory on XP (where I have 2 GB of RAM)

[Jean-Marie]

@qWord : Interesting : I'm gonna test that. Regarding precision, the bigger the better of course, but I don't mind hurting the compression ratio (too much) for the benefit of speed. Thanks!

[Jean-Marie]

@JJ2007: looks like you restored the registers in the wrong order :

Code Select Expand


@@Error:
deb 4, "@@Error", $Err$()
pop ebx ;  pop edi esi edx ecx ebx
pop ecx
pop edx
pop esi
pop edi
ret