FPU use, C vs MASM

[KeepingRealBusy]

Here is the result of the new output looking for rounding errors:

Code Select Expand


------------------------------------------------------------------ 17
Chi square distribution for 2048 samples is 6.13 (6.1250), and randomly
-------------------------------------------------------------- 17
Chi square distribution for 2048 samples is 6.12 (6.1250), and randomly


Now the question is, which one is correct? Should it round at .5 or at .5+.?
Note that my FPU output is via a slightly modified  FPULIB PROC FpuFLtoA, but because it is an embedded PROC in my code and not a library routine, I can modify it, but should I, or is ent wrong?

Dave.

[MichaelW]

The CRT could be changing the rounding mode inside some function and then restoring it before the function returns, but I can't see any ready way to detect this other than a debugger.

I have now tried optimizations up through /O2 /G7, and the output still matches that of the original EXE.

[dedndave]

Here is the result of the new output looking for rounding errors:

Code Select Expand


------------------------------------------------------------------ 17
Chi square distribution for 2048 samples is 6.13 (6.1250), and randomly
-------------------------------------------------------------- 17
Chi square distribution for 2048 samples is 6.12 (6.1250), and randomly


Now the question is, which one is correct? Should it round at .5 or at .5+.?
Note that my FPU output is via a slightly modified  FPULIB PROC FpuFLtoA, but because it is an embedded PROC in my code and not a library routine, I can modify it, but should I, or is ent wrong?

Dave.

macht nichts
rounding up to 6.13 is technically correct
but, 6.12 is just as close - lol
wouldn't hurt to add another decimal digit   :P

[KeepingRealBusy]

Here are a couple of  links I picked up at the CodeProject:

http://linuxgazette.net/144/misc/lg/a_question_of_rounding_in_issue_143.html
http://msdn.microsoft.com/en-us/library/ty9hx077(VS.100).aspx
http://www.stroustrup.com/

I could not find too much in the msdn documentation about rounding, other than they round up the last digit, but nothing about which rounding method. I did find that if you cast a float to a (short) then you get truncation.

So far, in ENT, I have found truncation, round to an even , and normal rounding (just add a 0.5 and truncate).

Dave.

[FORTRANS]

Hi Dave,

   That first link was interesting, thanks.

   In "The Art of Compuer Programming" by Donald E. Knuth,
there is a discussion on rounding.  He says that rounding up,
(rounding down), and truncation can cause drift in some cases.
Rounding to nearest or even (round to even) or round to odd
don't drift.  He says for base 2 or base 10 you should round
to even.  Drift is an accumulating error caused by the bias in
rounding in some repetitive algorithms.  (This is a bit simplfied.)

Regards,

Steve

[Gunther]

Hi Dave,

http://linuxgazette.net/144/misc/lg/a_question_of_rounding_in_issue_143.html

that's a very interesting article. Thank you.

Gunther

[KeepingRealBusy]

Steve,

I thought it was a very interesting  article also, even it it was talking about Linux. It did reference the  IEEE spec which identified the fact that rounding is implementation dependent.

I decided to just accept the differences I have found in the output when it is a difference of 1 in the last digit, and let it go at that. I removed the forced rounding for the cast in my modified ENT (since that could round an even to an odd). My MASMENT always uses round to nearest even, and it appears that the FpuLib FpuFLtoA does also.

Dave.

[KeepingRealBusy]

Gunther,

You responded while I was typing.

I agree.

Dave.

[dedndave]

it would be so much easier if CPU's spoke decimal as a native tongue - lol
if humans had 1 finger on each hand, we'd all be speaking binary
but, then, the computer may never have come about
it's kind of a zen parable thing

[dedndave]

found it
reminds me of a discussion Jochen and i were having a while back...
http://www.masmforum.com/board/index.php?topic=11781.0

[FORTRANS]

it would be so much easier if CPU's spoke decimal as a native tongue - lol

Hi Dave,

   That's what BCD is for.  But it would be a bit of a project
to implement IEEE floating point operations.

Cheers,

Steve N.

[dedndave]

BCD is a second language, with a poor translater   :P

il tuo asino sembra delizioso, oggi

lol
that turned out even worse than i expected