The MASM Forum

64 bit assembler => 64 Bit Assembler => Topic started by: Gunther on November 04, 2012, 09:52:42 PM

Title: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 04, 2012, 09:52:42 PM
I've attached the archive testbed.zip.

It contains the sources, the binary files, and the running applications for both 64 bit operating systems (Linux and Windows). For details and explanations, please check the files readme.txt and testbed.pdf.

A special thanks to Bill Cravener; he did a bit proof reading. But to make it clear: I'm responsible for all the errors in the files and documents - nobody else. Critical remarks and proposals for improvements are welcome.

Gunther
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Ryan on November 06, 2012, 07:18:50 AM
I've got the Express Edition of VS C++.  I was trying to compile it, and then I remembered I've had no luck linking 64-bit programs.
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 06, 2012, 07:21:27 AM
Ryan,

thank you for your effort. What's the problem? I used ANSI C?

Gunther
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: qWord on November 06, 2012, 07:21:36 AM
Quote
Processing REAL 4 numbers with the new XMM registers means that the
calculation is made only with 32 bit; that leads to the catastrophic digit
cancallation and is the reason for the false results. The data type float is not
very reliable under a 64 bit operating system, because the described design
decision seems to be unwise.
I can't agree with that:
1. floats have 24 precession bits, thus they can represents integers in the range from -16777216 to 16777216 exact. Your test produces numbers that are out of that rang thus rounding applies.
2. If I'm not wrong, most C/C++ compilers set the FPU's precision control to double by default. If you set the PC to single, you will get the same results as with the SIMD instructions.
3. I can't see why you claim this a 64Bit problem - you have simply chose the wrong data type

regards, qWord
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Ryan on November 06, 2012, 07:31:17 AM
Ryan,

thank you for your effort. What's the problem? I used ANSI C?

Gunther
I've tried multiple times to install the SDK as Microsoft recommends, but it gives a generic error message during the install and fails.  I've tried disabling my AV, but no effect.

http://msdn.microsoft.com/en-us/library/9yb4317s%28v=vs.80%29.aspx
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 06, 2012, 07:49:50 AM
Hi qword,

may be or not. We've the fact that the FPU has done a pretty good job in that number range, while the xmm code failed resoundingly. Period.

2. If I'm not wrong, most C/C++ compilers set the FPU's precision control to double by default.
That depends. The 32 bit OS starts the FPU with the full 80 bit wide in the rounding mode round to nearest (http://round to nearest); that's reasonable. The ANSI standard uses the rounding mode truncate for casts, for example, from floating point to integer. Therefore the FPU control word must be changed and restored after the cast. But there's no need to shrink the FPU precision. It has had serious reasons that William Kahan, the father of floating point, insist for a 80 bit wide FPU and Intel realized it.

3. I can't see why you claim this a 64Bit problem - you have simply chose the wrong data type

Why should be a REAL 4 (float) a wrong data type? The good old FPU makes a good job in the described data range. Why not using another approach? Let the FPU doing the floating point operations by default (with the full register width) and if one would like to use the XMM registers, set an appropriate compiler switch. This leads to better and more reliable results. That was the standard behaviour for more than 25 years.

Gunther
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: jj2007 on November 06, 2012, 08:17:00 AM
You get the precision you choose...

PI on FPU=      3.141592653589793238
PI ^ 9 WinCalc= 29809.09933344621166
PI ^ 9 FPU=     29809.09933344621167
PI ^ 9 R8=      29809.09933344620003
PI ^ 9 R4=      29809.10742187500000


Windows Calculator uses internally a very high precision, 30 digits or so.

include \masm32\MasmBasic\MasmBasic.inc   ; download (http://masm32.com/board/index.php?topic=94.0)
.data
res10   REAL10 29809.09933344621166650940240124   ; Windows Calculator
pi4   REAL4 3.1415926535897932384626433832795
pi8   REAL8 3.1415926535897932384626433832795
res4   REAL4 ?
res8   REAL8 ?

   Init
   expo=9
   fldpi
   Print Str$("PI on FPU=\t%Jf\n", ST(0))
   Print Str$("PI ^ 9 WinCalc=\t%Jf\n", res10)

   fldpi
   REPEAT expo-1
      fmul st, st(1)   ; FPU, Real10 precision
   ENDM
   Print Str$("PI ^ 9 FPU=\t%Jf\n", ST(0))

   movsd xmm0, pi8
   REPEAT expo-1
      mulsd xmm0, pi8   ; xmm, Real8 precision
   ENDM
   movsd res8, xmm0
   Print Str$("PI ^ 9 R8=\t%Jf\n", res8)

   movss xmm0, pi4
   REPEAT expo-1
      mulss xmm0, pi4   ; xmm, Real4 precision
   ENDM
   movss res4, xmm0
   Inkey Str$("PI ^ 9 R4=\t%Jf\n", res4)

   Exit
end start
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 06, 2012, 08:23:06 AM
Hi Jochen,

one thing is clear: MasmBasic is a good thing.  :t On the other hand are the numbers not inside a critical range.

Gunther
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: qWord on November 06, 2012, 08:29:25 AM
But there's no need to shrink the FPU precision.
Speed!
It has had serious reasons that William Kahan, the father of floating point, insist for a 80 bit wide FPU and Intel realized it.
The reason was that there are calculations that need such high precision. Even that requires to use the corresponding data type in HLLs.

Why should be a REAL 4 (float) a wrong data type? The good old FPU makes a good job in the described data range.
Why to use a data type that maybe can't represent the calculation's (intermediate) result(s)?
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: jj2007 on November 06, 2012, 08:34:56 AM
On the other hand are the numbers not inside a critical range.

But you can see how the precision decreases rapidly for Pi^9....
What do you get for your C compilers? Same as Real8, or Real4 precision? Under the hood of the compiler is assembler ;-)
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 06, 2012, 09:13:57 AM
Hi qword,

Speed!

What helps speed, if the result is garbage? You've fast garbage, nothing else.

The reason was that there are calculations that need such high precision. Even that requires to use the corresponding data type in HLLs.

There are a lot of calculations that needs such precision. What we have with floating point numbers is something like a comb with lesser and lesser tooths at the end of the range. So try to comb your hair with that equipment! And you would really like to shrink the precision?

Why to use a data type that maybe can't represent the calculation's (intermediate) result(s)?

That's the point: interim results must be exact enough to reduce the rounding errors. A lot of people made a lot of effort to reach that goal; check for example Kahan's summation algorithm https://en.wikipedia.org/wiki/Kahan_summation_algorithm (https://en.wikipedia.org/wiki/Kahan_summation_algorithm). And don't forget: Robert Hooke wrote it very precise: If I have been able to see further it is because I have stood on the Shoulders of Giants. There's no need to look arrogant at the work of our predecessors. Nobody alone is a pillar of wisdom.

Gunther

Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 06, 2012, 09:18:19 AM
Jochen,

But you can see how the precision decreases rapidly for Pi^9....
What do you get for your C compilers? Same as Real8, or Real4 precision? Under the hood of the compiler is assembler ;-)

that is very clear. What my compiler produces shows the example. You can't blame the underlying assembler for that. It has to do with the lack of thought by some compiler builders which haven't any clue of numerical mathematics.

Gunther
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: qWord on November 06, 2012, 09:38:12 AM
What helps speed, if the result is garbage? You've fast garbage, nothing else.
That's my point: using the correct data type doesn't yield  in garbage.

There's no need to look arrogant at the work of our predecessors. Nobody alone is a pillar of wisdom
That was not my intention. My thoughts was on the data type and not about the need of high precision calculations.

regards, qWord
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 07, 2012, 07:39:47 AM
Hi qWord,

That's my point: using the correct data type doesn't yield  in garbage.

That's your point of view - why not? But it seems to me that you're labouring under a misapprehension. In practical calculations it is very common that interim results are out of range. Let me give you an example: the multiplication of two 24 bit mantissas leads to a 48 bit mantissa - out of range! Do you really think that rounding down that interim result with brute force is exactly the ideal solution? The FPU width of 80 bit was made to accumulate such results; the appropriate rounding was done after finishing the computation. The record of success speaks for the FPU and not for the fancy new automatic XMM code generation.

That was not my intention. My thoughts was on the data type and not about the need of high precision calculations.

Of course, but a high accuracy arithmetic is only possible with good sense.

Gunther

Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: qWord on November 07, 2012, 08:58:48 AM
hi,
But it seems to me that you're labouring under a misapprehension. In practical calculations it is very common that interim results are out of range. Let me give you an example: the multiplication of two 24 bit mantissas leads to a 48 bit mantissa - out of range!
That is absolute correct: a FP multiplier must have at least a 48 Bit register. However, the results gets normalized, which also means that only redundant precision bit are rounded. Even, all this is done in the hardware regardless if we are using the FPU or SSEx instructions.

I see that we can't get on the same line - you expect that the compiler should always use highest precision regardless the used FP type. Even this sound a bit like you won't think about the error of an calculation and instead hope that the high precision solve that problem. I would estimate the error and then choose the correct data type or, if need, instead us an high/arbitrary precision library.

regards, qWord



Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 08, 2012, 02:51:18 AM
Hi qWord,

your later answers sounding better and better and I hope we're at the same line.

Even this sound a bit like you won't think about the error of an calculation and instead hope that the high precision solve that problem. I would estimate the error and then choose the correct data type or, if need, instead us an high/arbitrary precision library.

Excuse me please, but with the best will in the world, I can't see an error. In my testbed we've the following situation:

Therefore, the FPU makes a good job in that range and brings the right results. What you are calling "error" has only to do with the precision shrink. These shrink isn't a law of nature, but the work of man done by compiler builders.

Gunther
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: qWord on November 08, 2012, 09:36:45 AM
  • No number has fractional part. So, rounding errors which have to do with 0.1 and multiples of that can't occur.
all numbers are normalized to 1.xxx...*2^y

  • The valid range for REAL 4 is from 1.175 x 10^(-38) up to 3.403 x 10^38. We're far away from that limits
yes, with 24 precision bits. Even with 64 precision bits, you  will only get around 18 decimal digits of precision.

These shrink isn't a law of nature, but the work of man done by compiler builders.
no, it is an technical limit.

Therefore, the FPU makes a good job in that range and brings the right results.
no doubts that it does a good job, but you will also get it using double precision variables with SSEx.

If I use float or double variables in my programs, I assume that the compiler will create code that does the calculations at least with that precision. However I would never assume that higher precision is used. As your example shows, the compiler creators and most other SW developers share this opinion.
You assumption based on your experience that it was done that way in the past - I'm sure it was never the intention by any compiler builder to always use REAL10/8s  - they simply know that it is dam slow to switch the FPU flags.

regards, qWord
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: qWord on November 08, 2012, 10:02:47 AM
addendum: I've take a look in the latest C standard and found this:
Quote from: ISO/IEC 9899:201x Committee Draft — April 12, 2011
Except for assignment and cast (which remove all extra range and precision), the values
yielded by operators with floating operands and values subject to the usual arithmetic
conversions and of floating constants are evaluated to a format whose range and precision
may be greater
than required by the type. The use of evaluation formats is characterized
by the implementation-defined value of FLT_EVAL_METHOD
  • -1 indeterminable;
  • 0 evaluate all operations and constants just to the range and precision of the
    type;
  • 1 evaluate operations and constants of type float and double to the
    range and precision of the double type, evaluate long double
    operations and constants to the range and precision of the long double
    type;
  • 2 evaluate all operations and constants to the range and precision of the
    long double type.

All other negative values for FLT_EVAL_METHOD characterize implementation-defined
behavior.
What does your compiler returns for FLT_EVAL_METHOD?
I currently can't find similar statements for c++...
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 10, 2012, 09:35:29 PM
Hi qWord,

all numbers are normalized to 1.xxx...*2^y

That's clear, but I meant the decimal representation of the numbers.

yes, with 24 precision bits. Even with 64 precision bits, you  will only get around 18 decimal digits of precision.
My point was REAL 4 and by the way, I've strong doubts that you will have 18 valid decimal digits with REAL 8; probably 16 digits is real.

These shrink isn't a law of nature, but the work of man done by compiler builders.
no, it is an technical limit.
The decision to use the xmm registers and not the FPU is the work of man and has nothing to do with technical limits.

no doubts that it does a good job, but you will also get it using double precision variables with SSEx.
That is to check.

If I use float or double variables in my programs, I assume that the compiler will create code that does the calculations at least with that precision. However I would never assume that higher precision is used. As your example shows, the compiler creators and most other SW developers share this opinion.
Or, and that's another possibility, the compiler builders don't know enough about the dangers and risks.

What does your compiler returns for FLT_EVAL_METHOD?
I currently can't find similar statements for c++...

I've to check that.

Gunther
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: MichaelW on November 11, 2012, 05:59:55 AM
Everything I have is 32-bit. For my most recent installation of MinGW (with gcc version 4.5.2), __FLT_EVAL_METHOD__ is set to 2. This is the relevant section of math.h:
Code: [Select]
/* Use the compiler's builtin define for FLT_EVAL_METHOD to
   set float_t and double_t.  */
#if defined(__FLT_EVAL_METHOD__) 
# if ( __FLT_EVAL_METHOD__== 0)
typedef float float_t;
typedef double double_t;
# elif (__FLT_EVAL_METHOD__ == 1)
typedef double float_t;
typedef double double_t;
# elif (__FLT_EVAL_METHOD__ == 2)
typedef long double float_t;
typedef long double double_t;
#endif
#else /* ix87 FPU default */
typedef long double float_t;
typedef long double double_t;
#endif

I cannot find any reference to __FLT_EVAL_METHOD__ in the header files that shipped with my installation of the 2003 PSDK, the Windows Server 2003 SP1 DDK, or the Microsoft Visual C++ Toolkit 2003.
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 11, 2012, 09:48:18 AM
Hi Mike,

thank you for your investigation. My installed gcc version 4.7.2 contains the same section in math.h.

Gunther
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: MichaelW on November 11, 2012, 02:36:43 PM
It seems to me that there should be a way to control the value, a command line option or some such, but in my quick search of the GCC docs I did not find one. And then there is the question of how this relates to the effective deprecation of long double under Windows.
Title: Re: 64 bit OS and Real 4 (float) - not very reliable!
Post by: Gunther on November 11, 2012, 09:34:12 PM
Hi Mike,

It seems to me that there should be a way to control the value, a command line option or some such, but in my quick search of the GCC docs I did not find one. And then there is the question of how this relates to the effective deprecation of long double under Windows.

the gcc supports long double under Windows but can't print such values, because it uses the windows libc. It's a bit strange.

Gunther