Author Topic: FPU power  (Read 6496 times)

rrr314159

  • Member
  • *****
  • Posts: 1383
Re: FPU power
« Reply #45 on: October 11, 2015, 06:26:04 AM »
So ISO C99 says that at the midpoint round to even - in this case 38f00000 (down) not 38f00001. If it's above, no matter how little (10^-60, or 10^-60000) round up. So a compiler should take account of all the digits in your data statement (e.g. 5.028986508540491482566165849167001609232e-1, or 1.000000059604644775390625000000000000000000000000000000000001) to provide the correct rounding, even though of course they don't appear in the actual value your program will use. That's what I had thought. I wonder if ml.exe and/or JWasm does that. Easy enough to check it out ...
I am NaN ;)

qWord

  • Member
  • *****
  • Posts: 1471
  • The base type of a type is the type itself
    • SmplMath macros
Re: FPU power
« Reply #46 on: October 11, 2015, 07:59:21 AM »
sorry, saying that C99 describes such conversion was not right. For FP literals the standard says that the conversion should be identical to the runtime conversion (with the default runtime rounding mode). The number of significant digits is limit (DECIMAL_DIG@float.h, e.g. 21 for gcc). The standard than says:
Quote
If the subject sequence has the decimal form and at most DECIMAL_DIG significant digits, the result should be correctly rounded. If the subject sequence D has the decimal form and more than DECIMAL_DIG significant digits, consider the two bounding, adjacent decimal strings L and U, both having DECIMAL_DIG significant digits, such that the values of L, D, and U satisfy L ≤ D ≤ U.
The result should be one of the (equal or adjacent) values that would be obtained by correctly rounding L and U according to the current rounding direction, with the extra stipulation that the error with respect to D should have a correct sign for the current rounding direction
Also to say, the IEEE standard allows - but does not require - a limitation of significant digits.
MREAL macros - when you need floating point arithmetic while assembling!