Small favour needed, test precision with sprintf.

[Siekmanski]

Hi Raymond,

As far as I understand correctly, the maximum number of digits:

real4     7 digits (32/4-1)
real8   15 digits (64/4-1)
real10 19 digits (80/4-1)

Is this correct?

[nidud]

deleted

[aw27]

Hi Raymond,

As far as I understand correctly, the maximum number of digits:

real4     7 digits (32/4-1)
real8   15 digits (64/4-1)
real10 19 digits (80/4-1)

Is this correct?

Interesting rule of thumb, but is not based on the number of available decimal digits according to the IEEE format.
These are: 64, 52 and 23 bits for REAL10, REAL8 and REAL4 respectively
My calculation is the following (who knows better please contradict):

X=(ln 10)/(ln 2)=3.3219280948873623478703194294894

64/X = 19.26 => 19/20

52/X 15.65 => 15/16

23/X = 6.92 => 6/7

So, to be on the safe side, the number of guaranteed significant digits is: 19, 15, 6 respectively for REAl10, REAL8 and REAL4.

[Siekmanski]

From MSDN,

The mantissa is stored as a binary fraction greater than or equal to 1 and less than 2. For types float and double, there is an implied leading 1 in the mantissa in the most-significant bit position, so the mantissas are actually 24 and 53 bits long, respectively, even though the most-significant bit is never stored in memory.

https://msdn.microsoft.com/en-us/library/hd7199ke.aspx

[jj2007]

so the mantissas are actually 24 and 53 bits long

Test it... note the last digits in the range 8.xx and above (53/3.321928095=15.95 digits):

Code Select Expand

include \masm32\include\masm32rt.inc

.data
TmpR8 REAL8 ?
.code
start:
  xor ebx, ebx ; set two non-volatile
  finit
  fld1
  fld FP10(1.2345678901234567890)
  .Repeat
fld st
fstp TmpR8
printf("%.17f\n", TmpR8)
fadd ST, ST(1)
inc ebx
  .Until ebx>=8
  .Repeat
fld st
fstp TmpR8
printf("%.17f\n", TmpR8)
fsub ST, ST(1)
dec ebx
  .Until Sign?
  inkey
  exit
end start

[nidud]

deleted

[raymond]

Hi Raymond,

As far as I understand correctly, the maximum number of digits:

real4     7 digits (32/4-1)
real8   15 digits (64/4-1)
real10 19 digits (80/4-1)

Is this correct?

I generally agree with those limits, except that I consider that the range for the real8 to be 16 significant digits for all intents and purposes.

Same for real16, 113/112-bit. The bit is always set unless the value is zero.

I'm not familiar with the real16. However, for real4 and real8, the above statement is not 100% correct. It should read:
"The bit is always set unless the biased exponent is zero." i.e. the value of the real number is either 0 or that of a 'denormalized real number'. For those following this thread but not entirely familiar with this subject, some more info on 'denormalized real numbers' is available at
http://www.ray.masmcode.com/tutorial/fpuchap2.htm#denormal

[hutch--]

Hi Ray,

I well understand what can be fitted into a REAL8 variable but when defining a .data section entry, the assembler simply truncates values that are larger than REAL8 so I can routinely do things like this.

  .data
  align 16
  pi REAL8 3.14159265358979323846

A large number library is probably the only way to do really big numbers, REAL10 gets some extra precision but probably not enough for some of the scientific folk.

[jj2007]

that I consider that the range for the real8 to be 16 significant digits for all intents and purposes.

Simple pure Masm32 test, using REAL8 precision with CRT printf (source & exe attached):

Code Select Expand

include \masm32\include\masm32rt.inc

.data
TmpR8 REAL8 ?
.code
start:
  xor ebx, ebx ; set two non-volatile
  finit
  fld1
  fld FP10(1.0000000000000011111111)
  print "1.2345678901234567", 13, 10
  print "1.0000000000000011111 as REAL10", 13, 10
  .Repeat
fld st
fstp TmpR8
printf("%.16f\n", TmpR8)
fadd ST, ST(1)
inc ebx
  .Until ebx>=8
  .Repeat
fld st
fstp TmpR8
printf("%.16f\n", TmpR8)
fsub ST, ST(1)
dec ebx
  .Until Sign?
  inkey "0.0000000000000011111 as REAL10 on exit"
  exit
end start

Output:

Code Select Expand

1.2345678901234567
1.0000000000000011111 as REAL10
1.0000000000000011  +-0
2.0000000000000013  +2 at pos 17
3.0000000000000013  +2
4.0000000000000009  -2
5.0000000000000009
6.0000000000000009
7.0000000000000009
8.0000000000000018  +7
9.0000000000000018  +7    <<<<< max deviation
8.0000000000000018  +7
7.0000000000000009
6.0000000000000009
5.0000000000000009
4.0000000000000009  -2
3.0000000000000013  +2
2.0000000000000013
1.0000000000000011  +-0
0.0000000000000011111 as REAL10 on exit

It is the middle range where the 15.95 digits show a small distortion.

[hutch--]

I had to laugh, working with formulas involves stuff I learnt in primary school over 60 years ago. Back then before the digital era it was taught as fractions and little has changed apart from it being taught in digital format.

[raymond]

If the accuracy of that 16th digit with the real8 becomes sooooooooo important, the use of a real10 should then be investigated!!!!

The accuracy of a 17th digit with real8 is definitely not probable.

[Siekmanski]

Code Select Expand

1.2345678901234567
1.0000000000000011111 as REAL10
1.000000000000001
2.000000000000001
3.000000000000001
4.000000000000001
5.000000000000001
6.000000000000001
7.000000000000001
8.000000000000002
9.000000000000002
8.000000000000002
7.000000000000001
6.000000000000001
5.000000000000001
4.000000000000001
3.000000000000001
2.000000000000001
1.000000000000001
0.0000000000000011111 as REAL10 on exit

[aw27]

I am missing something for sure but when Raymond says  "I generally agree with those limits, except that I consider that the range for the real8 to be 16 significant digits for all intents and purposes." and I look at the JJ output where almost half the values have 15  significant digits, can I conclude that it is really bad luck for a calculated 15.95 value?

[nidud]

deleted

[jj2007]

almost half the values have 15  significant digits

When using rounding, only 3 out of 17 are off by one at the 16th digit - see Marinus' output.