News:

Masm32 SDK description, downloads and other helpful links
Message to All Guests

Main Menu

Small favour needed, test precision with sprintf.

Started by hutch--, September 05, 2018, 02:14:34 PM

Previous topic - Next topic

Siekmanski

Hi Raymond,

As far as I understand correctly, the maximum number of digits:

real4     7 digits (32/4-1)
real8   15 digits (64/4-1)
real10 19 digits (80/4-1)

Is this correct?
Creative coders use backward thinking techniques as a strategy.

nidud

#31
deleted

aw27

Quote from: Siekmanski on September 11, 2018, 04:05:42 AM
Hi Raymond,

As far as I understand correctly, the maximum number of digits:

real4     7 digits (32/4-1)
real8   15 digits (64/4-1)
real10 19 digits (80/4-1)

Is this correct?

Interesting rule of thumb, but is not based on the number of available decimal digits according to the IEEE format.
These are: 64, 52 and 23 bits for REAL10, REAL8 and REAL4 respectively
My calculation is the following (who knows better please contradict):

X=(ln 10)/(ln 2)=3.3219280948873623478703194294894

64/X = 19.26 => 19/20

52/X 15.65 => 15/16

23/X = 6.92 => 6/7

So, to be on the safe side, the number of guaranteed significant digits is: 19, 15, 6 respectively for REAl10, REAL8 and REAL4.



Siekmanski

From MSDN,

The mantissa is stored as a binary fraction greater than or equal to 1 and less than 2. For types float and double, there is an implied leading 1 in the mantissa in the most-significant bit position, so the mantissas are actually 24 and 53 bits long, respectively, even though the most-significant bit is never stored in memory.

https://msdn.microsoft.com/en-us/library/hd7199ke.aspx
Creative coders use backward thinking techniques as a strategy.

jj2007

Quote from: Siekmanski on September 11, 2018, 09:42:43 AMso the mantissas are actually 24 and 53 bits long

Test it... note the last digits in the range 8.xx and above (53/3.321928095=15.95 digits):
include \masm32\include\masm32rt.inc

.data
TmpR8 REAL8 ?
.code
start:
  xor ebx, ebx ; set two non-volatile
  finit
  fld1
  fld FP10(1.2345678901234567890)
  .Repeat
fld st
fstp TmpR8
printf("%.17f\n", TmpR8)
fadd ST, ST(1)
inc ebx
  .Until ebx>=8
  .Repeat
fld st
fstp TmpR8
printf("%.17f\n", TmpR8)
fsub ST, ST(1)
dec ebx
  .Until Sign?
  inkey
  exit
end start

nidud

#35
deleted

raymond

Quote from: Siekmanski on September 11, 2018, 04:05:42 AM
Hi Raymond,

As far as I understand correctly, the maximum number of digits:

real4     7 digits (32/4-1)
real8   15 digits (64/4-1)
real10 19 digits (80/4-1)

Is this correct?

I generally agree with those limits, except that I consider that the range for the real8 to be 16 significant digits for all intents and purposes.

QuoteSame for real16, 113/112-bit. The bit is always set unless the value is zero.

I'm not familiar with the real16. However, for real4 and real8, the above statement is not 100% correct. It should read:
"The bit is always set unless the biased exponent is zero." i.e. the value of the real number is either 0 or that of a 'denormalized real number'. For those following this thread but not entirely familiar with this subject, some more info on 'denormalized real numbers' is available at
http://www.ray.masmcode.com/tutorial/fpuchap2.htm#denormal
Whenever you assume something, you risk being wrong half the time.
http://www.ray.masmcode.com

hutch--

Hi Ray,

I well understand what can be fitted into a REAL8 variable but when defining a .data section entry, the assembler simply truncates values that are larger than REAL8 so I can routinely do things like this.

  .data
  align 16
  pi REAL8 3.14159265358979323846

A large number library is probably the only way to do really big numbers, REAL10 gets some extra precision but probably not enough for some of the scientific folk.

jj2007

Quote from: raymond on September 11, 2018, 12:37:12 PMthat I consider that the range for the real8 to be 16 significant digits for all intents and purposes.

Simple pure Masm32 test, using REAL8 precision with CRT printf (source & exe attached):
include \masm32\include\masm32rt.inc

.data
TmpR8 REAL8 ?
.code
start:
  xor ebx, ebx ; set two non-volatile
  finit
  fld1
  fld FP10(1.0000000000000011111111)
  print "1.2345678901234567", 13, 10
  print "1.0000000000000011111 as REAL10", 13, 10
  .Repeat
fld st
fstp TmpR8
printf("%.16f\n", TmpR8)
fadd ST, ST(1)
inc ebx
  .Until ebx>=8
  .Repeat
fld st
fstp TmpR8
printf("%.16f\n", TmpR8)
fsub ST, ST(1)
dec ebx
  .Until Sign?
  inkey "0.0000000000000011111 as REAL10 on exit"
  exit
end start


Output:
1.2345678901234567
1.0000000000000011111 as REAL10
1.0000000000000011  +-0
2.0000000000000013  +2 at pos 17
3.0000000000000013  +2
4.0000000000000009  -2
5.0000000000000009
6.0000000000000009
7.0000000000000009
8.0000000000000018  +7
9.0000000000000018  +7    <<<<< max deviation
8.0000000000000018  +7
7.0000000000000009
6.0000000000000009
5.0000000000000009
4.0000000000000009  -2
3.0000000000000013  +2
2.0000000000000013
1.0000000000000011  +-0
0.0000000000000011111 as REAL10 on exit


It is the middle range where the 15.95 digits show a small distortion.

hutch--

I had to laugh, working with formulas involves stuff I learnt in primary school over 60 years ago. Back then before the digital era it was taught as fractions and little has changed apart from it being taught in digital format.

raymond

If the accuracy of that 16th digit with the real8 becomes sooooooooo important, the use of a real10 should then be investigated!!!!

The accuracy of a 17th digit with real8 is definitely not probable.
Whenever you assume something, you risk being wrong half the time.
http://www.ray.masmcode.com

Siekmanski

1.2345678901234567
1.0000000000000011111 as REAL10
1.000000000000001
2.000000000000001
3.000000000000001
4.000000000000001
5.000000000000001
6.000000000000001
7.000000000000001
8.000000000000002
9.000000000000002
8.000000000000002
7.000000000000001
6.000000000000001
5.000000000000001
4.000000000000001
3.000000000000001
2.000000000000001
1.000000000000001
0.0000000000000011111 as REAL10 on exit
Creative coders use backward thinking techniques as a strategy.

aw27

I am missing something for sure but when Raymond says  "I generally agree with those limits, except that I consider that the range for the real8 to be 16 significant digits for all intents and purposes." and I look at the JJ output where almost half the values have 15  significant digits, can I conclude that it is really bad luck for a calculated 15.95 value?

nidud

#43
deleted

jj2007

Quote from: AW on September 12, 2018, 06:23:27 AMalmost half the values have 15  significant digits

When using rounding, only 3 out of 17 are off by one at the 16th digit - see Marinus' output.