### Author Topic: Math10 library  (Read 33498 times)

#### qWord

• Member
• Posts: 1476
• The base type of a type is the type itself
##### Re: Math10 library
« Reply #15 on: July 09, 2012, 05:17:12 AM »
I can't follow you: integers <> floating point values
As said, not all values can be exact represented.
e.g.: 1E-1 = 0.1001100110011001…
Such values must be rounded while conversion.
(see IEEE 754)
MREAL macros - when you need floating point arithmetic while assembling!

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #16 on: July 09, 2012, 06:19:23 AM »
Yes, nothing new.

I dont want to show 18 digits when i am sure the last
can be another digit

When i said
«we cannot convert 18 digits ASCII»

it means what i said before: we are not sure about
the last digit. So, i should not show it. Is this.
Or "It's that simple!" ;)

#### jj2007

• Member
• Posts: 11550
• Assembler is fun ;-)
##### Re: Math10 library
« Reply #17 on: July 09, 2012, 07:03:20 AM »
It's 18 or 19 digits. What does your routine show when you push the exact PI with fldpi?

t1   REAL10 1.2345678901234567890123
t2   REAL10 1.2345678901234567890123e-99
t3   REAL10 1.2345678901234567890123e+99
t4   REAL10 9.8765432109876543210987
t5   REAL10 9.8765432109876543210987e-99
t6   REAL10 9.8765432109876543210987e+99

Init
fldpi
DefNum 19  ; numbers starting with low digits
Print Str\$("\n%f", t1)
Print Str\$("\n%f", t2)
Print Str\$("\n%f", t3)
DefNum 18  ; numbers starting with high digits
Print Str\$("\n%f", t4)
Print Str\$("\n%f", t5)
Print Str\$("\n%f", t6)
Inkey Str\$("\nPI=%f", ST(0))
Exit
end start

1.234567890123456789
1.234567890123456789e-99
1.234567890123456789e+99
9.87654321098765432
9.87654321098765432e-99
9.87654321098765432e+99
PI=3.14159265358979324

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #18 on: July 09, 2012, 07:40:53 AM »
Jochen,
Quote
It's 18 or 19 digits. What does your routine show when you push the exact PI with fldpi?

It is not the case ! The case is this: we type an ASCII string
like this:

string  db "-123456789012345678.987654321098765432E4914", 0

with 18 INTEGER digits and we want to convert it to real10.
Here is the problem. Use TestMath10 and see it

Your routine uses FPU to convert ASCII to real10 ?

#### jj2007

• Member
• Posts: 11550
• Assembler is fun ;-)
##### Re: Math10 library
« Reply #19 on: July 09, 2012, 08:23:23 AM »
Sorry, Rui, where is the problem? Again, it is either 18 or 19 digits...

.data?
MyR10   REAL10 ?

Init
MovVal MyR10, Chr\$("-123456789012345678.901234e82")
DefNum 19
Print Str\$("The value is %f\n", MyR10)
MovVal MyR10, Chr\$("-987654.32109876543210e-104")
DefNum 18
Print Str\$("The value is %f\n", MyR10)
Inkey
Exit
end start

The value is -1.234567890123456789e+99
The value is -9.87654321098765432e-99

And yes, of course MB uses the FPU. The CRT routines are more limited in precision, and the same holds true for anything that uses XMM regs. Note that I cut off the max exponent at ca. -150...+150 for code size reasons, but that shouldn't have any impact on the number of digits correctly displayed.

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #20 on: July 09, 2012, 10:04:09 AM »
Sorry Jochen you are talking your method (i never see it )
not the way i am doind it.
If you use FBLD (Load BCD data from memory) you cannot get
that results. If you get can you prove it ?

#### jj2007

• Member
• Posts: 11550
• Assembler is fun ;-)
##### Re: Math10 library
« Reply #21 on: July 09, 2012, 03:24:24 PM »
We are both using Real10 precision, which implies a qword size mantissa. Can you show a fbld mem->fstp MyReal10 sequence where the 18th digit is wrong?

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #22 on: July 09, 2012, 07:33:19 PM »
We are both using Real10 precision, which implies a qword size mantissa. Can you show a fbld mem->fstp MyReal10 sequence where the 18th digit is wrong?
I will try, but please Jochen sit down,
i have some other things to do in this days

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #23 on: July 09, 2012, 07:46:08 PM »
some values may be able to yield 19 digits
however, it is somewhat senseless
if you are making a spreadsheet or something, you would retain the 10-byte float for precision, even if you display the decimal value in the GUI cell
i.e., the decimal string should never be used as an intermediate value

here are a couple of examples of what the 19th digit might mean in the real world

the USS Titanic displaced ~46,000 tons
.0000000000000000001 x 46,000 tons = 0.00000000422464 grams   :P
a typical drop of water would probably weigh more than .01 grams

the circumference of the earth at the equator is ~24,901.55 miles
.0000000000000000001 x 24,901.55 miles = 0.0000000001577762208 inches

it may be better to yield 18 digits for all the values
it makes it simpler to format columns of text

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #24 on: July 09, 2012, 08:09:02 PM »
when you convert a finite number of bits from binary to decimal, you can always yield a precise evaluation
however, when converting in the other direction, you may not be able to

a couple years ago, Jochen and i were discussing this stuff
i wrote this little program to evaluate extended reals to full precision for comparison
you can see what the difference is in +/- 1 LSB
i intended to expand on it - but never did - lol
Code: [Select]
`3FFE_A2F9836E_4E441529: +0.63661977236758134302213943400694518004456767812371253967285156253FFE_A2F9836E_4E44152A: +0.6366197723675813430763495426312204017449403181672096252441406253FFE_A2F9836E_4E44152B: +0.6366197723675813431305596512554956234453129582107067108154296875`

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #25 on: July 10, 2012, 04:38:53 AM »
Some things must be clear

1. The problem is not to know if a number has 17, 18, 19 or even 50 digits.
We want to decide how many significant digits we want to use

2. The problem is not if the method a) do this or the method b) do that

3. I am using one method to convert ascii to real10 and real10 to ascii
I find out one example and i got one particular result that i think
it is not correct. So, the procedure has a bug or we may not get best
results with that method. Is this what i think. This is the problem.

When we type those strings it seems that the procedure convert it
correctly from ascii to real10. We convert ascii to packed BCD format
and we use FBLD to load. This is the method used.

After we have the real10 number we use a procedure to convert it
to ascii again. We convert it to an integer of N digits and
we use fbstp to get the packed digits etc. This is the method used.
May be i have a bug here. Till now it seems to be correct and i
dont know what is wrong. If i do some "wrong things" i get the inverse
case (with the same example)

b) is incorrect
and a) is correct
Jochen,
I think you doesnt use those methods and i dont know if you
tried to use it before

Dave,
Find out other examples with significant digits not 0 and 0 etc.  ;)
Consider scientific notation

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #26 on: July 10, 2012, 05:08:00 AM »
it sounds to me as if you are experiencing ROSD (round-off stress disorder)

when you convert between bases and limit the number of digits, you are likely to experience some rounding errors

i might enter a number 1.234567890123456789 as a decimal string
when i convert that to a float, it will (should be) converted to the nearest binary value with m bits
now, i take that binary value and convert it back to decimal and round to 19 digits
(that is the nearest decimal value with n digits)
i am not going to be surprised if it comes out as 1.234567890123456788
there have been 2 opportunities for round-off errors to occur

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #27 on: July 10, 2012, 05:15:30 AM »
Dave,
Find out other examples with significant digits not 0 and 0 etc.  ;)
Consider scientific notation
the extra digits are only for demonstration - scientific notation is not what i was going for   :P

here are some examples you might like better - lol

the USS Titanic displaced ~46,000 tons
1.000000000000000001 x 46,000 tons = 46,000 tons + 0.0000000422464 grams
a typical drop of water would probably weigh more than .01 grams

the circumference of the earth at the equator is ~24,901.55 miles
1.000000000000000001 x 24,901.55 miles = 24,901.55 miles + 0.000000001577762208 inches

8)

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #28 on: July 10, 2012, 06:51:22 AM »
Dave
I think that sometimes you give answers that doesnt help
and you dont understand it or you dont want to understand it.
And it seems that when you dont like what we say you try to
insult us like saying "you are experiencing ROSD".
Sorry, Dave ...

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #29 on: July 10, 2012, 08:48:04 AM »
sorry you are offended, Rui - that was not my intention
i tried to explain why you may not get what you expect when converting between bases
i thought that was being helpful