### Author Topic: Math10 library  (Read 33494 times)

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #30 on: July 13, 2012, 04:17:49 AM »
Dave,

Ok, i misunderstood, no problems

I used your eval in this case

Here are the results:

In the case A) we use 18 integer digits
(not used in Math10 only in TestMath10)
and
in the case b) we use 17 integer digits only

case A)

String number X= "-1 234567890123456 78.987654321098765432E4914" <- INPUT
eval (by Dave) =    -1 234567890123456 54 7828547635625754786...   -> output
TestMath10    X=    -1.234567890123456 6E+4931                                -> output 17 digits

case b)

String number X= "-1 234567890123456 7.9 87654321098765432E4915" <- INPUT
eval (by Dave) =    -1 234567890123456 7 9 452373414429199518...   -> output
TestMath10    X=    -1.234567890123456 8E+4931                                -> output 17 digits

My conclusion is the same as before

As we can see, in case A), when we convert 18 interger digits from
ASCII to real10 "eval (by Dave)" gives the same 16 digits plus "54"

So, it seems that it makes no sense to convert 18 integer digits
(using the method i am using) when i cannot convert it to ascii
again correctly.

Does it make sense to show a lot of digits when the most significants
are not there ? My opinion is no.

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #31 on: July 13, 2012, 04:54:38 AM »
Code: [Select]
`case A)String number X= "-1 234567890123456 78.987654321098765432E4914" <- INPUTeval (by Dave) =    -1 234567890123456 54 7828547635625754786...   -> outputTestMath10    X=    -1.234567890123456 6E+4931                                -> output 17 digitscase b)String number X= "-1 234567890123456 7.9 87654321098765432E4915" <- INPUTeval (by Dave) =    -1 234567890123456 7 9 452373414429199518...   -> outputTestMath10    X=    -1.234567890123456 8E+4931                                -> output 17 digits`
there is a flaw in the logic here, Rui
before that number got from "String number X=" to my eval program, someone's code converted it from a decimal string into a real10 float

i believe that code made some errors that are larger than the normal round-off errors

my eval program accepts real10 floats as input - not decimal strings   :t
i think you will find that all real10's that begin with a "1" can yield 19 signifigant digits

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #32 on: July 13, 2012, 05:36:30 AM »
Code: [Select]
`4038_6DA6D2E9_8C3CD37F: +123456789012345677.9843754038_6DA6D2E9_8C3CD380: +1234567890123456784038_6DA6D2E9_8C3CD381: +123456789012345678.015625`
this shows that a routine that converts 123456789012345678.0 to real10 should yield
4038_6DA6D2E9_8C3CD380h

it also shows that there are 19 usable decimal digits

#### jj2007

• Member
• Posts: 11550
• Assembler is fun ;-)
##### Re: Math10 library
« Reply #33 on: July 13, 2012, 06:17:43 AM »
i think you will find that all real10's that begin with a "1" can yield 19 signifigant digits

.data
r10_1   REAL10 1234567890.1234567890e90
r10_2   REAL10 2234567890.1234567890e90
r10_3   REAL10 3234567890.1234567890e90
r10_4   REAL10 4234567890.1234567890e90
r10_5   REAL10 5234567890.1234567890e90
r10_6   REAL10 6234567890.1234567890e90
r10_7   REAL10 7234567890.1234567890e90
r10_8   REAL10 8234567890.1234567890e90
r10_9   REAL10 9134567890.1234567890e90
r10_10   REAL10 9234567890.1234567890e90
r10_11   REAL10 -1234567890.1234567890e90
r10_12   REAL10 -2234567890.1234567890e90
r10_13   REAL10 -3234567890.1234567890e90
r10_14   REAL10 -4234567890.1234567890e90
r10_15   REAL10 -5234567890.1234567890e90
r10_16   REAL10 -6234567890.1234567890e90
r10_17   REAL10 -7234567890.1234567890e90
r10_18   REAL10 -8234567890.1234567890e90
r10_19   REAL10 -9134567890.1234567890e90
Init
ct=0
DefNum 19
REPEAT 19
ct=ct+1
@CatStr(<PrintLine Str\$(r10_>, %ct, <)>)
ENDM
Inkey
Exit
end start

1.234567890123456789e+99
2.234567890123456789e+99
3.234567890123456789e+99
4.234567890123456789e+99
5.234567890123456788e+99
6.234567890123456789e+99
7.234567890123456789e+99
8.234567890123456789e+99
9.134567890123456789e+99
^v?
-1.234567890123456789e+99
-2.234567890123456789e+99
-3.234567890123456789e+99
-4.234567890123456789e+99
-5.234567890123456788e+99
-6.234567890123456789e+99
-7.234567890123456789e+99
-8.234567890123456789e+99
-9.134567890123456789e+99

The only range that has problems with the 19th digit is indeed the 9.22xxx that marks the QWORD limit. Note that DefNum 19 is an undocumented setting - DefNum -1 or DefNum 99 would set 18 digits, and all numbers above would correctly end with ...679. This is what the FPU can deliver, not less and not more.

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #34 on: July 14, 2012, 01:28:24 AM »
Dave,
I never said "eval (by Dave)" is incorrect

Quote
i believe that code made some errors that are larger than the normal round-off errors

Dave, run TestMath10 i posted and see the BCD strings to load
the FPU. Are they correct or not ? The problem is about
the code or the method ?
Do you Know a better converter (ascii to real10)  ?

Meanwhile i found this results:

_r10MASM  real10 -123456789012345678.9 8 7654321098765432 E4914
eval (by Dave)   = -123456789012345678 9 8 4781230850681807

In this case only the first 20 digits are equal.
Well i think you will answer: here we have the normal round-off errors.
And i dont say no. I say this:

Does it make sense to show a lot of digits when the most significants
are not there ? My opinion is no.

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #35 on: July 14, 2012, 01:53:20 AM »
to give you a simple answer - i have not written any better code to convert ASCII to float
Jochen has a nice routine for that, i think
also - Raymond's FPU lib is likely to have a pretty good routine
Drizz, lingo and others are also likely to have code for this
Drizz writes nice code that is pretty fast   :t
i seem to recall that he has a bignum library on his site

Quote
Does it make sense to show a lot of digits when the most significants are not there ?
My opinion is no.
i agree
however, i would try to find a better conversion routine
it only needs to examine the first 20 decimal digits - anything after that won't matter
of course, it also has to be able to parse a variety of floating format strings
simple: 0
not simple: -3.4564564563456e-23
also: +123456.78901234

i would think parsing the string would be harder than the base conversion   :redface:

#### qWord

• Member
• Posts: 1476
• The base type of a type is the type itself
##### Re: Math10 library
« Reply #36 on: July 14, 2012, 02:14:06 AM »
I'm not sure if I understand this discussion, but the number of significant digits depends on where the values come from and what they do represent. For example, if you measure the temperature with a thermometer, range -5 to 50°C and an error of +-0.5°C, it makes no sense to show more than 3 significant digits.
In fact, you can’t know this - the best solution is to let the user make this decision (number of shown digits) ;-D

qWord
MREAL macros - when you need floating point arithmetic while assembling!

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #37 on: July 14, 2012, 02:17:07 AM »
Quote
...
this shows that a routine that converts 123456789012345678.0 to real10 should yield
4038_6DA6D2E9_8C3CD380h

it also shows that there are 19 usable decimal digits

I get this:

_r10MASM2       real10 123456789012345678.0

4038_6DA6D2E9_8C3CD380h   <- Dave
; 123456789012345678.0  => 4037_DB4DA5D3_1879A700h   <- masm32
; 123456789012345678.0  => 4037_DB4DA5D3_1879A700h   <- my BCD converter

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #38 on: July 14, 2012, 02:22:45 AM »
yes - my mistake there, Rui
i can add another bit of resolution - oops
give me a minute....

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #39 on: July 14, 2012, 02:29:21 AM »
Code: [Select]
`4037_DB4DA5D3_1879A6FF: +123456789012345677.99218754037_DB4DA5D3_1879A700: +1234567890123456784037_DB4DA5D3_1879A701: +123456789012345678.0078125`
Quote
String number X= "-1 234567890123456 78.987654321098765432E4914" <- INPUT
what do you get as a REAL10 for that string ?

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #40 on: July 14, 2012, 02:32:06 AM »
I'm not sure if I understand this discussion, but the number of significant digits depends on where the values come from and what they do represent. For example, if you measure the temperature with a thermometer, range -5 to 50°C and an error of +-0.5°C, it makes no sense to show more than 3 significant digits.
In fact, you can’t know this - the best solution is to let the user make this decision (number of shown digits) ;-D

qWord
qWord, It is open. You understand. At the end we try to learn something !

#### jj2007

• Member
• Posts: 11550
• Assembler is fun ;-)
##### Re: Math10 library
« Reply #41 on: July 14, 2012, 02:50:21 AM »
There is actually a not so small range (...70h ... 79h) that produces identical 19-digit string representations:

.data
MyR10   REAL10 ?
x2low   dq 0DB4DA5D31879A769h
dw 4037h
xlow   dq 0DB4DA5D31879A770h
dw 4037h
xmid   dq 0DB4DA5D31879A775h
dw 4037h
xhi   dq 0DB4DA5D31879A779h
dw 4037h
x2hi   dq 0DB4DA5D31879A780h
dw 4037h

Init
DefNum 19   ; inofficial max setting

PrintLine Str\$(REAL10 ptr x2low)
PrintLine Str\$(REAL10 ptr xlow)
PrintLine Str\$(REAL10 ptr xmid)
PrintLine Str\$(REAL10 ptr xhi)
PrintLine Str\$(REAL10 ptr x2hi)

Inkey
Exit
end start

123456789012345678.8
123456789012345678.9
123456789012345678.9
123456789012345678.9
123456789012345679.0

Your string produces -1.234567890123456790e+.. - not surprising because ...78.987 is supposed to get rounded upwards. In contrast,
MovVal MyR10, "-123456789012345678.90123E123"
yields correctly -1.234567890123456789e+..

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #42 on: July 14, 2012, 02:57:15 AM »
Quote
what do you get as a REAL10 for that string ?

_r10MASM1       real10 -123456789012345678.987654321098765432E4914

7FFB_D484A3B0_44873E49h     <- masm
7FFB_D484A3B0_44874A03h     <- my BCD converter

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #43 on: July 14, 2012, 03:04:16 AM »
Quote
also - Raymond's FPU lib is likely to have a pretty good routine
Dave, it uses the BCD method

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #44 on: July 14, 2012, 03:09:19 AM »
Dave,
Quote
however, i would try to find a better conversion routine
it only needs to examine the first 20 decimal digits - anything after that won't matter
of course, it also has to be able to parse a variety of floating format strings
simple: 0
not simple: -3.4564564563456e-23
also: +123456.78901234

i would think parsing the string would be harder than the base conversion
Well, parsing the string seems to be easy.
I have one way to do that and it is not hard