Math10 library

[RuiLoureiro]

Dave,

    Ok, i misunderstood, no problems

    I used your eval in this case
   
Here are the results:

    In the case A) we use 18 integer digits
                   (not used in Math10 only in TestMath10)
    and
    in the case b) we use 17 integer digits only

case A)

String number X= "-1 234567890123456 78.987654321098765432E4914" <- INPUT
eval (by Dave) =    -1 234567890123456 54 7828547635625754786...   -> output
TestMath10    X=    -1.234567890123456 6E+4931                                -> output 17 digits
case b)

String number X= "-1 234567890123456 7.9 87654321098765432E4915" <- INPUT
eval (by Dave) =    -1 234567890123456 7 9 452373414429199518...   -> output
TestMath10    X=    -1.234567890123456 8E+4931                                -> output 17 digits
    My conclusion is the same as before

    As we can see, in case A), when we convert 18 interger digits from
    ASCII to real10 "eval (by Dave)" gives the same 16 digits plus "54"
    instead of "78".

    So, it seems that it makes no sense to convert 18 integer digits
    (using the method i am using) when i cannot convert it to ascii
    again correctly.   
    Does it make sense to show a lot of digits when the most significants
    are not there ? My opinion is no.

[dedndave]

Code Select Expand

case A)

String number X= "-1 234567890123456 78.987654321098765432E4914" <- INPUT
eval (by Dave) =    -1 234567890123456 54 7828547635625754786...   -> output
TestMath10    X=    -1.234567890123456 6E+4931                                -> output 17 digits

case b)

String number X= "-1 234567890123456 7.9 87654321098765432E4915" <- INPUT
eval (by Dave) =    -1 234567890123456 7 9 452373414429199518...   -> output
TestMath10    X=    -1.234567890123456 8E+4931                                -> output 17 digits

there is a flaw in the logic here, Rui
before that number got from "String number X=" to my eval program, someone's code converted it from a decimal string into a real10 float

i believe that code made some errors that are larger than the normal round-off errors

my eval program accepts real10 floats as input - not decimal strings   :t
i think you will find that all real10's that begin with a "1" can yield 19 signifigant digits

[dedndave]

Code Select Expand

4038_6DA6D2E9_8C3CD37F: +123456789012345677.984375

4038_6DA6D2E9_8C3CD380: +123456789012345678

4038_6DA6D2E9_8C3CD381: +123456789012345678.015625

this shows that a routine that converts 123456789012345678.0 to real10 should yield
4038_6DA6D2E9_8C3CD380h

it also shows that there are 19 usable decimal digits

[jj2007]

i think you will find that all real10's that begin with a "1" can yield 19 signifigant digits

include \masm32\MasmBasic\MasmBasic.inc   ; download
.data
r10_1   REAL10 1234567890.1234567890e90
r10_2   REAL10 2234567890.1234567890e90
r10_3   REAL10 3234567890.1234567890e90
r10_4   REAL10 4234567890.1234567890e90
r10_5   REAL10 5234567890.1234567890e90
r10_6   REAL10 6234567890.1234567890e90
r10_7   REAL10 7234567890.1234567890e90
r10_8   REAL10 8234567890.1234567890e90
r10_9   REAL10 9134567890.1234567890e90
r10_10   REAL10 9234567890.1234567890e90
r10_11   REAL10 -1234567890.1234567890e90
r10_12   REAL10 -2234567890.1234567890e90
r10_13   REAL10 -3234567890.1234567890e90
r10_14   REAL10 -4234567890.1234567890e90
r10_15   REAL10 -5234567890.1234567890e90
r10_16   REAL10 -6234567890.1234567890e90
r10_17   REAL10 -7234567890.1234567890e90
r10_18   REAL10 -8234567890.1234567890e90
r10_19   REAL10 -9134567890.1234567890e90
   Init
   ct=0
   DefNum 19
   REPEAT 19
      ct=ct+1
      @CatStr(<PrintLine Str$(r10_>, %ct, <)>)
   ENDM
   Inkey
   Exit
end start

1.234567890123456789e+99
2.234567890123456789e+99
3.234567890123456789e+99
4.234567890123456789e+99
5.234567890123456788e+99
6.234567890123456789e+99
7.234567890123456789e+99
8.234567890123456789e+99
9.134567890123456789e+99
^v?
-1.234567890123456789e+99
-2.234567890123456789e+99
-3.234567890123456789e+99
-4.234567890123456789e+99
-5.234567890123456788e+99
-6.234567890123456789e+99
-7.234567890123456789e+99
-8.234567890123456789e+99
-9.134567890123456789e+99

The only range that has problems with the 19th digit is indeed the 9.22xxx that marks the QWORD limit. Note that DefNum 19 is an undocumented setting - DefNum -1 or DefNum 99 would set 18 digits, and all numbers above would correctly end with ...679. This is what the FPU can deliver, not less and not more.

[RuiLoureiro]

Dave,
       I never said "eval (by Dave)" is incorrect
       

i believe that code made some errors that are larger than the normal round-off errors

       Dave, run TestMath10 i posted and see the BCD strings to load
       the FPU. Are they correct or not ? The problem is about
       the code or the method ?
       Do you Know a better converter (ascii to real10)  ?

       Meanwhile i found this results:

_r10MASM  real10 -123456789012345678.9 8 7654321098765432 E4914
eval (by Dave)   = -123456789012345678 9 8 4781230850681807

       In this case only the first 20 digits are equal.
       Well i think you will answer: here we have the normal round-off errors.
       And i dont say no. I say this:       
       Does it make sense to show a lot of digits when the most significants
       are not there ? My opinion is no.

[dedndave]

to give you a simple answer - i have not written any better code to convert ASCII to float   
Jochen has a nice routine for that, i think
also - Raymond's FPU lib is likely to have a pretty good routine
Drizz, lingo and others are also likely to have code for this
Drizz writes nice code that is pretty fast   :t
i seem to recall that he has a bignum library on his site

Does it make sense to show a lot of digits when the most significants are not there ?
My opinion is no.

i agree
however, i would try to find a better conversion routine
it only needs to examine the first 20 decimal digits - anything after that won't matter
of course, it also has to be able to parse a variety of floating format strings
simple: 0
not simple: -3.4564564563456e-23
also: +123456.78901234

i would think parsing the string would be harder than the base conversion   :redface:

[qWord]

I'm not sure if I understand this discussion, but the number of significant digits depends on where the values come from and what they do represent. For example, if you measure the temperature with a thermometer, range -5 to 50°C and an error of +-0.5°C, it makes no sense to show more than 3 significant digits.
In fact, you can't know this - the best solution is to let the user make this decision (number of shown digits) ;-D

qWord

[RuiLoureiro]

...
this shows that a routine that converts 123456789012345678.0 to real10 should yield
4038_6DA6D2E9_8C3CD380h

it also shows that there are 19 usable decimal digits

        I get this:

_r10MASM2       real10 123456789012345678.0

                                                4038_6DA6D2E9_8C3CD380h   <- Dave
; 123456789012345678.0  => 4037_DB4DA5D3_1879A700h   <- masm32
; 123456789012345678.0  => 4037_DB4DA5D3_1879A700h   <- my BCD converter

[dedndave]

yes - my mistake there, Rui
i can add another bit of resolution - oops
give me a minute....

[dedndave]

Code Select Expand

4037_DB4DA5D3_1879A6FF: +123456789012345677.9921875

4037_DB4DA5D3_1879A700: +123456789012345678

4037_DB4DA5D3_1879A701: +123456789012345678.0078125

String number X= "-1 234567890123456 78.987654321098765432E4914" <- INPUT

what do you get as a REAL10 for that string ?

[RuiLoureiro]

I'm not sure if I understand this discussion, but the number of significant digits depends on where the values come from and what they do represent. For example, if you measure the temperature with a thermometer, range -5 to 50°C and an error of +-0.5°C, it makes no sense to show more than 3 significant digits.
In fact, you can't know this - the best solution is to let the user make this decision (number of shown digits) ;-D

qWord

qWord, It is open. You understand. At the end we try to learn something !

[jj2007]

There is actually a not so small range (...70h ... 79h) that produces identical 19-digit string representations:

include \masm32\MasmBasic\MasmBasic.inc   ; download

.data
MyR10   REAL10 ?
x2low   dq 0DB4DA5D31879A769h
   dw 4037h
xlow   dq 0DB4DA5D31879A770h
   dw 4037h
xmid   dq 0DB4DA5D31879A775h
   dw 4037h
xhi   dq 0DB4DA5D31879A779h
   dw 4037h
x2hi   dq 0DB4DA5D31879A780h
   dw 4037h

   Init
   DefNum 19   ; inofficial max setting

   PrintLine Str$(REAL10 ptr x2low)
   PrintLine Str$(REAL10 ptr xlow)
   PrintLine Str$(REAL10 ptr xmid)
   PrintLine Str$(REAL10 ptr xhi)
   PrintLine Str$(REAL10 ptr x2hi)

   Inkey
   Exit
end start

123456789012345678.8
123456789012345678.9
123456789012345678.9
123456789012345678.9
123456789012345679.0

Your string produces -1.234567890123456790e+.. - not surprising because ...78.987 is supposed to get rounded upwards. In contrast,
   MovVal MyR10, "-123456789012345678.90123E123"
yields correctly -1.234567890123456789e+..

[RuiLoureiro]

what do you get as a REAL10 for that string ?

_r10MASM1       real10 -123456789012345678.987654321098765432E4914

                7FFB_D484A3B0_44873E49h     <- masm
                7FFB_D484A3B0_44874A03h     <- my BCD converter

[RuiLoureiro]

also - Raymond's FPU lib is likely to have a pretty good routine

Dave, it uses the BCD method     

[RuiLoureiro]

Dave,

however, i would try to find a better conversion routine
it only needs to examine the first 20 decimal digits - anything after that won't matter
of course, it also has to be able to parse a variety of floating format strings
simple: 0
not simple: -3.4564564563456e-23
also: +123456.78901234

i would think parsing the string would be harder than the base conversion   

Well, parsing the string seems to be easy.
            I have one way to do that and it is not hard