Math10 library

[qWord]

I can't follow you: integers <> floating point values
As said, not all values can be exact represented.
e.g.: 1E-1 = 0.1001100110011001...
Such values must be rounded while conversion.
(see IEEE 754)

[RuiLoureiro]

    Yes, nothing new.
     
    I dont want to show 18 digits when i am sure the last
    can be another digit

    When i said   
                 «we cannot convert 18 digits ASCII»
                                 
    it means what i said before: we are not sure about
    the last digit. So, i should not show it. Is this.
    Or "It's that simple!" ;)

[jj2007]

It's 18 or 19 digits. What does your routine show when you push the exact PI with fldpi?

include \masm32\MasmBasic\MasmBasic.inc   ; download
t1   REAL10 1.2345678901234567890123
t2   REAL10 1.2345678901234567890123e-99
t3   REAL10 1.2345678901234567890123e+99
t4   REAL10 9.8765432109876543210987
t5   REAL10 9.8765432109876543210987e-99
t6   REAL10 9.8765432109876543210987e+99

   Init
   fldpi
   DefNum 19  ; numbers starting with low digits
   Print Str$("\n%f", t1)
   Print Str$("\n%f", t2)
   Print Str$("\n%f", t3)
   DefNum 18  ; numbers starting with high digits
   Print Str$("\n%f", t4)
   Print Str$("\n%f", t5)
   Print Str$("\n%f", t6)
   Inkey Str$("\nPI=%f", ST(0))
   Exit
end start

1.234567890123456789
1.234567890123456789e-99
1.234567890123456789e+99
9.87654321098765432
9.87654321098765432e-99
9.87654321098765432e+99
PI=3.14159265358979324

[RuiLoureiro]

Jochen,

It's 18 or 19 digits. What does your routine show when you push the exact PI with fldpi?

            It is not the case ! The case is this: we type an ASCII string
            like this:

    string  db "-123456789012345678.987654321098765432E4914", 0

            with 18 INTEGER digits and we want to convert it to real10.
            Here is the problem. Use TestMath10 and see it

            Your routine uses FPU to convert ASCII to real10 ?

[jj2007]

Sorry, Rui, where is the problem? Again, it is either 18 or 19 digits...

include \masm32\MasmBasic\MasmBasic.inc   ; download

.data?
MyR10   REAL10 ?

   Init
   MovVal MyR10, Chr$("-123456789012345678.901234e82")
   DefNum 19
   Print Str$("The value is %f\n", MyR10)
   MovVal MyR10, Chr$("-987654.32109876543210e-104")
   DefNum 18
   Print Str$("The value is %f\n", MyR10)
   Inkey
   Exit
end start

The value is -1.234567890123456789e+99
The value is -9.87654321098765432e-99

And yes, of course MB uses the FPU. The CRT routines are more limited in precision, and the same holds true for anything that uses XMM regs. Note that I cut off the max exponent at ca. -150...+150 for code size reasons, but that shouldn't have any impact on the number of digits correctly displayed.

[RuiLoureiro]

Sorry Jochen you are talking your method (i never see it )
not the way i am doind it.
If you use FBLD (Load BCD data from memory) you cannot get
that results. If you get can you prove it ?

[jj2007]

We are both using Real10 precision, which implies a qword size mantissa. Can you show a fbld mem->fstp MyReal10 sequence where the 18th digit is wrong?

[RuiLoureiro]

We are both using Real10 precision, which implies a qword size mantissa. Can you show a fbld mem->fstp MyReal10 sequence where the 18th digit is wrong?

I will try, but please Jochen sit down,
               i have some other things to do in this days 

[dedndave]

some values may be able to yield 19 digits
however, it is somewhat senseless
if you are making a spreadsheet or something, you would retain the 10-byte float for precision, even if you display the decimal value in the GUI cell
i.e., the decimal string should never be used as an intermediate value

here are a couple of examples of what the 19th digit might mean in the real world

the USS Titanic displaced ~46,000 tons
.0000000000000000001 x 46,000 tons = 0.00000000422464 grams   :P
a typical drop of water would probably weigh more than .01 grams

the circumference of the earth at the equator is ~24,901.55 miles
.0000000000000000001 x 24,901.55 miles = 0.0000000001577762208 inches

it may be better to yield 18 digits for all the values
it makes it simpler to format columns of text

[dedndave]

when you convert a finite number of bits from binary to decimal, you can always yield a precise evaluation
however, when converting in the other direction, you may not be able to

a couple years ago, Jochen and i were discussing this stuff
i wrote this little program to evaluate extended reals to full precision for comparison
you can see what the difference is in +/- 1 LSB
i intended to expand on it - but never did - lol

Code Select Expand

3FFE_A2F9836E_4E441529: +0.6366197723675813430221394340069451800445676781237125396728515625

3FFE_A2F9836E_4E44152A: +0.636619772367581343076349542631220401744940318167209625244140625

3FFE_A2F9836E_4E44152B: +0.6366197723675813431305596512554956234453129582107067108154296875

[RuiLoureiro]

Some things must be clear

    1. The problem is not to know if a number has 17, 18, 19 or even 50 digits.
       We want to decide how many significant digits we want to use

    2. The problem is not if the method a) do this or the method b) do that
       
    3. I am using one method to convert ascii to real10 and real10 to ascii
       The problem is about this method or it is related with this method.
       I find out one example and i got one particular result that i think
       it is not correct. So, the procedure has a bug or we may not get best
       results with that method. Is this what i think. This is the problem.

       When we type those strings it seems that the procedure convert it
       correctly from ascii to real10. We convert ascii to packed BCD format
       and we use FBLD to load. This is the method used.

       After we have the real10 number we use a procedure to convert it
       to ascii again. We convert it to an integer of N digits and
       we use fbstp to get the packed digits etc. This is the method used.
       May be i have a bug here. Till now it seems to be correct and i
       dont know what is wrong. If i do some "wrong things" i get the inverse
       case (with the same example)

                                   b) is incorrect
                            and a) is correct       
Jochen,
       I think you doesnt use those methods and i dont know if you
       tried to use it before
   
Dave,
       Find out other examples with significant digits not 0 and 0 etc.  ;)
       Consider scientific notation

[dedndave]

it sounds to me as if you are experiencing ROSD (round-off stress disorder)

when you convert between bases and limit the number of digits, you are likely to experience some rounding errors

i might enter a number 1.234567890123456789 as a decimal string
when i convert that to a float, it will (should be) converted to the nearest binary value with m bits
now, i take that binary value and convert it back to decimal and round to 19 digits
(that is the nearest decimal value with n digits)
i am not going to be surprised if it comes out as 1.234567890123456788
there have been 2 opportunities for round-off errors to occur

[dedndave]

Dave,
       Find out other examples with significant digits not 0 and 0 etc.  ;)
       Consider scientific notation

the extra digits are only for demonstration - scientific notation is not what i was going for   :P

here are some examples you might like better - lol

the USS Titanic displaced ~46,000 tons
1.000000000000000001 x 46,000 tons = 46,000 tons + 0.0000000422464 grams
a typical drop of water would probably weigh more than .01 grams

the circumference of the earth at the equator is ~24,901.55 miles
1.000000000000000001 x 24,901.55 miles = 24,901.55 miles + 0.000000001577762208 inches

8)

[RuiLoureiro]

Dave
        I think that sometimes you give answers that doesnt help
        and you dont understand it or you dont want to understand it.
        And it seems that when you dont like what we say you try to
        insult us like saying "you are experiencing ROSD".
        Sorry, Dave ... 

[dedndave]

sorry you are offended, Rui - that was not my intention
i tried to explain why you may not get what you expect when converting between bases
i thought that was being helpful