### Author Topic: Math10 library  (Read 33075 times)

#### jj2007

• Member
• Posts: 11527
• Assembler is fun ;-)
##### Re: Math10 library
« Reply #45 on: July 14, 2012, 03:47:58 AM »
Quote
also - Raymond's FPU lib is likely to have a pretty good routine
Dave, it uses the BCD method

Test yourself:
MovVal MyR10, "-123456789012345678.90123E123"
PrintLine Str\$(MyR10)
invoke FpuAtoFL, Chr\$("-123456789012345678.90123E123"), offset MyR10, DEST_MEM
PrintLine Str\$(MyR10)

-1.234567890123456789e+140
-1.234567890123456780e+140

It seems one digit gets lost in translation.

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #46 on: July 14, 2012, 04:06:35 AM »
Jochen,
«It seems one digit gets lost in translation»

yes it lose digits !
I tested some cases

#### jj2007

• Member
• Posts: 11527
• Assembler is fun ;-)
##### Re: Math10 library
« Reply #47 on: July 14, 2012, 04:15:32 AM »
Indeed. For example,

invoke FpuAtoFL, chr\$("-123456789012345678.99E123"), 0, DEST_FPU

It could have to do with limits of the BCD conversion (the MasmBasic Val algo uses qwords, not BCD numbers).

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #48 on: July 14, 2012, 04:57:05 AM »
yuppers - from Ray's FpuAtoFL...
Code: [Select]
`integer:      cmp   al,"."      jnz   @F      lea   edi,bcdstri      call  load_integer      lodsb      lea   edi,bcdstrf+8      mov   cl,18              ;<----      and   bh,4      jmp   decimals`personally, i would accept up to 21 user-entered decimal digits - and round the last one to 20 digits
that should always yield the closest binary float

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #49 on: July 14, 2012, 06:26:15 AM »
Dave,

If i get only 3 decimals from the string below
the BCD converter gives the same as MASM

_r10MASM1       real10 -123456789012345678.987654321098765432E4914

7FFB_D484 A3B0_4487 4A03h     <- masm
7FFB_D484 A3B0_4487 4A03h     <- my BCD converter

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #50 on: July 14, 2012, 01:04:30 PM »
i was just reading the Sun Numerical Computation Guide
(it was on my desktop, and i clicked on it by accident - lol)

it is telling me that x86 extended reals can yield 18 to 21 signifigant decimal digits
that's news to me   :redface:

#### jj2007

• Member
• Posts: 11527
• Assembler is fun ;-)
##### Re: Math10 library
« Reply #51 on: July 14, 2012, 04:14:01 PM »
21 sounds unlikely - no miracles under the Sun
In reply #41 above, you can see that 3 LSBs cover the range of 1.234567890123456789. With some imagination, you could get something like
1.2345678901234567892
1.2345678901234567895
1.2345678901234567898
when analysing the LSB, but that is still 20 digits, not 21. And remember that we got a theoretical limit of 18 for numbers that start with 9.22xxx and higher. Statistically speaking, you could neglect these 7% of all numbers, but I wouldn't trust a library that suggests this strategy

#### dedndave

• Member
• Posts: 8829
• Still using Abacus 2.0
##### Re: Math10 library
« Reply #52 on: July 14, 2012, 06:24:25 PM »
i haven't tried it
but, maybe if the number is something like 1.00000000000000000001, it might work   ::)
you'd have to try a few adjacent values to see
1.00000000000000000001
1.00000000000000000002
1.00000000000000000003

#### MichaelW

• Global Moderator
• Member
• Posts: 1209
##### Re: Math10 library
« Reply #53 on: July 14, 2012, 06:46:01 PM »
i haven't tried it
but, maybe if the number is something like 1.00000000000000000001, it might work   ::)
you'd have to try a few adjacent values to see
1.00000000000000000001
1.00000000000000000002
1.00000000000000000003

It seems to me that the adjacent values of interest are the representable values.
Well Microsoft, here’s another nice mess you’ve gotten us into.

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #54 on: July 16, 2012, 03:55:56 AM »
I wrote a new procedure to convert ascii (19 digits) to real10 format

Here are some results (first word is the exponent than qword)
_r10MASM_  -> converted by masm
_r10RCL_     ->         "        by my proc StrToRelCnv
In some cases the last byte is not equal

Code: [Select]
`_r10MASM_1   real10 123456789012345678.0     4037_DB4D A5D3_1879 A700h  _r10RCL_1                                    4037_DB4D A5D3_1879 A700h_r10MASM_2  real10 1.23456789012345678E17 4037_DB4D A5D3_1879 A700h     _r10RCL_2                                 4037_DB4D A5D3_1879 A700h_r10MASM_3   real10 123456789012345678.1     4037_DB4D A5D3_1879 A70Dh_r10RCL_3                                    4037_DB4D A5D3_1879 A70Eh_r10MASM_4   real10 123456789012345678.2     4037_DB4D A5D3_1879 A71Ah_r10RCL_4                                    4037_DB4D A5D3_1879 A71Bh_r10MASM_5   real10 123456789012345678.3     4037_DB4D A5D3_1879 A726h_r10RCL_5                                    4037_DB4D A5D3_1879 A728h_r10MASM_6   real10 123456789012345678.4     4037_DB4D A5D3_1879 A733h_r10RCL_6                                    4037_DB4D A5D3_1879 A734h_r10MASM_7   real10 123456789012345678.5     4037_DB4D A5D3_1879 A740h_r10RCL_7                                    4037_DB4D A5D3_1879 A741h_r10MASM_8   real10 123456789012345678.6     4037_DB4D A5D3_1879 A74Dh_r10RCL_8                                    4037_DB4D A5D3_1879 A74Eh_r10MASM_9   real10 123456789012345678.7     4037_DB4D A5D3_1879 A75Ah_r10RCL_9                                    4037_DB4D A5D3_1879 A75Bh_r10MASM_10  real10 123456789012345678.8     4037_DB4D A5D3_1879 A766h_r10RCL_10                                   4037_DB4D A5D3_1879 A768h_r10MASM_11 real10 123456789012345678.9     4037_DB4D A5D3_1879 A773h_r10RCL_11                                  4037_DB4D A5D3_1879 A774h_r10MASM_12 real10 123456789012345678.98    4037_DB4D A5D3_1879 A77Dh_r10RCL_12                                  4037_DB4D A5D3_1879 A77Eh_r10MASM_13   real10 -123456789.0E320         4440_EE43 C218_CA57 BB04h_r10RCL_13      db "-123456789.0E320", 0      4440_EE43 C218_CA57 BB62h_r10MASM_14   real10 -123456789.12345E320     4440_EE43 C21C_C9A0 1C66h_r10RCL_14      db "-123456789.12345E320",0   4440_EE43 C21C_C9A0 1A1Ah  **_r10MASM_15   real10 1.0E320                  4426_8184 2F29_F2CC E376h_r10RCL_15      db "1.0E320",0                4426_8184 2F29_F2CC E3A9h;-----------------------------------------------_r10MASM_30   real10 -0.1                     3FFB_CCCC CCCC_CCCC CCCDh_r10RCL_30      db "-0.1",0                   3FFB_CCCC CCCC_CCCC CCCEh_r10MASM_31   real10 -0.01                    3FF8_A3D7 0A3D_70A3 D70Ah_r10RCL_31      db "-0.01",0                  3FF8_A3D7 0A3D_70A3 D70Bh_r10MASM_32   real10 -0.001                   3FF5_8312 6E97_8D4F DF3Bh_r10RCL_32      db "-0.001",0                 3FF5_8312 6E97_8D4F DF3Fh_r10MASM_33   real10 -0.0001                  3FF1_D1B7 1758_E219 652Ch_r10RCL_33      db "-0.0001",0                3FF1_D1B7 1758_E219 652Eh_r10MASM_34   real10 -0.00001                 3FEE_A7C5 AC47_1B47 8423h_r10RCL_34      db "-0.00001",0               3FEE_A7C5 AC47_1B47 8422h_r10MASM_35   real10 -0.000001                3FEB_8637 BD05_AF6C 69B6h_r10RCL_35      db "-0.000001",0              3FEB_8637 BD05_AF6C 69BEh_r10MASM_36   real10 -0.0000001               3FE7_D6BF 94D5_E57A 42BCh_r10RCL_36      db "-0.0000001",0             3FE7_D6BF 94D5_E57A 42C6h_r10MASM_37   real10 -0.00000001              3FE4_ABCC 7711_8461 CEFDh_r10RCL_37      db "-0.00000001",0            3FE4_ABCC 7711_8461 CF01h  **_r10MASM_38   real10 -0.000000001             3FE1_8970 5F41_36B4 A597h_r10RCL_38      db "-0.000000001",0           3FE1_8970 5F41_36B4 A598h_r10MASM_39   real10 -0.0000000001            3FDD_DBE6 FECE_BDED D5BFh_r10RCL_39      db "-0.0000000001",0          3FDD_DBE6 FECE_BDED D5BCh_r10MASM_40   real10 -0.00000000001           3FDA_AFEB FF0B_CB24 AAFFh_r10RCL_40      db "-0.00000000001",0         3FDA_AFEB FF0B_CB24 AAFAh_r10MASM_41   real10 -0.000000000001          3FD7_8CBC CC09_6F50 88CCh_r10RCL_41      db "-0.000000000001",0        3FD7_8CBC CC09_6F50 88DDh_r10MASM_42   real10 -0.0000000000001         3FD3_E12E 1342_4BB4 0E13h_r10RCL_42      db "-0.0000000000001",0       3FD3_E12E 1342_4BB4 0E2Bh..._r10MASM_47   real10 -0.000000000000000001    3FC3_9392 EE8E_921D 5D07h_r10RCL_47      db "-0.000000000000000001",0  3FC3_9392 EE8E_921D 5D09h_r10MASM_50   real10 -12345678901234567.987654321098765432E4915_r10RCL_50      db "-12345678901234567.987654321098765432E4915", 0                             _r10MASM_50 ->     7FFB_D484 A3B0_4487 4A71h                              _r10rcl_50  ->    7FFB_D484 A3B0_4487 3331h_r10MASM_51   real10 -0.1234567890123456789     3FFB_FCD6 E9BA_37B2 F8E2h_r10RCL_51      db "-0.1234567890123456789", 0  3FFB_FCD6 E9BA_37B2 F8E1h_r10RCL_52      real10 -1.2345678912345678934E-4930                                                  0006_92E1 63E1_EAE4 BDF5h_r10RCL_52      db "-1.2345678912345678934E-4930", 0                                                 0006_92E1 63E1_EAE4 BDEFh`
« Last Edit: July 18, 2012, 01:32:22 AM by RuiLoureiro »

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #55 on: July 17, 2012, 12:04:15 AM »
More data starting with 922.., 93...

Code: [Select]
`_r10MASM_60     real10 9223372036854775807.0    403D_FFFF FFFF_FFFF FFFEh_r10RCL_60      db "9223372036854775807.0",0    403D_FFFF FFFF_FFFF FFFEh      _r10MASM_61     real10 9223372036854775808.0    403E_8000 0000_0000 0000h_r10RCL_61      db "9223372036854775808.0",0    403E_8000 0000_0000 0000h_r10MASM_62     real10 9233372036854775808.0    403E_8023 86F2_6F61 0000h_r10RCL_62      db "9233372036854775808.0", 0   403E_8023 86F2_6F61 0000h_r10MASM_63     real10 9333372036854775808.0    403E_8186 CC6A_CD4B 0000h_r10RCL_63      db "9333372036854775808.0", 0   403E_8186 CC6A_CD4B 0000h        _r10MASM_64     real10 9333372036854775809.9333372036854775808                                                        403E_8186 CC6A_CD4B 0001h_r10RCL_64      db"9333372036854775809.9333372036854775808", 0                                                        403E_8186 CC6A_CD4A FFF9h_r10MASM_65     real10 9333372036854775809.9333372036854775808E4913                                                7FFE_C8D4 9910_070B BD08h_r10RCL_65      db "9333372036854775809.9333372036854775808E4913", 0                                                7FFE_C8D4 9910_070B B1E9h`

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #56 on: July 28, 2012, 05:58:45 AM »
Hi
Here we have a math library of real10 numbers version 0.04

We can test each function using TestMath10

We can find the documentation in the file Math10.txt

We have more 2 procedures to convert real10 to ascci
and ascii to real10 with 19 digits

StrToRealCnv and RealToStrCnv

Test it with TestMath10.exe. Choose Table1 to Table12
and ConvertDn or ConvertUp.
All values are also converted by MASM. Compare them.

Good Luck

Jochen,
Now you can see PI with 19 digits (use TestMath10.exe)
Try it and say something !

#### jj2007

• Member
• Posts: 11527
• Assembler is fun ;-)
##### Re: Math10 library
« Reply #57 on: July 28, 2012, 07:13:39 AM »
Rui,

Congrats, so you made it to full precision :t
How do you handle the 9.22x case? Check for the 18-digits range?

Best, Jochen

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Math10 library
« Reply #58 on: July 28, 2012, 08:01:29 AM »
Jochen,
Thanks ! :t

«How do you handle the 9.22x case? Check for the 18-digits range?»

No. I convert and test for 7FFFFFFFh. If above i move the last digit
from integer part to the decimal part, so i convert only 18 digits.

#### jj2007

• Member
• Posts: 11527
• Assembler is fun ;-)
##### Re: Math10 library
« Reply #59 on: July 28, 2012, 08:14:47 AM »
Yes, that is what I meant. Good job, Rui :t