Hi
Here we have a math library of real10 numbers
We can test each function using TestMath10
We can find the documentation in the file Math10.txt
Good Luck
EDIT : i replaced version 0.03 by the last version 0.04
We have more 2 procedures to convert real10 to ascci
and ascii to real10 with 19 digits
StrToRealCnv and RealToStrCnv
Quote from: RuiLoureiro on June 29, 2012, 12:47:24 AM
We can test each function using TestMath10
We can find the documentation in the file Math10.txt
We can not find the source :icon_confused:
i would say....
DLL ? - at that size, why not just make a static LIB :P
It's the version 0.01 !
I need to write more functions to that .dll
and it is not completed tested.
Questions:
Are those procedures useful ?
Is there some errors ?
it's a nice collection of functions, Rui :t
Quote from: dedndave on June 30, 2012, 05:31:15 AM
it's a nice collection of functions, Rui :t
Only you to say something,
DaveThank you Dave :t
We need a procedure to convert real10 to string
It will be in the next
Quote from: RuiLoureiro on June 30, 2012, 05:49:00 AM
We need a procedure to convert real10 to string
REAL10 is a QWORD plus a dot or a dot and an exponent. You can use the known Qword2Ascii routines and insert a dot somewhere.
include \masm32\MasmBasic\MasmBasic.inc ; download (http://masm32.com/board/index.php?topic=94.0)
Init
Inkey Str$("PI=%Jf", PI)
Exit
end start
Output:
PI=3.141592653589793238The problem is to know where to insert the dot...
Jochen,
These are the results that i get using FPU ;)
e = 2.71828182845904509 18 digits
PI= 3.14159265358979324 18 "
Output: PI=3.141592653589793238 <<-- Jochen
some values can have 20 usable digits
most can have only 19 usable digits
i forget the numbers, but if the signifigant digits start with something like 18765.... or less, 20 usable digits are available
you can find the leading digits through a little experimentation
http://www.masmforum.com/board/index.php?topic=11781.msg89823#msg89823 (http://www.masmforum.com/board/index.php?topic=11781.msg89823#msg89823)
The mantissa of a REAL10 is 64 bit aka QWORD. So the number of usable digits depends on what a QWORD can deliver...
but - it is scaled by the exponent
when converting binary to decimal, the number of usable digits shifts a little
Hi
Here we have a math library of real10 numbers
version 0.02 We can test each function using TestMath10
We can find the
documentation in the file
Math10.txt In this version we have the functions:
FncXaddY, FncXsubY,FncXmulY,FncXdivY
and FncXcmpY
We have also 4 procedures to convert real10
to ascci.
In TestMath10.exe, to change the number of digits
type a number 6 up to 17 and press compute
Example:
Quote
.data
_decimal dd 5
_digits dd 17
X dt -3.1345
; dont forget this DWORD before XBuffer
dd 0 ;<--- to length
XBuffer db 27 dup (?) ; we need 27 here
.code
invoke CnvRelToRclT, addr X, addr XBuffer
jc error
invoke CnvRelToRclZ, addr X, addr XBuffer
jc error
invoke CnvRelToRcl, addr X, addr XBuffer, _decimal
jc error
invoke CnvRelToRclD, addr X, addr XBuffer, _digits
jc error
error:
Good Luck
Hi
2 questions or 1 question 1 problem
1.
Here we have a math library of real10 numbers version 0.03
We can test each function using TestMath10
We can find the documentation in the file Math10.txt
We have more functions
and we have a procedure to convert ASCII to REAL10:
invoke CnvRclToRel, pRcl, pReal
jc error
In TestMath10.exe, if we want to show only N digits
type N in the input box and press compute
2.
When we run TestMath10 and press PREVIOUS we find
the function (it is not used in Math10 library
and when we run it we can see the BCD strings)
a) RclToRelCnv(-123456789012345678.987654321098765432E4914)
If we press PREVIOUS again we find
the function (it is in the Math10 library)
b) CnvRclToRel(-12345678901234567.987654321098765432E4915)
In the case a) we use 18 integer digits and
in the case b) we use 17 integer digits only
-123456789012345678 <---- a)
-12345678901234567 <---- b)
It means that RclToRelCnv tries to convert 18 integer digits
and CnvRclToRel only 17 integer digits
In both cases press COMPUTE and see the results with 17 digits
-1 23456789012345678.9 <- input in RclToRelCnv
a) X= -1.2345678901234566E+4931
-1 2345678901234567.9 <- input in CnvRclToRel
b) X= -1.2345678901234568E+4931
As we can see, the result of b) is correct
and the result of a) is INcorrect
I dont know why, but my first conclusion is this:
«we cannot convert 18 digits ASCII»
Have you any other opinion ?
Please answer this post
Good Luck
and thanks
As you surely know, not all decimal numbers can be exactly represented as binary numbers. So the answer is:
"We can maybe convert N digits ASCII"
The shown results look correct (rounded).
qWord,
Thanks for reply !
Quote
The shown results look correct (rounded).
Yes, rounded . ( we are talking of FPU only)
It seems to me that i made nothing wrong
and i show all details. We can change the numbers
and compute to see what we get
Quote
So the answer is:
"We can maybe convert N digits ASCII"
I dont like the "maybe" when i know 1 case
where it gives incorrect results. I think
there are a lot of cases like that and
i want to avoid a lot of them if possible
So i prefer my conclusion.
See the next
Quote
As you surely know, not all decimal numbers can be exactly represented
as binary numbers.
From Ray tutorial:
INTEGER DATA TYPES
QWORD range ±(2^63-1) or ±9223372036854775807
I tried to think that if the limit is this 20 digit number
if i use FBLD (Load BCD data from memory) with only 18 digits
it should give the correct result
Meanwhile, we can read the following, in the file
FpuAtoFL in the fpulib. Raymond wrote this:
«
; ... The number of digits (excluding all leading 0's and trailing
; decimal 0's) must not exceed 18.
»
and that told me we should have not problems in using 18 digits.
But im wrong.
This is why i want to show the problem in TestMath10
Math10 library uses CnvRclToRel to convert 17 integer digits only.
I can't follow you: integers <> floating point values
As said, not all values can be exact represented.
e.g.: 1E-1 = 0.1001100110011001...
Such values must be rounded while conversion.
(see IEEE 754)
Yes, nothing new.
I dont want to show 18 digits when i am sure the last
can be another digit
When i said
«we cannot convert 18 digits ASCII»
it means what i said before: we are not sure about
the last digit. So, i should not show it. Is this.
Or "It's that simple!" ;)
It's 18 or 19 digits. What does your routine show when you push the exact PI with fldpi?
include \masm32\MasmBasic\MasmBasic.inc ; download (http://masm32.com/board/index.php?topic=94.0)
t1 REAL10 1.2345678901234567890123
t2 REAL10 1.2345678901234567890123e-99
t3 REAL10 1.2345678901234567890123e+99
t4 REAL10 9.8765432109876543210987
t5 REAL10 9.8765432109876543210987e-99
t6 REAL10 9.8765432109876543210987e+99
Init
fldpi
DefNum 19 ; numbers starting with low digits
Print Str$("\n%f", t1)
Print Str$("\n%f", t2)
Print Str$("\n%f", t3)
DefNum 18 ; numbers starting with high digits
Print Str$("\n%f", t4)
Print Str$("\n%f", t5)
Print Str$("\n%f", t6)
Inkey Str$("\nPI=%f", ST(0))
Exit
end start
1.234567890123456789
1.234567890123456789e-99
1.234567890123456789e+99
9.87654321098765432
9.87654321098765432e-99
9.87654321098765432e+99
PI=3.14159265358979324
Jochen,
Quote
It's 18 or 19 digits. What does your routine show when you push the exact PI with fldpi?
It is not the case ! The case is this: we type an ASCII string
like this:
string db "-123456789012345678.987654321098765432E4914", 0
with 18 INTEGER digits and we want to convert it to real10.
Here is the problem. Use TestMath10 and see it
Your routine uses FPU to convert ASCII to real10 ?
Sorry, Rui, where is the problem? Again, it is either 18 or 19 digits...
include \masm32\MasmBasic\MasmBasic.inc ; download (http://masm32.com/board/index.php?topic=94.0)
.data?
MyR10 REAL10 ?
Init
MovVal MyR10, Chr$("-123456789012345678.901234e82")
DefNum 19
Print Str$("The value is %f\n", MyR10)
MovVal MyR10, Chr$("-987654.32109876543210e-104")
DefNum 18
Print Str$("The value is %f\n", MyR10)
Inkey
Exit
end start
The value is -1.234567890123456789e+99
The value is -9.87654321098765432e-99
And yes, of course MB uses the FPU. The CRT routines are more limited in precision, and the same holds true for anything that uses XMM regs. Note that I cut off the max exponent at ca. -150...+150 for code size reasons, but that shouldn't have any impact on the number of digits correctly displayed.
Sorry Jochen you are talking your method (i never see it )
not the way i am doind it.
If you use FBLD (Load BCD data from memory) you cannot get
that results. If you get can you prove it ?
We are both using Real10 precision, which implies a qword size mantissa. Can you show a fbld mem->fstp MyReal10 sequence where the 18th digit is wrong?
Quote from: jj2007 on July 09, 2012, 03:24:24 PM
We are both using Real10 precision, which implies a qword size mantissa. Can you show a fbld mem->fstp MyReal10 sequence where the 18th digit is wrong?
I will try, but please Jochen sit down,
i have some other things to do in this days :biggrin:
some values may be able to yield 19 digits
however, it is somewhat senseless
if you are making a spreadsheet or something, you would retain the 10-byte float for precision, even if you display the decimal value in the GUI cell
i.e., the decimal string should never be used as an intermediate value
here are a couple of examples of what the 19th digit might mean in the real world
the USS Titanic displaced ~46,000 tons
.0000000000000000001 x 46,000 tons = 0.00000000422464 grams :P
a typical drop of water would probably weigh more than .01 grams
the circumference of the earth at the equator is ~24,901.55 miles
.0000000000000000001 x 24,901.55 miles = 0.0000000001577762208 inches
it may be better to yield 18 digits for all the values
it makes it simpler to format columns of text
when you convert a finite number of bits from binary to decimal, you can always yield a precise evaluation
however, when converting in the other direction, you may not be able to
a couple years ago, Jochen and i were discussing this stuff
i wrote this little program to evaluate extended reals to full precision for comparison
you can see what the difference is in +/- 1 LSB
i intended to expand on it - but never did - lol
3FFE_A2F9836E_4E441529: +0.6366197723675813430221394340069451800445676781237125396728515625
3FFE_A2F9836E_4E44152A: +0.636619772367581343076349542631220401744940318167209625244140625
3FFE_A2F9836E_4E44152B: +0.6366197723675813431305596512554956234453129582107067108154296875
Some things must be clear
1. The problem is not to know if a number has 17, 18, 19 or even 50 digits.
We want to decide how many significant digits we want to use
2. The problem is not if the method a) do this or the method b) do that
3. I am using one method to convert ascii to real10 and real10 to ascii
The problem is about this method or it is related with this method.
I find out one example and i got one particular result that i think
it is not correct. So, the procedure has a bug or we may not get best
results with that method. Is this what i think. This is the problem.
When we type those strings it seems that the procedure convert it
correctly from ascii to real10. We convert ascii to packed BCD format
and we use FBLD to load. This is the method used.
After we have the real10 number we use a procedure to convert it
to ascii again. We convert it to an integer of N digits and
we use fbstp to get the packed digits etc. This is the method used.
May be i have a bug here. Till now it seems to be correct and i
dont know what is wrong. If i do some "wrong things" i get the inverse
case (with the same example)
b) is incorrect
and a) is correct
Jochen,
I think you doesnt use those methods and i dont know if you
tried to use it before
Dave,
Find out other examples with significant digits not 0 and 0 etc. ;)
Consider scientific notation
it sounds to me as if you are experiencing ROSD (round-off stress disorder)
when you convert between bases and limit the number of digits, you are likely to experience some rounding errors
i might enter a number 1.234567890123456789 as a decimal string
when i convert that to a float, it will (should be) converted to the nearest binary value with m bits
now, i take that binary value and convert it back to decimal and round to 19 digits
(that is the nearest decimal value with n digits)
i am not going to be surprised if it comes out as 1.234567890123456788
there have been 2 opportunities for round-off errors to occur
Quote from: RuiLoureiro on July 10, 2012, 04:38:53 AM
Dave,
Find out other examples with significant digits not 0 and 0 etc. ;)
Consider scientific notation
the extra digits are only for demonstration - scientific notation is not what i was going for :P
here are some examples you might like better - lol
the USS Titanic displaced ~46,000 tons
1.000000000000000001 x 46,000 tons = 46,000 tons + 0.0000000422464 grams
a typical drop of water would probably weigh more than .01 grams
the circumference of the earth at the equator is ~24,901.55 miles
1.000000000000000001 x 24,901.55 miles = 24,901.55 miles + 0.000000001577762208 inches
8)
Dave
I think that sometimes you give answers that doesnt help
and you dont understand it or you dont want to understand it.
And it seems that when you dont like what we say you try to
insult us like saying "you are experiencing ROSD".
Sorry, Dave ...
sorry you are offended, Rui - that was not my intention
i tried to explain why you may not get what you expect when converting between bases
i thought that was being helpful
Dave,
Ok, i misunderstood, no problems
I used your eval in this case
Here are the results:
In the case A) we use 18 integer digits
(not used in Math10 only in TestMath10)
and
in the case b) we use 17 integer digits only
case A)
String number X= "-1 234567890123456 78.987654321098765432E4914" <- INPUT
eval (by Dave) = -1 234567890123456 54 7828547635625754786... -> output
TestMath10 X= -1.234567890123456 6E+4931 -> output 17 digits
case b)
String number X= "-1 234567890123456 7.9 87654321098765432E4915" <- INPUT
eval (by Dave) = -1 234567890123456 7 9 452373414429199518... -> output
TestMath10 X= -1.234567890123456 8E+4931 -> output 17 digits
My conclusion is the same as before
As we can see, in case A), when we convert 18 interger digits from
ASCII to real10 "eval (by Dave)" gives the same 16 digits plus "54"
instead of "78".
So, it seems that it makes no sense to convert 18 integer digits
(using the method i am using) when i cannot convert it to ascii
again correctly.
Does it make sense to show a lot of digits when the most significants
are not there ? My opinion is no.
case A)
String number X= "-1 234567890123456 78.987654321098765432E4914" <- INPUT
eval (by Dave) = -1 234567890123456 54 7828547635625754786... -> output
TestMath10 X= -1.234567890123456 6E+4931 -> output 17 digits
case b)
String number X= "-1 234567890123456 7.9 87654321098765432E4915" <- INPUT
eval (by Dave) = -1 234567890123456 7 9 452373414429199518... -> output
TestMath10 X= -1.234567890123456 8E+4931 -> output 17 digits
there is a flaw in the logic here, Rui
before that number got from "String number X=" to my eval program, someone's code converted it from a decimal string into a real10 float
i believe that code made some errors that are larger than the normal round-off errors
my eval program accepts real10 floats as input - not decimal strings :t
i think you will find that all real10's that begin with a "1" can yield 19 signifigant digits
4038_6DA6D2E9_8C3CD37F: +123456789012345677.984375
4038_6DA6D2E9_8C3CD380: +123456789012345678
4038_6DA6D2E9_8C3CD381: +123456789012345678.015625
this shows that a routine that converts 123456789012345678.0 to real10 should yield
4038_6DA6D2E9_8C3CD380h
it also shows that there are 19 usable decimal digits
Quote from: dedndave on July 13, 2012, 04:54:38 AMi think you will find that all real10's that begin with a "1" can yield 19 signifigant digits
include \masm32\MasmBasic\MasmBasic.inc ; download (http://masm32.com/board/index.php?topic=94.0)
.data
r10_1 REAL10 1234567890.1234567890e90
r10_2 REAL10 2234567890.1234567890e90
r10_3 REAL10 3234567890.1234567890e90
r10_4 REAL10 4234567890.1234567890e90
r10_5 REAL10 5234567890.1234567890e90
r10_6 REAL10 6234567890.1234567890e90
r10_7 REAL10 7234567890.1234567890e90
r10_8 REAL10 8234567890.1234567890e90
r10_9 REAL10 9134567890.1234567890e90
r10_10 REAL10 9234567890.1234567890e90
r10_11 REAL10 -1234567890.1234567890e90
r10_12 REAL10 -2234567890.1234567890e90
r10_13 REAL10 -3234567890.1234567890e90
r10_14 REAL10 -4234567890.1234567890e90
r10_15 REAL10 -5234567890.1234567890e90
r10_16 REAL10 -6234567890.1234567890e90
r10_17 REAL10 -7234567890.1234567890e90
r10_18 REAL10 -8234567890.1234567890e90
r10_19 REAL10 -9134567890.1234567890e90
Init
ct=0
DefNum 19
REPEAT 19
ct=ct+1
@CatStr(<PrintLine Str$(r10_>, %ct, <)>)
ENDM
Inkey
Exit
end start
1.234567890123456789e+99
2.234567890123456789e+99
3.234567890123456789e+99
4.234567890123456789e+99
5.234567890123456788e+99
6.234567890123456789e+99
7.234567890123456789e+99
8.234567890123456789e+99
9.134567890123456789e+99
^v?
-1.234567890123456789e+99
-2.234567890123456789e+99
-3.234567890123456789e+99
-4.234567890123456789e+99
-5.234567890123456788e+99
-6.234567890123456789e+99
-7.234567890123456789e+99
-8.234567890123456789e+99
-9.134567890123456789e+99The only range that has problems with the 19th digit is indeed the 9.22xxx that marks the QWORD limit. Note that DefNum 19 is an undocumented setting - DefNum -1 or DefNum 99 would set 18 digits, and all numbers above would correctly end with ...679. This is what the FPU can deliver, not less and not more.
Dave,
I never said "eval (by Dave)" is incorrect
Quote
i believe that code made some errors that are larger than the normal round-off errors
Dave, run TestMath10 i posted and see the BCD strings to load
the FPU. Are they correct or not ? The problem is about
the code or the method ?
Do you Know a better converter (ascii to real10) ?
Meanwhile i found this results:
_r10MASM real10 -123456789012345678.9 8 7654321098765432 E4914
eval (by Dave) = -123456789012345678 9 8 4781230850681807
In this case only the first 20 digits are equal.
Well i think you will answer: here we have the normal round-off errors.
And i dont say no. I say this: Does it make sense to show a lot of digits when the most significants
are not there ? My opinion is no.
to give you a simple answer - i have not written any better code to convert ASCII to float :biggrin:
Jochen has a nice routine for that, i think
also - Raymond's FPU lib is likely to have a pretty good routine
Drizz, lingo and others are also likely to have code for this
Drizz writes nice code that is pretty fast :t
i seem to recall that he has a bignum library on his site
QuoteDoes it make sense to show a lot of digits when the most significants are not there ?
My opinion is no.
i agree
however, i would try to find a better conversion routine
it only needs to examine the first 20 decimal digits - anything after that won't matter
of course, it also has to be able to parse a variety of floating format strings
simple: 0
not simple: -3.4564564563456e-23
also: +123456.78901234
i would think parsing the string would be harder than the base conversion :redface:
I'm not sure if I understand this discussion, but the number of significant digits depends on where the values come from and what they do represent. For example, if you measure the temperature with a thermometer, range -5 to 50°C and an error of +-0.5°C, it makes no sense to show more than 3 significant digits.
In fact, you can't know this - the best solution is to let the user make this decision (number of shown digits) ;-D
qWord
Quote
...
this shows that a routine that converts 123456789012345678.0 to real10 should yield
4038_6DA6D2E9_8C3CD380h
it also shows that there are 19 usable decimal digits
I get this:
_r10MASM2 real10 123456789012345678.0
4038_6DA6D2E9_8C3CD380h <- Dave
; 123456789012345678.0 => 4037_DB4DA5D3_1879A700h <- masm32
; 123456789012345678.0 => 4037_DB4DA5D3_1879A700h <- my BCD converter
yes - my mistake there, Rui
i can add another bit of resolution - oops
give me a minute....
4037_DB4DA5D3_1879A6FF: +123456789012345677.9921875
4037_DB4DA5D3_1879A700: +123456789012345678
4037_DB4DA5D3_1879A701: +123456789012345678.0078125QuoteString number X= "-1 234567890123456 78.987654321098765432E4914" <- INPUT
what do you get as a REAL10 for that string ?
Quote from: qWord on July 14, 2012, 02:14:06 AM
I'm not sure if I understand this discussion, but the number of significant digits depends on where the values come from and what they do represent. For example, if you measure the temperature with a thermometer, range -5 to 50°C and an error of +-0.5°C, it makes no sense to show more than 3 significant digits.
In fact, you can't know this - the best solution is to let the user make this decision (number of shown digits) ;-D
qWord
qWord, It is open. You understand. At the end we try to learn something !
There is actually a not so small range (...70h ... 79h) that produces identical 19-digit string representations:
include \masm32\MasmBasic\MasmBasic.inc ; download (http://masm32.com/board/index.php?topic=94.0)
.data
MyR10 REAL10 ?
x2low dq 0DB4DA5D31879A769h
dw 4037h
xlow dq 0DB4DA5D31879A770h
dw 4037h
xmid dq 0DB4DA5D31879A775h
dw 4037h
xhi dq 0DB4DA5D31879A779h
dw 4037h
x2hi dq 0DB4DA5D31879A780h
dw 4037h
Init
DefNum 19 ; inofficial max setting
PrintLine Str$(REAL10 ptr x2low)
PrintLine Str$(REAL10 ptr xlow)
PrintLine Str$(REAL10 ptr xmid)
PrintLine Str$(REAL10 ptr xhi)
PrintLine Str$(REAL10 ptr x2hi)
Inkey
Exit
end start
123456789012345678.8
123456789012345678.9
123456789012345678.9
123456789012345678.9
123456789012345679.0
Your string produces -1.234567890123456790e+.. - not surprising because ...78.987 is supposed to get rounded upwards. In contrast,
MovVal MyR10, "-123456789012345678.90123E123"
yields correctly -1.234567890123456789e+..
Quote
what do you get as a REAL10 for that string ?
_r10MASM1 real10 -123456789012345678.987654321098765432E4914
7FFB_D484A3B0_44873E49h <- masm
7FFB_D484A3B0_44874A03h <- my BCD converter
Quote
also - Raymond's FPU lib is likely to have a pretty good routine
Dave, it uses the BCD method
Dave,
Quote
however, i would try to find a better conversion routine
it only needs to examine the first 20 decimal digits - anything after that won't matter
of course, it also has to be able to parse a variety of floating format strings
simple: 0
not simple: -3.4564564563456e-23
also: +123456.78901234
i would think parsing the string would be harder than the base conversion
Well, parsing the string seems to be easy.
I have one way to do that and it is not hard
Quote from: RuiLoureiro on July 14, 2012, 03:04:16 AM
Quote
also - Raymond's FPU lib is likely to have a pretty good routine
Dave, it uses the BCD method
Test yourself:
MovVal MyR10, "-123456789012345678.90123E123"
PrintLine Str$(MyR10)
invoke FpuAtoFL, Chr$("-123456789012345678.90123E123"), offset MyR10, DEST_MEM
PrintLine Str$(MyR10)
-1.23456789012345678
9e+140
-1.23456789012345678
0e+140
It seems one digit gets lost in translation.
Jochen,
«It seems one digit gets lost in translation»
yes it lose digits !
I tested some cases
Indeed. For example,
invoke FpuAtoFL, chr$("-123456789012345678.99E123"), 0, DEST_FPU
yields -1.234567890123456781e+140 instead of ...79
It could have to do with limits of the BCD conversion (the MasmBasic Val algo uses qwords, not BCD numbers).
yuppers - from Ray's FpuAtoFL...
integer:
cmp al,"."
jnz @F
lea edi,bcdstri
call load_integer
lodsb
lea edi,bcdstrf+8
mov cl,18 ;<----
and bh,4
jmp decimals
personally, i would accept up to 21 user-entered decimal digits - and round the last one to 20 digits
that should always yield the closest binary float
Dave,
If i get only 3 decimals from the string below
the BCD converter gives the same as MASM
_r10MASM1 real10 -123456789012345678.987654321098765432E4914
7FFB_D484 A3B0_4487 4A03h <- masm
7FFB_D484 A3B0_4487 4A03h <- my BCD converter
i was just reading the Sun Numerical Computation Guide
(it was on my desktop, and i clicked on it by accident - lol)
it is telling me that x86 extended reals can yield 18 to 21 signifigant decimal digits
that's news to me :redface:
21 sounds unlikely - no miracles under the Sun :biggrin:
In reply #41 (http://masm32.com/board/index.php?topic=378.msg3175#msg3175) above, you can see that 3 LSBs cover the range of 1.234567890123456789. With some imagination, you could get something like
1.2345678901234567892
1.2345678901234567895
1.2345678901234567898
when analysing the LSB, but that is still 20 digits, not 21. And remember that we got a theoretical limit of 18 for numbers that start with 9.22xxx and higher. Statistically speaking, you could neglect these 7% of all numbers, but I wouldn't trust a library that suggests this strategy :greensml:
i haven't tried it
but, maybe if the number is something like 1.00000000000000000001, it might work ::)
you'd have to try a few adjacent values to see
1.00000000000000000001
1.00000000000000000002
1.00000000000000000003
Quote from: dedndave on July 14, 2012, 06:24:25 PM
i haven't tried it
but, maybe if the number is something like 1.00000000000000000001, it might work ::)
you'd have to try a few adjacent values to see
1.00000000000000000001
1.00000000000000000002
1.00000000000000000003
It seems to me that the adjacent values of interest are the
representable values.
I wrote a new procedure to convert ascii (19 digits) to real10 format
Here are some results (first word is the exponent than qword)
_r10MASM_ -> converted by masm
_r10RCL_ -> " by my proc StrToRelCnv
In some cases the last byte is not equal
_r10MASM_1 real10 123456789012345678.0 4037_DB4D A5D3_1879 A700h
_r10RCL_1 4037_DB4D A5D3_1879 A700h
_r10MASM_2 real10 1.23456789012345678E17 4037_DB4D A5D3_1879 A700h
_r10RCL_2 4037_DB4D A5D3_1879 A700h
_r10MASM_3 real10 123456789012345678.1 4037_DB4D A5D3_1879 A70Dh
_r10RCL_3 4037_DB4D A5D3_1879 A70Eh
_r10MASM_4 real10 123456789012345678.2 4037_DB4D A5D3_1879 A71Ah
_r10RCL_4 4037_DB4D A5D3_1879 A71Bh
_r10MASM_5 real10 123456789012345678.3 4037_DB4D A5D3_1879 A726h
_r10RCL_5 4037_DB4D A5D3_1879 A728h
_r10MASM_6 real10 123456789012345678.4 4037_DB4D A5D3_1879 A733h
_r10RCL_6 4037_DB4D A5D3_1879 A734h
_r10MASM_7 real10 123456789012345678.5 4037_DB4D A5D3_1879 A740h
_r10RCL_7 4037_DB4D A5D3_1879 A741h
_r10MASM_8 real10 123456789012345678.6 4037_DB4D A5D3_1879 A74Dh
_r10RCL_8 4037_DB4D A5D3_1879 A74Eh
_r10MASM_9 real10 123456789012345678.7 4037_DB4D A5D3_1879 A75Ah
_r10RCL_9 4037_DB4D A5D3_1879 A75Bh
_r10MASM_10 real10 123456789012345678.8 4037_DB4D A5D3_1879 A766h
_r10RCL_10 4037_DB4D A5D3_1879 A768h
_r10MASM_11 real10 123456789012345678.9 4037_DB4D A5D3_1879 A773h
_r10RCL_11 4037_DB4D A5D3_1879 A774h
_r10MASM_12 real10 123456789012345678.98 4037_DB4D A5D3_1879 A77Dh
_r10RCL_12 4037_DB4D A5D3_1879 A77Eh
_r10MASM_13 real10 -123456789.0E320 4440_EE43 C218_CA57 BB04h
_r10RCL_13 db "-123456789.0E320", 0 4440_EE43 C218_CA57 BB62h
_r10MASM_14 real10 -123456789.12345E320 4440_EE43 C21C_C9A0 1C66h
_r10RCL_14 db "-123456789.12345E320",0 4440_EE43 C21C_C9A0 1A1Ah **
_r10MASM_15 real10 1.0E320 4426_8184 2F29_F2CC E376h
_r10RCL_15 db "1.0E320",0 4426_8184 2F29_F2CC E3A9h
;-----------------------------------------------
_r10MASM_30 real10 -0.1 3FFB_CCCC CCCC_CCCC CCCDh
_r10RCL_30 db "-0.1",0 3FFB_CCCC CCCC_CCCC CCCEh
_r10MASM_31 real10 -0.01 3FF8_A3D7 0A3D_70A3 D70Ah
_r10RCL_31 db "-0.01",0 3FF8_A3D7 0A3D_70A3 D70Bh
_r10MASM_32 real10 -0.001 3FF5_8312 6E97_8D4F DF3Bh
_r10RCL_32 db "-0.001",0 3FF5_8312 6E97_8D4F DF3Fh
_r10MASM_33 real10 -0.0001 3FF1_D1B7 1758_E219 652Ch
_r10RCL_33 db "-0.0001",0 3FF1_D1B7 1758_E219 652Eh
_r10MASM_34 real10 -0.00001 3FEE_A7C5 AC47_1B47 8423h
_r10RCL_34 db "-0.00001",0 3FEE_A7C5 AC47_1B47 8422h
_r10MASM_35 real10 -0.000001 3FEB_8637 BD05_AF6C 69B6h
_r10RCL_35 db "-0.000001",0 3FEB_8637 BD05_AF6C 69BEh
_r10MASM_36 real10 -0.0000001 3FE7_D6BF 94D5_E57A 42BCh
_r10RCL_36 db "-0.0000001",0 3FE7_D6BF 94D5_E57A 42C6h
_r10MASM_37 real10 -0.00000001 3FE4_ABCC 7711_8461 CEFDh
_r10RCL_37 db "-0.00000001",0 3FE4_ABCC 7711_8461 CF01h **
_r10MASM_38 real10 -0.000000001 3FE1_8970 5F41_36B4 A597h
_r10RCL_38 db "-0.000000001",0 3FE1_8970 5F41_36B4 A598h
_r10MASM_39 real10 -0.0000000001 3FDD_DBE6 FECE_BDED D5BFh
_r10RCL_39 db "-0.0000000001",0 3FDD_DBE6 FECE_BDED D5BCh
_r10MASM_40 real10 -0.00000000001 3FDA_AFEB FF0B_CB24 AAFFh
_r10RCL_40 db "-0.00000000001",0 3FDA_AFEB FF0B_CB24 AAFAh
_r10MASM_41 real10 -0.000000000001 3FD7_8CBC CC09_6F50 88CCh
_r10RCL_41 db "-0.000000000001",0 3FD7_8CBC CC09_6F50 88DDh
_r10MASM_42 real10 -0.0000000000001 3FD3_E12E 1342_4BB4 0E13h
_r10RCL_42 db "-0.0000000000001",0 3FD3_E12E 1342_4BB4 0E2Bh
...
_r10MASM_47 real10 -0.000000000000000001 3FC3_9392 EE8E_921D 5D07h
_r10RCL_47 db "-0.000000000000000001",0 3FC3_9392 EE8E_921D 5D09h
_r10MASM_50 real10 -12345678901234567.987654321098765432E4915
_r10RCL_50 db "-12345678901234567.987654321098765432E4915", 0
_r10MASM_50 -> 7FFB_D484 A3B0_4487 4A71h
_r10rcl_50 -> 7FFB_D484 A3B0_4487 3331h
_r10MASM_51 real10 -0.1234567890123456789 3FFB_FCD6 E9BA_37B2 F8E2h
_r10RCL_51 db "-0.1234567890123456789", 0 3FFB_FCD6 E9BA_37B2 F8E1h
_r10RCL_52 real10 -1.2345678912345678934E-4930
0006_92E1 63E1_EAE4 BDF5h
_r10RCL_52 db "-1.2345678912345678934E-4930", 0
0006_92E1 63E1_EAE4 BDEFh
More data starting with 922.., 93...
_r10MASM_60 real10 9223372036854775807.0 403D_FFFF FFFF_FFFF FFFEh
_r10RCL_60 db "9223372036854775807.0",0 403D_FFFF FFFF_FFFF FFFEh
_r10MASM_61 real10 9223372036854775808.0 403E_8000 0000_0000 0000h
_r10RCL_61 db "9223372036854775808.0",0 403E_8000 0000_0000 0000h
_r10MASM_62 real10 9233372036854775808.0 403E_8023 86F2_6F61 0000h
_r10RCL_62 db "9233372036854775808.0", 0 403E_8023 86F2_6F61 0000h
_r10MASM_63 real10 9333372036854775808.0 403E_8186 CC6A_CD4B 0000h
_r10RCL_63 db "9333372036854775808.0", 0 403E_8186 CC6A_CD4B 0000h
_r10MASM_64 real10 9333372036854775809.9333372036854775808
403E_8186 CC6A_CD4B 0001h
_r10RCL_64 db"9333372036854775809.9333372036854775808", 0
403E_8186 CC6A_CD4A FFF9h
_r10MASM_65 real10 9333372036854775809.9333372036854775808E4913
7FFE_C8D4 9910_070B BD08h
_r10RCL_65 db "9333372036854775809.9333372036854775808E4913", 0
7FFE_C8D4 9910_070B B1E9h
Hi
Here we have a math library of real10 numbers version 0.04
We can test each function using TestMath10
We can find the documentation in the file Math10.txt
We have more 2 procedures to convert real10 to ascci
and ascii to real10 with 19 digits
StrToRealCnv and RealToStrCnv
Test it with TestMath10.exe. Choose Table1 to Table12
and ConvertDn or ConvertUp.
All values are also converted by MASM. Compare them.
Good Luck
:biggrin:
Jochen,
Now you can see PI with 19 digits (use TestMath10.exe)
Try it and say something !
Rui,
Congrats, so you made it to full precision :t
How do you handle the 9.22x case? Check for the 18-digits range?
Best, Jochen
Jochen,
Thanks ! :t
«How do you handle the 9.22x case? Check for the 18-digits range?»
No. I convert and test for 7FFFFFFFh. If above i move the last digit
from integer part to the decimal part, so i convert only 18 digits.
Yes, that is what I meant. Good job, Rui :t
remember
we can use the functions in this Math10 library
Only to say
Hello all :t
:icon14:
Hello! :icon14:
Olá, Rui!
Hi Rui! :t