### Author Topic: FPU Rounding  (Read 2112 times)

#### raymond

• Member
• Posts: 327
##### Re: FPU Rounding
« Reply #30 on: February 13, 2023, 02:21:15 PM »
src1 REAL10 123.45678901234567890
src2 REAL10 123456789012345.678

From a purely mathematical point of view, if both those values must be rounded to the same accuracy, let's use 12 decimals digits as an example, the results should be:
src1  123.456789012   (or in the other standard way: 1.23456789012 x 102)
src2  123456789012000 (or as above 1.23456789012 x 1015)

@ JJ: Can you imagine reporting, as you suggested, the largest number which can be represented with the FPU in REAL10 mode (1.19x104932) with an accuracy of 3 decimal digits?
Whenever you assume something, you risk being wrong half the time.
http://www.ray.masmcode.com/

#### daydreamer

• Member
• Posts: 2395
• my kind of REAL10 Blonde
##### Re: FPU Rounding
« Reply #31 on: February 13, 2023, 07:11:29 PM »
Hutch great

What happens with SendMessage string to gui control with restricted length of string displays can only fit 9.123 out of 9.123456789?
Does it display first or last part of digits?
Related problem is display fusion reactor million kelvin 150 millions kelvin instead of 150123456 kelvin or millions of dollars
Jochen packed SIMD might be useful for statistics with curves and tables
Also a weather map with lots of temperatures on it,from weather forecast sources like satellites, computer simulation etc to no display with no decimals
Raymond I thought infinity is biggest fpu number?

my none asm creations
http://masm32.com/board/index.php?topic=6937.msg74303#msg74303
I am an Invoker
"An Invoker is a mage who specializes in the manipulation of raw and elemental energies."
Like SIMD coding

#### jj2007

• Member
• Posts: 13944
• Assembly is fun ;-)
##### Re: FPU Rounding
« Reply #32 on: February 13, 2023, 08:05:58 PM »
@ JJ: Can you imagine reporting, as you suggested, the largest number which can be represented with the FPU in REAL10 mode (1.19x104932) with an accuracy of 3 decimal digits?

Raymond,

I can surely imagine that, but I didn't suggest that. As we both know, that would be utter nonsense. However, the whole thread deals with "two digits after the dot", and that's what the multiply by 100 method can do.

Achieving a given precision or number of valid digits is an entirely different story:

include \masm32\MasmBasic\MasmBasic.inc   ; *** round to a given number of valid digits ***
src1 REAL10 123.45678901234567890
src2 REAL10 123456789012345.67890
src3 REAL10 123.45678901234567890e-99
src4 REAL10 123456789012345.67890e99
Init
Print Str\$("Result s14=%4f\t", src1), Str\$("s19=%9f  \t", src1), Str\$("s1J=%Jf\n", src1)
Print Str\$("Result s24=%4f\t", src2), Str\$("s29=%9f\t", src2), Str\$("s2J=%Jf\n", src2)
Print Str\$("Result s34=%4f\t", src3), Str\$("s39=%9f\t", src3), Str\$("s3J=%Jf\n", src3)
Print Str\$("Result s44=%4f\t", src4), Str\$("s49=%9f\t", src4), Str\$("s4J=%Jf\n", src4)

EndOfCode

Output:
Code: [Select]
`Result s14=123.5        s19=123.456789          s1J=123.4567890123456789Result s24=1.235e+14    s29=1.23456789e+14      s2J=123456789012345.6789Result s34=1.235e-97    s39=1.23456789e-97      s3J=1.234567890123456789e-97Result s44=1.235e+113   s49=1.23456789e+113     s4J=1.234567890123456789e+113`

#### mineiro

• Member
• Posts: 946
##### Re: FPU Rounding
« Reply #33 on: February 13, 2023, 11:10:30 PM »
Well, using a sliding window!!!
Code: [Select]
`123 23.  3.4   .45...`
When we read a text full of alphabetic digits, do we read the whole text at once or sequentially? Letter by letter, word by word...
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

#### HSE

• Member
• Posts: 2494
• AMD 7-32 / i3 10-64
##### Re: FPU Rounding
« Reply #34 on: February 14, 2023, 12:15:45 AM »
Same but with Print Str\$("Result=%Hf\n", dest), where dest is REAL10: Result=123456789012345.68

The FPU has its limits (and SIMD instructions are worse)

Code: [Select]
`  fld src1  fmul x100  frndint  fld x001  fmul  fstp dest  invoke vc_printf, chr\$("Result =%24f", 13, 10), dest  fld src2  fstp dest  invoke vc_printf, chr\$("Result =%24f", 13, 10), dest`
Code: [Select]
`Result =              123.460000Result =  123456789012345.670000Press any key to continue...`
Close to the limit You have to use same number:
Code: [Select]
`  fld src2  fmul x100  frndint  fdiv x100  fstp dest`because there is no guarantee that internally 0.01 equals 1/100 at full precision.
Equations in Assembly: SmplMath

#### jj2007

• Member
• Posts: 13944
• Assembly is fun ;-)
##### Re: FPU Rounding
« Reply #35 on: February 14, 2023, 12:21:20 AM »
Close to the limit You have to use same number:
Code: [Select]
`  fld src2  fmul x100  frndint  fdiv x100  fstp dest`because there is no guarantee that internally 0.01 equals 1/100 at full precision.

You may not have noted that I used two different multipliers, for a reason:

Beware of rounding errors ;-)

Code: [Select]
`include \masm32\include\masm32rt.inc.dataf1 REAL10 123.45678901234567890.codestart:  fld f1  fmul FP4(100.0)  frndint  fld FP10(0.01)  fmul  int 3 ; check here with your debugger  fstp f1  exitend start`

100.0 is ok as REAL4, but 0.01 requires a REAL10. Btw the correct result is 68, not 67; you are using a REAL8, I suppose.

#### HSE

• Member
• Posts: 2494
• AMD 7-32 / i3 10-64
##### Re: FPU Rounding
« Reply #36 on: February 14, 2023, 12:39:09 AM »
you are using a REAL8, I suppose.

Yes. FPU is not designed to store/retrieve REAL10 numbers, they are for internal use and external checks. If you need more precision than real8 you have to use big numbers libraries.
Equations in Assembly: SmplMath

#### jj2007

• Member
• Posts: 13944
• Assembly is fun ;-)
##### Re: FPU Rounding
« Reply #37 on: February 14, 2023, 12:42:00 AM »
FPU is not designed to store/retrieve REAL10 numbers

That is an interesting theory

#### HSE

• Member
• Posts: 2494
• AMD 7-32 / i3 10-64
##### Re: FPU Rounding
« Reply #38 on: February 14, 2023, 12:57:18 AM »
FPU is not designed to store/retrieve REAL10 numbers

That is an interesting theory

30 years old. At that time I have an emulator, because you was having to buy FPU chip appart, and was so fantastic real8 precision that nobody was thinking in real10.

I never see that explanations again.
Equations in Assembly: SmplMath

#### jj2007

• Member
• Posts: 13944
• Assembly is fun ;-)
##### Re: FPU Rounding
« Reply #39 on: February 14, 2023, 12:59:43 AM »
30 years old

In the meantime, real geniusesTM at Intel and AMD have developed the fld and fstp instructions

#### HSE

• Member
• Posts: 2494
• AMD 7-32 / i3 10-64
##### Re: FPU Rounding
« Reply #40 on: February 14, 2023, 01:58:22 AM »
In the meantime, real geniusesTM at Intel and AMD have developed the fld and fstp instructions

I presume always were there to store/retrieve full internal FPU states, but I don't know.
« Last Edit: February 15, 2023, 04:37:09 AM by HSE »
Equations in Assembly: SmplMath

#### mineiro

• Member
• Posts: 946
##### Re: FPU Rounding
« Reply #41 on: February 15, 2023, 04:04:29 AM »
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/printf-printf-l-wprintf-wprintf-l?view=msvc-170

Quote
Important
Starting in Windows 10 version 2004 (build 19041), the printf family of functions prints exactly representable floating point numbers according to the IEEE 754 rules for rounding. In previous versions of Windows, exactly representable floating point numbers ending in '5' would always round up. IEEE 754 states that they must round to the closest even digit (also known as "Banker's Rounding"). For example, both printf("%1.0f", 1.5) and printf("%1.0f", 2.5) should round to 2. Previously, 1.5 would round to 2 and 2.5 would round to 3. This change only affects exactly representable numbers. For example, 2.35 (which, when represented in memory, is closer to 2.35000000000000008) continues to round up to 2.4. Rounding done by these functions now also respects the floating point rounding mode set by fesetround. Previously, rounding always chose FE_TONEAREST behavior. This change only affects programs built using Visual Studio 2019 version 16.2 and later. To use the legacy floating point rounding behavior, link with legacy_stdio_float_rounding.obj.
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

#### mineiro

• Member
• Posts: 946
##### Re: FPU Rounding
« Reply #42 on: February 15, 2023, 06:33:10 AM »
hello sir JK;

I found an approximation for the first 400 digits of pi on the internet.
After arithmetically encoding it, I got the 181 bytes below:

51 fa 11 a8 43 67 0f d2 82 47 3e 43 e3 5b 92 51
f9 9f 8d cf 79 d9 d9 b5 09 1a 6d 36 01 b1 58 02
...
d4 cf 80 0e 3c

After decoding it, I got the digits of pi again.

I used 64-bit integer registers in the program, but nothing prevents you from updating the program to use 32-bit or 16-bit for this particular purpose.
This encoding/decoding demonstrates that we can simulate an infinite floating point using integers.
It is up to you, if you wish, to update the program to use floating point.

My level in the English language is poor, for a better understanding of the source code I recommend:
Arithmetic Coding revealed : A guided tour from theory to praxis
Eric Bodden, Malte Clasen, Joachim Kneis.
The text above is focused on data compression, however, I eliminated from the source code the part that "compresses" the data, resulting only in the arithmetic encoder.

Follow a win64 and linux64 source code and executables. I used uasm in both O.S.
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

#### JK

• Member
• Posts: 193
##### Re: FPU Rounding
« Reply #43 on: February 21, 2023, 02:48:23 AM »
Ok, after some reading, thinking and coding i now have this one (attached)

Summary: there are two methods you can go:

1.) round to a total # of digits
make mantissa an integer by fmul/fdiv 10^exp-> round mantissa (fdiv, frndint, fmul) + restore

In case of floats beyond 10^18 rounding to decimal places after decimal point doesn´t make sense in FPU, because such numbers require more than 19 digital places in total, the original number is just returned. It doesn´t do any harm, but it also has no effect!!! In scientific format there is always one digit before the decimal point and all the rest after it, you can round these numbers to a total number of digits.

2.) round to a # of places after the decimal point
round mantissa (fmul, frndint, fdiv)

This works works only in integer range (19 digits in total = 10^18), in this range rounding to decimal digits after decimal point is possible respecting the FPU´s internal accuracy of 19 digits, i.e. 1234567890.xxxx... can be rounded to a max of 8 decimal places. In case of "overflow" (digits before decimal point + digits to round to > 18) rounding is done as far as possible, numbers beyond 10^18 are returned unchanged.

Please tell me, if there are problems or errors

JK