### Author Topic: Floating point arithmetic question  (Read 2618 times)

#### Ascended

• Member
• Posts: 332
##### Floating point arithmetic question
« on: April 16, 2018, 08:20:24 PM »
Hi guys,

I am trying to work out how to do a simple floating point subtraction but the result is incorrect.

Code: [Select]
` fild valA fild valB fsub fstp result`
valA is a real4 of 1000.0 and valB is a real4 of 1.0

I was expecting a result of 999.0 but I get -8.34929E+07.

Any advice would be much appreciated

#### Ascended

• Member
• Posts: 332
##### Re: Floating point arithmetic question
« Reply #1 on: April 16, 2018, 08:32:37 PM »
Ah! Worked it out. I should have been using fld not fild

#### daydreamer

• Member
• Posts: 561
• reach for the stars
##### Re: Floating point arithmetic question
« Reply #2 on: April 16, 2018, 08:58:39 PM »
For simple math and sqrt and rsqrt use SSE instead,its easier to make code faster in a loop
movss xmm0,val1
subss xmm0,val2
movss result,xmm0
And when you need speedup,use movaps,subps etc instead
Quote from Flashdance
Nick  :  When you give up your dream, you die.
*wears a flameproof asbestos suit*

#### Ascended

• Member
• Posts: 332
##### Re: Floating point arithmetic question
« Reply #3 on: April 16, 2018, 09:05:22 PM »
Awesome! Thanks for the tip

#### jj2007

• Member
• Posts: 8885
• Assembler is fun ;-)
##### Re: Floating point arithmetic question
« Reply #4 on: April 16, 2018, 09:22:25 PM »
For simple math and sqrt and rsqrt use SSE instead,its easier to make code faster in a loop
movss xmm0,val1
subss xmm0,val2
movss result,xmm0

And if you are not absolutely sure, time it:
Code: [Select]
`Intel(R) Core(TM) i5-2450M CPU @ 2.50GHz (SSE4)87      cycles for 100 * movss59      cycles for 100 * fpu80      cycles for 100 * movss59      cycles for 100 * fpu80      cycles for 100 * movss59      cycles for 100 * fpu84      cycles for 100 * movss59      cycles for 100 * fpu80      cycles for 100 * movss60      cycles for 100 * fpu24      bytes for movss18      bytes for fpu`
movss:
Code: [Select]
` movss xmm0, val1 subss xmm0, val2 movss result, xmm0`
fpu:
Code: [Select]
` fld val1 fsub val2 fstp result`

#### Ascended

• Member
• Posts: 332
##### Re: Floating point arithmetic question
« Reply #5 on: April 16, 2018, 09:33:46 PM »
Yep. Been timing routines heaps today.

Trying to make them as tight as possible as they are performance critical for my project.

Lot's of testing, timing, and learning going on

#### Ascended

• Member
• Posts: 332
##### Re: Floating point arithmetic question
« Reply #6 on: April 16, 2018, 10:22:06 PM »
Having another small issue.

I am trying to calculate the tangent of a value using the fp* commands.

Code: [Select]
` fld VALUE ;// contains 0.523599 fptan fstp RESULT ;// should be 0.57735`
According to ConverterDD (http://masm32.com/board/index.php?topic=1819.0) the value in RESULT is NaN.

ConverterDD has being displaying all results correctly so far.

Am I using fptan correctly?

#### jj2007

• Member
• Posts: 8885
• Assembler is fun ;-)
##### Re: Floating point arithmetic question
« Reply #7 on: April 16, 2018, 10:43:20 PM »
Am I using fptan correctly?
What does the help file say?

Init
fld FP4(0.523599)
fptan
deb 4, "The FPU:", ST(0), ST(1)
EndOfCode

Code: [Select]
`The FPU:ST(0)           1.000000000000000000ST(1)           0.5773506065083982818`

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Floating point arithmetic question
« Reply #8 on: April 16, 2018, 11:45:22 PM »
Having another small issue.

I am trying to calculate the tangent of a value using the fp* commands.

Quote
fld VALUE         ;// contains 0.523599
fptan
fstp  st            ; remove 1.0 from st(0)<<-- See Simply FPU by Raymond
fstp RESULT      ;// should be 0.57735

According to ConverterDD (http://masm32.com/board/index.php?topic=1819.0) the value in RESULT is NaN.

ConverterDD has being displaying all results correctly so far.

Am I using fptan correctly?    <<<--- NO, it seems you want tan(VALUE)=0.5773505683919327
Hi
Could you post this simple example (the asm file) ?
« Last Edit: April 17, 2018, 08:41:15 AM by RuiLoureiro »

#### Ascended

• Member
• Posts: 332
##### Re: Floating point arithmetic question
« Reply #9 on: April 17, 2018, 10:05:25 AM »
What does the help file say?

Help file says I am. The result says otherwise.

@RuiLoureiro - Will do

#### Ascended

• Member
• Posts: 332
##### Re: Floating point arithmetic question
« Reply #10 on: April 17, 2018, 10:12:49 AM »
This works though.

Code: [Select]
` fld number ;// contains 0.523599 fptan fstp result ;// contains 1 why? fstp result ;// should be 0.57735`
Why is it that two fstp calls are required?

#### raymond

• Member
• Posts: 218
##### Re: Floating point arithmetic question
« Reply #11 on: April 17, 2018, 10:27:52 AM »
Quote
Why is it that two fstp calls are required?

If you follow up on reading the recommended FPU tutorial (more specifically the part relating to the fptan instruction at http://www.ray.masmcode.com/tutorial/fpuchap10.htm#fptan) you would get your answer to your questions, including the last.

It may also give you a hint to explain one of your previous comment:
Quote
According to ConverterDD (http://masm32.com/board/index.php?topic=1819.0) the value in RESULT is NaN.
That may possibly be due to valid data already being in the ST(7) and/or the ST(0) register when attempting to compute the tangent. Otherwise
a value of "1" should have been returned by ConverterDD.
Whenever you assume something, you risk being wrong half the time.
http://www.ray.masmcode.com/

#### Ascended

• Member
• Posts: 332
##### Re: Floating point arithmetic question
« Reply #12 on: April 17, 2018, 02:03:21 PM »
Thanks for the link to the tutorial. Clears it up well

#### RuiLoureiro

• Member
• Posts: 819
##### Re: Floating point arithmetic question
« Reply #13 on: April 17, 2018, 10:41:22 PM »
This works though.

Quote
fld number         ;// contains 0.523599
fptan
fstp resultX         ;// contains 1 why?  <<<- because fptan gives 2 results, not 1
fstp result         ;// should be 0.57735

Why is it that two fstp calls are required?
What do you get for the new resultX variable ? Try to see. Print it. It should be 1.0 as Raymond said.
fstp  st should be used in this case, it is faster. It removes the current st(0) when we dont need it.

« Last Edit: April 18, 2018, 04:08:42 AM by RuiLoureiro »

#### jj2007

• Member
• Posts: 8885
• Assembler is fun ;-)
##### Re: Floating point arithmetic question
« Reply #14 on: April 17, 2018, 10:55:42 PM »
fstp  st should be used, it is faster. It removes the current st(0) when we dont need it.

Btw there is also fincstp, which at first sight has the same effect. But try a simple fldpi afterwards, and you'll see the difference. Olly has a section with the FPU regs, you must scroll down a little bit the upper right pane to see it.