FPU Rounding

[JK]

How would i round a floating point value in the FPU to a specific number of decimal places using the standard FPU rounding mode?
... shouldn´t be too hard to do, i guess, but i don´t get it.

Thanks

JK

[hutch--]

I cheat, I truncate the display results but leave the FP number unmodified.

[HSE]

multiplication, rounding, division

[mineiro]

x*0,1
x*1/100

x*10
x*100

[jj2007]

Beware of rounding errors ;-)

Code: [Select]

include \masm32\include\masm32rt.inc
.data
f1 REAL10 123.45678901234567890
.code
start:
  fld f1
  fmul FP4(100.0)
  frndint
  fld FP10(0.01)
  fmul
  int 3 ; check here with your debugger
  fstp f1
  exit
end start

[HSE]

Beware of rounding errors ;-)

  Perfect.

The default FPU rounding mode is round to nearest. You can change that.

[jj2007]

Beware of rounding errors ;-)

  Perfect.

The default FPU rounding mode is round to nearest. You can change that.

Code: [Select]

start:
  finit
Default? The finit instruction changes the rounding mode from near 53 to near 64.

[daydreamer]

I cheat, I truncate the display results but leave the FP number unmodified.

thanks
never thought of that simple ,invoke fptoascii and search for "." and mov zero termination into the string+ number of decimals

 

[HSE]

Default? The finit instruction changes the rounding mode from near 53 to near 64.

  What I can say?

[hutch--]

 

Magnus,


; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

 NOSTACKFRAME

 truncate proc
  ; -----------------------------------------------------
  ; truncate the number of digits after the decimal point
  ; -----------------------------------------------------
  ; rcx = string address
  ; rdx = decimal place count

    sub rcx, 1
  @@:
    add rcx, 1
    cmp BYTE PTR [rcx], 0           ; exit on zero if no period
    je bye
    cmp BYTE PTR [rcx], "."         ; scan until the decimal point
    jne @B

    test rdx, rdx                   ; if rdx not 0
    jnz nxt                         ; bypass "0" settings
    mov BYTE PTR [rcx], "."         ; write decimal point
    mov BYTE PTR [rcx+1], "0"       ; a trailing 0
    mov BYTE PTR [rcx+2], 0         ; terminate the string
    ret                             ; return to caller

  nxt:
    add rdx, 1                      ; correct for period

  @@:
    add rcx, 1
    cmp BYTE PTR [rcx], 0           ; scan until rdx count
    je bye
    sub rdx, 1
    jnz @B

    mov BYTE PTR [rcx], 0           ; terminate string on rdx count
  bye:

    ret

 truncate endp

 STACKFRAME

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

[mineiro]

Code: [Select]

.data
align 16
f1 REAL8 123.45678901234567890
align 16
f2 real8 100.0
align 16
f3 REAL8 0.01

.code
movsd xmm0,f1
invoke printf,CStr("%f",10),xmm0
movsd xmm0,f1
cvtsd2si rcx,xmm0
invoke printf,CStr("%d",10),rcx
movsd xmm0,f1
cvtsd2si rcx,xmm0
mulsd xmm0,f2
invoke printf,CStr("%f",10),xmm0
movsd xmm0,f1
cvtsd2si rcx,xmm0
mulsd xmm0,f2
cvtsd2si rax,xmm0
invoke printf,CStr("cvtsd2si: %d",10),rax
movsd xmm0,f1
cvtsd2si rcx,xmm0
mulsd xmm0,f2
cvttsd2si rax,xmm0
invoke printf,CStr("cvttsd2si: %d",10),rax
movsd xmm0,f1
cvtsd2si rcx,xmm0
mulsd xmm0,f2
mulsd xmm0,f3
invoke printf,CStr("%f",10),xmm0
movsd xmm0,f1
cvtsd2si rcx,xmm0
mulsd xmm0,f2
mulsd xmm0,f3
cvtsd2si rbx,xmm0
invoke printf,CStr("%d",10),rbx
ret


Code: [Select]

123.456789
123
12345.678901
cvtsd2si: 12346
cvttsd2si: 12345
123.456789
123


[HSE]

Code: [Select]

    movsd xmm0,f1
    mulsd xmm0,f2
    cvtsd2si rcx,xmm0
    cvtsi2sd xmm0, rcx
    mulsd xmm0,f3


[mineiro]

Code: [Select]

.data
align 16
f1 REAL8 123.45678901234567890
one dq 1
hundred dq 100
_1_100 dq 0 ;1/100
temp dq 0

.code
movsd xmm0,f1           ;123.45678901234567890
cvtsi2sd xmm1,one       ;1
cvtsi2sd xmm2,hundred   ;100
divsd xmm1,xmm2         ;1/100
movsd _1_100,xmm1       
mulsd xmm0,xmm2         ;12345.6789...
cvttsd2si rcx,xmm0      ;12345
movsd temp,xmm0
invoke printf,CStr("%d",10),rcx
movsd xmm0,temp         ;12345.6789...
mulsd xmm0,_1_100       ;1/100=0.01
cvttsd2si rcx,xmm0      ;123
invoke printf,CStr("%d",10),rcx
ret


Code: [Select]

12345
123


[mineiro]

Advice 0-> 1/10 in decimal base can't fit exactly in binary base, precision lost. (1÷1010) == 0,0001100... (24 (float) or 53 (double) bits)
f real4 0.1  == 0,00011001100110011001100
d real8 0.1 == 0,000110011001100110011001100110011001100110011001100
Converting:
f real4 0.1  == 1.10011001100110011001100 * 10^−100 (2^-4)
d real8 0.1 == 1.100110011001100110011001100110011001100110011001100 * 10^-100 (2^-4)
Rounding:
f real4 0.1  == 1.10011001100110011001100"1" * 10^−100 (2^-4)
d real8 0.1 == 1.100110011001100110011001100110011001100110011001100"1" * 10^-100 (2^-4)
after rounding, that "1" (right most binary digit) will be rounded to the left side.
f real4 0.1  == 1.10011001100110011001101 * 10^−100 (2^-4)
d real8 0.1 == 1.100110011001100110011001100110011001100110011001101 * 10^-100 (2^-4)

Advice 1-> don't mix real4 with real8 while comparing.

Code: [Select]

.data
f real4 0.1     ;float
d real8 0.1     ;double

.code
movss xmm0,f
cvtss2sd xmm0,xmm0  ;convert float to double, so printf will work fine
invoke printf,CStr("%.15f",10),xmm0
movsd xmm0,d
invoke printf,CStr("%.15f",10),xmm0

movss xmm0,f
movsd xmm1,d
cvtss2sd xmm0,xmm0  ;float to double
ucomisd xmm0,xmm1   ;cmp xmm0,xmm1
je equal
ja bigger
jb lower
ret
equal:
invoke printf,CStr("float == double",10)
ret
bigger:
invoke printf,CStr("float > double",10)
ret
lower:
invoke printf,CStr("float < double",10)
ret


Code: [Select]

0.100000001490116
0.100000000000000
float > double


[jj2007]

Code: [Select]

12345
123


Hi mineiro,

I've tried to test your code, which looks like Masm64 SDK code, but I get different results, see attachment. Where are printf and CStr defined? I can't find them in the Masm64 SDK, so I used vc_printf and chr$().