Author Topic: Rounding Mode in FPU question  (Read 1968 times)

guga

  • Member
  • *****
  • Posts: 1043
  • Assembly is a state of art.
    • RosAsm
Re: Rounding Mode in FPU question
« Reply #30 on: February 16, 2019, 01:13:38 AM »
What about the ranges for a TenByte be considered a NAN/SNAN, infinite etc, what are they ?

According to this site https://www.doc.ic.ac.uk/~eedwards/compsys/float/nan.html , the ranges (In little endian) for a Real4 and Real8 are:

Quote
For single-precision values:

Positive infinity is represented by the bit pattern 7F800000
Negative infinity is represented by the bit pattern FF800000

A signalling NaN (NANS) is represented by any bit pattern
between 7F800001 and 7FBFFFFF or between FF800001 and FFBFFFFF

A quiet NaN (NANQ) is represented by any bit pattern
between 7FC00000 and 7FFFFFFF or between FFC00000 and FFFFFFFF

For double-precision values:

Positive infinity is represented by the bit pattern 7FF0000000000000
Negative infinity is represented by the bit pattern FFF0000000000000

A signalling NaN is represented by any bit pattern

between 7FF0000000000001 and 7FF7FFFFFFFFFFFF or
between FFF0000000000001 and FFF7FFFFFFFFFFFF

A quiet NaN is represented by any bit pattern

between 7FF8000000000000 and 7FFFFFFFFFFFFFFF or
between FFF8000000000000 and FFFFFFFFFFFFFFFF


So, from Raymond´s specifications is it correct assume the ranges for a TenByte be like this ? IF not, what are the ranges for a TenByte be considered, NAN, QNAN, Infinite, etc ?


Quote
For extended precision

bias  = 03FFF; Max = 07FFF . Exponent = 07FFF-Bias-1 ? . Range from NAN etc starts from 07FFF to 0FFFF ?

Positive infinity is represented by the bit pattern 7FFF_00000000_00000000
Negative infinity is represented by the bit pattern FFFF_00000000_00000000

A signalling NaN is represented by any bit pattern

between 7FFF_00000000_00000001 and 7FF7_FFFFFFFF_FFFFFFFF or
between FFFF_00000000_00000001 and FFF7_FFFFFFFF_FFFFFFFF

A quiet NaN is represented by any bit pattern

between 7FFF_80000000_00000000 and 7FFF_FFFFFFFF_FFFFFFFF or
between FFFF_80000000_00000000 and FFFF_FFFFFFFF_FFFFFFFF

Note:
Assuming a TenByte to be viewed on the format of a structure we should have:
[TenByteFormat:
 TenByteFormat.Fraction1: D$ 0
 TenByteFormat.Fraction2: D$ 0 ; It must contains a minimum of 080000000 to be considered a valid number ?
 TenByteFormat.Exponent: W$ 0] ; including the sign on the last bit

Since for a TenByte the 31th bit of the second dword must contains a "1" (080000000) So, all of the 2nd dword should contains at least 080000000 but...what if it don´t have this "1" value as a part of the number ?

Coding in Assembly requires a mix of:
80% of brain, passion, intuition, creativity
10% of programming skills
10% of alcoholic levels in your blood.

My Code Sites:
http://rosasm.freeforums.org
http://winasm.tripod.com

raymond

  • Member
  • **
  • Posts: 234
    • Raymond's page
Re: Rounding Mode in FPU question
« Reply #31 on: February 16, 2019, 06:01:36 AM »
According to the data which I had gathered at the time, I would have the following for the REAL10 format:

Positive INFINITY is represented by the bit pattern 7FFF_80000000_00000000
Negative INFINITY is represented by the bit pattern FFFF_80000000_00000000

INDEFINITE is represented by the bit pattern FFFF_C0000000_00000000

A Quiet NaN would be represented by any bit pattern
between 7FFF_C0000000_00000001 and 7FFF_FFFFFFFF_FFFFFFFF or
between FFFF_C0000000_00000001 and FFFF_FFFFFFFF_FFFFFFFF

A Signaling NaN would be represented by any bit pattern
between 7FFF_80000000_00000001 and 7FFF_BFFFFFFF_FFFFFFFF or
between FFFF_80000000_00000001 and FFFF_BFFFFFFF_FFFFFFFF

The explicit 1 in bit 63 always remains set for the REAL10 format, including all the NANs. Otherwise, the FPU seems to reject it as an unknown format.
Whenever you assume something, you risk being wrong half the time.
http://www.ray.masmcode.com/

guga

  • Member
  • *****
  • Posts: 1043
  • Assembly is a state of art.
    • RosAsm
Re: Rounding Mode in FPU question
« Reply #32 on: February 16, 2019, 09:40:34 AM »
Thank you so much, Raymond. That´s what i was looking for.  :t :t :t
Coding in Assembly requires a mix of:
80% of brain, passion, intuition, creativity
10% of programming skills
10% of alcoholic levels in your blood.

My Code Sites:
http://rosasm.freeforums.org
http://winasm.tripod.com

raymond

  • Member
  • **
  • Posts: 234
    • Raymond's page
Re: Rounding Mode in FPU question
« Reply #33 on: February 16, 2019, 12:02:00 PM »
You're welcome. Best wishes for your project.

One minor clarification to my last statement.
Quote
The explicit 1 in bit 63 always remains set for the REAL10 format
The one exception is when all exponent and fraction bits are 0's, defining the value of +/-0 depending on the sign bit.
Whenever you assume something, you risk being wrong half the time.
http://www.ray.masmcode.com/

daydreamer

  • Member
  • ****
  • Posts: 904
  • watch Chebyshev on the backside of the Moon
Re: Rounding Mode in FPU question
« Reply #34 on: February 16, 2019, 08:52:15 PM »
According to the data which I had gathered at the time, I would have the following for the REAL10 format:

Positive INFINITY is represented by the bit pattern 7FFF_80000000_00000000
Negative INFINITY is represented by the bit pattern FFFF_80000000_00000000

thanks raymond,its good to know infinitys so I can choose
;x/y

.IF y!=0
fld x
fdiv y
.ELSE
fld infinity ;humans thinks x/divbyzero=infinity
or
fldz ;humans think 0*0=0 as div by zero
depending on what works best in the formula you code

Quote from Flashdance
Nick  :  When you give up your dream, you die
*wears a flameproof asbestos suit*
what cpu handle "press any key"? any cpu of course(from C#) :D

guga

  • Member
  • *****
  • Posts: 1043
  • Assembly is a state of art.
    • RosAsm
Re: Rounding Mode in FPU question
« Reply #35 on: February 17, 2019, 12:04:09 AM »
You're welcome. Best wishes for your project.

One minor clarification to my last statement.
Quote
The explicit 1 in bit 63 always remains set for the REAL10 format
The one exception is when all exponent and fraction bits are 0's, defining the value of +/-0 depending on the sign bit.

Thanks  :t :t :t :t  :)


About the integer being set i did that and the cases of zero, i did those as you described :)  For negative zeros i put those on the same category as zero, since there´s no such a thing as a -0

I just added 4 new categories on the rare situations where the integer bit is not present on the TenByte (rare, but happens when you analyse some old libraries from a disassembler) resulting in an error on the FPU computations that will reject it.

[SpecialFPU_SpecialIndefQNan 10] ; Special INDEFINITE QNAN . Same as QNAN, but without the integer bit set
[SpecialFPU_SpecialIndefSNan 11] ; Special INDEFINITE SNAN. Same as SNAN, but without the integer bit set
[SpecialFPU_SpecialIndefNegInfinite 12] ; Special INDEFINITE Negative Infinite. Same as Negative Infinite, but without the integer bit set
[SpecialFPU_SpecialIndefPosInfinite 13] ; Special INDEFINITE Positive Infinite. Same as Positive Infinite, but without the integer bit set

For the disassembler and debugger purposes, i added those just to better categorize the error type, rather then throw an Unknown Format flag.

The updated code looks like this:

Code: [Select]

[SpecialFPU_PosValid 0] ; The FPU contains a valid positive result
[SpecialFPU_NegValid 1] ; The FPU contains a valid negative result
[SpecialFPU_PosSubNormal 2] ; The FPU produced a positive Subnormal (denormalized) number. So, although it´ range is outside the range 3.6...e-4932, the number lost it´ precision, but it is still valid
[SpecialFPU_NegSubNormal 3] ; The FPU produced a negative Subnormal (denormalized) number. So, although it´ range is outside the range -3.6...e-4932, the number lost it´ precision, but it is still valid
[SpecialFPU_Zero 4] ; The FPU contains a valid zero number
[SpecialFPU_QNAN 5] ; QNAN
[SpecialFPU_SNAN 6] ; SNAN
[SpecialFPU_NegInf 7] ; Negative Infinite
[SpecialFPU_PosInf 8] ; Positive Infinite
[SpecialFPU_Indefinite 9] ; Indefinite
[SpecialFPU_SpecialIndefQNan 10] ; Special INDEFINITE QNAN
[SpecialFPU_SpecialIndefSNan 11] ; Special INDEFINITE SNAN
[SpecialFPU_SpecialIndefNegInfinite 12] ; Special INDEFINITE Negative Infinite
[SpecialFPU_SpecialIndefPosInfinite 13] ; Special INDEFINITE Positive Infinite

Proc RealTenFPUNumberCategory:
    Arguments @Float80Pointer
    Local @FPUErrorMode, @TenByteErr
    Uses ebx


    mov ebx D@Float80Pointer
    mov D@FPUErrorMode SpecialFPU_PosValid

    ; 1st located all zero numbers (no bits settled on the whole TenByte or, only the Sign bit (15th of the last word = 79th bit of the Tenbyte ) was settled.
    ; Ex: D$ 0, 0, W$ 0 ; (0)
    ; Ex: D$ 0, 0, W$ 08000 ; negative zero is only zero
    .If_and D$ebx = 0, D$ebx+4 = 0
        ; ebx+8 = 08000 means a negative zero. But, there´s no such a thing as a negative zero, thus, it should be considered only 0
        If_Or W$ebx+8 = 0, W$ebx+8 = 08000
            mov eax SpecialFPU_Zero ; eax wil exist as Zero
            ExitP
        End_If
    .End_If

    ; Check if the TenByte is valid. Check for the integer bit (bit 63 of the tenbyte = bit 31 of the 2nd dword)
    mov D@TenByteErr &FALSE
    Test_If_Not B$ebx+7 080 ; See if the integer bit is present on the tenbyte. No integer bit ? Flag it as an error
        mov D@TenByteErr &TRUE
    Test_End


    ; Possible NANs contains always the 1st 2 bits of the 2nd word settled and the last 2 bit settled (pg 91/100 on Intel Manual)
    ; The biased exponent is on this form for NAN 11..11

     ; 2nd located all denormalized, but possible positive numbers.
     ; Ex: D$ 0FFFFFFFF, 0, W$ 0         ; (1.56560127731845315e-4941)
     ; Ex: D$ 0, 01, W$ 0                ; (1.56560127768297311e-4941)
     ; Ex: D$ 0FFFFFFFF, 0FFFFFFFF, W$ 0 ; (6.72420628622418417e-4932)
                                         ; (6.7242062862241870120e-4932) (Olly)

    ...If_And W$ebx+8 = 0, D$ebx+4 >= 0, D@TenByteErr = &FALSE ; No errors in the tenbyte. It also do contains the integer bit

        mov D@FPUErrorMode SpecialFPU_PosSubNormal

    ...Else_If_And W$ebx+8 = 08000, D$ebx+4 >= 0, D@TenByteErr = &FALSE ; No errors in the tenbyte. It also do contains the integer bit

    ; 3rd located all denormalized, but possible negative numbers. Bit 15th of the last word (Bit79 of the tenbyte) is settled
    ; Ex: D$ 0FFFFFFFF, 0, W$ 08000 ; (-1.56560127731845315e-4941)
    ; Ex: D$ 0, 01000000, W$ 08000  ; (-2.626643080565632194e-4934)  in olly dbg = (-0.2626643080556323050e-4933)
    ; Ex: D$ 01, 0FFFFFFFF, W$ 08000 ; (-6.72420628465858289e-4932)  in olly dbg = (-6.7242062846585857350e-4932)

        mov D@FPUErrorMode SpecialFPU_NegSubNormal

    ...Else_If W$ebx+8 = 07FFF ; Locate all positive infinite, QNAN, Special indefinite 00__0111_1111__1111_1111
    ; 00__0110_0000__0000_0011 06003 ; error happen here
    ; 00__0111_1111__1111_1111 07FFF ; error happen here

    ; 00__0111_1111__1111_1110 07FFE ; ok normal
    ; 00__0110_0000__0000_0010 06002 ; ok normal
    ; 00__0000_0000__0000_0001 01 ; ok normal

        ; locate all: Positive Infinite (Bit 15 of the last word is not set), the second dword are zero and the 1st dword contains only the integer bit 00__1000_0000__0000_0000
        ; Ex: D$ 0, 0, W$ 07FFF  ; 07FFF 00000000 00000000 when bit15 is not set it is positive 00__0111_1111__1111_1111

        .If_And D$ebx+4 = 080000000, D$ebx = 0; 2nd dword = 00__1000_0000__0000_0000__0000_0000__0000_0000 1st dword = 0

            mov D@FPUErrorMode SpecialFPU_PosInf

        .Else_If_And D$ebx+4 >= 0C0000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_QNAN

        .Else_If_And D$ebx+4 >= 080000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_SNAN

        .Else_If_And D$ebx+4 >= 040000000, D$ebx >= 01
            ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
            mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

        .Else_If D$ebx >= 01
            ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

            mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

        .Else
            ; all remaining cases, result only in D$ebx = 0. The lack of the 63th bit on this case could represente also a Indefinite Positive, but we are here labeling it as Indefinite infinite to be more
            ; logical with the Indefinite category
            mov D@FPUErrorMode SpecialFPU_SpecialIndefPosInfinite

        .End_If

    ...Else_If W$ebx+8 = 0FFFF ; Locate all negative infinite, QNAN, indefinite

        .If_And D$ebx+4 = 080000000, D$ebx = 0; 2nd dword = 00__1000_0000__0000_0000__0000_0000__0000_0000 1st dword = 0

            mov D@FPUErrorMode SpecialFPU_NegInf

        .Else_If_And D$ebx+4 >= 0C0000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_QNAN

        .Else_If_And D$ebx+4 >= 080000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_SNAN

        .Else_If_And D$ebx+4 = 0C0000000, D$ebx = 0

            mov D@FPUErrorMode SpecialFPU_Indefinite

        .Else_If_And D$ebx+4 >= 040000000, D$ebx >= 01
            ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
            mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

        .Else_If D$ebx >= 01
            ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

            mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

        .Else
            ; all remaining cases, result only in D$ebx = 0.

            mov D@FPUErrorMode SpecialFPU_SpecialIndefNegInfinite

        .End_If

    ...Else

        ; Now we must analyse the cases where the integer Bit (Bit 63 of tenbyte or 31th of the 2nd dword) is not settled by some error
        ; and the FPU will simply refuse to process
        ; Ex: [NumTest18: D$ 0, 013900F00, W$ 07ED]
        ..If D$ebx+4 < 080000000 ; Integer bit is never settled

            .If_And D$ebx+4 >= 040000000, D$ebx >= 01
                ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
                mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

            .Else_If D$ebx >= 01
                ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

                mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

            .Else
                ; all remaining cases, result only in D$ebx = 0 and we must only check if it is positive or negative
                ; Sign bit is settled
                Test_If B$ebx+9 0_80
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefNegInfinite
                Test_Else
                    ; Sign bit is never settled
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefPosInfinite
                Test_End
            .End_If
        ..End_If

    ...End_If

    .If D@FPUErrorMode = SpecialFPU_PosValid
        Test_If B$ebx+9 0_80
            mov D@FPUErrorMode SpecialFPU_NegValid
        Test_End
    .End_If

    mov eax D@FPUErrorMode

EndP

« Last Edit: February 17, 2019, 02:13:06 AM by guga »
Coding in Assembly requires a mix of:
80% of brain, passion, intuition, creativity
10% of programming skills
10% of alcoholic levels in your blood.

My Code Sites:
http://rosasm.freeforums.org
http://winasm.tripod.com

guga

  • Member
  • *****
  • Posts: 1043
  • Assembly is a state of art.
    • RosAsm
Re: Rounding Mode in FPU question
« Reply #36 on: February 17, 2019, 09:17:15 AM »
For exploring the mysteries of REAL10 notation 8)

picked=5
ife picked
  NaN   REAL10 0x07ED13900F0000000000
elseif picked eq 1
  NaN   REAL10 0x00000000000F9013ED07
elseif picked eq 2
  NaN   db 07, 0EDh, 13h, 90h, 0Fh, 00, 00, 00, 00, 00
elseif picked eq 3
  NaN   db 0, 0, 1, 0, 0, 0, 0, 0, 0, 0
elseif picked eq 4
  NaN   REAL10 0x00000000000000000001
elseif picked eq 5
  NaN   db 1, 0, 0, 0, 0, 0, 0, 0, 0, 0
elseif picked eq 6
  NaN   db 00, 00h, 00h, 00h, 00h, 00h, 00, 80h, 00, 00
endif
...
  int 3
  fld REAL10 ptr NaN
  fld FP10(1.0e4900)
  fmul


picked 5 produces 3.6452e-51 after multiplying with 1.0e4900 - remember what Wiki says about the range?

Specific Watcom mysteries: I expected picked 4 & 5 to be identical. What exactly does the REAL10 0x111 notation mean?

Hi JJ

from my new routines, i identify this as:

REAL10 0x111 ; If this is the order [JJReal10: D$ 0111 D$ 0 W$ 0] =  9.95139472203915145e-4949 denormal, but it lacks the integer bit. ?
REAL10 0x111 ; Or, if the order is this: [JJReal10: D$ 0 D$ 0 W$ 0111] = Special INDEFINITE Positive Infinite. It lacks the integer bit, but the category is similar to a Positive Infinite (if that bit was settled.)
Real10: D$ 0, D$ 00, W$ 01] ; 0x000000000000000001 = Special INDEFINITE Positive Infinite. It lacks the integer bit, but the category is similar to a Positive Infinite (if that bit was settled.)
[JJ3Real10: D$ 01, D$ 00, W$ 0] ; 0x100000000000000000 = Special INDEFINITE SNAN meaning, something that should looks like a SNAN, but it lacks the integer bit.

In all 4 cases, the FPU should not be able to process this, due to the lack of the integer bit, but it is performing the multiplication even thought the bit is not settled.

About this:

  fld REAL10 ptr NaN
  fld FP10(1.0e4900)
  fmul

Here is resulting in a Indefinite [JJReal410: D$ 0, D$ 0 W$ 0111]*1e4900 ; [JJReal410: D$ 0, D$ 0 W$ 0111] = Special Indefinite Positive infinite

or if the order is the reversed one [JJReal10: D$ 0111 D$ 0 W$ 0]  9.95139472203915145e-4949 (denormal)

  fld REAL10 ptr NaN
  fld FP10(1.0e4900)
  fmul

It then results on a 9.95139472203915133e-49

That´s weird. Maybe for a Subnormal, the number may still be valid even if the integer bit is not set ? From page 91 in Intel Manual the integer bit (For a subnormal) is set to zero. I´ll confirme that...hold on :)

Btw. Many tks for the help, JJ  :t :t

« Last Edit: February 17, 2019, 10:31:07 AM by guga »
Coding in Assembly requires a mix of:
80% of brain, passion, intuition, creativity
10% of programming skills
10% of alcoholic levels in your blood.

My Code Sites:
http://rosasm.freeforums.org
http://winasm.tripod.com

guga

  • Member
  • *****
  • Posts: 1043
  • Assembly is a state of art.
    • RosAsm
Re: Rounding Mode in FPU question
« Reply #37 on: February 17, 2019, 10:21:27 AM »
Hi JJ.. just confirmed the last post.

Denormal values does not have the integer bit set. So

[JJReal10: D$ 0111 D$ 0 W$ 0] = 9.95139472203915145e-4949

when multiply by 1e4900 results in:  9.95139472203915145e-49

Updated the function again to detect properly the denormal. Thanks a lot


Code: [Select]

[SpecialFPU_PosValid 0] ; The FPU contains a valid positive result
[SpecialFPU_NegValid 1] ; The FPU contains a valid negative result
[SpecialFPU_PosSubNormal 2] ; The FPU produced a positive Subnormal (denormalized) number. So, although it´ range is outside the range 3.6...e-4932, the number lost it´ precision, but it is still valid
[SpecialFPU_NegSubNormal 3] ; The FPU produced a negative Subnormal (denormalized) number. So, although it´ range is outside the range -3.6...e-4932, the number lost it´ precision, but it is still valid
[SpecialFPU_Zero 4] ; The FPU contains a valid zero number
[SpecialFPU_QNAN 5] ; QNAN
[SpecialFPU_SNAN 6] ; SNAN
[SpecialFPU_NegInf 7] ; Negative Infinite
[SpecialFPU_PosInf 8] ; Positive Infinite
[SpecialFPU_Indefinite 9] ; Indefinite
[SpecialFPU_SpecialIndefQNan 10] ; Special INDEFINITE QNAN
[SpecialFPU_SpecialIndefSNan 11] ; Special INDEFINITE SNAN
[SpecialFPU_SpecialIndefNegInfinite 12] ; Special INDEFINITE Negative Infinite
[SpecialFPU_SpecialIndefPosInfinite 13] ; Special INDEFINITE Positive Infinite

Proc RealTenFPUNumberCategory:
    Arguments @Float80Pointer
    Local @FPUErrorMode
    Uses ebx


    mov ebx D@Float80Pointer
    mov D@FPUErrorMode SpecialFPU_PosValid

    ; 1st located all zero numbers (no bits settled on the whole TenByte or, only the Sign bit (15th of the last word = 79th bit of the Tenbyte ) was settled.
    ; Ex: D$ 0, 0, W$ 0 ; (0)
    ; Ex: D$ 0, 0, W$ 08000 ; negative zero is only zero
    .If_and D$ebx = 0, D$ebx+4 = 0
        ; ebx+8 = 08000 means a negative zero. But, there´s no such a thing as a negative zero, thus, it should be considered only 0
        If_Or W$ebx+8 = 0, W$ebx+8 = 08000
            mov eax SpecialFPU_Zero ; eax will exit as Zero FPU category
            ExitP
        End_If
    .End_If



    ; Possible NANs contains always the 1st 2 bits of the 2nd word settled and the last 2 bit settled (pg 91/100 on Intel Manual)
    ; The biased exponent is on this form for NAN 11..11

     ; 2nd located all denormalized, but possible positive numbers.
     ; Ex: D$ 0FFFFFFFF, 0, W$ 0         ; (1.56560127731845315e-4941)
     ; Ex: D$ 0, 01, W$ 0                ; (1.56560127768297311e-4941)
     ; Ex: D$ 0FFFFFFFF, 0FFFFFFFF, W$ 0 ; (6.72420628622418417e-4932)
                                         ; (6.7242062862241870120e-4932) (Olly)

    ...If_And W$ebx+8 = 0, D$ebx+4 >= 0 ; On denormal (Subnormal) values, the integer bit is not set

        mov D@FPUErrorMode SpecialFPU_PosSubNormal

    ...Else_If_And W$ebx+8 = 08000, D$ebx+4 >= 0 ; On denormal (Subnormal) values, the integer bit is not set

    ; 3rd located all denormalized, but possible negative numbers. Bit 15th of the last word (Bit79 of the tenbyte) is settled
    ; Ex: D$ 0FFFFFFFF, 0, W$ 08000 ; (-1.56560127731845315e-4941)
    ; Ex: D$ 0, 01000000, W$ 08000  ; (-2.626643080565632194e-4934)  in olly dbg = (-0.2626643080556323050e-4933)
    ; Ex: D$ 01, 0FFFFFFFF, W$ 08000 ; (-6.72420628465858289e-4932)  in olly dbg = (-6.7242062846585857350e-4932)

        mov D@FPUErrorMode SpecialFPU_NegSubNormal

    ...Else_If W$ebx+8 = 07FFF ; Locate all positive infinite, QNAN, Special indefinite 00__0111_1111__1111_1111
    ; 00__0110_0000__0000_0011 06003 ; error happen here
    ; 00__0111_1111__1111_1111 07FFF ; error happen here

    ; 00__0111_1111__1111_1110 07FFE ; ok normal
    ; 00__0110_0000__0000_0010 06002 ; ok normal
    ; 00__0000_0000__0000_0001 01 ; ok normal

        ; locate all: Positive Infinite (Bit 15 of the last word is not set), the 1st dword are zero and the 2nd dword contains only the integer bit 00__1000_0000__0000_0000
        ; Ex: D$ 0, 0, W$ 07FFF  ; 07FFF 00000000 00000000 when bit15 is not set it is positive 00__0111_1111__1111_1111

        .If_And D$ebx+4 = 080000000, D$ebx = 0; 2nd dword = 00__1000_0000__0000_0000__0000_0000__0000_0000 1st dword = 0

            mov D@FPUErrorMode SpecialFPU_PosInf

        .Else_If_And D$ebx+4 >= 0C0000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_QNAN

        .Else_If_And D$ebx+4 >= 080000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_SNAN

        .Else_If_And D$ebx+4 >= 040000000, D$ebx >= 01
            ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
            mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

        .Else_If D$ebx >= 01
            ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

            mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

        .Else
            ; all remaining cases, result only in D$ebx = 0. The lack of the 63th bit on this case could represent also a Indefinite Positive, identified here as a Special Indefinite Positive since it is similar to the Indefinite Positive (But, without the integer bit set)
            mov D@FPUErrorMode SpecialFPU_SpecialIndefPosInfinite

        .End_If

    ...Else_If W$ebx+8 = 0FFFF ; Locate all negative infinite, QNAN, indefinite

        .If_And D$ebx+4 = 080000000, D$ebx = 0; 2nd dword = 00__1000_0000__0000_0000__0000_0000__0000_0000 1st dword = 0

            mov D@FPUErrorMode SpecialFPU_NegInf

        .Else_If_And D$ebx+4 >= 0C0000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_QNAN

        .Else_If_And D$ebx+4 >= 080000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_SNAN

        .Else_If_And D$ebx+4 = 0C0000000, D$ebx = 0

            mov D@FPUErrorMode SpecialFPU_Indefinite

        .Else_If_And D$ebx+4 >= 040000000, D$ebx >= 01
            ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
            mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

        .Else_If D$ebx >= 01
            ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

            mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

        .Else
            ; all remaining cases, result only in D$ebx = 0.

            mov D@FPUErrorMode SpecialFPU_SpecialIndefNegInfinite

        .End_If

    ...Else

        ; Now we must analyse the cases where the integer Bit (Bit 63 of tenbyte or 31th of the 2nd dword) is not settled by some error
        ; and the FPU will simply refuse to process
        ; Ex: [NumTest18: D$ 0, 013900F00, W$ 07ED]
        ..If D$ebx+4 < 080000000 ; Integer bit is never settled

            .If_And D$ebx+4 >= 040000000, D$ebx >= 01
                ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
                mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

            .Else_If D$ebx >= 01
                ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

                mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

            .Else
                ; all remaining cases, result only in D$ebx = 0 and we must only check if it is positive or negative
                ; Sign bit is settled
                Test_If B$ebx+9 0_80
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefNegInfinite
                Test_Else
                    ; Sign bit is never settled
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefPosInfinite
                Test_End
            .End_If
        ..End_If

    ...End_If

    .If D@FPUErrorMode = SpecialFPU_PosValid
        Test_If B$ebx+9 0_80
            mov D@FPUErrorMode SpecialFPU_NegValid
        Test_End
    .End_If

    mov eax D@FPUErrorMode

EndP

« Last Edit: February 17, 2019, 11:02:13 PM by guga »
Coding in Assembly requires a mix of:
80% of brain, passion, intuition, creativity
10% of programming skills
10% of alcoholic levels in your blood.

My Code Sites:
http://rosasm.freeforums.org
http://winasm.tripod.com

jj2007

  • Member
  • *****
  • Posts: 9684
  • Assembler is fun ;-)
    • MasmBasic
Re: Rounding Mode in FPU question
« Reply #38 on: February 17, 2019, 02:46:45 PM »
Many tks for the help, JJ  :t :t

When it comes to fpu stuff, Raymond is the best source of advice, as usual :t