The MASM Forum

General => The Workshop => Topic started by: guga on February 11, 2019, 10:03:13 PM

Title: Rounding Mode in FPU question
Post by: guga on February 11, 2019, 10:03:13 PM
Hi Guys

what is the minimum acceptable value for rounding in FPU ?
I mean, consider we have 2 numbers loaded in ST0 and ST1, respectivelly

Number1 = 0403A B1A2 BC2E C4FF FFF7
Number2 = 0413A B1A2 BC2E C4FF FFF7

When the numbers are compared using operands such as:

fcom/fcompp and friends, what is the minimum value for those numbers be considered equal ?

If both are TenBytes, they are different ? Or the difference (error mode due to rounding) only happens  on Real8/real4 ? Or there isn´t any difference at all, and the functions after being compared will be considered different ?



Title: Re: Rounding Mode in FPU question
Post by: jj2007 on February 12, 2019, 12:54:04 AM
what is the minimum acceptable value for rounding in FPU ?

None! Here is a testbed:

include \masm32\MasmBasic\MasmBasic.inc         ; download (http://masm32.com/board/index.php?topic=94.0)

.data
align 16
Num1    REAL10 1234567890.1234567890
align 16
Num2    REAL10 1234567890.1234567891
align 16

.data?
Dest1   REAL10 ?
Dest2   REAL10 ?
DestX1  REAL8 ?
DestX2  REAL8 ?

  Init
  movaps xmm1, OWORD PTR Num1
  movaps xmm2, OWORD PTR Num2
  deb 4, "Test xmm", x:xmm1, x:xmm2
  comisd xmm1, xmm2
  deb 4, "comisd (lowercase = flag not set)", flags
  fld Num1
  fld Num2
  deb 4, "Test FPU", ST(0), ST(1)
  fst DestX1
  fst DestX2
  movlps xmm1, DestX1
  movlps xmm2, DestX2
  comisd xmm1, xmm2
  deb 4, "comisd (lowercase = not set)", flags
  deb 4, "Test xmm", x:xmm1, x:xmm2, x:DestX1, x:DestX2
  fstp Dest2
  fstp Dest1
  mov eax, dword ptr Dest1
  mov edx, dword ptr Dest2
  deb 4, "lowest 4 bytes", x:eax, x:edx
  fld Num1
  fld Num2
  fcomip ST, ST(1)
  deb 4, "fcomip (lowercase = not set)", flags
  Inkey "ok"
EndOfCode


Code: [Select]
Test xmm
x:xmm1          00000000 0000401D 932C05A4 3F35BA6E
x:xmm2          00000000 0000401D 932C05A4 3F35BA6F
comisd (lowercase = not set)    flags:          czso

Test FPU
ST(0)           1234567890.123456789
ST(1)           1234567890.123456789
comisd (lowercase = not set)    flags:          cZso

Test xmm
x:xmm1          00000000 0000401D 41D26580 B487E6B7
x:xmm2          00000000 0000401D 41D26580 B487E6B7
x:DestX1        41D26580 B487E6B7
x:DestX2        41D26580 B487E6B7

lowest 4 bytes
x:eax           3F35BA6E
x:edx           3F35BA6F
fcomip (lowercase = not set)    flags:          czso

Note in particular that the  fst DestX1 + movlps xmm1, DestX1 results in a ZERO flag after comisd!

In case you want to build the snippet & play with it: It needs the very latest version of MB of today, right now (http://masm32.com/board/index.php?topic=94.0), archive ending with *_f.zip. The reason being that your request made me find a tiny little bug in deb - the flags were not reported correctly :icon_redface:
Title: Re: Rounding Mode in FPU question
Post by: Raistlin on February 12, 2019, 01:17:07 AM
I had a similar (exactly the same) issue 16 years ago
writing a system in asm for the SA Reserve bank. See
old forum: FPU rounding. The answer came from raymond
if I remember correctly re: convert real4 within his LIB,
or face the convoluted answer via inspecting the code.
Title: Re: Rounding Mode in FPU question
Post by: AW on February 12, 2019, 04:32:50 AM
writing a system in asm for the SA Reserve bank.
Great opportunity for a salami slicing (fraudulent numbers are always round numbers).
Title: Re: Rounding Mode in FPU question
Post by: raymond on February 12, 2019, 04:55:50 AM
Quote
what is the minimum acceptable value for rounding in FPU ?

a) Whatever value is held in FPU data registers ALWAYS has 10 bytes (80 bits) consisting of the sign bit, 15 exponent bits and 64 significand bits. This is the REAL10 format.

b) When storing such values in memory as REAL4 or REAL8, some of the significand bits would get lost due to truncation and rounding would be performed on the remaining significand bits according to the selected rounding mode.

c) If you reload a "truncated" float into an FPU data register, it would be converted into the 80-bit format but without remembering whatever bits may have been truncated previously.

d) If you compare the truncated value to its original 80-bit value, it would be considered different UNLESS ALL THE TRUNCATED BITS HAD BEEN 0's.
Title: Re: Rounding Mode in FPU question
Post by: Raistlin on February 12, 2019, 05:16:04 AM
VERY funny AW27 ... inspired  :lol:
But YES ! At lasT!  To magistrate Raymond, after 16 years, thanks it now makes sense - thank you ever so much, my nightmares on this very topic will now end. Thanks sincerely.

Pps guga: the ladies in your avatar are too distracting, perhaps stars in certain places are appropriate ? 😈 
Title: Re: Rounding Mode in FPU question
Post by: guga on February 12, 2019, 06:44:58 AM
Quote
None! Here is a testbed:

I was afraid you said that :icon_mrgreen: :icon_mrgreen: :icon_mrgreen: I´m rebuilding the FPU functions for disassembler and debugger, and will have to make they work differently. I´ll need to update the debugger to it avoid those rounding anyway :( :(  The disassembler, i made a couple of fixes too, but allowing the rounding mode due to errors produced by some compilers throwing things like:  07ED 7FFFFFFF 00000000, or 07ED 13900F00 00000000 whenm the correct should be 07ED 80000000 00000000

Tks, JJ !  :t :t

Thanks a lot Raymond. :t :t :t i´ll be forced to make the function works without rounding or truncating (at least, for the debugger).  Btw...do you know where i can find a scheme for Real10 values similar as this ? https://en.wikipedia.org/wiki/Extended_precision I didn´t fully understood how to correctly categorize the error modes. Wiki said about 80 bit values, but the documentation seems to be related to 64 ?

I´m trying to redesign an old function in RosAsm, but i´m not sure about some values. On the FPU to string, it is converting values such as:

[Value1: D$ 0, 0FFFFFFFF W$ 0] ; This number exceeds the limit for FPU. It is converted to 6.724206...e-4932 Does it exceeds the limit ?
[Value2: B$ 0FE, 07F, 0, 0, 0C0, 07F, 0, 0, 0, 0] ; This number exceeds the limit for FPU. It is converted to 5.1201424......e-4937 Does it exceeds the limit ?

But i´m not sure, if those values are correct. Aren´t they exceeding the limit ? If so, how to force them to be handled according to the error mode/category ? Are they QNAN, indefinite etc ? If so, what are the bits related to settled QNAN etc for those 2 numbers in particular ?

Code: [Select]

[SpecialFPU_QNAN 1] ; QNAN
[SpecialFPU_SNAN 2] ; SNAN
[SpecialFPU_NegInf 3] ; Negative Infinite
[SpecialFPU_PosInf 4] ; Positive Infinite
[SpecialFPU_Indefinite 5] ; Indefinite
[SpecialFPU_SpecialIndefQNan 6] ; Special INDEFINITE QNAN
[SpecialFPU_SpecialIndefSNan 7] ; Special INDEFINITE SNAN
[SpecialFPU_SpecialIndefInfinite 8] ; Special INDEFINITE Infinite

Proc RealTenFPUNumberCategory:
    Arguments @Float80Pointer
    Local @FPUErrorMode
    Uses edi, ebx


    mov ebx D@Float80Pointer
    mov D@FPUErrorMode &FALSE

    ...If_And W$ebx+8 = 0, D$ebx+4 = 0 ; This is denormalized, but it is possible.
        ; 0000 00000000 00000000
        ; 0000 00000000 FFFFFFFF
    ...Else_If_And W$ebx+8 = 0, D$ebx+4 > 0 ; This is Ok.
        ; 0000 00000001 00000000
        ; 0000 FFFFFFFF FFFFFFFF
    ...Else_If_And W$ebx+8 > 0, W$ebx+8 < 07FFF; This is ok only if the fraction Dword is bigger or equal to 080000000
        .If D$ebx+4 < 080000000
            .Test_If D$ebx+4 040000000
                ; QNAN 40000000
                mov D@FPUErrorMode SpecialFPU_QNAN
            .Test_Else
                If_And D$ebx+4 > 0, D$ebx > 0
                    ; SNAN only if at least 1 bit is set
                    mov D@FPUErrorMode SpecialFPU_SNAN
                Else ; All fraction Bits are 0
                    ; Bit 15 is never reached. The bit is 0 from W$ebx+8
                    ; -INFINITE ; Bit15 = 0
                    mov D@FPUErrorMode SpecialFPU_NegInf
                End_If
            .Test_End
        .End_If
    ...Else_If W$ebx+8 = 07FFF; This is ok only if the fraction Dword is bigger or equal to 080000000
        .Test_If D$ebx+4 040000000
            ; QNAN 40000000
            mov D@FPUErrorMode SpecialFPU_QNAN
        .Test_Else
            If_And D$ebx+4 > 0, D$ebx > 0
                ; SNAN only if at least 1 bit is set
                mov D@FPUErrorMode SpecialFPU_SNAN
            Else ; All fraction Bits are 0
                ; Bit 15 is never reached. The bit is 0 from W$ebx+8
                ; -INFINITE ; Bit15 = 0
;               Test_If W$ebx+8 = 0FFFF ; we need to see if Bit 15 is set
 ;                  ; -INFINITE ; Bit15 = 0
  ;             Test_Else
   ;                ; +INFINITE ; Bit15 = 1
    ;           Test_End
                ;mov D$edi '-INF', B$edi+4 0
                mov D@FPUErrorMode SpecialFPU_NegInf
            End_If
        .Test_End
        ; Below is similar to W$ebx+8 = 0
    ...Else_If_And W$ebx+8 = 08000, D$ebx+4 = 0 ; This is denormalized, but possible.
        ; 8000 00000000 00000000 (0)
        ; 8000 00000000 FFFFFFFF (-0.0000000156560127730E-4933)
    ...Else_If_And W$ebx+8 = 08000, D$ebx+4 > 0 ; This is Ok.
        ; 8000 01000000 00000000 (-0.2626643080556322880E-4933)
        ; 8000 FFFFFFFF 00000001 (-6.7242062846585856000E-4932)
    ...Else_If_And W$ebx+8 > 08000, W$ebx+8 < 0FFFF; This is ok only if the fraction Dword is bigger or equal to 080000000
        .If D$ebx+4 < 080000000
            .Test_If D$ebx+4 040000000
                ; QNAN 40000000
                mov D@FPUErrorMode SpecialFPU_QNAN
            .Test_Else
                If_And D$ebx+4 > 0, D$ebx > 0
                    ; SNAN only if at least 1 bit is set
                    ;mov D$edi 'SNaN', B$edi+4 0
                    mov D@FPUErrorMode SpecialFPU_SNAN
                Else ; All fraction Bits are 0
                    ; Bit 15 is always reached. The bit is 1 from W$ebx+8
                    ; +INFINITE ; Bit15 = 1
                    ;mov D$edi '+INF', B$edi+4 0
                    mov D@FPUErrorMode SpecialFPU_PosInf
                End_If
            .Test_End
        .End_If

    ...Else_If W$ebx+8 = 0FFFF; This is to we identify indefined or other NAN values

        .If_And D$ebx+4 >= 040000000, D$ebx = 0
            ; INDEFINITE
            mov D@FPUErrorMode SpecialFPU_Indefinite
        .Else
            .Test_If D$ebx+4 040000000
                ; Special INDEFINITE QNAN 40000000
                mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan
            .Test_Else
                If_And D$ebx+4 > 0, D$ebx > 0
                    ; Special INDEFINITE SNAN only if at least 1 bit is set
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan
                Else ; All fraction Bits are 0
                    ; Bit 15 is always reached. The bit is 1 from W$ebx+8
                    ; Special INDEFINITE +INFINITE ; Bit15 = 1
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefInfinite
                End_If
            .Test_End
        .End_If
    ...End_If

    ..If D@FPUErrorMode <> 0

        On B$edi-1 = '-', dec edi

        .If D@FPUErrorMode = SpecialFPU_QNAN
            push esi | zcopy {"QNAN ", 0} | pop esi
            mov B$edi 0
        .Else_If D@FPUErrorMode = SpecialFPU_SNAN
            push esi | zcopy {"SNAN ", 0} | pop esi
            mov B$edi 0
        .Else_If D@FPUErrorMode = SpecialFPU_NegInf
            push esi | zcopy {"-INFINITE ", 0} | pop esi
            mov B$edi 0
        .Else_If D@FPUErrorMode = SpecialFPU_PosInf
            push esi | zcopy {"+INFINITE ", 0} | pop esi
            mov B$edi 0
        .Else_If D@FPUErrorMode = SpecialFPU_Indefinite
            push esi | zcopy {"INDEFINITE ", 0} | pop esi
            mov B$edi 0
        .Else_If D@FPUErrorMode = SpecialFPU_SpecialIndefQNan
            push esi | zcopy {"Special INDEFINITE QNAN ", 0} | pop esi
            mov B$edi 0
        .Else_If D@FPUErrorMode = SpecialFPU_SpecialIndefSNan
            push esi | zcopy {"Special INDEFINITE SNAN ", 0} | pop esi
            mov B$edi 0
        .Else_If D@FPUErrorMode = SpecialFPU_SpecialIndefInfinite
            push esi | zcopy {"Special INDEFINITE +INFINITE ", 0} | pop esi
            mov B$edi 0
        .End_If

    ..End_If

    mov eax D@FPUErrorMode

EndP

Raistlin
Quote
Pps guga: the ladies in your avatar are too distracting, perhaps stars in certain places are appropriate ? 😈

 :greensml: :greensml: :greensml: :greensml: :greensml: Considering the time i made the avatar, those ladies must be kind old right now (or dead):greensml: :greensml: :greensml: :greensml: :greensml: (They are not naked, you know...Only wearing a bikini)  :bgrin: :bgrin:
Title: Re: Rounding Mode in FPU question
Post by: K_F on February 12, 2019, 07:02:27 AM
A possible way of re-introducing the lost resolution is to run a 'random' generator at the 'resolution of lost bits', and then add that to the truncated Real4 (Real8).
A way of minimising FPU error propagation. ;)
Title: Re: Rounding Mode in FPU question
Post by: raymond on February 12, 2019, 07:39:55 AM
NANs (including QNAN's and SNAN's) are sufficiently described in Simply FPU and would be a waste of time to start copying that section here.

See http://www.ray.masmcode.com/tutorial/fpuchap2.htm#nans
Title: Re: Rounding Mode in FPU question
Post by: AW on February 12, 2019, 08:37:17 PM
If f is a NaN then f !=f is true.
This is according to IEEE, so compilers/assemblers must abide.
Title: Re: Rounding Mode in FPU question
Post by: guga on February 14, 2019, 03:45:59 AM
If f is a NaN then f !=f is true.
This is according to IEEE, so compilers/assemblers must abide.

Yes, but some compilers generates errors when converting string to FPU and it is hard to identify them while you are disassembling or debugging. For example, the programmer is using Float as a global variable, but when it compiled it, the compiler generated a NAN, like this:
07ED 13900F00 00000000

So, if that variable was supposed to be used on a multiplication or division operation, for example, it will produce an error. And this error is hard to identify or fix.  On the disassembler we needed to make fixes when found bugs like these ones forcing the numbers of that variable to be restricted on their own boundaries. On such situations, to fix that we needed to check the next good value that is, on this case: 07ED 80000000 00000000 (1.0361967820008025600E-4321)

Sure, we could also look for the previous good value as well, but, they are the same (on this case) 07EC FFFFFFFF FFFFFFFF (1.0361967820008025600E-4321), showing us that the NAN was, in fact: 1.0361967820008025600E-4321 (07ED 80000000 00000000) and we could safely use that value, rather then accepting the NAN as a good value.

This kind of gap, is for the disassembler and debugger purposes and not for defining a NAN or checking for zero divisions etc. For the disassembler point of view, its useless trying to "guess" what the number was originally, because we are analyzing a defined value with certain Bytes that certainly was badly generated due to an error on the linker/compiler that failed to check for the NANs or error mode values or by any other unknown error. For example, if the user was trying to build things like:
T$ 1.0361967820008025600E-4321, instead the compiler output this bytes: 07ED 80000000 00000000, it outputed bad ones, like 07ED 7FFFFFFF 00000000, or 07ED 13900F00 00000000 like in the example showed.

Those kind of errors were not supposed to happen, but it do happens, specially in older compilers or old libraries.
Title: Re: Rounding Mode in FPU question
Post by: AW on February 14, 2019, 04:08:20 AM
Well, let me try to reformulate because I had not taken that from the horse's mouth. Now, I have read IEEE Std 754 -2008.
What it says, when talking about comparisons of floating point numbers, is:
Quote
Four mutually exclusive relations are possible: less than, equal, greater than, and unordered. The last case arises when at least one operand is NaN. Every NaN shall compare unordered with everything, including itself.
....
Languages define how the result of a comparison shall be delivered, in one of two ways: either as a relation identifying one of the four relations listed above, or as a true-false response to a predicate that names the specific comparison desired.
So, for a few high-level language, including C (at least in Visual Studio), it is true what I said:
If f is a NaN then f !=f is true

But, what happened behind the scenes is not that, what happens is an unordered relation:

This is the MASM equivalent of what really happens:

Code: [Select]
.model flat, stdcall
option casemap:none

includelib \masm32\lib\msvcrt.lib
printf proto C :ptr, :VARARG
includelib \masm32\lib\kernel32.lib
ExitProcess proto :dword

NANFactory UNION
q QWORD ?
f REAL8 ?
NANFactory ENDS

.data
nan NANFactory   <7FF0000000000001h> ; Generates SNaN
;nan NANFactory  <7FFF100000000001h> ; Generates QNaN

isNotNaNMsg db 'Value is not a NaN',10,0
isNaNMsg db 'Value is a NaN',10,0

.code
main proc
; ************ SSE *************
movsd xmm0, qword ptr nan.f
ucomisd xmm0, xmm0
COMMENT £
The UCOMISD instruction differs from the COMISD instruction in that it signals a SIMD floating-point invalid operation exception (#I) only when a source operand is an SNaN. But both QNaN and SNaN return unordered results.
£
lahf ; For NaN: SF=0 ZF=1 0 AF=0 0 PF=1 1 CF=1 , UNORDERED: ZF,PF,CF←111;
; For non NaN:
; GREATER_THAN: ZF,PF,CF←000
; LESS_THAN: ZF,PF,CF←001;
; EQUAL: ZF,PF,CF←100;
test ah, 01000100b ; 44h - Test ZF and PF
.if PARITY?
invoke printf, offset isNaNMsg ; zf=0 pf=1
.else
invoke printf, offset isNotNaNMsg ; zf=0 pf=0
.endif

; ************ X87 *************
fld qword ptr nan.f
fld qword ptr nan.f
fucompp ; unordered comparison of the contents of register ST(0) and ST(1)
COMMENT £
Unordered. Flags C0, C2 and C3 set when SNaN or an unsupported format.
£
fnstsw ax
test ah, 01000100b ; 44h - Test ZF and PF
.if PARITY?
invoke printf, offset isNaNMsg ; zf=0 pf=1
.else
invoke printf, offset isNotNaNMsg ; zf=0 pf=0
.endif
@exit:
invoke ExitProcess, 0
main endp

end



Title: Re: Rounding Mode in FPU question
Post by: Adamanteus on February 14, 2019, 04:58:59 AM
A possible way of re-introducing the lost resolution is to run a 'random' generator at the 'resolution of lost bits', and then add that to the truncated Real4 (Real8).
A way of minimising FPU error propagation. ;)
Particular way of solving such type of problems, is to have ONE type for all calculations, I'm using this :
Code: [Select]
MACE TYPEDEF REAL10
Title: Re: Rounding Mode in FPU question
Post by: raymond on February 14, 2019, 05:06:14 AM
Quote
the compiler generated a NAN, like this:
07ED 13900F00 00000000

Would you be kind enough to clarify if that hex value
a) is taken from an FPU register or from a memory section
b) if the latter, is it reported in the little-endian style or would it be as an ordered ten-byte value

Quote
the programmer is using Float as a global variable

Many HLL programmers consider the term "float" as meaning REAL4 variables. Then you use a REAL10 example number for discussion. Could you clarify the discrepancy.
Title: Re: Rounding Mode in FPU question
Post by: guga on February 14, 2019, 06:26:56 AM
Hi raymond

Sorry, i forgot to fix the previous examples before posting. They are TenByte in little endian. They are taken from memory section. So, the Ten Byte they looks like:

D$ 0
D$ 013900F00
W$ 07ED

In byte sequence:  0, 0, 0, 0, 0, 0F, 090, 013, 0ED, 07

Quote
Many HLL programmers consider the term "float" as meaning REAL4 variables. Then you use a REAL10 example number for discussion. Could you clarify the discrepancy.

I said Float (In general), but i was referring to the Real10 Variable on the example i used.
Title: Re: Rounding Mode in FPU question
Post by: raymond on February 14, 2019, 06:50:30 AM
Whenever all the bits are set to 1 in the exponent field of a real number format, the value is designated as a NAN.

The number you used as an example would thus not qualify as a NAN. However, it would qualify as a denormalized number (http://www.ray.masmcode.com/tutorial/fpuchap2.htm#denormal) which is very different from a NAN.
Title: Re: Rounding Mode in FPU question
Post by: AW on February 14, 2019, 07:08:18 AM
To be sincere I have no idea what guga is talking about and can't even see how he obtains the values he mentions.  :dazzled:
Title: Re: Rounding Mode in FPU question
Post by: guga on February 14, 2019, 07:12:45 AM
Hi Raymond, not sure if i´m fully undertanding this.

From what i understood on the documentation, the Word 07ED (last 2 bytes from the TenByte) represents the sign bit (79) and the exponent, right ? And so, they can´t be denormalized since the last Word (of the tenbyte) is not zero and the second Dword also is not zero. Demnormalized numbers are the ones where the second Dword of the tenbyte is zero and the last word is either zero or 08000, right ?

So, is this number qualified as an -inifinte ? I don´t understand why you say that this number is denormalized.
Title: Re: Rounding Mode in FPU question
Post by: guga on February 14, 2019, 07:16:50 AM
I´m quite confused right now. :dazzled: :dazzled:

I created a function to categorize the FPU numbers (Tenbyte in memory). Is this correct ?

Code: [Select]

    RealTenFPUNumberCategory
        This function identifies the Errors existant in a Real10 FPU data.

    Parameters:
        Float80Pointer - A pointer to a variable containing a TenByte (80 bit) value

    Returned Values:
   
        The function will return one of the following equates:

        Equate                          Value   Description
       
        SpecialFPU_PosValid             0       The FPU contains a valid positive number.
        SpecialFPU_NegValid             1       The FPU contains a valid negative number.
        SpecialFPU_PosSubNormal         2       The FPU produced a positive Subnormal (denormalized) number.
                                                Although it´s range is outside the range 3.6...e-4932, the number lost it´ precision, but it is still valid
                                                Ex: 0000 00000000 00000000
                                                    0000 00000000 FFFFFFFF
                                                    0000 00000000 00008000
                                                    0000 00000001 00000000
                                                    0000 FFFFFFFF FFFFFFFF
        SpecialFPU_NegSubNormal         3       The FPU produced a negative Subnormal (denormalized) number.
                                                Although it´s range is outside the range -3.6...e-4932, the number lost it´ precision, but it is still valid
                                                Ex: 8000 00000000 00000000 (0) (Negative zero must be considered only as zero)
                                                    8000 00000000 FFFFFFFF (-0.0000000156560127730E-4933)
                                                    8000 01000000 00000000 (-0.2626643080556322880E-4933)
                                                    8000 FFFFFFFF 00000001 (-6.7242062846585856000E-4932)
        SpecialFPU_QNAN                 4       QNAN - Quite NAN (Not a number)
        SpecialFPU_SNAN                 5       SNAN - Signaling NAN (Not a number)
        SpecialFPU_NegInf               6       Negative Infinite
        SpecialFPU_PosInf               7       Positive Infinite
        SpecialFPU_Indefinite           8       Indefinite
        SpecialFPU_SpecialIndefQNan     9       Special INDEFINITE QNAN
        SpecialFPU_SpecialIndefSNan     10      Special INDEFINITE SNAN
        SpecialFPU_SpecialIndefInfinite 11      Special INDEFINITE Infinite
__________________________________________________________________________

; Equates related to the function

[SpecialFPU_PosValid 0] ; The FPU contains a valid positive result
[SpecialFPU_NegValid 1] ; The FPU contains a valid negative result
[SpecialFPU_PosSubNormal 2] ; The FPU produced a positive Subnormal (denormalized) number. So, although it´ range is outside the range 3.6...e-4932, the number lost it´ precision, but it is still valid
[SpecialFPU_NegSubNormal 3] ; The FPU produced a negative Subnormal (denormalized) number. So, although it´ range is outside the range -3.6...e-4932, the number lost it´ precision, but it is still valid
[SpecialFPU_QNAN 4] ; QNAN
[SpecialFPU_SNAN 5] ; SNAN
[SpecialFPU_NegInf 6] ; Negative Infinite
[SpecialFPU_PosInf 7] ; Positive Infinite
[SpecialFPU_Indefinite 8] ; Indefinite
[SpecialFPU_SpecialIndefQNan 9] ; Special INDEFINITE QNAN
[SpecialFPU_SpecialIndefSNan 10] ; Special INDEFINITE SNAN
[SpecialFPU_SpecialIndefInfinite 11] ; Special INDEFINITE Infinite

; Updated in 12/02/2019
Proc RealTenFPUNumberCategory:
    Arguments @Float80Pointer
    Local @FPUErrorMode
    Uses edi, ebx


    mov ebx D@Float80Pointer
    mov D@FPUErrorMode SpecialFPU_PosValid

    ...If_And W$ebx+8 = 0, D$ebx+4 = 0 ; This is denormalized, but it is possible.
        ; 0000 00000000 00000000
        ; 0000 00000000 FFFFFFFF
        mov D@FPUErrorMode SpecialFPU_PosSubNormal
    ...Else_If_And W$ebx+8 = 0, D$ebx+4 > 0 ; This is Ok.
        ; 0000 00000001 00000000
        ; 0000 FFFFFFFF FFFFFFFF
        mov D@FPUErrorMode SpecialFPU_PosSubNormal
    ...Else_If_And W$ebx+8 > 0, W$ebx+8 < 07FFF; This is ok only if the fraction Dword is bigger or equal to 080000000
        .If D$ebx+4 < 080000000
            .Test_If D$ebx+4 040000000
                ; QNAN 40000000
                mov D@FPUErrorMode SpecialFPU_QNAN
            .Test_Else
                If_And D$ebx+4 > 0, D$ebx > 0
                    ; SNAN only if at least 1 bit is set
                    mov D@FPUErrorMode SpecialFPU_SNAN
                Else ; All fraction Bits are 0
                    ; Bit 15 is never reached. The bit is 0 from W$ebx+8
                    ; -INFINITE ; Bit15 = 0
                    mov D@FPUErrorMode SpecialFPU_NegInf
                End_If
            .Test_End
        .End_If
    ...Else_If W$ebx+8 = 07FFF; This is ok only if the fraction Dword is bigger or equal to 080000000
        .Test_If D$ebx+4 040000000
            ; QNAN 40000000
            mov D@FPUErrorMode SpecialFPU_QNAN
        .Test_Else
            If_And D$ebx+4 > 0, D$ebx > 0
                ; SNAN only if at least 1 bit is set
                mov D@FPUErrorMode SpecialFPU_SNAN
            Else ; All fraction Bits are 0
                ; Bit 15 is never reached. The bit is 0 from W$ebx+8
                ; -INFINITE ; Bit15 = 0
                mov D@FPUErrorMode SpecialFPU_NegInf
            End_If
        .Test_End
        ; Below is similar to W$ebx+8 = 0
    ...Else_If_And W$ebx+8 = 08000, D$ebx+4 = 0 ; This is denormalized, but possible.
        ; 8000 00000000 00000000 (0)
        ; 8000 00000000 FFFFFFFF (-0.0000000156560127730E-4933)
        mov D@FPUErrorMode SpecialFPU_NegSubNormal
    ...Else_If_And W$ebx+8 = 08000, D$ebx+4 > 0 ; This is Ok.
        ; 8000 01000000 00000000 (-0.2626643080556322880E-4933)
        ; 8000 FFFFFFFF 00000001 (-6.7242062846585856000E-4932)
        mov D@FPUErrorMode SpecialFPU_NegSubNormal
    ...Else_If_And W$ebx+8 > 08000, W$ebx+8 < 0FFFF; This is ok only if the fraction Dword is bigger or equal to 080000000
        .If D$ebx+4 < 080000000
            .Test_If D$ebx+4 040000000
                ; QNAN 40000000
                mov D@FPUErrorMode SpecialFPU_QNAN
            .Test_Else
                If_And D$ebx+4 > 0, D$ebx > 0
                    ; SNAN only if at least 1 bit is set
                    ;mov D$edi 'SNaN', B$edi+4 0
                    mov D@FPUErrorMode SpecialFPU_SNAN
                Else ; All fraction Bits are 0
                    ; Bit 15 is always reached. The bit is 1 from W$ebx+8
                    ; +INFINITE ; Bit15 = 1
                    ;mov D$edi '+INF', B$edi+4 0
                    mov D@FPUErrorMode SpecialFPU_PosInf
                End_If
            .Test_End
        .End_If

    ...Else_If W$ebx+8 = 0FFFF; This is to we identify indefined or other NAN values

        .If_And D$ebx+4 >= 040000000, D$ebx = 0
            ; INDEFINITE
            mov D@FPUErrorMode SpecialFPU_Indefinite
        .Else
            .Test_If D$ebx+4 040000000
                ; Special INDEFINITE QNAN 40000000
                mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan
            .Test_Else
                If_And D$ebx+4 > 0, D$ebx > 0
                    ; Special INDEFINITE SNAN only if at least 1 bit is set
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan
                Else ; All fraction Bits are 0
                    ; Bit 15 is always reached. The bit is 1 from W$ebx+8
                    ; Special INDEFINITE +INFINITE ; Bit15 = 1
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefInfinite
                End_If
            .Test_End
        .End_If
    ...End_If

    .If D@FPUErrorMode = SpecialFPU_PosValid
        Test_If B$ebx+9 0_80
            mov D@FPUErrorMode SpecialFPU_NegValid
        Test_End
    .End_If
    mov eax D@FPUErrorMode

EndP
Title: Re: Rounding Mode in FPU question
Post by: AW on February 14, 2019, 07:33:46 AM
In my computer when I load the 0x07ED13900F0000000000 from memory I get +1.5836591183212933e-4322
I don't see neither a NaN nor a denormalized number. I can see a denormalized number if I look at the number backwards.

BTW, since we are talking about backwards, coffee spelled backwards is eeffoc. Just know that I don't give eeffoc until I've had some coffee.
Title: Re: Rounding Mode in FPU question
Post by: K_F on February 14, 2019, 08:35:24 AM
A possible way of re-introducing the lost resolution is to run a 'random' generator at the 'resolution of lost bits', and then add that to the truncated Real4 (Real8).
A way of minimising FPU error propagation. ;)
Particular way of solving such type of problems, is to have ONE type for all calculations, I'm using this :
Code: [Select]
MACE TYPEDEF REAL10
True, Sometimes you have to convert down from Real10 ;)
Title: Re: Rounding Mode in FPU question
Post by: guga on February 14, 2019, 10:14:35 AM
AW, you are feeding an inverted order of the numbers, i presume. That´s why you are having different results from me and Raymond


I was talking about this sequence: db  0, 0, 0, 0, 0, 0F, 090, 013, 0ED, 07

The exponent and the sign are the last 2 bytes of the tenbyte.

I succeeded to convert the function to be ported to masm. Sorry for the lack os macros, i actually don´t remember the syntax in masm, but ported the whole functions if someone is interested in convert it to masm syntax more properly.

I´m not understanding why Raymond says it is denormalized, while on mine version and on ollydbg, this number shows as an -Infinite. What i´m doing wrong ?

Btw, the specifications of usage of the function is as follows:

Code: [Select]
    FloatToUString - Updated in 10/02/2019
   
    This function converts a FPU Number to decimal string (positive or negative) to be displayed on the debugger

    Parameters:
       
        Float80Pointer - A pointer to a variable containing a TenByte (80 bit) value to be converted to decimal string.
       
        DestinationPointer - A buffer to hold the converted string. The size of the buffer must be at least 32 bytes.
                             A maximum of 19 chars (including the dot) will be converted.
                             If the number cannot be converted, the buffer will contain a string representing the proper
                             category of the FPU error, such as: QNAN, SNAN, Infinite, Indefinite.
       
        TruncateBytes - The total amount of bytes to truncate. You can truncate a maximum of 3 numbers
                        on the end of a string. The truncation is to prevent rounding errors of some
                        compilers when tries to convert Floating Point Units.
                        For Terabytes we can discard the last 3 Bytes that are related to a diference in the error mode.
                        But, if you want to maintain accuracy, leave this parameter as 0.
       
        AddSubNormalMsg - A flag to enable append a warning message at the end of the number stored on the buffer at the DestinationPointer,
                          labeling it as a "(Bad)" number (positive or negative) meaning that the number is way too below the limit for
                          the FPU TenByte and is decreasing precision.
                           
                          To append the warning message, set this flag to &TRUE. Otherwise, set it to &FALSE.
                                                     
                          The 80-bit floating point format has a range (including subnormals) from approximately 3.65e-4951 to 1.18e4932.
                          Normal numbers from within the range 3.36210314311209208e-4932 to 1.18e4932) keeps their accuracy.
                          Numbers below that limit are called "SubNormal" (or denormalized) on a range from 3.65e-4951 to 3.362103...e-4932
                           
                          All subnormal numbers decreases their precision as they are going away from the limit of a normal number.
                          It have an approximated amount of 2^63 subnormal numbers that are way too close to zero and decreasing precision.
                           
                          The limit of a normal number is: 3.36210314311209208e-4932 (equivalent to declare it as: "FPULimit: D$ 0, 080000000, W$ 01")
                         
                         

    Return Values:

        The function will return one of the following equates:

        Equate                          Value   Description
       
        SpecialFPU_PosValid             0       The FPU contains a valid positive number.
        SpecialFPU_NegValid             1       The FPU contains a valid negative number.
        SpecialFPU_PosSubNormal         2       The FPU produced a positive Subnormal (denormalized) number.
                                                Although it´s range is outside the range 3.6...e-4932, the number lost it´ precision, but it is still valid
                                                Ex: 0000 00000000 00000000
                                                    0000 00000000 FFFFFFFF
                                                    0000 00000000 00008000
                                                    0000 00000001 00000000
                                                    0000 FFFFFFFF FFFFFFFF
        SpecialFPU_NegSubNormal         3       The FPU produced a negative Subnormal (denormalized) number.
                                                Although it´s range is outside the range -3.6...e-4932, the number lost it´ precision, but it is still valid
                                                Ex: 8000 00000000 00000000 (0) (Negative zero must be considered only as zero)
                                                    8000 00000000 FFFFFFFF (-0.0000000156560127730E-4933)
                                                    8000 01000000 00000000 (-0.2626643080556322880E-4933)
                                                    8000 FFFFFFFF 00000001 (-6.7242062846585856000E-4932)
        SpecialFPU_QNAN                 4       QNAN - Quite NAN (Not a number)
        SpecialFPU_SNAN                 5       SNAN - Signaling NAN (Not a number)
        SpecialFPU_NegInf               6       Negative Infinite
        SpecialFPU_PosInf               7       Positive Infinite
        SpecialFPU_Indefinite           8       Indefinite
        SpecialFPU_SpecialIndefQNan     9       Special INDEFINITE QNAN
        SpecialFPU_SpecialIndefSNan     10      Special INDEFINITE SNAN
        SpecialFPU_SpecialIndefInfinite 11      Special INDEFINITE Infinite

Title: Re: Rounding Mode in FPU question
Post by: jj2007 on February 14, 2019, 12:15:43 PM
For exploring the mysteries of REAL10 notation 8)

picked=5
ife picked
  NaN   REAL10 0x07ED13900F0000000000
elseif picked eq 1
  NaN   REAL10 0x00000000000F9013ED07
elseif picked eq 2
  NaN   db 07, 0EDh, 13h, 90h, 0Fh, 00, 00, 00, 00, 00
elseif picked eq 3
  NaN   db 0, 0, 1, 0, 0, 0, 0, 0, 0, 0
elseif picked eq 4
  NaN   REAL10 0x00000000000000000001
elseif picked eq 5
  NaN   db 1, 0, 0, 0, 0, 0, 0, 0, 0, 0
elseif picked eq 6
  NaN   db 00, 00h, 00h, 00h, 00h, 00h, 00, 80h, 00, 00
endif
...
  int 3
  fld REAL10 ptr NaN
  fld FP10(1.0e4900)
  fmul


picked 5 produces 3.6452e-51 after multiplying with 1.0e4900 - remember what Wiki says about the range?

Specific Watcom mysteries: I expected picked 4 & 5 to be identical. What exactly does the REAL10 0x111 notation mean?
Title: Re: Rounding Mode in FPU question
Post by: raymond on February 14, 2019, 12:43:15 PM
Hi Raymond, not sure if i´m fully undertanding this.

From what i understood on the documentation, the Word 07ED (last 2 bytes from the TenByte) represents the sign bit (79) and the exponent, right ? And so, they can´t be denormalized since the last Word (of the tenbyte) is not zero and the second Dword also is not zero. Demnormalized numbers are the ones where the second Dword of the tenbyte is zero and the last word is either zero or 08000, right ?

Maybe 'right' or 'wrong' depending on what you happen to consider the last 2 bytes. You would be right if they are the last 2 bytes in memory; wrong if you consider the 80 bits as a whole number.

I think we are getting confused with the actual location of the 10 bytes. The most significant byte of a TBYTE would have the sign bit followed by the 7 most significant bits of the biased exponent. The next byte would have the remaining 8 bits of the 15-bit biased exponent.

In the case we are discussing, the most significant byte appears to be either 7 or ED (the 7 indicating a positive number or the ED indicating a negative number), with the other byte being the remaining bits of the exponent depending on how your display of W$ 07ED is interpreted. Either way, such could never be considered as a NAN.
Title: Re: Rounding Mode in FPU question
Post by: raymond on February 14, 2019, 02:44:29 PM
I may have solved your mystery. If you look closely at the description of a REAL10, you will notice the following:

As opposed to the REAL4 and REAL8 formats, the first bit of the number is explicitly included in the significand field and followed by the fraction bits f1, f2, etc.

Therefore, if bit #63 of a REAL10 in memory is not set to a 1, it would not be considered as a valid extended precision "float". I would neither have the format of a NAN if the exponent bits are not all 1s, nor of a valid denormalized number if the exponent bits are not all 0s. The FPU would simply refuse to process it.

Such could happen easily if you try using a float from memory and referring to it as a REAL10 when it is not.
Title: Re: Rounding Mode in FPU question
Post by: hutch-- on February 14, 2019, 03:55:07 PM
 :biggrin:

> coffee spelled backwards is eeffoc.

I wonder if this is 00EEFF0Ch  :P
Title: Re: Rounding Mode in FPU question
Post by: AW on February 14, 2019, 06:50:05 PM
AW, you are feeding an inverted order of the numbers, i presume. That´s why you are having different results from me and Raymond
I was talking about this sequence: db  0, 0, 0, 0, 0, 0F, 090, 013, 0ED, 07
I am talking about the same thing as you are.
0x07ED13900F0000000000 is laid out in memory exactly like that in all little endian systems.
The number can't be a NaN because the exponent is not all ones. It can't be subnormal because the exponent is not all zeros. Is that difficult?
Title: Re: Rounding Mode in FPU question
Post by: jj2007 on February 14, 2019, 08:09:54 PM
0x07ED13900F0000000000 is laid out in memory exactly like that in all little endian systems.
Code: [Select]
include \masm32\MasmBasic\MasmBasic.inc ; download
align 16
R10a REAL10 0x11223344556677889900
db 0AAh, 0BBh, 0CCh, 0DDh, 0EEh, 0FFh ; fill with AA BB CC DD EE FF
R10b db 00h, 99h, 88h, 77h, 66h, 55h, 44h, 33h, 22h, 11h
db 0AAh, 0BBh, 0CCh, 0DDh, 0EEh, 0FFh ; fill with AA BB CC DD EE FF
R8a REAL8 0x1122334455667788
REAL8 0
R8b db 88h, 77h, 66h, 55h, 44h, 33h, 22h, 11h
REAL8 0

  Init
  Inkey HexDump$(offset R10a, 64, notext)
EndOfCode

Result:
Code: [Select]
00000000  00 00 00 00 00 32 11 EF 1D 40 AA BB CC DD EE FF
00000010  00 99 88 77 66 55 44 33 22 11 AA BB CC DD EE FF
00000020  00 00 00 E0 9D 59 D5 41 00 00 00 00 00 00 00 00
00000030  88 77 66 55 44 33 22 11 00 00 00 00 00 00 00 00
Title: Re: Rounding Mode in FPU question
Post by: HSE on February 15, 2019, 01:44:50 AM
It's in this way :
Code: [Select]
include \masm32\MasmBasic\MasmBasic.inc ; download
align 16
R10a REAL10 11223344556677889900r
db 0AAh, 0BBh, 0CCh, 0DDh, 0EEh, 0FFh ; fill with AA BB CC DD EE FF
R10b db 00h, 99h, 88h, 77h, 66h, 55h, 44h, 33h, 22h, 11h
db 0AAh, 0BBh, 0CCh, 0DDh, 0EEh, 0FFh ; fill with AA BB CC DD EE FF
R8a REAL8 1122334455667788r
REAL8 0.0
R8b db 88h, 77h, 66h, 55h, 44h, 33h, 22h, 11h
REAL8 0.0
NaN REAL8  7FF8000000000000r     ;>> I use a lot this

  Init
  Inkey HexDump$(offset R10a, 64+8, notext)
EndOfCode

Result:
Code: [Select]
00000000  00 99 88 77 66 55 44 33 22 11 AA BB CC DD EE FF
00000010  00 99 88 77 66 55 44 33 22 11 AA BB CC DD EE FF
00000020  88 77 66 55 44 33 22 11 00 00 00 00 00 00 00 00
00000030  88 77 66 55 44 33 22 11 00 00 00 00 00 00 00 00
00000040  00 00 00 00 00 00 F8 7F
Title: Re: Rounding Mode in FPU question
Post by: jj2007 on February 15, 2019, 03:07:50 AM
It's in this way

Yep, the "r" thing, thanks for reminding me. Qword used this notation years ago.
Title: Re: Rounding Mode in FPU question
Post by: guga on February 16, 2019, 01:13:38 AM
What about the ranges for a TenByte be considered a NAN/SNAN, infinite etc, what are they ?

According to this site https://www.doc.ic.ac.uk/~eedwards/compsys/float/nan.html , the ranges (In little endian) for a Real4 and Real8 are:

Quote
For single-precision values:

Positive infinity is represented by the bit pattern 7F800000
Negative infinity is represented by the bit pattern FF800000

A signalling NaN (NANS) is represented by any bit pattern
between 7F800001 and 7FBFFFFF or between FF800001 and FFBFFFFF

A quiet NaN (NANQ) is represented by any bit pattern
between 7FC00000 and 7FFFFFFF or between FFC00000 and FFFFFFFF

For double-precision values:

Positive infinity is represented by the bit pattern 7FF0000000000000
Negative infinity is represented by the bit pattern FFF0000000000000

A signalling NaN is represented by any bit pattern

between 7FF0000000000001 and 7FF7FFFFFFFFFFFF or
between FFF0000000000001 and FFF7FFFFFFFFFFFF

A quiet NaN is represented by any bit pattern

between 7FF8000000000000 and 7FFFFFFFFFFFFFFF or
between FFF8000000000000 and FFFFFFFFFFFFFFFF


So, from Raymond´s specifications is it correct assume the ranges for a TenByte be like this ? IF not, what are the ranges for a TenByte be considered, NAN, QNAN, Infinite, etc ?


Quote
For extended precision

bias  = 03FFF; Max = 07FFF . Exponent = 07FFF-Bias-1 ? . Range from NAN etc starts from 07FFF to 0FFFF ?

Positive infinity is represented by the bit pattern 7FFF_00000000_00000000
Negative infinity is represented by the bit pattern FFFF_00000000_00000000

A signalling NaN is represented by any bit pattern

between 7FFF_00000000_00000001 and 7FF7_FFFFFFFF_FFFFFFFF or
between FFFF_00000000_00000001 and FFF7_FFFFFFFF_FFFFFFFF

A quiet NaN is represented by any bit pattern

between 7FFF_80000000_00000000 and 7FFF_FFFFFFFF_FFFFFFFF or
between FFFF_80000000_00000000 and FFFF_FFFFFFFF_FFFFFFFF

Note:
Assuming a TenByte to be viewed on the format of a structure we should have:
[TenByteFormat:
 TenByteFormat.Fraction1: D$ 0
 TenByteFormat.Fraction2: D$ 0 ; It must contains a minimum of 080000000 to be considered a valid number ?
 TenByteFormat.Exponent: W$ 0] ; including the sign on the last bit

Since for a TenByte the 31th bit of the second dword must contains a "1" (080000000) So, all of the 2nd dword should contains at least 080000000 but...what if it don´t have this "1" value as a part of the number ?

Title: Re: Rounding Mode in FPU question
Post by: raymond on February 16, 2019, 06:01:36 AM
According to the data which I had gathered at the time, I would have the following for the REAL10 format:

Positive INFINITY is represented by the bit pattern 7FFF_80000000_00000000
Negative INFINITY is represented by the bit pattern FFFF_80000000_00000000

INDEFINITE is represented by the bit pattern FFFF_C0000000_00000000

A Quiet NaN would be represented by any bit pattern
between 7FFF_C0000000_00000001 and 7FFF_FFFFFFFF_FFFFFFFF or
between FFFF_C0000000_00000001 and FFFF_FFFFFFFF_FFFFFFFF

A Signaling NaN would be represented by any bit pattern
between 7FFF_80000000_00000001 and 7FFF_BFFFFFFF_FFFFFFFF or
between FFFF_80000000_00000001 and FFFF_BFFFFFFF_FFFFFFFF

The explicit 1 in bit 63 always remains set for the REAL10 format, including all the NANs. Otherwise, the FPU seems to reject it as an unknown format.
Title: Re: Rounding Mode in FPU question
Post by: guga on February 16, 2019, 09:40:34 AM
Thank you so much, Raymond. That´s what i was looking for.  :t :t :t
Title: Re: Rounding Mode in FPU question
Post by: raymond on February 16, 2019, 12:02:00 PM
You're welcome. Best wishes for your project.

One minor clarification to my last statement.
Quote
The explicit 1 in bit 63 always remains set for the REAL10 format
The one exception is when all exponent and fraction bits are 0's, defining the value of +/-0 depending on the sign bit.
Title: Re: Rounding Mode in FPU question
Post by: daydreamer on February 16, 2019, 08:52:15 PM
According to the data which I had gathered at the time, I would have the following for the REAL10 format:

Positive INFINITY is represented by the bit pattern 7FFF_80000000_00000000
Negative INFINITY is represented by the bit pattern FFFF_80000000_00000000

thanks raymond,its good to know infinitys so I can choose
;x/y

.IF y!=0
fld x
fdiv y
.ELSE
fld infinity ;humans thinks x/divbyzero=infinity
or
fldz ;humans think 0*0=0 as div by zero
depending on what works best in the formula you code

Title: Re: Rounding Mode in FPU question
Post by: guga on February 17, 2019, 12:04:09 AM
You're welcome. Best wishes for your project.

One minor clarification to my last statement.
Quote
The explicit 1 in bit 63 always remains set for the REAL10 format
The one exception is when all exponent and fraction bits are 0's, defining the value of +/-0 depending on the sign bit.

Thanks  :t :t :t :t  :)


About the integer being set i did that and the cases of zero, i did those as you described :)  For negative zeros i put those on the same category as zero, since there´s no such a thing as a -0

I just added 4 new categories on the rare situations where the integer bit is not present on the TenByte (rare, but happens when you analyse some old libraries from a disassembler) resulting in an error on the FPU computations that will reject it.

[SpecialFPU_SpecialIndefQNan 10] ; Special INDEFINITE QNAN . Same as QNAN, but without the integer bit set
[SpecialFPU_SpecialIndefSNan 11] ; Special INDEFINITE SNAN. Same as SNAN, but without the integer bit set
[SpecialFPU_SpecialIndefNegInfinite 12] ; Special INDEFINITE Negative Infinite. Same as Negative Infinite, but without the integer bit set
[SpecialFPU_SpecialIndefPosInfinite 13] ; Special INDEFINITE Positive Infinite. Same as Positive Infinite, but without the integer bit set

For the disassembler and debugger purposes, i added those just to better categorize the error type, rather then throw an Unknown Format flag.

The updated code looks like this:

Code: [Select]

[SpecialFPU_PosValid 0] ; The FPU contains a valid positive result
[SpecialFPU_NegValid 1] ; The FPU contains a valid negative result
[SpecialFPU_PosSubNormal 2] ; The FPU produced a positive Subnormal (denormalized) number. So, although it´ range is outside the range 3.6...e-4932, the number lost it´ precision, but it is still valid
[SpecialFPU_NegSubNormal 3] ; The FPU produced a negative Subnormal (denormalized) number. So, although it´ range is outside the range -3.6...e-4932, the number lost it´ precision, but it is still valid
[SpecialFPU_Zero 4] ; The FPU contains a valid zero number
[SpecialFPU_QNAN 5] ; QNAN
[SpecialFPU_SNAN 6] ; SNAN
[SpecialFPU_NegInf 7] ; Negative Infinite
[SpecialFPU_PosInf 8] ; Positive Infinite
[SpecialFPU_Indefinite 9] ; Indefinite
[SpecialFPU_SpecialIndefQNan 10] ; Special INDEFINITE QNAN
[SpecialFPU_SpecialIndefSNan 11] ; Special INDEFINITE SNAN
[SpecialFPU_SpecialIndefNegInfinite 12] ; Special INDEFINITE Negative Infinite
[SpecialFPU_SpecialIndefPosInfinite 13] ; Special INDEFINITE Positive Infinite

Proc RealTenFPUNumberCategory:
    Arguments @Float80Pointer
    Local @FPUErrorMode, @TenByteErr
    Uses ebx


    mov ebx D@Float80Pointer
    mov D@FPUErrorMode SpecialFPU_PosValid

    ; 1st located all zero numbers (no bits settled on the whole TenByte or, only the Sign bit (15th of the last word = 79th bit of the Tenbyte ) was settled.
    ; Ex: D$ 0, 0, W$ 0 ; (0)
    ; Ex: D$ 0, 0, W$ 08000 ; negative zero is only zero
    .If_and D$ebx = 0, D$ebx+4 = 0
        ; ebx+8 = 08000 means a negative zero. But, there´s no such a thing as a negative zero, thus, it should be considered only 0
        If_Or W$ebx+8 = 0, W$ebx+8 = 08000
            mov eax SpecialFPU_Zero ; eax wil exist as Zero
            ExitP
        End_If
    .End_If

    ; Check if the TenByte is valid. Check for the integer bit (bit 63 of the tenbyte = bit 31 of the 2nd dword)
    mov D@TenByteErr &FALSE
    Test_If_Not B$ebx+7 080 ; See if the integer bit is present on the tenbyte. No integer bit ? Flag it as an error
        mov D@TenByteErr &TRUE
    Test_End


    ; Possible NANs contains always the 1st 2 bits of the 2nd word settled and the last 2 bit settled (pg 91/100 on Intel Manual)
    ; The biased exponent is on this form for NAN 11..11

     ; 2nd located all denormalized, but possible positive numbers.
     ; Ex: D$ 0FFFFFFFF, 0, W$ 0         ; (1.56560127731845315e-4941)
     ; Ex: D$ 0, 01, W$ 0                ; (1.56560127768297311e-4941)
     ; Ex: D$ 0FFFFFFFF, 0FFFFFFFF, W$ 0 ; (6.72420628622418417e-4932)
                                         ; (6.7242062862241870120e-4932) (Olly)

    ...If_And W$ebx+8 = 0, D$ebx+4 >= 0, D@TenByteErr = &FALSE ; No errors in the tenbyte. It also do contains the integer bit

        mov D@FPUErrorMode SpecialFPU_PosSubNormal

    ...Else_If_And W$ebx+8 = 08000, D$ebx+4 >= 0, D@TenByteErr = &FALSE ; No errors in the tenbyte. It also do contains the integer bit

    ; 3rd located all denormalized, but possible negative numbers. Bit 15th of the last word (Bit79 of the tenbyte) is settled
    ; Ex: D$ 0FFFFFFFF, 0, W$ 08000 ; (-1.56560127731845315e-4941)
    ; Ex: D$ 0, 01000000, W$ 08000  ; (-2.626643080565632194e-4934)  in olly dbg = (-0.2626643080556323050e-4933)
    ; Ex: D$ 01, 0FFFFFFFF, W$ 08000 ; (-6.72420628465858289e-4932)  in olly dbg = (-6.7242062846585857350e-4932)

        mov D@FPUErrorMode SpecialFPU_NegSubNormal

    ...Else_If W$ebx+8 = 07FFF ; Locate all positive infinite, QNAN, Special indefinite 00__0111_1111__1111_1111
    ; 00__0110_0000__0000_0011 06003 ; error happen here
    ; 00__0111_1111__1111_1111 07FFF ; error happen here

    ; 00__0111_1111__1111_1110 07FFE ; ok normal
    ; 00__0110_0000__0000_0010 06002 ; ok normal
    ; 00__0000_0000__0000_0001 01 ; ok normal

        ; locate all: Positive Infinite (Bit 15 of the last word is not set), the second dword are zero and the 1st dword contains only the integer bit 00__1000_0000__0000_0000
        ; Ex: D$ 0, 0, W$ 07FFF  ; 07FFF 00000000 00000000 when bit15 is not set it is positive 00__0111_1111__1111_1111

        .If_And D$ebx+4 = 080000000, D$ebx = 0; 2nd dword = 00__1000_0000__0000_0000__0000_0000__0000_0000 1st dword = 0

            mov D@FPUErrorMode SpecialFPU_PosInf

        .Else_If_And D$ebx+4 >= 0C0000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_QNAN

        .Else_If_And D$ebx+4 >= 080000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_SNAN

        .Else_If_And D$ebx+4 >= 040000000, D$ebx >= 01
            ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
            mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

        .Else_If D$ebx >= 01
            ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

            mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

        .Else
            ; all remaining cases, result only in D$ebx = 0. The lack of the 63th bit on this case could represente also a Indefinite Positive, but we are here labeling it as Indefinite infinite to be more
            ; logical with the Indefinite category
            mov D@FPUErrorMode SpecialFPU_SpecialIndefPosInfinite

        .End_If

    ...Else_If W$ebx+8 = 0FFFF ; Locate all negative infinite, QNAN, indefinite

        .If_And D$ebx+4 = 080000000, D$ebx = 0; 2nd dword = 00__1000_0000__0000_0000__0000_0000__0000_0000 1st dword = 0

            mov D@FPUErrorMode SpecialFPU_NegInf

        .Else_If_And D$ebx+4 >= 0C0000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_QNAN

        .Else_If_And D$ebx+4 >= 080000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_SNAN

        .Else_If_And D$ebx+4 = 0C0000000, D$ebx = 0

            mov D@FPUErrorMode SpecialFPU_Indefinite

        .Else_If_And D$ebx+4 >= 040000000, D$ebx >= 01
            ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
            mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

        .Else_If D$ebx >= 01
            ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

            mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

        .Else
            ; all remaining cases, result only in D$ebx = 0.

            mov D@FPUErrorMode SpecialFPU_SpecialIndefNegInfinite

        .End_If

    ...Else

        ; Now we must analyse the cases where the integer Bit (Bit 63 of tenbyte or 31th of the 2nd dword) is not settled by some error
        ; and the FPU will simply refuse to process
        ; Ex: [NumTest18: D$ 0, 013900F00, W$ 07ED]
        ..If D$ebx+4 < 080000000 ; Integer bit is never settled

            .If_And D$ebx+4 >= 040000000, D$ebx >= 01
                ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
                mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

            .Else_If D$ebx >= 01
                ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

                mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

            .Else
                ; all remaining cases, result only in D$ebx = 0 and we must only check if it is positive or negative
                ; Sign bit is settled
                Test_If B$ebx+9 0_80
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefNegInfinite
                Test_Else
                    ; Sign bit is never settled
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefPosInfinite
                Test_End
            .End_If
        ..End_If

    ...End_If

    .If D@FPUErrorMode = SpecialFPU_PosValid
        Test_If B$ebx+9 0_80
            mov D@FPUErrorMode SpecialFPU_NegValid
        Test_End
    .End_If

    mov eax D@FPUErrorMode

EndP

Title: Re: Rounding Mode in FPU question
Post by: guga on February 17, 2019, 09:17:15 AM
For exploring the mysteries of REAL10 notation 8)

picked=5
ife picked
  NaN   REAL10 0x07ED13900F0000000000
elseif picked eq 1
  NaN   REAL10 0x00000000000F9013ED07
elseif picked eq 2
  NaN   db 07, 0EDh, 13h, 90h, 0Fh, 00, 00, 00, 00, 00
elseif picked eq 3
  NaN   db 0, 0, 1, 0, 0, 0, 0, 0, 0, 0
elseif picked eq 4
  NaN   REAL10 0x00000000000000000001
elseif picked eq 5
  NaN   db 1, 0, 0, 0, 0, 0, 0, 0, 0, 0
elseif picked eq 6
  NaN   db 00, 00h, 00h, 00h, 00h, 00h, 00, 80h, 00, 00
endif
...
  int 3
  fld REAL10 ptr NaN
  fld FP10(1.0e4900)
  fmul


picked 5 produces 3.6452e-51 after multiplying with 1.0e4900 - remember what Wiki says about the range?

Specific Watcom mysteries: I expected picked 4 & 5 to be identical. What exactly does the REAL10 0x111 notation mean?

Hi JJ

from my new routines, i identify this as:

REAL10 0x111 ; If this is the order [JJReal10: D$ 0111 D$ 0 W$ 0] =  9.95139472203915145e-4949 denormal, but it lacks the integer bit. ?
REAL10 0x111 ; Or, if the order is this: [JJReal10: D$ 0 D$ 0 W$ 0111] = Special INDEFINITE Positive Infinite. It lacks the integer bit, but the category is similar to a Positive Infinite (if that bit was settled.)
Real10: D$ 0, D$ 00, W$ 01] ; 0x000000000000000001 = Special INDEFINITE Positive Infinite. It lacks the integer bit, but the category is similar to a Positive Infinite (if that bit was settled.)
[JJ3Real10: D$ 01, D$ 00, W$ 0] ; 0x100000000000000000 = Special INDEFINITE SNAN meaning, something that should looks like a SNAN, but it lacks the integer bit.

In all 4 cases, the FPU should not be able to process this, due to the lack of the integer bit, but it is performing the multiplication even thought the bit is not settled.

About this:

  fld REAL10 ptr NaN
  fld FP10(1.0e4900)
  fmul

Here is resulting in a Indefinite [JJReal410: D$ 0, D$ 0 W$ 0111]*1e4900 ; [JJReal410: D$ 0, D$ 0 W$ 0111] = Special Indefinite Positive infinite

or if the order is the reversed one [JJReal10: D$ 0111 D$ 0 W$ 0]  9.95139472203915145e-4949 (denormal)

  fld REAL10 ptr NaN
  fld FP10(1.0e4900)
  fmul

It then results on a 9.95139472203915133e-49

That´s weird. Maybe for a Subnormal, the number may still be valid even if the integer bit is not set ? From page 91 in Intel Manual the integer bit (For a subnormal) is set to zero. I´ll confirme that...hold on :)

Btw. Many tks for the help, JJ  :t :t

Title: Re: Rounding Mode in FPU question
Post by: guga on February 17, 2019, 10:21:27 AM
Hi JJ.. just confirmed the last post.

Denormal values does not have the integer bit set. So

[JJReal10: D$ 0111 D$ 0 W$ 0] = 9.95139472203915145e-4949

when multiply by 1e4900 results in:  9.95139472203915145e-49

Updated the function again to detect properly the denormal. Thanks a lot


Code: [Select]

[SpecialFPU_PosValid 0] ; The FPU contains a valid positive result
[SpecialFPU_NegValid 1] ; The FPU contains a valid negative result
[SpecialFPU_PosSubNormal 2] ; The FPU produced a positive Subnormal (denormalized) number. So, although it´ range is outside the range 3.6...e-4932, the number lost it´ precision, but it is still valid
[SpecialFPU_NegSubNormal 3] ; The FPU produced a negative Subnormal (denormalized) number. So, although it´ range is outside the range -3.6...e-4932, the number lost it´ precision, but it is still valid
[SpecialFPU_Zero 4] ; The FPU contains a valid zero number
[SpecialFPU_QNAN 5] ; QNAN
[SpecialFPU_SNAN 6] ; SNAN
[SpecialFPU_NegInf 7] ; Negative Infinite
[SpecialFPU_PosInf 8] ; Positive Infinite
[SpecialFPU_Indefinite 9] ; Indefinite
[SpecialFPU_SpecialIndefQNan 10] ; Special INDEFINITE QNAN
[SpecialFPU_SpecialIndefSNan 11] ; Special INDEFINITE SNAN
[SpecialFPU_SpecialIndefNegInfinite 12] ; Special INDEFINITE Negative Infinite
[SpecialFPU_SpecialIndefPosInfinite 13] ; Special INDEFINITE Positive Infinite

Proc RealTenFPUNumberCategory:
    Arguments @Float80Pointer
    Local @FPUErrorMode
    Uses ebx


    mov ebx D@Float80Pointer
    mov D@FPUErrorMode SpecialFPU_PosValid

    ; 1st located all zero numbers (no bits settled on the whole TenByte or, only the Sign bit (15th of the last word = 79th bit of the Tenbyte ) was settled.
    ; Ex: D$ 0, 0, W$ 0 ; (0)
    ; Ex: D$ 0, 0, W$ 08000 ; negative zero is only zero
    .If_and D$ebx = 0, D$ebx+4 = 0
        ; ebx+8 = 08000 means a negative zero. But, there´s no such a thing as a negative zero, thus, it should be considered only 0
        If_Or W$ebx+8 = 0, W$ebx+8 = 08000
            mov eax SpecialFPU_Zero ; eax will exit as Zero FPU category
            ExitP
        End_If
    .End_If



    ; Possible NANs contains always the 1st 2 bits of the 2nd word settled and the last 2 bit settled (pg 91/100 on Intel Manual)
    ; The biased exponent is on this form for NAN 11..11

     ; 2nd located all denormalized, but possible positive numbers.
     ; Ex: D$ 0FFFFFFFF, 0, W$ 0         ; (1.56560127731845315e-4941)
     ; Ex: D$ 0, 01, W$ 0                ; (1.56560127768297311e-4941)
     ; Ex: D$ 0FFFFFFFF, 0FFFFFFFF, W$ 0 ; (6.72420628622418417e-4932)
                                         ; (6.7242062862241870120e-4932) (Olly)

    ...If_And W$ebx+8 = 0, D$ebx+4 >= 0 ; On denormal (Subnormal) values, the integer bit is not set

        mov D@FPUErrorMode SpecialFPU_PosSubNormal

    ...Else_If_And W$ebx+8 = 08000, D$ebx+4 >= 0 ; On denormal (Subnormal) values, the integer bit is not set

    ; 3rd located all denormalized, but possible negative numbers. Bit 15th of the last word (Bit79 of the tenbyte) is settled
    ; Ex: D$ 0FFFFFFFF, 0, W$ 08000 ; (-1.56560127731845315e-4941)
    ; Ex: D$ 0, 01000000, W$ 08000  ; (-2.626643080565632194e-4934)  in olly dbg = (-0.2626643080556323050e-4933)
    ; Ex: D$ 01, 0FFFFFFFF, W$ 08000 ; (-6.72420628465858289e-4932)  in olly dbg = (-6.7242062846585857350e-4932)

        mov D@FPUErrorMode SpecialFPU_NegSubNormal

    ...Else_If W$ebx+8 = 07FFF ; Locate all positive infinite, QNAN, Special indefinite 00__0111_1111__1111_1111
    ; 00__0110_0000__0000_0011 06003 ; error happen here
    ; 00__0111_1111__1111_1111 07FFF ; error happen here

    ; 00__0111_1111__1111_1110 07FFE ; ok normal
    ; 00__0110_0000__0000_0010 06002 ; ok normal
    ; 00__0000_0000__0000_0001 01 ; ok normal

        ; locate all: Positive Infinite (Bit 15 of the last word is not set), the 1st dword are zero and the 2nd dword contains only the integer bit 00__1000_0000__0000_0000
        ; Ex: D$ 0, 0, W$ 07FFF  ; 07FFF 00000000 00000000 when bit15 is not set it is positive 00__0111_1111__1111_1111

        .If_And D$ebx+4 = 080000000, D$ebx = 0; 2nd dword = 00__1000_0000__0000_0000__0000_0000__0000_0000 1st dword = 0

            mov D@FPUErrorMode SpecialFPU_PosInf

        .Else_If_And D$ebx+4 >= 0C0000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_QNAN

        .Else_If_And D$ebx+4 >= 080000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_SNAN

        .Else_If_And D$ebx+4 >= 040000000, D$ebx >= 01
            ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
            mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

        .Else_If D$ebx >= 01
            ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

            mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

        .Else
            ; all remaining cases, result only in D$ebx = 0. The lack of the 63th bit on this case could represent also a Indefinite Positive, identified here as a Special Indefinite Positive since it is similar to the Indefinite Positive (But, without the integer bit set)
            mov D@FPUErrorMode SpecialFPU_SpecialIndefPosInfinite

        .End_If

    ...Else_If W$ebx+8 = 0FFFF ; Locate all negative infinite, QNAN, indefinite

        .If_And D$ebx+4 = 080000000, D$ebx = 0; 2nd dword = 00__1000_0000__0000_0000__0000_0000__0000_0000 1st dword = 0

            mov D@FPUErrorMode SpecialFPU_NegInf

        .Else_If_And D$ebx+4 >= 0C0000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_QNAN

        .Else_If_And D$ebx+4 >= 080000000, D$ebx >= 01

            mov D@FPUErrorMode SpecialFPU_SNAN

        .Else_If_And D$ebx+4 = 0C0000000, D$ebx = 0

            mov D@FPUErrorMode SpecialFPU_Indefinite

        .Else_If_And D$ebx+4 >= 040000000, D$ebx >= 01
            ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
            mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

        .Else_If D$ebx >= 01
            ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

            mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

        .Else
            ; all remaining cases, result only in D$ebx = 0.

            mov D@FPUErrorMode SpecialFPU_SpecialIndefNegInfinite

        .End_If

    ...Else

        ; Now we must analyse the cases where the integer Bit (Bit 63 of tenbyte or 31th of the 2nd dword) is not settled by some error
        ; and the FPU will simply refuse to process
        ; Ex: [NumTest18: D$ 0, 013900F00, W$ 07ED]
        ..If D$ebx+4 < 080000000 ; Integer bit is never settled

            .If_And D$ebx+4 >= 040000000, D$ebx >= 01
                ; 00__0100_0000__0000_0000__0000_0000__0000_0000 If the compiler made an error and didn´t inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)
                mov D@FPUErrorMode SpecialFPU_SpecialIndefQNan

            .Else_If D$ebx >= 01
                ; 2nd Dword = 0, and at least 1 bit settled on the 1st dword. If the compiler made an error and didn´ inserted the integer bit (bit 63 of the tenbyte or 31 of the 2nd dword)

                mov D@FPUErrorMode SpecialFPU_SpecialIndefSNan

            .Else
                ; all remaining cases, result only in D$ebx = 0 and we must only check if it is positive or negative
                ; Sign bit is settled
                Test_If B$ebx+9 0_80
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefNegInfinite
                Test_Else
                    ; Sign bit is never settled
                    mov D@FPUErrorMode SpecialFPU_SpecialIndefPosInfinite
                Test_End
            .End_If
        ..End_If

    ...End_If

    .If D@FPUErrorMode = SpecialFPU_PosValid
        Test_If B$ebx+9 0_80
            mov D@FPUErrorMode SpecialFPU_NegValid
        Test_End
    .End_If

    mov eax D@FPUErrorMode

EndP

Title: Re: Rounding Mode in FPU question
Post by: jj2007 on February 17, 2019, 02:46:45 PM
Many tks for the help, JJ  :t :t

When it comes to fpu stuff, Raymond is the best source of advice, as usual :t