Author Topic: Small favour needed, test precision with sprintf.  (Read 7985 times)

HSE

  • Member
  • *****
  • Posts: 1377
  • <AMD>< 7-32>
Re: Small favour needed, test precision with sprintf.
« Reply #15 on: September 06, 2018, 06:25:34 AM »
What do you mean exactly?

If code that work with ML don't work with AsmC,
   then AsmC is not compatible with ML

jj2007

  • Member
  • *****
  • Posts: 10543
  • Assembler is fun ;-)
    • MasmBasic
Re: Small favour needed, test precision with sprintf.
« Reply #16 on: September 06, 2018, 06:28:08 AM »
Code: [Select]
include \masm32\MasmBasic\MasmBasic.inc ; download
.data
MyR10 REAL10 1234567890.1234567890123

  Init
  Inkey Str$("This is the highest resolution possible without a bignum library:\n%Jf", MyR10)
EndOfCode

This is the highest resolution possible without a bignum library:
1234567890.123456789

mineiro

  • Member
  • ****
  • Posts: 615
Re: Small favour needed, test precision with sprintf.
« Reply #17 on: September 06, 2018, 07:59:24 AM »
123456.1234567890060134
Press any key to continue...
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

nidud

  • Member
  • *****
  • Posts: 1980
    • https://github.com/nidud/asmc
Re: Small favour needed, test precision with sprintf.
« Reply #18 on: September 06, 2018, 08:24:55 AM »
What do you mean exactly?

If code that work with ML don't work with AsmC,
   then AsmC is not compatible with ML

That seems to be correct. New version uploaded.

Test case:
Code: [Select]
;
; v2.28.05 -- HSE
;
    .x64
    .model flat

    .data
    m64 label qword

    .code

    andpd   xmm0,m64
    andnpd  xmm0,m64
    xorpd   xmm0,m64
    orpd    xmm0,m64

    end

HSE

  • Member
  • *****
  • Posts: 1377
  • <AMD>< 7-32>
Re: Small favour needed, test precision with sprintf.
« Reply #19 on: September 06, 2018, 08:47:13 AM »
New version uploaded.

Thanks Nidud  :t

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 7537
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: Small favour needed, test precision with sprintf.
« Reply #20 on: September 06, 2018, 12:38:24 PM »
I think I know what Marinus was trying to do, split the input data between the lead integer and the decimal fraction and while you cannot extract more out of 64 bit maths in terms of precision, split into 2 components you may be able to handle each one separately to get a 64 x 64 precision level.

ALSO : Thanks to those folks who fed back the results on the sprints test piece, it means sprintf is safe to use from XP64 upwards.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :skrewy:

Siekmanski

  • Member
  • *****
  • Posts: 2326
Re: Small favour needed, test precision with sprintf.
« Reply #21 on: September 06, 2018, 02:31:19 PM »
I think I know what Marinus was trying to do, split the input data between the lead integer and the decimal fraction and while you cannot extract more out of 64 bit maths in terms of precision, split into 2 components you may be able to handle each one separately to get a 64 x 64 precision level.

You are correct, but I was wrong in assuming it could work that way.
Creative coders use backward thinking techniques as a strategy.

raymond

  • Member
  • **
  • Posts: 244
    • Raymond's page
Re: Small favour needed, test precision with sprintf.
« Reply #22 on: September 07, 2018, 03:00:32 AM »
Quote
split into 2 components you may be able to handle each one separately to get a 64 x 64 precision level.

NOT UNLESS you use some bignum library.

As far as I know, MASM can only handle the equivalent of 64 bits with REAL10, AND that would hold only if you modify the control word for precision in your program (finit does that); otherwise the default precision under Windows is 53 bits for floats.

If you declare a float in the data section with more than the maximum usable number of digits for a REAL10, the surplus ones are simply disregarded in the resulting binary equivalent which is stored.
If you try to split a float into its integer and fractional components as two separate entries in your data and later attempt to recombine them as an entity, any surplus 'digit' would again get dropped out, negating any intended gain in accuracy.
Whenever you assume something, you risk being wrong half the time.
http://www.ray.masmcode.com/

AW

  • Member
  • *****
  • Posts: 2583
  • Let's Make ASM Great Again!
Re: Small favour needed, test precision with sprintf.
« Reply #23 on: September 07, 2018, 03:36:03 AM »
Working with REAL10 we have 19 significant digits.
So, is not possible to print 123456.1234567890123456 even using the trick of subtracting the integer part from the whole number and print in 2 parts.
It is however possible to print: 123.1234567890123456 (eventually also 1234.1234567890123456, but is not guaranteed and we have to confirm).

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 7537
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: Small favour needed, test precision with sprintf.
« Reply #24 on: September 07, 2018, 04:56:51 AM »
You are only ever going to get the number of digits that the data size handles at its maximum. My own view is that in most instances, REAL4 or REAL8 do the job and while there are application for REAL10 or big number libraries but unless you are plotting points in other galaxies, the extra precision is not much use to you. The "sprintf" CRT function I have used has a number of flags for truncation or reduced precision levels, the alternative is to truncate the decimal fraction to an arbitrary count depending one what is required.

With the distinction between floating point string display and the actual floating point value, you can simply truncate the string to whatever is needed in the application but retain the actual fp value in the variable it is stored in. This would not be all that hard to code as you allocate the memory for the string plus 8 bytes, write the fp value at the start of the memory and write the string after it.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :skrewy:

AW

  • Member
  • *****
  • Posts: 2583
  • Let's Make ASM Great Again!
Re: Small favour needed, test precision with sprintf.
« Reply #25 on: September 07, 2018, 05:11:41 AM »
The windows runtime does not provide any function to print an 80-bit floating point value (Delphi and Free Pascal have functions for that). We can do it with ASM, but as you said it is not needed at all.  If we really need a huge amount of decimals, for example to calculate the value of PI with 1 billion decimals we will need to resort to a Big Number library which will do it in software.

jj2007

  • Member
  • *****
  • Posts: 10543
  • Assembler is fun ;-)
    • MasmBasic
Re: Small favour needed, test precision with sprintf.
« Reply #26 on: September 07, 2018, 05:52:03 AM »
in most instances, REAL4 or REAL8 do the job

Right. My Str$() loads whatever size on the FPU and spits out variable precision up to REAL10, simply because a single routine can handle all sizes, and the performance penalty REAL10 vs REAL4 is negligible (if any). But of course, if you want to determine the "exact" position of a pixel on the screen, then REAL4 is more than enough. GdiPlus takes integer and REAL4 args but never REAL8.

OTOH "in assembly, we can print 19 digits" can be an argument when discussing advantages over the CRT ;)

nidud

  • Member
  • *****
  • Posts: 1980
    • https://github.com/nidud/asmc
Re: Small favour needed, test precision with sprintf.
« Reply #27 on: September 10, 2018, 10:45:40 PM »
Integer version.
Code: [Select]
    mov         rax,123456.1234567890123456
    movq        xmm0,rax
    cvtpd2dq    xmm1,xmm0
    movq        rdx,xmm1
    cvtdq2pd    xmm1,xmm1
    subpd       xmm0,xmm1
    mov         rax,10000000000.0
    movq        xmm1,rax
    mulpd       xmm0,xmm1
    cvtpd2dq    xmm0,xmm0
    movq        r8,xmm0

Using cvtpd2dq and cvtdq2pd will fail here given the result is a 32-bit integer value. The correct version should be something this:
Code: [Select]
DoubleToInt proc val:real8

    cvtsd2si    rdx,xmm0
    cvtsi2sd    xmm1,rdx
    subsd       xmm0,xmm1
    mov         rcx,10000000000000000.0
    movq        xmm1,rcx
    mulsd       xmm0,xmm1
    cvtsd2si    rax,xmm0
    ret

DoubleToInt endp

Code: [Select]
main proc

    DoubleToInt(9007199254740991.0)
    printf("%I64d.%I64d\n", rdx, rax)
    DoubleToInt(1.2345678901234567)
    printf("%I64d.%I64d\n", rdx, rax)
    DoubleToInt(0.1234567890123456)
    printf("%I64d.%I64d\n", rdx, rax)
    DoubleToInt(123456.1234567890123456)
    printf("%I64d.%I64d\n", rdx, rax)
    xor eax,eax
    ret

main endp

9007199254740991.0
1.2345678901234567
0.1234567890123456
123456.1234567890060134

raymond

  • Member
  • **
  • Posts: 244
    • Raymond's page
Re: Small favour needed, test precision with sprintf.
« Reply #28 on: September 11, 2018, 03:11:54 AM »
Quote
DoubleToInt proc val:real8
DoubleToInt(123456.1234567890123456)
->  123456.1234567890060134

This proves the point I tried to make in my previous post. The data supplied as input for a REAL8 was cropped after the initial 16 significant digits in the decimal system. Trying to recover the cropped digits in print only resulted in garbage being displayed (060134) for those dropped digits (123456), certainly not improving any precision and possibly misleading the recipient.

But, as programmers, let's not mix "apples and oranges" and forget about mathematical principles. The precision limit of a real8 is stated as having a maximum of 16 significant digits in the decimal system, which means the total number of the digits in both the integer and fractional portions.

For example, 5000000000000000000.10203 would have 23 significant digits,
while, 0.00000000000000000010203 would have only 5 significant digits.
Whenever you assume something, you risk being wrong half the time.
http://www.ray.masmcode.com/

nidud

  • Member
  • *****
  • Posts: 1980
    • https://github.com/nidud/asmc
Re: Small favour needed, test precision with sprintf.
« Reply #29 on: September 11, 2018, 04:02:12 AM »
The problem here was to fit the value into a 64-bit register. The total storage is 53-bit but fractions may extend way beyond 64 as your last sample shows. However, this may also be converted:
Code: [Select]
    cvtsd2si    rdx,xmm0
    cvtsi2sd    xmm1,rdx
    subsd       xmm0,xmm1
    mov         rcx,10000000000000000.0
    movq        xmm1,rcx
    mulsd       xmm0,xmm1
    cvtsd2si    rax,xmm0
    mulsd       xmm0,xmm1
    cvtsd2si    r9,xmm0
    ret

Code: [Select]
    DoubleToInt(0.00000000000000000010203)
    printf("%I64d.%016I64d%016I64d\n", rdx, rax, r9)

0.00000000000000000010203000000000

The former value need some help thought.
Code: [Select]
    .data
    q real16 5000000000000000000.10203
...
    printf("%llf\n", q)

5000000000000000000.102030