Author Topic: Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?  (Read 8006 times)

nidud

  • Member
  • *****
  • Posts: 2390
    • https://github.com/nidud/asmc
deleted
« Last Edit: February 26, 2022, 04:30:18 AM by nidud »

jj2007

  • Member
  • *****
  • Posts: 12322
  • Assembler is fun ;-)
    • MasmBasic
Re: Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #1 on: March 04, 2021, 06:23:53 AM »
Microsoft and GCC do this in reverse, in push-order which is right-to-left

Normally, the order is not an issue - except if you use those registers as arguments which are used for the 4 fastcall arguments:
Code: [Select]
include \Masm32\MasmBasic\Res\JBasic.inc ; ## builds in 32- or 64-bit mode with UAsm, ML, AsmC ##
BytesWritten dq ?
Init ; OPT_64 1 ; put 0 for 32 bit, 1 for 64 bit assembly
  PrintLine Chr$("This program was assembled with ", @AsmUsed$(1), " in ", jbit$, "-bit format.")
  jinvoke DeleteFile, Chr$("CreateFile.opened")
  if 1
mov eax, GENERIC_WRITE
mov ecx, FILE_SHARE_WRITE
mov edx, CREATE_ALWAYS
mov r8, FILE_ATTRIBUTE_NORMAL

jinvoke CreateFile, Chr$("CreateFile.opened"), eax, ecx, NULL, edx, r8, 0
  else
jinvoke CreateFile, Chr$("CreateFile.opened"), GENERIC_WRITE, FILE_SHARE_WRITE, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0
  endif
  xchg rax, rbx
  PrintLine Err$()
  jinvoke WriteFile, rbx, Chr$("Hello World, how are you?"), c$Len, addr BytesWritten, 0
  jinvoke CloseHandle, rbx
EndOfCode

Under the hood:
Code: [Select]
int3                                                      |
mov eax,40000000                                          |
mov ecx,2                                                 |
mov edx,2                                                 |
mov r8,80                                                 |

and qword ptr ss:[rsp+30],0                               |
mov r10,r8                                                |
mov qword ptr ss:[rsp+28],r10                             |
mov r10d,edx                                              |
mov qword ptr ss:[rsp+20],r10                             |
xor r9d,r9d                                               |
mov r8d,ecx                                               |
mov edx,eax                                               |
lea rcx,qword ptr ds:[1400013A9]                          | 1400013A9:"CreateFile.opened"
call qword ptr ds:[<sub_140001740>]                       |

Obviously, left-to-right order would fail when passing arguments in rcx and rdx.

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 9240
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #2 on: March 04, 2021, 06:32:43 AM »
Write order is not specific, only argument location.

        mov     dword ptr [rsp+28H], edx
        mov     dword ptr [rsp+30H], eax
        mov     dword ptr [rsp+20H], ecx
        mov     r9d, 4                 
        mov     r8d, 3                 
        mov     edx, 2                 
        mov     ecx, 1                 
        call    foo


Microsoft and GCC do this in reverse, in push-order which is right-to-left.

x64 does not have a PUSH order.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :skrewy:

nidud

  • Member
  • *****
  • Posts: 2390
    • https://github.com/nidud/asmc
Re: Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #3 on: March 04, 2021, 08:02:16 AM »
deleted
« Last Edit: February 26, 2022, 04:30:34 AM by nidud »

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 9240
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #4 on: March 04, 2021, 10:54:06 AM »
> Of course it has. This is the main part of the convention which determine the position of each argument on the stack for a function call.

You may have a deviant terminology but I suggest that it has more to do with you supporting 32 bit STDCALL than 64 bit FASTCALL. With the x64 ABI you can have left to right, up to down, inside out and upside down or even a random distribution, as long as all the arguments end up in the right sequence, it will work but PUSH order, forget it.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :skrewy:

jj2007

  • Member
  • *****
  • Posts: 12322
  • Assembler is fun ;-)
    • MasmBasic
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #5 on: March 04, 2021, 12:28:43 PM »
With the x64 ABI you can have left to right, up to down, inside out and upside down or even a random distribution, as long as all the arguments end up in the right sequence, it will work

See Reply #1. Remember the "register gets overwritten" error?

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 9240
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #6 on: March 05, 2021, 12:22:21 AM »
Yes, it was a bad example but I understood what he was aiming at. It seems that many paddling around trying to understand x64 FASTCALL have yet to fully understand how it works.

This is among the reasons why shadow space is required in some situations but the general drift is don't use any of the first four registers in the first four arguments.

The layout of x64 FASTCALL is primarily designed for a 64 bit C compiler but MASM can be configured to properly use x64 FASTCALL.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :skrewy:

nidud

  • Member
  • *****
  • Posts: 2390
    • https://github.com/nidud/asmc
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #7 on: March 05, 2021, 01:14:22 AM »
deleted
« Last Edit: February 26, 2022, 04:30:45 AM by nidud »

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 9240
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #8 on: March 05, 2021, 12:28:45 PM »
The problem as I see it is that you are shifting across a large variety of calling conventions and effectively blurring the distinctions. We all know how 32 bit Windows STDCALL worked, push/call notation and the specs are publically available for Microsoft x64 Windows and there is no leakage across the two.

The Microsoft x64 ABI FASTCALL does not use PUSH at all and the notion of a PUSH order may be convenient when you are dealing with a multitude of different calling conventions but it misrepresents the 64 bit Windows FASTCALL. Now as I am sure that you understand the M$ FASTCALL, the obvious is that when you write the 1st 4 args to registers,

mov rcx, 1
mov rdx, 2
mov r8, 3
mov r9, 4

is the same as

mov r8, 3
mov rcx, 1
mov r9, 4
mov rdx, 2

While the identical data is written to the 1st 4 registers, they are not written in the same order as it simply does not matter, the 4 arguments are written to the correct registers. The idea of PUSH order is incorrect here, its the WRITE order that matters. With the 5th and higher arguments written to the stack after the 4 shadow space positions, they don't have to be written in PUSH order either. For each 64 bit location on the stack, if you write the correct data to each 64 bit location, you can write it in any order you like.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :skrewy:

nidud

  • Member
  • *****
  • Posts: 2390
    • https://github.com/nidud/asmc
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #9 on: March 05, 2021, 09:55:56 PM »
deleted
« Last Edit: February 26, 2022, 04:30:55 AM by nidud »

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 9240
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #10 on: March 05, 2021, 10:52:05 PM »
 :biggrin:
Quote
It actually changes everything and it's done so by design. RCX is the most important register as it is used as a counter (hence the C) and operand for various shift operations. RDX is also a better choice than R8..Rn as it has two byte registers. Compilers will therefor use (spill) RAX, RCX, and RDX for loading arguments from the right and assign value to RCX at the end.
This is MS-DOS level thinking and it is not x64 Windows ABI compliant.

ax = accumulator
bx = base address
cx = counter
dx = data
si = source index
di = destination address
sp = stack pointer
bp = base pointer

With Win64 ABI you are free from this ancient thinking, in the 1st 4 registers in FASTCALL you can put anything you like as long as you don't over write registers with other registers.

Now when you make a call in 64 bit Windows FASTCALL, you have written values to the first 4 registers but in the procedure that you are calling you can directly use them, assign them to locals, copy them to other registers and even save some of the system registers and save them there.

There is no reason why a compiler cannot be fully ABI compliant, slopping data around with unsound assumptions produces junk code, something a compiler should not do.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :skrewy:

jj2007

  • Member
  • *****
  • Posts: 12322
  • Assembler is fun ;-)
    • MasmBasic
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #11 on: March 05, 2021, 11:29:49 PM »
as long as you don't over write registers with other registers.

Yep, and that's the only reason why the "push order" matters a little bit.

nidud

  • Member
  • *****
  • Posts: 2390
    • https://github.com/nidud/asmc
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #12 on: March 06, 2021, 01:20:46 AM »
deleted
« Last Edit: February 26, 2022, 04:31:07 AM by nidud »

TimoVJL

  • Member
  • ****
  • Posts: 998
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #13 on: March 06, 2021, 01:34:59 AM »
Windows x64 ABI and C/C++ rules, so be calm.

BTW:
Nice feature of programming, when we old people go to pension, our software still runs, even after we die or vanish :smiley:
Hardware workers are in better case, their work are usable a lot of longer.
« Last Edit: March 06, 2021, 02:48:23 AM by TimoVJL »
May the source be with you

HSE

  • Member
  • *****
  • Posts: 2019
  • AMD 7-32 / i3 10-64
Re: 64-bit: Why Can't I get "CreateFileA" to Access a File or Device?
« Reply #14 on: March 06, 2021, 01:59:52 AM »
as this is not a "real" FASTCALL convention.

Things begin to make sense  :thumbsup:

Thanks.
Equations in Assembly: SmplMath