Calling WriteFile in x64

[markallyn]

Hello everyone,

I am lousy at writing assemby language and I apologize for the stupidity I'm about to display.

The code assembles and links without a problem.  The code runs without a crash but fails to print the message. 

extrn GetStdHandle  : PROC
extrn WriteFile    : PROC

.data
msg   db   "The damned thing worked",0
len   EQU   SIZEOF msg
lpReserved    dq   0

.data?
hndl      dq   ?
lpcchWritten   dq    ?

.code
main   PROC   C
sub   rsp,   40
mov   rcx,     -11
call   GetStdHandle
mov   hndl,   rax
lea   rcx,    hndl
lea   rdx,    msg
mov   r8d,   len
lea   r9,     lpcchWritten
push   lpReserved
call   WriteFile
add   rsp,    40
ret   0
main   ENDP
END

I've scoured the internet and looked at previous posts on this forum to no avail.  Could someone put me right?

Mark Allyn

[hutch--]

Mark,

If its MASM x64, it does not use PUSH / CALL syntax. You have to get a couple of things right, correctly align the stack and write the arguments according to x64 FASTCALL calling convention. Have a look at the MASM64 project as it has the right techniques to work in 64 bit Windows.

[markallyn]

Hello Hutch,

Thanks for your reply.  I do appreciate the attention you have given my query. 

Honestly, I thought I was following the FASTCALL convention in passing registers to WriteFile, including pushing 0 onto the stack as the fifth parameter.  I also thought I had got the stack alignment correct as well with sub rsp, 40.  But, I will have to go back and review again.

Your advice about the masm 64 project is good indeed and I will endeavor to do so.

Regards,
Mark

[hutch--]

Mark,

Here is a simple example for you using the 64 bit MASM project.

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    include \masm32\include64\masm64rt.inc

    .code

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

entry_point proc

    USING r12

    SaveRegs

    .data
      tmsg db "This is a test",13,10,0
      pmsg dq tmsg
    .code

    rcall StdOut,pmsg

    waitkey
    RestoreRegs
    .exit

entry_point endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

STACKFRAME

StdOut proc

  ; rcx = input text address

    USING r12,r13                               ; select registers to use
    LOCAL bwrt  :QWORD

    SaveRegs                                    ; save them

    mov r12, rcx                                ; store address in r12
    mov rax, r12
    sub rax, 1
  @@:
    add rax, 1
    cmp BYTE PTR [rax], 0                       ; loop to get the text length
    jne @B

    sub rax, r12                                ; sub original address from RAX
    mov r13, rax                                ; save string length into r13

    rcall GetStdHandle,STD_OUTPUT_HANDLE
    invoke WriteFile,rax,r12,r13,ADDR bwrt,NULL

    mov rax, bwrt                               ; return value is bytes written

    RestoreRegs                                 ; restore the registers

    ret

StdOut endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    end

[markallyn]

Good morning (evening) Hutch,

Thanks very much indeed for taking the time and trouble to create the detailed reply to my last post.

I need to study very carefully what you created.  I will revise my code according to your model and resubmit it for the forum's review.

Regards,
Mark

[jj2007]

Mark,
It works with mov rcx, hndl.

[markallyn]

Hello JJ and good morning,

You're absolutely right!  Now, the question that is bothering me is (the usual one), WHY?  In general, I have been troubled by the difference between lea and mov.  Something rather subtle is going on here that I am too stupid to understand.

But, I have to say that I am delighted by the results of the revision.  I was going somewhat nuts.  I really thought I understood the 64 bit ABI and then I ran into this business.

Regards,
Mark

[markallyn]

Hello again, JJ,

Well, looking at the signature for the WriteFile function, the first value is a HANDLE, i.e. a number, not an address.  (BTW, in this case the number happens to be "7").  I kept ignoring this simple fact and treated the number as if it was an address.

Mark

[jj2007]

Yep. And it has been 7 for quite a while

[markallyn]

JJ,

Ah, yet another interesting piece of info you masm pros have tucked away.

I learned something else in this exercise, btw. I don't understand it, but here's what I found.  If you attempt to run similar code using printf rather than WriteFile under the Visual Studio 2019 x64 native tools command prompt, the program will attempt to write in a forbidden section of memory.  You will be unceremoniously ejected, in short.

However, if you simply assemble and link the same code under either a standard command prompt, or under 2019 administrator command prompt, it runs perfectly.  That apparent fact may be why some earlier OPs were complaining that printf didn't work, and displayed the same behavior as I'm describing.

Why this should be so is something I don't know.  Perhaps you do, seeing that you know that "7" is always the handle for an output file.

Regards,
Mark

[jj2007]

If you attempt to run similar code using printf rather than WriteFile under the Visual Studio 2019 x64 native tools command prompt, the program will attempt to write in a forbidden section of memory.  You will be unceremoniously ejected, in short.

That's a VS problem then. This works just fine:

Code Select Expand

jinvoke printf, Chr$("%s"), Chr$("Hello World")