I have been trying to get up to speed with writing 64-bit routines, particularly those that call other routines. It took me a long while to figure out how to call standard library functions like printf from within some 64-bit code. Right after I figured the code below out, I found some interesting examples in, of all places, a site for art and music, by a fellow named Sydney Grew -
http://artmusic.smfforfree.com/index.php/topic,249.0.html.
I've looked through the examples here in the 64-bit section (but not in the archived posts on the old masm site), and didn't find very much that pertained to calling external routines from 64-bit code. Hopefully what I have here will be helpful to someone else who is just getting started.
One important point is how the parameters are passed to the external function. In 32-bit code, I'm used to pushing the parameters on the stack in reverse order. In 64-bit programming (edit: in Windows), parameters are passed using the registers, with RCX, RDX, and R8 being used for the first three integer parameters, and XMM0, XMM1, and XMM2 being used for the first three floating point parameters. As I am passing a format string (address) and two integer parameters, I set RCX, RDX, and R8 with the appropriate values in the following code.
Another thing that's important is the use of a prologue to set up space on the stack for the function, and the epilogue at the end to put the stack back in its original state. I'm still somewhat vague on exactly how much to allocate. For my example, setting aside 64 bytes (40h bytes) seems to do the trick, although I could probably pare that down. I think it's important to use a number that is a multiple of 16 bytes.
extern printf:PROC
.data
fmtString db "test string: %lld and %lld", 0Dh, 0Ah, 0h
valueForPrintf dq 5
value2ForPrintf dq 6
.code
TestPrintf PROC C
; Prologue
push rdi
sub rsp, 40h
mov rdi, rsp
; Before calling printf, pass the format string in RCX, and int values in RDX and R8.
; Notice that the parameters are passed in registers rather than being pushed on the stack in reverse order.
lea rcx, [fmtString]
mov rdx, [valueForPrintf]
mov r8, [value2ForPrintf]
call printf
; Epilogue
add rsp, 40h
pop rdi
ret
TestPrintf ENDP
END
I call TestPrintf from some C code. The prototype for TestPrintf is void TestPrintf(void);
Output from the above is
test string: 5 and 6