Author Topic: Question about calling conventions and parameters passing  (Read 595 times)

enzechen

  • Regular Member
  • *
  • Posts: 9
Question about calling conventions and parameters passing
« on: November 16, 2016, 10:33:30 PM »
Currently, I am confused of the calling convention and parameters passing in win64:

According to MSDN (https://msdn.microsoft.com/en-us/library/zthk2dkh.aspx), win64 calling convention is to pass first 4 integer parameters into RCX, RDX, R8, and R9, first 4 floating point into XMM0 ~ XMM3. Others are needed to pushed onto stack.

How does ML64 handle this calling convention by using INVOKE? Does INVOKE macro passes parameters using above conventions? or it's the same as win32 to push all parameters onto stack? How about shadow stack?

If I only use pure assembly without macro, how shall I handle this by using CALL?

Thanks a lot.

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 4886
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: Question about calling conventions and parameters passing
« Reply #1 on: November 16, 2016, 11:15:49 PM »
ML64 does not have a native "invoke" directive. It has been implemented as a macro in conjunction with a prologue/epilogue pair of macros as ML64 comes unconfigured. Look in the subforum above for ML64 code that works with "invoke" in the normal MASM manner. Note that the project for 64 bit MASM is not complete and is still under development but you can get it going and produce 64 bit binaries with it.

LATER: Have a look at the two HTML files attached to this post. They explain how the win64 calling convention works in assembler code and how the invoke macro works. You can code this stuff manually but the stack addressing is very complex and unless you properly align the stack the app simply will not start. I would suggest that you use the supplied prologue / epilogue as it get all of this correct. You can manually write no stack frame procedures when the arg count to the procedure is 4 or less. Also note that you generally do not use PUSH / POP to preserve registers, you write them to local variables as the preferred way to write win64 procedures.
« Last Edit: November 17, 2016, 03:14:30 AM by hutch-- »
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :biggrin: