The MASM Forum

64 bit assembler => 64 bit assembler. Conceptual Issues => Topic started by: markallyn on October 26, 2017, 03:29:22 AM

Title: prolog and epilog in 64 bit code functions
Post by: markallyn on October 26, 2017, 03:29:22 AM
Hello everyone,

I'm fighting my way up the 64 bit learning curve.  Here's one more question.

If I write a frame function and need to do an PROLOG and EPILOG, do I need the .dot directives (.pushreg, etc.) for anything other than SEH?  The documentation I've seen seems to indicate that their only function is for adding unwind code. 

Thanks,
Mark Allyn
Title: Re: prolog and epilog in 64 bit code functions
Post by: hutch-- on October 26, 2017, 01:52:55 PM
Mark,

The sad tale but true is that to get a prologue/epilogue working in 64 bit MASM, you write them by test piece in conjunction with a debugger/disassembler. This is what I did with MASM and used ArkDasm to keep looking at the results. I am not even vaguely interested in structured exception handling as I am a fan of writing code that does not have mistakes in it in the first place.

The existing reference material is appalling and often wrong and you need to get a very good understanding of how the 64 bit Windows stack works in conjunction with the actual details of the Win 64 FASTCALL calling convention. You know you have got it right when you can successfully call any Windows API and don't be afraid to test the content of RSP before and after calls to make sure you have not modified the stack.
Title: Re: prolog and epilog in 64 bit code functions
Post by: jj2007 on October 26, 2017, 02:07:41 PM
Quote from: hutch-- on October 26, 2017, 01:52:55 PMI am not even vaguely interested in structured exception handling as I am a fan of writing code that does not have mistakes in it in the first place.

Amen :t
Title: Re: prolog and epilog in 64 bit code functions
Post by: aw27 on October 26, 2017, 02:40:00 PM
Quote from: markallyn on October 26, 2017, 03:29:22 AM
If I write a frame function and need to do an PROLOG and EPILOG, do I need the .dot directives (.pushreg, etc.) for anything other than SEH?  The documentation I've seen seems to indicate that their only function is for adding unwind code. 

The subject is difficult and the documentation you can find is sparse, incomplete and even wrong. I have been working on the subject for the past few weeks and will come up with something that will publish on Codeproject.com today or tomorrow.
Title: Re: prolog and epilog in 64 bit code functions
Post by: felipe on October 26, 2017, 03:54:10 PM
Wow, seems like 64 bit is like moving to mars.  :dazzled:

:biggrin:
Title: Re: prolog and epilog in 64 bit code functions
Post by: hutch-- on October 26, 2017, 08:40:27 PM
Hi felipe,

One you get the bare mechanics up and working the rest of it is a bit different but reasonably straight forward. Even with 32 bit MASM you mainly relied on the built in default PROC/ENDP prologue and epilogue and usually just turned it off and on when you wrote code with no stack frame. With 64 bit MASM it did not have the default prologue/epilogue so you had to write your own which was OK as it had the preprocessor capacity to do this. The real problem was the reference data was very poor and often wrong and the win 64 FASTCALL calling convention was different to the win32 STDCALL and C calling conventions.

On the bright side, you have many more registers to work with and laying out more complicated algorithms is a lot cleaner and more straight forward and you just don't need to modify the stack at all to produce viable fast code.
Title: Re: prolog and epilog in 64 bit code functions
Post by: felipe on October 27, 2017, 03:19:22 AM
I want to start 64 bit, but i think i should know much more of 32 bit yet. For example more complex instruction sets like mmx and sse and all that. I think a have a feeling of ignorance that i can't stand for it if a don't know those things and just jump to the newest bits.  :dazzled:

:biggrin: