x64 Directx COM calls

[satpro]

Recently I made the decision to try an x64 program with DirectDraw, just to get the easy surface and a bit for old times sake.  In the past (with 32-bit code) I have managed to call any DDraw method using our standard 'CoInvoke' macro, and even some custom ones I made with jumbled params, and THEN even some straight Jeremy Gordon-style hand-coded old-school stuff.  After fixing the usual bugs, my 32-bit code eventually always worked fine.

I started having problems as soon as I got to the x64 method calls.  Lots of crashes.  So, then I switched over to Dx 9, thinking maybe 64-bit DDraw was maybe not really a thing that would work on Win 11???  Dx 9 was a problem for me now, too, which led me to wonder about MY stack alignment or even something else, 'cuz I just know it's me somehow ... and then maybe hope for an easier way to view the whole subject.

A question:

When we get to the 'Start' label our stack is always something xxxxxx08h, correct?  Would any 16-byte stack alignment adjustment (including the call) work at that point?  Or, does there need to be another 08h subtraction?  In other words, what would be the way to look at this with 4 params (or 5) and a return address?  Could one adjust once by say, 120 (or 128) bytes, and then just correct afterwards, while being careful with further sub calls (because of another 8 bytes SP adj.), could one just leave those "extra" 128 bytes on the stack until the end?

I have looked at some com-call EXEs in IDA and I don't see alignment (like I am expecting to see) within many of those com calls.

Does anyone have something I could take to my brain and explain this, perhaps a rule of thumb or something else?  It is obvious I do not yet grasp this completely.   

Thank you,
Bert

[Caché GB]

@ satpro

Obviously the info I provided was not what you where looking for.
Anyway, good luck.

[satpro]

Spectacular advice!    Your words reminded me of two very important items that are spending most of their IN-PRINT time sitting idly directly behind me:  Agner and the Intel Developer manuals -- all of them.  Maybe it's an "old dog" thing, but at least now I have a real "handle" on it.   

Hey! I have been writing 32-bit software for a very long time and am quite comfortable with any API and 100% assembly on my end, but recently found myself with a real need for 64-bit... it was either that, or slogging through 64-bit data the 32-bit way -- in a program emulating 64 mhz operation IN up to 64-bit data.  And there I was, frozen like a deer in the headlights... Google really, really sux these days and I had literally forgotten about the absolute best assembly language reference sections of all!  LOL  So, thank you very much for snapping me back-to, 'cuz that is what it took this time. It may be a lesson for many of us.

It was stack alignment, and I was stuck in 32-bit World, using a 32-bit mindset for everything except the registers.  All better now.  Thank you, whew!


The second comment?  Ouch, man.    We don't help others like that here, do we?  Well, of course not.  Just know that I am 100.0% elated about the spark you sent my way in the first comment... and, more importantly, that a thank-you means absolutely nothing when it has been coerced.  -->  See?    Now you have no idea if I was being truthful above.

Sometimes the thank-you comes in only a minute, but at other times?  It can take two seconds short of an entire day.

[mineiro]

Hello sir satpro;
If you think that can be stack not aligned, link bellow it's a good advice of doing by hands.
https://masm32.com/board/index.php?topic=10870.msg120814#msg120814

[jj2007]

Does anyone have something I could take to my brain and explain this, perhaps a rule of thumb or something else?

Hi Bert,

Below a typical disassembly starting at the entry point 140001000. Note that after the enter 100,0, the stack remains at xxx0 before each call. Btw it could be enter 20, 0 - the purpose is just to reserve a few bytes of shadow space and align the stack to "0", i.e. 16 bytes.

Hope this helps for the alignment issue

Code Select Expand

0000000140001000 | C8 0001 00                 | enter 100,0                     |
0000000140001004 | E8 41030000                | call 14000134A                  |
0000000140001009 | E8 77020000                | call 140001285                  |
000000014000100E | 33C9                       | xor ecx,ecx                     |
0000000140001010 | FF15 46340000              | call [<&SetLastError>]          |
0000000140001016 | FF15 48340000              | call [<&GetProcessHeap>]        |
000000014000101C | 48:8905 DD220000           | mov [140003300],rax             |
0000000140001023 | 48:8D1D 66230000           | lea rbx,[140003390]             |
000000014000102A | 48:8975 58                 | mov [rbp+58],rsi                |
000000014000102E | 48:897D 50                 | mov [rbp+50],rdi                |
0000000140001032 | 48:C7C1 F5FFFFFF           | mov rcx,FFFFFFFFFFFFFFF5        |
0000000140001039 | FF15 2D340000              | call [<&GetStdHandle>]          |

[Caché GB]

satpro

I am truly happy for the spark sent your way. I hope your 64 bit journey is most rewarding.

Once again, good luck.

[satpro]

Hello sir satpro;
If you think that can be stack not aligned, link bellow it's a good advice of doing by hands.
https://masm32.com/board/index.php?topic=10870.msg120814#msg120814

Thanks!  Yes, it was just that.    I could feel dumb, but I don't.  I got you guys!

Below a typical disassembly starting at the entry point 140001000. Note that after the enter 100,0, the stack remains at xxx0 before each call. Btw it could be enter 20, 0 - the purpose is just to reserve a few bytes of shadow space and align the stack to "0", i.e. 16 bytes.

Jochen, do you ever (or always) use Enter or Enter/Leave?  I have not (probably only because of all the bad stuff written by bad asm programmers) and have been starting to wonder if maybe Hutch was right in thinking we should be using the instructions as they do in C/C++.

This might also be a great time for that person who understands this subject well enough to write an article about it, an article that is agnostic and easy to follow.  Obviously, calling functions and 64-bit stack alignment is a SCARCE subject here and everywhere else.  Almost NO ONE out here knows enough about it, and in my opinion it's the main reason why most software today stinks, among others.  The Stack.  The one series of articles on CodeProject (about DirectX written in 64-bit assembly) is really just some old 8-bit guy trying to show off his cryptic macros and spaghetti code.  A goose egg on proper alignment.  It's terrible.  In these articles he saves all 16 registers in WndProc, so....  enough said about that.

Thank you all!  I think the lights are back on over here, so maybe it's time for me to get reading.

[jj2007]

Jochen, do you ever (or always) use Enter or Enter/Leave?

I use enter once to start a 64-bit program. It's a short but relatively slow instruction, so don't use it in an innermost loop.

In the 32-bit MasmBasic library, I don't use enter, but leave appears about ten times (among about 200 procs). The leave instruction is short and fast, the main reason why people don't use it is the ret macro: it seems easier to use.

[Caché GB]