rbp based stack frame and stack space allocation

[Vortex]

Reading the JWasm manual :

3.9 Directive OPTION WIN64
.
.
INVOKE Stack Space Reservation [bit 1]:
.
.
- 1: the maximum stack space required by all INVOKEs inside a procedure is computed by the assembler and reserved once on the procedure's entry. It's released when the procedure is exited.

The explanation above is valid for rsp based stack frames. My aim is to create rbp based stack frames allocating the maximum space at the entry of a procedure. Is there a specific HJWasm construct to accomplish this? My method is to employ a custom invoke macro :

Code Select Expand

option casemap:none
option frame:auto
option win64:1

include DlgBox.inc
include invoke.inc

.data

DlgBox      db 'DLGBOX',0

.code

start PROC

    sub     rsp,8+5*8+8

   _invoke  GetModuleHandle,0
   _invoke  DialogBoxParam,rax,ADDR DlgBox,0,ADDR DlgProc,0
   _invoke  ExitProcess,rax

start ENDP

DlgProc PROC hWnd:QWORD,uMsg:QWORD,wParam:QWORD,lParam:QWORD

    sub         rsp,4*8

    .IF uMsg==WM_CLOSE

       _invoke  EndDialog,hWnd,0

    .ELSE

        xor     rax,rax
        ret

    .ENDIF

    mov     eax,1
    ret

DlgProc ENDP

END

Disassembling the object module :

Code Select Expand

_text   SEGMENT PARA 'CODE'         

start   PROC
        sub     rsp, 56
        xor     rcx, rcx
        call    GetModuleHandleA
        mov     rcx, rax       
        mov     rdx, offset DlgBox
        xor     r8, r8           
        mov     r9, offset DlgProc
        mov     qword ptr [rsp+20H], 0
        call    DialogBoxParamA       
        mov     rcx, rax             
        call    ExitProcess           

DlgProc LABEL NEAR
        mov     qword ptr [rsp+8H], rcx
        mov     qword ptr [rsp+10H], rdx
        mov     qword ptr [rsp+18H], r8
        mov     qword ptr [rsp+20H], r9
        push    rbp                     
        mov     rbp, rsp               
        sub     rsp, 32                 
        cmp     qword ptr [rbp+18H], 16
        jnz     ?_001                   
        mov     rcx, qword ptr [rbp+10H]
        xor     rdx, rdx               
        call    EndDialog               
        jmp     ?_002                   

?_001:  xor     rax, rax
        leave           
        ret             
start   ENDP

?_002   LABEL NEAR
        mov     eax, 1                                 
        leave                                           
        ret

rbp based stack frame are allowing the insertion of push\pop pairs with the condition of respecting the 16-bytes stack alignment.

[aw27]

Vortex,

bit 1 is not bit 0, you need  option win64:2

[Vortex]

Hi aw27,

I need to use the shadow space for the first arguments rcx,rdx,r8 and r9. This is why I specify bit 0.  Option win64:2 or option win64:3 will switch to the rsp based stack frame.

[johnsa]

I'm busy looking into decoupling all of this and sorting it out..

so that all you have to do is select RBP or RSP as stackbase .. and the rest is automatic.

[Vortex]

Hi johnsa,

so that all you have to do is select RBP or RSP as stackbase .. and the rest is automatic.

Exactly. ml64 selects rbp as stackbase by default. Attached is an example. Selecting rbp or rsp could provide flexibility.