The MASM Forum

64 bit assembler => UASM Assembler Development => Topic started by: aw27 on July 20, 2017, 12:11:18 AM

Title: Stack issues
Post by: aw27 on July 20, 2017, 12:11:18 AM
A few stack related issues:


; x64

option casemap:none

.code

proc1 proc
ret
proc1 endp

main proc

; Stack is not realigned on entry

invoke proc1

ret
main endp

end ;main

end

COMMENT &
proc1
00000001`3ffa1000 4883c408   add     rsp,8 <- ????????????????????????
00000001`3ffa1004 c3              ret
main
               Stack is not realigned on entry
00000001`3ffa1005 4883ec20   sub    rsp,20h
00000001`3ffa1009 e8f2ffffff     call    test+0x1000 (00000001`3ffa1000)
00000001`3ffa100e 4883c420   add    rsp,20h
00000001`3ffa1021 4883c408   add    rsp,8 <- ????????????????????
00000001`3ffa1025 c3              ret
&
Title: Re: Stack issues
Post by: johnsa on July 20, 2017, 06:06:48 PM
Hi,

This one is already fixed in 2.38 which will be out soon :)
Title: Re: Stack issues
Post by: aw27 on July 20, 2017, 06:30:36 PM
Quote from: johnsa on July 20, 2017, 06:06:48 PM
Hi,

This one is already fixed in 2.38 which will be out soon :)
:t
Title: Re: Stack issues
Post by: johnsa on July 26, 2017, 10:56:47 PM
2.38 for (win32 and win64) is up, please test :)
Title: Re: Stack issues
Post by: jj2007 on July 26, 2017, 11:35:18 PM
Spits loads of errors: \masm32\include\windows.inc(20234) : Error A2279: EVEX encoding not enabled. Use option evex directive?
Title: Re: Stack issues
Post by: johnsa on July 27, 2017, 12:50:12 AM
Found the issue, due to improvements in the handling of evex instructions which use { } has caused this regression, will fix and update asap.
Title: Re: Stack issues
Post by: johnsa on July 27, 2017, 05:40:04 PM
Packages are updated on the site, dated today the 27th. Please try again.

Thanks,
John
Title: Re: Stack issues
Post by: jj2007 on July 27, 2017, 11:31:24 PM
Thanks, now it works perfectly, tested on 1.5MB of sources :t
Title: Re: Stack issues
Post by: LiaoMi on September 22, 2018, 01:08:11 AM
Hi,

why does an exception occur when I call the MessageBox? In the first experiment, I manually do the alignment, in the other I hope that uasm64 will do it, but something is not going according to plan :( Two binary files can be downloaded below .. In general, I have noticed before that uasm does not align stack ?! Or I'm wrong?

.686
.MMX
.XMM
.x64

option casemap:none
option win64:15;win64:11
option frame:auto
option stackbase:rsp
option dotname
option evex:1
option proc:private
WIN32_LEAN_AND_MEAN equ 1
.nolist
.nocref

    include C:\masm64\VS2017\include_x86_x64\translate64.inc
    include C:\masm64\sdkrc100\um\windows.inc
    include macros64_Uasm.inc
;MessageBox proto :QWORD,:QWORD,:QWORD,:DWORD
;ExitProcess proto :DWORD
; -----------------------------------------
  ; non branching version with no ELSE clause
  ; -----------------------------------------
      chr$ MACRO any_text:VARARG
        LOCAL txtname
        .data
          IFDEF __UNICODE__
            WSTR txtname,any_text
            ;align 4
            .code
            EXITM <OFFSET txtname>
          ENDIF

          txtname db any_text,0
          ;align 4
          .code
          EXITM <OFFSET txtname>
      ENDM

      memalign MACRO reg, number
        sub reg, 8
        add reg, number - 1
        and reg, -number
      ENDM

Prologue64 MACRO       
                push        rdi 
sub         rsp, 100h 
mov         rdi, rsp
ENDM

Epilogue64 MACRO
add       rsp, 100h
                    pop       rdi 
          ENDM

;     ~~~~~~~~~
;     libraries
;     ~~~~~~~~~
    ; ------------------------------------------
    ; import libraries for Windows API functions
    ; ------------------------------------------

includelib Kernel32.lib
includelib  User32.lib
includelib  Msimg32.lib         
includelib  Comctl32.lib
includelib  Comdlg32.lib
includelib  Shell32.lib           
includelib  OleAut32.lib             
includelib  Ole32.lib
includelib  Advapi32.lib

.data
Me db "Test UASM Debug Compiling",0
.code

Main proc FRAME
    ;Prologue64
    invoke MessageBox,0,chr$("Test UASM Debug Compiling"),chr$("Test"),0
    ;Epilogue64
    ret
Main endp


start:
    enter 16,0
    ;memalign rsp,16
    invoke Main
    invoke ExitProcess,0
    leave
end start
Title: Re: Stack issues
Post by: hutch-- on September 22, 2018, 04:10:01 AM
Give this technique a blast in UASM, I can only test in MASM but I have turned off the automatic stack frame and have coded it manually and I think UASM produces the same start alignment at the entry point.

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    include \masm32\include64\masm64rt.inc

    .code

NOSTACKFRAME

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

entry_point proc

    push rbp
    mov rbp, rsp
    sub rsp, 128

    call testproc
    invoke ExitProcess,0

    leave
    ret

entry_point endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

testproc proc

    push rbp
    mov rbp, rsp
    sub rsp, 32

    invoke MessageBox,0,"Called within a stack frame","Title",MB_OK

    leave
    ret

testproc endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

STACKFRAME

    end
Title: Re: Stack issues
Post by: LiaoMi on September 22, 2018, 06:59:47 PM
Quote from: hutch-- on September 22, 2018, 04:10:01 AM
Give this technique a blast in UASM

Hi Hutch,

of course, I can do so, but by default the binary file is not working. Here are the differences of the binary files that the dumpbin utility was able to create ..
(https://image.ibb.co/iH3Zmp/Image_3.png)
Title: Re: Stack issues
Post by: aw27 on September 22, 2018, 08:03:58 PM
@LiaoMi

You are putting too much irrelevant code into your demo. After stripping most of the unneeded code, some still there, I can not second your claim - it does indeed align the stack. This works.



.686
.MMX
.XMM
.x64

option casemap:none
option win64:15;win64:11
option frame:auto
option stackbase:rsp
option dotname
option evex:1
option proc:private
WIN32_LEAN_AND_MEAN equ 1
.nolist
.nocref

HWND typedef ptr
includelib \masm32\lib64\kernel32.lib
ExitProcess proto :dword
includelib \masm32\lib64\user32.lib
MessageBoxA proto :HWND,:PTR,:PTR,:DWORD

; -----------------------------------------
  ; non branching version with no ELSE clause
  ; -----------------------------------------
      chr$ MACRO any_text:VARARG
        LOCAL txtname
        .data
          IFDEF __UNICODE__
            WSTR txtname,any_text
            ;align 4
            .code
            EXITM <OFFSET txtname>
          ENDIF

          txtname db any_text,0
          ;align 4
          .code
          EXITM <OFFSET txtname>
      ENDM


.data
Me db "Test UASM Debug Compiling",0
.code

Main proc FRAME
    invoke MessageBoxA,0,chr$("Test UASM Debug Compiling"),chr$("Test"),0
    ret
Main endp


start:
    enter 16,0
    invoke Main
    invoke ExitProcess,0
    leave
end start

Title: Re: Stack issues
Post by: LiaoMi on September 22, 2018, 09:16:06 PM
Quote from: AW on September 22, 2018, 08:03:58 PM
@LiaoMi

You are putting too much irrelevant code into your demo. After stripping most of the unneeded code, some still there, I can not second your claim - it does indeed align the stack. This works.



.686
.MMX
.XMM
.x64

option casemap:none
option win64:15;win64:11
option frame:auto
option stackbase:rsp
option dotname
option evex:1
option proc:private
WIN32_LEAN_AND_MEAN equ 1
.nolist
.nocref

HWND typedef ptr
includelib \masm32\lib64\kernel32.lib
ExitProcess proto :dword
includelib \masm32\lib64\user32.lib
MessageBoxA proto :HWND,:PTR,:PTR,:DWORD

; -----------------------------------------
  ; non branching version with no ELSE clause
  ; -----------------------------------------
      chr$ MACRO any_text:VARARG
        LOCAL txtname
        .data
          IFDEF __UNICODE__
            WSTR txtname,any_text
            ;align 4
            .code
            EXITM <OFFSET txtname>
          ENDIF

          txtname db any_text,0
          ;align 4
          .code
          EXITM <OFFSET txtname>
      ENDM


.data
Me db "Test UASM Debug Compiling",0
.code

Main proc FRAME
    invoke MessageBoxA,0,chr$("Test UASM Debug Compiling"),chr$("Test"),0
    ret
Main endp


start:
    enter 16,0
    invoke Main
    invoke ExitProcess,0
    leave
end start


Hi AW,

that's right, the code like yours is the same as my working example, now try to compile the same code, only without enter 16,0 and leave frame.
Title: Re: Stack issues
Post by: nidud on September 22, 2018, 09:31:17 PM
deleted
Title: Re: Stack issues
Post by: LiaoMi on September 22, 2018, 09:53:17 PM
Quote from: nidud on September 22, 2018, 09:31:17 PM
Quote from: LiaoMi on September 22, 2018, 09:16:06 PMnow try to compile the same code, only without enter 16,0 and leave frame.

There's no real alignment done by assemblers or compilers so this is based serialization: one misaligned call will destroy the whole chain.

Hi nidud,

but in 32-bit version its possible without alignment?!  :icon_redface: I dont remember that I had such problems, this feature is only for x64 systems?!
Title: Re: Stack issues
Post by: hutch-- on September 22, 2018, 10:13:41 PM
Have a look at this, its why you are having problems. At the entry point the stack is not correctly aligned, the first PUSH aligns the stack. You can use the ENTER LEAVE pair to do the same task but you must align the entry point for the app to start.

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    include \masm32\include64\masm64rt.inc

    .data
      txt db "MessageBox message text",0
      ttl db "The Title",0

    .code

NOSTACKFRAME

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

entry_point proc           ; < stack is misaligned by 8

    push rbp                ; push aligns the stack with extra 8 bytes
    mov rbp, rsp            ; preserve the stack pointer

    mov rcx, 0
    lea rdx, txt
    lea r8, ttl
    mov r9, MB_OK
    call testproc           ; call a remote MessageBox

    xor rcx, rcx
    call ExitProcess

    mov rsp, rbp            ; never called after ExitProcess
    pop rbp

    ret

entry_point endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

testproc proc

    push rbp                ; push aligns the stack with extra 8 bytes
    mov rbp, rsp            ; preserve the stack pointer

    rcall MessageBox        ; registers already loaded

    mov rsp, rbp            ; restore the stack pointer
    pop rbp                 ; restore rbp

    ret

testproc endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

STACKFRAME

    end
Title: Re: Stack issues
Post by: aw27 on September 22, 2018, 10:30:08 PM
@LiaoMi

I see now what you mean. What Johnsa told me, and it makes all sense, is that start is just a label. Only the linker knows it will be Entry Point.
So, you either have to manually align the stack on the label that will be entry point or use a procedure for the entry point. In the last case  it will align the stack on entry to the procedure, at least it used to.
Title: Re: Stack issues
Post by: nidud on September 22, 2018, 10:38:06 PM
deleted
Title: Re: Stack issues
Post by: LiaoMi on September 23, 2018, 12:43:18 AM
Quote from: hutch-- on September 22, 2018, 10:13:41 PM
Have a look at this, its why you are having problems. At the entry point the stack is not correctly aligned, the first PUSH aligns the stack. You can use the ENTER LEAVE pair to do the same task but you must align the entry point for the app to start.

Thank you!  :t

Quote from: AW on September 22, 2018, 10:30:08 PM
@LiaoMi

I see now what you mean. What Johnsa told me, and it makes all sense, is that start is just a label. Only the linker knows it will be Entry Point.
So, you either have to manually align the stack on the label that will be entry point or use a procedure for the entry point. In the last case  it will align the stack on entry to the procedure, at least it used to.

Thanks for the explanation! Everything is clear now :icon14:  :eusa_dance: In this case, the skeleton will be similar to this

.686
.MMX
.XMM
.x64

option casemap:none
option win64:15;win64:11
option frame:auto
option stackbase:rsp
option dotname
option evex:1
option proc:private
option LITERALS:ON
option PROCALIGN:16
OPTION FIELDALIGN:16; 1|2|4|8|16|32 - The default value is 1 or the value set by cmdline switch -Zp
OPTION FRAME:AUTO;NOAUTO

WIN32_LEAN_AND_MEAN equ 1
.nolist
.nocref

    include C:\masm64\VS2017\include_x86_x64\translate64.inc
    include C:\masm64\sdkrc100\um\windows.inc
    include macros64_Uasm.inc

MBox PROTO
; -----------------------------------------
  ; non branching version with no ELSE clause
  ; -----------------------------------------
      chr$ MACRO any_text:VARARG
        LOCAL txtname
        .data
          IFDEF __UNICODE__
            WSTR txtname,any_text
            ;align 4
            .code
            EXITM <OFFSET txtname>
          ENDIF

          txtname db any_text,0
          ;align 4
          .code
          EXITM <OFFSET txtname>
      ENDM

;     libraries
;     ~~~~~~~~~
    ; ------------------------------------------
    ; import libraries for Windows API functions
    ; ------------------------------------------
   
includelib Kernel32.lib
includelib  User32.lib
includelib  Msimg32.lib
includelib  Comctl32.lib
includelib  Comdlg32.lib
includelib  Shell32.lib
includelib  OleAut32.lib
includelib  Ole32.lib
includelib  Advapi32.lib

.code

WinMainCRTStartup Proc public

    ;enter 16,0 no longer required
    invoke MBox
    invoke ExitProcess,0
    ;leave no longer required
ret

WinMainCRTStartup  Endp

MBox proc

    invoke MessageBox,0,chr$("Test UASM Debug Compiling"),chr$("Test"),0
    ret
   
MBox endp

end WinMainCRTStartup


Quote from: nidud on September 22, 2018, 10:38:06 PM
Quote from: LiaoMi on September 22, 2018, 09:53:17 PM
but in 32-bit version its possible without alignment?!

Not really, no. In 32-bit you may or may not get away with a misaligned stack. You could test this
    inc  esp
    call API

and see what happens.

In 32-bit the alignment is equal to push-size (4) so the stack is (normally) always aligned. In 64-bit the push-size is 8 and the required alignment is 16 so this creates the problem. This means that all calls has to be made with a 16-byte aligned stack. The call pushes IP and jumps which makes all proc-entries aligned 8:

start:          ; aligned 8 on entry
    push rbp    ; add 8 byte
                ; the stack is now aligned 16


All my misunderstandings are corrected, thanks!  :t