A MessageBox Anomaly

[raleep]

Attempting to find a bug, which I thought might be causing an improper value for esp on the return from my main windows procedure, I saved the value of esp at the start of the message loop and compared the saved value with the value just before the return.  The code is:

Code Select Expand


WProc proc wp_hWnd,uMsg,wParam,lParam

WP_0:
    mov    eax,[wp_hWnd]
    mov    [hWnd],eax

    mov    [esp0],esp

    mov    eax,[uMsg]

... etc., and

Code Select Expand


WP_end::
    cmp    esp,[esp0]
      jz    @F

    int    3
@@:
    xor    eax,eax

    ret


I did not find the bug, but I did find an anomaly in the message box routine.
Here is the code:

Code Select Expand


cnfrm:
    mov    esi, OFFSET ldcftxt
    mov    edi, OFFSET ldcfcap

cnfrm_0:
    invoke    SetCursorPos,3C4,250

    mov    edx,[esp0]
    push    edx

    invoke    MessageBox,hwnd,esi,edi,MB_OKCANCEL or MB_DEFBUTTON1

    pop    edx
    mov    [esp0],edx

    cmp    eax,IDCANCEL
    retn


The code using edx should be superfluous, but it is not, because MessageBox changes [esp0].  And what really puzzles me is that it does not seem to matter where in memory esp0 is declared.  The first thing I tried when I discovered the change was to move esp0 out of the .DATA? segment entirely and into a new location in .DATA.  MessageBox found esp0 in its new location and changed it there too.

Can anyone throw some light on what is going on here?


PS a few minutes later:

The relevent declarations are:

Code Select Expand


ldcftxt    DB    "Warning: This file may",0A
    DB    " already be in memory",0A
    DB    " Load Anyway (OK/Enter)",0A
    DB    " Or Abort (Cancel/Escape)?",0
ldcfcap    DB    "Filename & path found",0

and

Code Select Expand


esp0    DD    ?
.

The values for SetCursorPos were arrived at by trial and error and will not work as intended with different screen metrics.

[jj2007]

Post complete code.

[raleep]

Post complete code.

OK.  Here it is.  But I won't ask you to plow through my idiosyncratic code.  I will work on and post a shorter program in more standard coding that shows the problem.

[ragdog]

Your code make the problem is very confused and to understand

[jj2007]

OK.  Here it is.

Thanks. Neither WP_end:: nor cnfrm_0: show this problem in my tests (but I also don't have the file your code expects).
The only reason why this could happen is one push (or pop) too much (too little). Check if the stack is always balanced; one reason might be a push or pop (or a missing one) inside a branch.

P.S.:

Code Select Expand

CALLA MACRO dest ;Call if NCNZ
LOCAL end
  jna end
call dest
end:
ENDM

Nice idea, reminds me of 68000 assembly :t

[dedndave]

i didn't make my way through the entire WndProc code - lol

but - the biggest problem i foresee is that the WndProc return value in EAX is destroyed by POPAD

in many cases, EAX will be 0
but that is not true for messages that pass through DefWindowProc
the value returned by DefWindowProc must be passed when WndProc exits

[dedndave]

your plan to PUSHAD, store ESP, [whatever happens to ESP], then restore ESP, POPAD is kind of messy

the only 4 registers that must be preserved are EBX, EBP, ESI, and EDI
the assembler takes care of EBP (as long as you don't use it for anything else)

so....
1) maintain a balanced stack
2) preserve and restore EBX, ESI, EDI, as required

[raleep]

your plan to PUSHAD, store ESP, [whatever happens to ESP], then restore ESP, POPAD is kind of messy

I agree, it is messy, but I did not restore esp, only compared it to the saved value to test for an unbalanced stack.  The values matched in every case but when MessageBox was called.  That subtracted 0A58h from the saved value, but the actual value of esp always matched the saved value as it existed before being changed by MessageBox.

[raleep]

i didn't make my way through the entire WndProc code - lol

but - the biggest problem i foresee is that the WndProc return value in EAX is destroyed by POPAD

in many cases, EAX will be 0
but that is not true for messages that pass through DefWindowProc
the value returned by DefWindowProc must be passed when WndProc exits

The popad comes before the call to DefWindowProc, which is immediately followed by ret.

[hutch--]

Sad to say you have a "coding" anomaly, not a MessageBox anomaly.

Here is the simple code to test the API.

Code Select Expand


; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
    include \masm32\include\masm32rt.inc
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    .code

start:
   
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    call main

    exit

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

main proc

    LOCAL espreg    :DWORD

    mov espreg, esp         ; copy ESP before any MessageBox call
    fn MessageBox,0,str$(espreg),"ESP before",MB_OK

    mov espreg, esp         ; copy ESP after a MessageBox call
    fn MessageBox,0,str$(espreg),"ESP after",MB_OK

    ret

main endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

end start