MASM INVOKE with a WORD may cause crashes, wild branches, bad RCs, etc

[MtheK]

  I wanted to colorize a line of STDOUT, but I couldn't get the
'color' DOS Prompt command to do so (all or none?), so, to the
assembler we go...

  I can't say I've ever done this with a WORD of storage, but when I
code this to ref a variable from GetConsoleScreenBufferInfo:

Code Select Expand



WFLASHSTAT DWORD  1                       ; 0=set color OK (dft is bad)
SOCOLORA WORD  ?   
         INVOKE SetConsoleTextAttribute,
               SOHANDLE,
               SOCOLORA                   ; try x'b0' (see WFLASH.BAT)

as opposed to a fixed value:

         INVOKE SetConsoleTextAttribute,
               SOHANDLE,
               BACKGROUND_BLUE+BACKGROUND_GREEN+BACKGROUND_RED+BACKGROUND_INTENSITY  ; (reverse)



and using WinDbg to verify the ESP, I get a mis-match of the former but not the latter:

Code Select Expand



;           push   000h
;eax=00000001 ebx=0041120d ecx=f5ae5e69 edx=77866344 esi=00000000 edi=00000000
;eip=0040382f esp=0012ff64 ebp=0012ff94 iopl=0         nv up ei pl zr na pe nc
;cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
;OPENWAIT!mainCRTStartup+0x282f:
;0040382f 6a00            push    0
;0:000>
;eax=00000001 ebx=0041120d ecx=f5ae5e69 edx=77866344 esi=00000000 edi=00000000
;eip=00403831 esp=0012ff60 ebp=0012ff94 iopl=0         nv up ei pl zr na pe nc
;cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
;              ESP loses 4

;           push   SOCOLORA
;eax=00000001 ebx=0041120d ecx=f5ae5e69 edx=77866344 esi=00000000 edi=00000000
;eip=00403838 esp=0012ff5e ebp=0012ff94 iopl=0         nv up ei pl zr na pe nc
;cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
;              ESP loses 2

;           push   SOHANDLE
;eax=00000001 ebx=0041120d ecx=f5ae5e69 edx=77866344 esi=00000000 edi=00000000
;eip=0040383e esp=0012ff5a ebp=0012ff94 iopl=0         nv up ei pl zr na pe nc
;cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
;              ESP loses 4

;           call   SetConsoleTextAttribute
;eax=00000000 ebx=0041120d ecx=f5ae5e69 edx=000003e6 esi=00000000 edi=00000000
;eip=00403843 esp=0012ff62 ebp=0012ff94 iopl=0         nv up ei pl zr na pe nc
;cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
;              ESP is NOT(!!!) back to starting point !!!??? LOSS OF 2! Caused:
;* S0C4s (INT3s in 'clear PID entry', a wild branch?), rcs 87 and 998, etc...

---

;           push   +0000000F0h
;eax=00000001 ebx=0041120d ecx=fe49a87c edx=77866344 esi=00000000 edi=00000000
;eip=0040382f esp=0012ff64 ebp=0012ff94 iopl=0         nv up ei pl zr na pe nc
;cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
;OPENWAIT!mainCRTStartup+0x282f:
;0040382f 68f0000000      push    0F0h
;0:000> p
;eax=00000001 ebx=0041120d ecx=fe49a87c edx=77866344 esi=00000000 edi=00000000
;eip=00403834 esp=0012ff60 ebp=0012ff94 iopl=0         nv up ei pl zr na pe nc
;cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
;              ESP loses 4

;           push   SOHANDLE
;eax=00000001 ebx=0041120d ecx=fe49a87c edx=77866344 esi=00000000 edi=00000000
;eip=0040383a esp=0012ff5c ebp=0012ff94 iopl=0         nv up ei pl zr na pe nc
;cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
;      ESP loses another 4

;           call   SetConsoleTextAttribute
;eax=00000001 ebx=0041120d ecx=fe49a87c edx=77866344 esi=00000000 edi=00000000
;eip=0040383f esp=0012ff64 ebp=0012ff94 iopl=0         nv up ei pl zr na pe nc
;cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
;         ESP back to starting point; colors work properly by line



for some reason, the former INVOKE has an extra PUSH.  So, after the
call, the stack is off by 2, causing later havoc.  All my programs
deliberately go back to the OS via RET in the main thread, to catch
things like this.

  I'm obviously not coding the INVOKE correctly, but I'm not sure
what will work. I can't even think of an API that I use with a WORD
(not OFFSET), or even decent keywords to use in a search. Most of
the examples I looked at use DWORDs. I ended up taking the
instructions in the .lst (latter) and hard-coded them in my global
macro and replaced the first PUSH with a 'PUSH EAX' to allow
variability. This works correctly; the ESP also is correct.

  Comments? Thankx...

[Vortex]

Concerning 32-bit coding, the stack is aligned to 32-bit so you must be careful why passing parameters to the stack :

Code Select Expand

BOOL WINAPI SetConsoleTextAttribute(
  _In_  HANDLE hConsoleOutput,
  _In_  WORD wAttributes
);


http://msdn.microsoft.com/en-us/library/windows/desktop/ms686047%28v=vs.85%29.aspx

HANDLE hConsoleOutput is a DWORD ( 32-bit ) value but you are passing a WORD value to the stack :

Code Select Expand

SOCOLORA WORD  ?

[MtheK]

  Well, yea, since the doc seems to indicate it's a WORD. When getting the info,
SOCOLORA gets a WORD  returned (x'0700' backwards). So is 'wAttributes' a WORD or a DWORD? And in any case, why the extra PUSH? 3 PUSHes for 2 variables?

[Vortex]

Here is a quick example for you :

Code Select Expand

include     \masm32\include\masm32rt.inc

; The value of BACKGROUND_BLUE is 10h

SOCOLORA equ 10h

.data

string      db 'This is a SetConsoleTextAttribute test.',0

.code

start:

    invoke  GetStdHandle,STD_OUTPUT_HANDLE

    invoke  SetConsoleTextAttribute,eax,SOCOLORA

    invoke  StdOut,ADDR string

    invoke  ExitProcess,0

END start

Disassembling or debugging the code :

Code Select Expand


push    -11
call    _GetStdHandle@4
push    16
push    eax
call    _SetConsoleTextAttribute@8

As you can see, SOCOLORA can be a 8 , 16 or a 32-bit value but it's passed as a

[Vortex]

Here is a quick example for you :

Code Select Expand

include     \masm32\include\masm32rt.inc

; The value of BACKGROUND_BLUE is 10h

SOCOLORA equ 10h

.data

string      db 'This is a SetConsoleTextAttribute test.',0

.code

start:

    invoke  GetStdHandle,STD_OUTPUT_HANDLE

    invoke  SetConsoleTextAttribute,eax,SOCOLORA

    invoke  StdOut,ADDR string

    invoke  ExitProcess,0

END start

Disassembling or debugging the code :

Code Select Expand


push    -11
call    _GetStdHandle@4
push    16
push    eax
call    _SetConsoleTextAttribute@8

As you can see, SOCOLORA can be a 8 , 16 or a 32-bit value but it's pushed as a DWORD value to the stack. The trick is that a WORD value can fit a DWORD value.

[MtheK]

  Ah, I see. 'wAttributes' is NOT a piece of storage, it s/b just an EQUATE.
Perhaps INVOKE got confused expecting a value, not a reference?

[MtheK]

  In your example, if u make SOCOLORA a WORD, how many PUSHes are generated in your listing? I get 3...

[dedndave]

it may help to look at \masm32\include\kernel32.inc

you will see that it's prototyped as a DWORD
older versions of masm did have problems with byte and word INVOKE arguments
the stack must be 32-bit aligned for 32-bit code  :P

[jj2007]

older versions of masm did have problems with byte and word INVOKE arguments

On top of the list of known Masm bugs

[MtheK]

 I probably have an older ML.exe, cause this is what came up from that example:

00400fff 006af5          add     byte ptr [edx-0Bh],ch
00401002 e827000000      call    test!GetStdHandle (0040102e)
00401007 6a00            push    0
00401009 66ff3500304000  push    word ptr [test!SOCOLORA (00403000)]
00401010 50              push    eax
00401011 e81e000000      call    test!SetConsoleTextAttribute (00401034)

3 PUSHes.

Thankx.

[Vortex]

  In your example, if u make SOCOLORA a WORD, how many PUSHes are generated in your listing? I get 3...

Exactly. A quick fix is like the following :

Code Select Expand

.data

SOCOLORA    dw 10h,0 ; the 0 value is required to construct a DWORD value. This will also avoid
                                     ; hitting the next string definition

string      db 'This is a SetConsoleTextAttribute test.',0
.
.

.code

    invoke  SetConsoleTextAttribute,eax,DWORD PTR SOCOLORA

The result is :

Code Select Expand


push    dword ptr [SOCOLORA]
push    eax
call    _SetConsoleTextAttribute@8

More logical :

Code Select Expand


SOCOLORA    dd 10h
.
.

.code

.
.
    invoke  SetConsoleTextAttribute,eax,SOCOLORA

[dedndave]

there are a few ways to fix it   

if you are pushing a constant, masm should take care of it, automatically

of course, you can just push a dword, including the word that follows in the data area
chances are, the high word is ignored

or, set the high word to 0, as Vortex shows

another way.....

Code Select Expand

    INVOKE  GetStdHandle,STD_OUTPUT_HANDLE
    movzx   edx,word ptr wColors
    INVOKE  SetConsoleTextAttribute,eax,edx

[MtheK]

  WORD PTR; I forgot about that. I see that a lot  (xxx PTR) in WinDbg tracing in
other programs. Thankx all.

[Antariy]

Some info here: http://www.masmforum.com/board/index.php?topic=15342.0

ML9 and above has fixed this bug

[dedndave]

hi Alex   :t