DialogBoxParam weird problem, please help

[PsYcHoCoDe]

Hi there, gyus... I'm having a strange problem with DialogBoxParam, called from inside a DLL... it returns -1 (eax == 0FFFFFFFFh) and GetLastError says ERROR_SUCCESS (0)... the dialog procedure actually get's called a few times... received messages are WM_SETFONT, WA_CLICKACTIVE and WM_NCDESTROY (in this order). I'm seriously stuck this time, as i fail to spot anything possibly wrong...  Any help on this issue will be greatly appreciated! I'm attaching the complete source code (MASM32) of the dll, along with the make script...

P.S: forgot to mention, the OS is windows 2009 Embedded (XP-alike)...

[dedndave]

EmuGetLastError returns error success
what about the real GetLastError ?   :P

[PsYcHoCoDe]

dedndave, i thought someone would ask this question  I've tested it with the original GetLastError, returns 100% the same... ;)

[dedndave]

then something is happening to cause the dialog to close
it is returning no error, because there is no error condition

[PsYcHoCoDe]

then something is happening to cause the dialog to close -> any ideas about what the problem could be?! i double & triple checked my DlgProc code and i see nothing wrong about it...  :icon_eek:

[qWord]

Shouldn't the resource script contain keywords such BEGIN/END for the declaration of the Dialog?

[dedndave]

it's hard for us to troubleshoot, as it is in a DLL and we would have to write code to call it

but - this code does not look right

Code: [Select]

@its_command:
    mov eax, wParam
    mov edx, eax
    shr edx, 16
    cmp dx, WM_CLOSE
    jz @close_dialog
the high word of wParam is the control identifier if it comes from a control
WM_CLOSE = 10h
perhaps you are getting a message from a control with ID=10h ???

Code: [Select]

Message Source wParam (high word) wParam (low word) lParam
Menu 0 Menu identifier (IDM_*) 0
Accelerator 1 Accelerator identifier (IDM_*) 0
Control Control-defined notification code Control identifier Handle to the control windowsorry for the messed-up table - lol
it appears here, correctly:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms647591%28v=vs.85%29.aspx

on a side note....
a shortcut to get the high word of wParam.....

Code: [Select]

        movzx   edx,word ptr wParam+2

[PsYcHoCoDe]

@qWord: nope, it's compeletely valid under masm32 with no begin/end stuff... actually never saw a masm32 source, using begin/end statements... @least 'till v.10 of masm32 (the version i'm actually using, with ml v.10.x)

@dedndave: i'm attaching the executable, that loads it... it's still a barebone skeleton, so no problem about that... btw the suspicious code works perfectly well for another similar situation, i've coded earlier... and i got no control with an ID value == 10h...

P.S: the attachment contains the binary executable+the icon (ico) file, defined inside rsrc.rc of the dll source... the binary loads the dll in question from the 'current folder'...

[qWord]

The problem are the richedit controls: you must explicit load the corresponding DLL before creating the dialog:

Code: [Select]

fn LoadLibrary,"Riched32.dll"

[PsYcHoCoDe]

god damn it! i've never used richedit controls before, so that could be it, indeed... :/ i'll check it out and report immed...

[PsYcHoCoDe]

hahahaha, pretty much thanks, dudes... the problem was indeed not having explicitly loaded riched32.dll before creation of dialog, while having rich edit controls inside that dialogue! thank u very much, saved my life 

[jj2007]

Riched32.dll is the oldest version around. You have more features with RichEd20.dll, and "RichEdit20A" or "RichEdit20W" as control type.

[PsYcHoCoDe]

I aim at compatibility too, so this should do the job, in spite of being old, but thanks for the info, i didn't knew that... never been much in GUI programming before... :t