The MASM Forum

General => The Workshop => Topic started by: PsYcHoCoDe on September 17, 2012, 11:47:14 PM

Title: DialogBoxParam weird problem, please help
Post by: PsYcHoCoDe on September 17, 2012, 11:47:14 PM
Hi there, gyus... I'm having a strange problem with DialogBoxParam, called from inside a DLL... it returns -1 (eax == 0FFFFFFFFh) and GetLastError says ERROR_SUCCESS (0)... the dialog procedure actually get's called a few times... received messages are WM_SETFONT, WA_CLICKACTIVE and WM_NCDESTROY (in this order). I'm seriously stuck this time, as i fail to spot anything possibly wrong...  :dazzled: Any help on this issue will be greatly appreciated! I'm attaching the complete source code (MASM32) of the dll, along with the make script...

P.S: forgot to mention, the OS is windows 2009 Embedded (XP-alike)...
Title: Re: DialogBoxParam weird problem, please help
Post by: dedndave on September 18, 2012, 01:13:21 AM
EmuGetLastError returns error success
what about the real GetLastError ?   :P
Title: Re: DialogBoxParam weird problem, please help
Post by: PsYcHoCoDe on September 18, 2012, 01:15:11 AM
dedndave, i thought someone would ask this question  :biggrin: I've tested it with the original GetLastError, returns 100% the same... ;)
Title: Re: DialogBoxParam weird problem, please help
Post by: dedndave on September 18, 2012, 01:18:26 AM
then something is happening to cause the dialog to close
it is returning no error, because there is no error condition
Title: Re: DialogBoxParam weird problem, please help
Post by: PsYcHoCoDe on September 18, 2012, 01:35:56 AM
then something is happening to cause the dialog to close -> any ideas about what the problem could be?! i double & triple checked my DlgProc code and i see nothing wrong about it...  :icon_eek:
Title: Re: DialogBoxParam weird problem, please help
Post by: qWord on September 18, 2012, 02:06:03 AM
Shouldn't the resource script contain keywords such BEGIN/END for the declaration of the Dialog?
Title: Re: DialogBoxParam weird problem, please help
Post by: dedndave on September 18, 2012, 02:16:21 AM
it's hard for us to troubleshoot, as it is in a DLL and we would have to write code to call it

but - this code does not look right
@its_command:
    mov eax, wParam
    mov edx, eax
    shr edx, 16
    cmp dx, WM_CLOSE
    jz @close_dialog


the high word of wParam is the control identifier if it comes from a control
WM_CLOSE = 10h
perhaps you are getting a message from a control with ID=10h ???

Message Source wParam (high word) wParam (low word) lParam
Menu 0 Menu identifier (IDM_*) 0
Accelerator 1 Accelerator identifier (IDM_*) 0
Control Control-defined notification code Control identifier Handle to the control window

sorry for the messed-up table - lol
it appears here, correctly:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms647591%28v=vs.85%29.aspx (http://msdn.microsoft.com/en-us/library/windows/desktop/ms647591%28v=vs.85%29.aspx)

on a side note....
a shortcut to get the high word of wParam.....
        movzx   edx,word ptr wParam+2
Title: Re: DialogBoxParam weird problem, please help
Post by: PsYcHoCoDe on September 18, 2012, 02:55:33 AM
@qWord: nope, it's compeletely valid under masm32 with no begin/end stuff... actually never saw a masm32 source, using begin/end statements... @least 'till v.10 of masm32 (the version i'm actually using, with ml v.10.x)

@dedndave: i'm attaching the executable, that loads it... it's still a barebone skeleton, so no problem about that... btw the suspicious code works perfectly well for another similar situation, i've coded earlier... and i got no control with an ID value == 10h...

P.S: the attachment contains the binary executable+the icon (ico) file, defined inside rsrc.rc of the dll source... the binary loads the dll in question from the 'current folder'...
Title: Re: DialogBoxParam weird problem, please help
Post by: qWord on September 18, 2012, 03:21:11 AM
The problem are the richedit controls: you must explicit load the corresponding DLL before creating the dialog:
fn LoadLibrary,"Riched32.dll"
Title: Re: DialogBoxParam weird problem, please help
Post by: PsYcHoCoDe on September 18, 2012, 03:24:58 AM
god damn it! i've never used richedit controls before, so that could be it, indeed... :/ i'll check it out and report immed...
Title: Re: DialogBoxParam weird problem, please help
Post by: PsYcHoCoDe on September 18, 2012, 03:31:04 AM
hahahaha, pretty much thanks, dudes... the problem was indeed not having explicitly loaded riched32.dll before creation of dialog, while having rich edit controls inside that dialogue! thank u very much, saved my life  :eusa_clap:
Title: Re: DialogBoxParam weird problem, please help
Post by: jj2007 on September 18, 2012, 05:44:40 AM
Riched32.dll is the oldest version around. You have more features with RichEd20.dll, and "RichEdit20A" or "RichEdit20W" as control type.
Title: Re: DialogBoxParam weird problem, please help
Post by: PsYcHoCoDe on September 18, 2012, 02:58:37 PM
I aim at compatibility too, so this should do the job, in spite of being old, but thanks for the info, i didn't knew that... never been much in GUI programming before... :t