Debugging

[jj2007]

- extract the attached *.asm and *.rc files to some folder on your Masm32 drive
- open MiniWinMasm32.asm in \Masm32\qEditor.exe (it is a very simple window template)
- use menu Project/Build all to assemble the file
- use menu Project/Run to see the window

Now, just in case you have a recent MasmBasic installation:
- change the if 1 in line 2 to if 0
- press Ctrl S to save the file
- use menu Project/Console build all to assemble the file
- use menu Project/Run to see the window 

With if 1, your project is a pure Masm32 project. Absolutely no extra code added.

With if 0, you add the full potential of the deb macro to your project. In the attached example, it will
a) tell you if setting the window text was successful, through deb 1, "CreateWindowEx", hEdit, $Err$(), and it will
b) show you (with some exceptions like WM_MOUSEMOVE) all windows messages on entry to your WndProc

[askm]

Currently evaluating your project.
I've shortened actual code to the begin, after using GetLastError further along.
Even got a Trojan:Win32/Wacatac.C!ml flag at some test run.
Is there this and other expected difficulties using your project 64bit ?

Why does this...

include \Masm32\MasmBasic\Res\JBasic.inc
usedeb=1
Init      ; OPT_64 1
deb 4, "error init ", $Err$()
EndOfCode

give this ...

'error init      $Err$()
A dynamic link library (DLL) initialization routine failed.'

( however
'error init      $Err$()         The operation completed successfully.__'
on 32bit with masmbasic.inc)

on 64bit when generated by...
uasm64 -win64 /c /Zp8 /nologo
polink /SUBSYSTEM:CONSOLE /MERGE:.data=.rdata /MERGE:.rdata=.text /entry:start


using your current archive ?

[jj2007]

This is Windows... sometimes there are intermediate errors. In this case, LoadLibrary sets somewhere deep in the guts of Windows an error, but since it does return a handle (otherwise you wouldn't see the error message: printing requires some functions called with the LoadLibrary handle), there is no error. That happens with quite a number of WinAPI functions, and the docs explain this confusing behaviour in a confusing way: "if the function returns zero, then GetLastError will tell you about the error". The docs will not tell you "if the function returns a handle, GetLastError will return meaningless crap". That you will have to find out yourself ;-)

Therefore you must check the return value, and if it's non-zero (e.g. a valid handle), then GetLastError returns an irrelevant non-error. Example:

Code Select Expand

include \Masm32\MasmBasic\Res\JBasic.inc
usedeb=1
Init      ; OPT_64 1
PrintLine "Hello World"
deb 4, "error init ", $Err$()
jinvoke SetLastError, 3 ; can't find that path
jinvoke LoadLibrary, Chr$("Shell32")
deb 4, "error shell32", x:rax, $Err$()
EndOfCode

Before calling LoadLibrary here, you set the error to "can't find that path".
After calling LoadLibrary you got a handle in rax but still the meaningless error message. So LoadLibrary does not set error 0.

P.S.: For the trojan message, we have a dedicated AV Software sh*t list sub-forum.