How do I compile a SUBSYSTEM:WINDOWS for masm32?

[cyrus]

I'm new to masm32. I am using ml.exe, not the other masm32 that exists that can be downloaded. I guess the syntax is a bit different? For ml64.exe I just simply add /SUBSYSTEM:WINDOWS. I'm now programming in masm32 and everything is a lot different here. Just compiling, I need to add many flags I'm not familiar with like /c and /coff and then linking the object files but I was able to do so in console mode. For windows mode, I am getting this error:

msvcrt.lib(exe_winmain.obj) : error LNK2019: unresolved external symbol _WinMain@16 referenced in function "int __cdecl __scrt_common_main_seh(void)" (?__scrt_common_main_seh@@YAHXZ)

I tried putting WinMain instead of main in proc and endp but no difference.

[NoCforMe]

Looks like you're using some kind of external library (containing a function "__scrt_common_main_seh") that might be the problem. What libraries or external object modules are you using?

[cyrus]

Ok I was able to figure this out. I'm using these libraries:

includelib ucrt.lib
includelib msvcrt.lib
includelib legacy_stdio_definitions.lib

I changed main to WinMain but before WinMain proc, I put entry: like so:

entry:
;main proc
WinMain proc

At the end:

WinMain endp
;main endp
end entry

[NoCforMe]

Yes; remember that

Code Select Expand

end [label]at the end of your code sets the entry point for your executable to label. Nothing "magic" about that name.

[cyrus]

oh yes I know I could use anything for the label, I guess when they moved to ml64, they probably realized it was a pain to use WinMain and just allowed people to go with the traditional main proc and main endp

[NoCforMe]

Nothing magic about that name either. My standard top o'code:

Code Select Expand

.code

start: INVOKE GetModuleHandle, NULL
MOV InstanceHandle, EAX

CALL WinMain
INVOKE ExitProcess, EAX

You could name it "Turkey" if you wanted to and it would still work. It's just an internal function name.

[cyrus]

I'm going to have to study all the INVOKE stuff because typically I just include the libraries. That means I am not including from a masm directory like I see a lot of code here is. I just do includelib kernel32.lib. I am just coding it in a more raw form I guess. It's more fun to code in 32 bit lately to be honest but the majority of my code is 64-bit. I wrote a pretty complex program just now with GNU as (GAS) on Windows using TDM-GCC as and ld and just converted it to masm. I don't think it's possible to run the GAS program with a subsystem:windows way just because it such an old compiler, unless I don't know of any. I like coding in either AT&T or intel, I know both syntaxes but intel is just easier to type lol.

[NoCforMe]

Well, if you're going to use INVOKE, then you need prototypes for all your functions. Do you have those? (Should be in an .include file.) Otherwise you'll need to do the ugly old-school push-push-call.

[cyrus]

I'm doing the ugly old school push push call but i like doing that. The only thing that trips me up a bit is when I add esp, 8 for 2 push calls, for some functions, the entire program does not work. I know I absolutely need to do that inside a loop such as recursion, but for other functions it varies. It's real strange. I'll have to post some code later in another thread to explain.

[NoCforMe]

I get it. So long as you're comfortable with that style (and assuming you absolutely know which functions take which parameters), then there should be no problem.

Of course push-push-call is very much less maintainable and very much more error-prone than INVOKE. (I'm an anti-macro guy for the most part, but I couldn't see coding without INVOKE!)

[cyrus]

Yeah I'm fine with that coding style. I'm using the x86 developer command prompt from Visual Studio to run ml.exe so I can either just import the libraries directly with includelib or use PEB which I had to do with GNU as

Surprisingly, I went back and just uncommented all the add esp, 8 or whatever relative to any push (4 bytes each push), and recompiled and seems to work. i had a lot of bugs in my code. just getting the recursive function (to recurse into directories), I had to use the stack on one part of it because using variables would not work. It was the most difficult function I ever wrote. I used some guidance from disassembling from c++ code but the machine uses the stack entirely and so when I tried to make mine all stack, it didn't work. i'm pretty sure i did something wrong. but 100% i needed the stack for the actual recursion section where it calls itself over and over to reach all directories.

I do enjoy coding in 32-bits for ease but also for portability with both windows and linux systems since I could just install multilib for gcc.

[NoCforMe]

I just finished (part of) a project that uses recursion; it was super-easy. I did have to increase the stack size to make it work. Is your stack large enough? (Using MASM, it's .stack <stack size>).

[cyrus]

I just finished (part of) a project that uses recursion; it was super-easy. I did have to increase the stack size to make it work. Is your stack large enough? (Using MASM, it's .stack <stack size>).

that just shows how much of a newbie i am after 2 years of assembly (and i first learned it 6 years ago but only 32-bit linux AT&T. just basic stuff). i didn't even know i could increase the stack size but the default is more than plenty for what i am doing. the recursion was difficult for me because i was just so frustrated why it didn't work. i ended up cheating by using the disassembler but i find it useful for other projects as well. you have to use the stack and it is just so strange. for sure coding in C or C++ is such a joke compared to assembly. and i thought C was intimidating when my college professor told me to learn it 12 years ago!

[NoCforMe]

Welll, drifting off-topic a bit maybe, but C isn't very much different from assembly language. In fact, it's damn close to assembly, with a few layers of abstraction on top. Stack usage is pretty much the same as C in any assembly program that uses functions, which have their parameters pushed on the stack (and that use local variables, also on the stack).

BTW, using a debugger is definitely not cheating (unless you're some kind of friggin' genius!).

C++, fuggedaboudit ...

[tenkey]

... when I add esp, 8 for 2 push calls, ...

That's two pops.
Two pushes is

Code Select Expand

sub esp, 8