The MASM Forum

General => The Campus => Topic started by: asmator on December 03, 2015, 01:40:57 AM

Title: Demo programs hang after compilation
Post by: asmator on December 03, 2015, 01:40:57 AM
Hi,

I have a problem executing the demos.

I have compiled (console&Assemble console) the \console\demo1 program successfully. When I execute it, it totally hangs doing nothing. The process stays as a zombie in the tasks. However if I run the pre-compiled "hello.exe" that comes with MASM32, it works just fine. At the beginning I thought it was because I got this error during the installation in my Win8: http://masm32.com/board/index.php?topic=1110.0.

So, to overcome the problem I installed MASM32 in a WinXP, everything ran OK (I didn't get the error I got in Win8) and I overwrote the whole MASM32 directory into my Win8. Now I compile and execute demo1 with no problems.

However, when I try with demo2, I have the same problem again: it compiles but it hangs when running, whereas the pre-compiled proc.exe runs perfectly. I have compared both .exe files to find an explanation, and I get a few differences in the PE header "Virtual Size" of .text and .rdata, and then some differences in what seems addresses for JMP instructions, which I guess is normal.

How can I fix this?  Is it a problem about my installation?

Thanks.
Title: Re: Demo programs hang after compilation
Post by: hutch-- on December 03, 2015, 01:49:51 AM
If you build a console app that has the "inkey" macro without a console, it can never receive the keystroke to make it exit. If its a console app, build it from the Build menu as a console app. Then run it from the command line to see what happens. The "inkey" macro is there so you can run a console app from QE and see the results.
Title: Re: Demo programs hang after compilation
Post by: asmator on December 03, 2015, 02:06:40 AM
Yes, I have done a "console Assembly & Link".  Regardless of how I run the programs (cmd, from qeditor, from Windows explorer) I always get the same results: my proc.exe hangs (and I am not able to kill it manually) and the original proc.exe runs perfectly.   When I mean it hangs I mean that I only get a black screen, nothing is even printed, like if no instruction has been executed. I tried to use a debugger and it also hangs trying to open the file. That is why I think it might be something to do with the PE, because the code seems the same in both programs.

Thanks
Title: Re: Demo programs hang after compilation
Post by: jj2007 on December 03, 2015, 02:15:01 AM
Weird. Can you zip the exe and post it here? Thanks.
Title: Re: Demo programs hang after compilation
Post by: asmator on December 03, 2015, 02:35:33 AM
Find it attached. I included some screenshots of what happens.

Thanks.
Title: Re: Demo programs hang after compilation
Post by: jj2007 on December 03, 2015, 04:53:03 AM
They work just fine. The problem seems to be your machine. Do you have an active antivirus that could intervene?

Btw I also launched it with OllyDbg (http://www.ollydbg.de/version2.html), no problems.
Title: Re: Demo programs hang after compilation
Post by: asmator on December 03, 2015, 07:38:33 PM
Then it is definitely something with my machine... not sure what, though. It is certainly not the code, since it is too simple and it does not even execute the first instruction... So could be something related to the imported functions?  And actually, now I try to run demo1 (hello.exe) and the pre-compiled version still works and mine doesn't - if launched from Explorer it stays like in the photo attached, with a permanent "working" Windows icon . So something is messing with it.

I have compared again both files and overwrite all bytes in my version with the ones that work except for the compilation date in the PE header and any of the code, and it stills hangs. That is strange because I would not think the code has something to do, but some PE value. If using Olly, Olly itself hangs with my version when it works fine on the original (where is good old Sice in this times) - photo attached.

I have compared both codes and strangely, my compilation has many more bytes and instructions for some unknown reason (photo attached). So I could think it is something with the code, but however, one more detail to make my problem fuzzier... I have tested my version in my VM XP and it works!  So there is something funny between the code and my Win8. The only issue I might think is in the directions the code is using to call kernel32.dll, something that in Win XP would work fine but not in Win8.....

Any ideas?
Title: Re: Demo programs hang after compilation
Post by: asmator on December 03, 2015, 08:02:11 PM
jj2007, I think you are right.

After testing every possible change in the executables to isolate the portion that was not working, I got some inconsistencies (like identical programs with different compilation times and one works and the other doesn't). So, many tests after, I disable the antivirus and it seems to work.  ::)  And this is probably the reason why I could not kill the process - even though I could not find any handle taken by the antivirus. Now I have two questions:

- Why my compiller generates much more code to do such a simple thing? It seems there are a lot of instructions that are not really necessary, it does not seem to be translating the assembly directly.
- What can be the reason the antivirus breaks it? Maybe a sandbox that for some reason gets stuck?

Thanks
Title: Re: Demo programs hang after compilation
Post by: GoneFishing on December 03, 2015, 08:08:38 PM
edited


   
Title: Re: Demo programs hang after compilation
Post by: TWell on December 03, 2015, 08:29:03 PM
Quote from: asmator on December 03, 2015, 08:02:11 PM
- Why my compiller generates much more code to do such a simple thing? It seems there are a lot of instructions that are not really necessary, it does not seem to be translating the assembly directly.
You are using masm32 RTL, not API directly.
Title: Re: Demo programs hang after compilation
Post by: asmator on December 03, 2015, 09:45:13 PM
Quote from: TWell on December 03, 2015, 08:29:03 PM
Quote from: asmator on December 03, 2015, 08:02:11 PM
- Why my compiller generates much more code to do such a simple thing? It seems there are a lot of instructions that are not really necessary, it does not seem to be translating the assembly directly.
You are using masm32 RTL, not API directly.

I will look into this to see how it works.

Thank you everyone for the help! That was awesome.
Title: Re: Demo programs hang after compilation
Post by: jj2007 on December 03, 2015, 09:47:50 PM
One thing I see in diff_code.jpg is that "original" has a stack frame (push ebp ... leave) while "my version" doesn't have one.
Title: Re: Demo programs hang after compilation
Post by: hutch-- on December 03, 2015, 10:41:19 PM
asmator,

The trick if you are having AV scanner problems is to either get a better one OR set up another user profile that you can perform normal development in without either the OS or an AV scanner interfering with what you write. Also make sure that you don't have any virus or trojan damage on your computer as this can effect how things work. It would do you no harm to download MalwareBytes, update the data files for it and do a detailed scan of your computer. If there is any damage you can delete the damaged files and if anything has been wrecked you should be able to repair it with the install disk if you have one for the OS version you use.
Title: Re: Demo programs hang after compilation
Post by: asmator on December 03, 2015, 11:16:59 PM
Quote from: jj2007 on December 03, 2015, 09:47:50 PM
One thing I see in diff_code.jpg is that "original" has a stack frame (push ebp ... leave) while "my version" doesn't have one.

Mmm... and how do I configure this behaviour? Is it something coming from "masm32 RTL" somehow?


Quote from: hutch-- on December 03, 2015, 10:41:19 PM
asmator,

The trick if you are having AV scanner problems is to either get a better one OR set up another user profile that you can perform normal development in without either the OS or an AV scanner interfering with what you write. Also make sure that you don't have any virus or trojan damage on your computer as this can effect how things work. It would do you no harm to download MalwareBytes, update the data files for it and do a detailed scan of your computer. If there is any damage you can delete the damaged files and if anything has been wrecked you should be able to repair it with the install disk if you have one for the OS version you use.

Thanks. I use Avast actually (the one blocking), and never had this kind of problem before. I have run MalwareBytes a few times during the past days and it reported nothing, thanks for the suggestion. I have contacted Avast to see why they are preventing the execution.
Title: Re: Demo programs hang after compilation
Post by: jj2007 on December 03, 2015, 11:35:23 PM
Quote from: asmator on December 03, 2015, 11:16:59 PM
Quote from: jj2007 on December 03, 2015, 09:47:50 PM
One thing I see in diff_code.jpg is that "original" has a stack frame (push ebp ... leave) while "my version" doesn't have one.

Mmm... and how do I configure this behaviour? Is it something coming from "masm32 RTL" somehow?

The same source should never produce different code. All our assemblers (ML 6.14 ... 11.0, JWasm, AsmC) do exactly the same. If there is no stack frame, the source was different (see also OPTION PROLOGUE). That is why it is so weird...

QuoteI have contacted Avast to see why they are preventing the execution.

No chance, they won't even answer. Their crappy software has a built-in heuristic analyser that shouts foul if it doesn't find the typical footprints of C/C++ code. Solution is to mark your \masm32 and all its subfolders as "don't touch" in the AV settings. Which is not ideal if you plan to sell the software, of course, but for coding as a hobby it's perfectly OK.
Title: Re: Demo programs hang after compilation
Post by: TWell on December 04, 2015, 12:44:40 AM
Example using msvcrt.dll.386
.model flat, stdcall
option casemap :none

printf proto cdecl :vararg
exit proto cdecl :dword
includelib msvcrt.lib

.data
mystring db "Hello, world!",13,10,0

.code
main:
invoke printf, addr mystring
invoke exit, 0
end main
Title: Re: Demo programs hang after compilation
Post by: asmator on December 04, 2015, 02:11:05 AM
Quote from: jj2007 on December 03, 2015, 11:35:23 PM
The same source should never produce different code. All our assemblers (ML 6.14 ... 11.0, JWasm, AsmC) do exactly the same. If there is no stack frame, the source was different (see also OPTION PROLOGUE). That is why it is so weird...

This is then very strange, because what I compile is the simple hello.asm example (see below), but I can see the compiled code is longer that it should be, and running the file on IDA shows all sort of instructions I did not place there... but the code seems to do the same. I can only think of two sensible things:

1. Would this be related to what I said in my 1st post? i.e. that I have installed MASM32 over XP and then copied it to my Win8 computer.

2. OR I am making some stupid mistake. What I do is opening the .asm in qeditor and pressing "Project - Console Assemble & Link"   (I have also tried" Project - Console Build All" and "Project - Build All"). Perhaps I am not linking in the right way or I have not done any necessary steps after installation (I did not modify anything, I am using all default options).


; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««

;                 Build this with the "Project" menu using
;                       "Console Assemble and Link"

; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««

    .486                                    ; create 32 bit code
    .model flat, stdcall                    ; 32 bit memory model
    option casemap :none                    ; case sensitive

    include \masm32\include\windows.inc     ; always first
    include \masm32\macros\macros.asm       ; MASM support macros

  ; -----------------------------------------------------------------
  ; include files that have MASM format prototypes for function calls
  ; -----------------------------------------------------------------
    include \masm32\include\masm32.inc
    include \masm32\include\gdi32.inc
    include \masm32\include\user32.inc
    include \masm32\include\kernel32.inc

  ; ------------------------------------------------
  ; Library files that have definitions for function
  ; exports and tested reliable prebuilt code.
  ; ------------------------------------------------
    includelib \masm32\lib\masm32.lib
    includelib \masm32\lib\gdi32.lib
    includelib \masm32\lib\user32.lib
    includelib \masm32\lib\kernel32.lib

    .code                       ; Tell MASM where the code starts

; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««

start:                          ; The CODE entry point to the program

    print chr$("Hey, this actually works.",13,10)
    exit

; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««

end start                       ; Tell MASM where the program ends


Result from the Console Assemble nd Link:


Microsoft (R) Macro Assembler Version 6.14.8444
Copyright (C) Microsoft Corp 1981-1997.  All rights reserved.

Assembling: E:\masm32\tutorial\console\demo1\hello.asm

***********
ASCII build
***********

Microsoft (R) Incremental Linker Version 5.12.8078
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.

Volume in drive E is E
Volume Serial Number is BAD6-0CEF

Directory of E:\masm32\tutorial\console\demo1

07/02/04  00:59             1,776 hello.asm
03/12/15  16:13             2,560 hello.exe
03/12/15  16:13               669 hello.obj
               3 File(s)          5,005 bytes
               0 Dir(s)  166,597,718,016 bytes free
Press any key to continue . . .
Title: Re: Demo programs hang after compilation
Post by: dedndave on December 04, 2015, 02:21:50 AM
it shouldn't make any difference which OS you install on - the results should be the same

there are, however, differences in assemblers
in some cases, there are different forms of the same instruction (places where either form will work)
and - some improvements (hopefully) along the way

i suggest you place macros.asm at the end of the includes
(before or after includelibs - doesn't matter)

as for the build....
perhaps you have some other version of ML or LINK in the PATH
check the chain of batch files (qeditor menus reference batch files)
you might verify by building directly from the command line, rather than QE menus
Title: Re: Demo programs hang after compilation
Post by: asmator on December 04, 2015, 06:51:24 PM
Quote from: dedndave on December 04, 2015, 02:21:50 AM
i suggest you place macros.asm at the end of the includes
(before or after includelibs - doesn't matter)

as for the build....
perhaps you have some other version of ML or LINK in the PATH
check the chain of batch files (qeditor menus reference batch files)
you might verify by building directly from the command line, rather than QE menus

I tried to compile & link manually with the same result.

e:\masm32\tutorial\console\demo1>\masm32\bin\ml /c /coff hello.asm
e:\masm32\tutorial\console\demo1>\masm32\bin\Link /SUBSYSTEM:CONSOLE /OPT:NOREF hello.obj


The funny thing is that I did not mess anything. Just install MASM32 and compile the examples in the "tutorial" dir, which I suppose where compiled with the same ml and link I am using. What I am thinking is if the macros are being interpreted in some way that generates the code in the way it does. I imagine "print" will call:

invoke StdOut,expand_prefix(reparg(arg1))

Correct? Because in that case I think my approach should be to try to go one level deeper each time (using invoke -> using invoke without expand_prefix() and reparg() -> using CALL directly) to see what is generating the different instructions.
Title: Re: Demo programs hang after compilation
Post by: dedndave on December 05, 2015, 05:51:19 AM
i can't answer that specific question - never used that macro

but, let's try this...

Microsoft (R) Macro Assembler Version 6.15.8803
Copyright (C) Microsoft Corp 1981-2000.  All rights reserved.

Microsoft (R) Incremental Linker Version 5.12.8078
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.

Title: Re: Demo programs hang after compilation
Post by: dedndave on December 05, 2015, 05:53:43 AM
i am able to build that tutorial program with no problem
it runs in the console (and quickly disappears)

to see it, open a console window first, then type "hello" at the prompt
Title: Re: Demo programs hang after compilation
Post by: asmator on December 07, 2015, 08:28:36 PM
That is not a problem for me either, I am able to execute it when the AV is off. What is strange is that the compiler is producing a very different code (and much bigger to do the same thing) from the one pre-compiled one.

Can you please send me the .exe to verify the code?  (or tell me if the one you compiled is equal to the code in the .exe that was included in masm32)

Not sure this will have any impact, but can you please confirm if you are using MASM32 SDK Version 11 as I do?
Title: Re: Demo programs hang after compilation
Post by: hutch-- on December 07, 2015, 09:46:53 PM
Default build in MASM32.


Microsoft (R) Macro Assembler Version 6.14.8444
Copyright (C) Microsoft Corp 1981-1997.  All rights reserved.

Assembling: K:\masm32\tutorial\console\demo1\hello.asm

***********
ASCII build
***********

Microsoft (R) Incremental Linker Version 5.12.8078
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.

Volume in drive K is disk3_k
Volume Serial Number is F679-CB12

Directory of K:\masm32\tutorial\console\demo1

02/07/2004  10:59 AM             1,776 hello.asm
12/07/2015  09:45 PM             2,560 hello.exe
12/07/2015  09:45 PM               669 hello.obj
               3 File(s)          5,005 bytes
               0 Dir(s)  975,485,345,792 bytes free
Press any key to continue . . .


This is the total code in disassembly.


    start:
    push offset off_00403000
    call fn_00401014

    push 0
    call ExitProcess

    db 3 dup (0CCh)
; ----------------------------
fn_00401014:

    push ebp
    mov ebp, esp
    add esp, 0FFFFFFF4h
    push 0FFFFFFF5h
    call GetStdHandle

    mov [ebp-4], eax
    push DWORD PTR [ebp+8]
    call fn_00401050

    mov [ebp-0Ch], eax
    push 0
    lea eax, [ebp-8]
    push eax
    push DWORD PTR [ebp-0Ch]
    push DWORD PTR [ebp+8]
    push DWORD PTR [ebp-4]
    call WriteFile

    mov eax, [ebp-8]
    leave
    ret 4

    db 6 dup (0CCh)
; ----------------------------
fn_00401050:

    mov eax, [esp+4]
    lea edx, [eax+3]
    push ebp
    push edi
    mov ebp, 80808080h

  lbl0:
    mov edi, [eax]
    add eax, 4
    lea ecx, [edi-1010101h]
    not edi
    and ecx, edi
    and ecx, ebp
    jnz lbl1
    mov edi, [eax]
    add eax, 4
    lea ecx, [edi-1010101h]
    not edi
    and ecx, edi
    and ecx, ebp
    jnz lbl1
    mov edi, [eax]
    add eax, 4
    lea ecx, [edi-1010101h]
    not edi
    and ecx, edi
    and ecx, ebp
    jnz lbl1
    mov edi, [eax]
    add eax, 4
    lea ecx, [edi-1010101h]
    not edi
    and ecx, edi
    and ecx, ebp
    jz lbl0

  lbl1:
    test ecx, 8080h
    jnz lbl2
    shr ecx, 10h
    add eax, 2

  lbl2:
    shl cl, 1
    sbb eax, edx
    pop edi
    pop ebp
    ret 4
Title: Re: Demo programs hang after compilation
Post by: asmator on December 08, 2015, 02:23:24 AM
Thanks, that is reassuring... because this is the exact same output I get.

However, the code seems longer than it should be (I know that in this particular case it does not take more space in the file, tough). This is what jj2007 was mentioning here:

QuoteQuote from: jj2007 on December 03, 2015, 09:47:50 PM
One thing I see in diff_code.jpg is that "original" has a stack frame (push ebp ... leave) while "my version" doesn't have one.

Mmm... and how do I configure this behaviour? Is it something coming from "masm32 RTL" somehow?

The same source should never produce different code. All our assemblers (ML 6.14 ... 11.0, JWasm, AsmC) do exactly the same. If there is no stack frame, the source was different (see also OPTION PROLOGUE). That is why it is so weird...

In particular I see the produced code has this repeating block, not sure why:

mov edi, [eax]
    add eax, 4
    lea ecx, [edi-1010101h]
    not edi
    and ecx, edi
    and ecx, ebp
    jnz lbl1


And this does not really exist neither in the original .asm file (unless the macro is inserting such code) nor in the pre-compiled hello.exe that comes with MASM32 by default. And this "mysterious" code is what makes my AV complain. Not sure why MASM creates such code.

By the way, the generated 0xCC are created for alignment?
Title: Re: Demo programs hang after compilation
Post by: nidud on December 08, 2015, 03:02:13 AM
deleted
Title: Re: Demo programs hang after compilation
Post by: asmator on December 16, 2015, 01:25:22 AM
Ok, it is perhaps worth it to leave the discussion here, concluding: