I have just encountered a very strange Visual Studio problem. My system is running Win 7. I am using VS 2008 as a JIT debugger.
My program is assembled with MASM 8.0 (I tried both with MASM 8.0 and also Masm 9.0, both reacted the same way). This program is structured the same way as I structure my other programs. I have just checked another program and it works correctly. Only this one program fails.
The program structure (same structure as other programs, just different data):
INCLUDE <.\pkg\prog.pkg
INCLUDE <.\inc\prog.inc
INCLUDE <.\inc\WINAPI
INCLUDE <.\inc\Runtime
etc.
INCLUDE <.\pkg\DataDef.pkg
.code
main PROC
int 3
call GetMemory
.
.
.
The int 3 is the first thing executed in the program proper. VS comes up, but instead of displaying the program package code, it is displaying the empty lines at the end of the DataDef.pkg. If I bring up the dissassembly window, VS properly displays the dissassembly code with the int 3 and the call GetMemory. As soon as I step into the GetMemory call, VS properly pops up the code window with the GetMemory source code. If I step out of the GetMemory code, I am back at the dissassembly window again. If I switch to the DataDef.pkg and step through the "code", VS walks through the empty lines at the end of the DataDef.pkg.
For some reason, MASM or Link or VS has forgotten where the main PROC code is (everything following the INCLUDE <.\pkg\DataDef.pkg.
If I manually open the prog.pkg file in VS and try to set a breakpoint, I get the nasty message that there is no code associated with those locations so the breakpoint is disabled.
Has anyone else run into this? If so, what did you "fix" to get out of it?
Dave.
Hi,
What is the command line for ML ?
Once I had much the same problem with WinDbg. It seems like the debugger knows about the associated source code files but can not see the lines' numbers.
I checked ML options and added /Zf switch to already existing /Zi.
HTH
Regards
vertograd
Thanks for the hint, but it did no good. Same problem. In addition, my other programs that work use the same ML options. I tried it anyway, no help.
Dave.
DAVE,
What in general terms is the content of the file: DataDef.pkg ???
I searched over at MSDN, and found this page: File Types Created for Visual C++ Projects (http://msdn.microsoft.com/en-US/library/vstudio/3awe4781.aspx). I have never seen a file with the .pkg extension in one of my Visual Studio project directories.
Zen,
.pkg is just a private extension. MASM doesn't care, it is just an INCLUDE of a file.extension. I use it to indicate a package of procedures or data definitions. I do not use libraries, just INCLUDES of my source (except for kernel32.lib), I roll my own runtime.
Everything (since 2001 when I started my MASM career) has run, until now with this program.
The data in the DataDef.pkg is nothing but data definitions (dxxxx DWORD 0) (szString BYTE "msg",10,13,0) etc.
Dave
Can you post the content of Output (debug) window?
vertograd,
I have attached a zip of 4 screens. The first is the first interruption at the start of Main and it shows that VS thinks the code is at the end of the DataDef.pkg INCLUDE file. The second is with a disassembly window and it shows the actual code at the start of Main. The third is a disassembly window after I stepped into the Get Memory procedure which shows that VS knows about the GetMemory source file. The fourth is the code in GetMemory. If I step out of GetMemory and return to the caller, I end back at the disassembly window but there is no code window onlt the end of the DataDef.pkg.
Dave.
Dave,
I can't see if the Output window is empty . Though it's likely to be.
If you say that :
Quote from: KeepingRealBusy on July 30, 2013, 02:12:54 AM
This program is structured the same way as I structure my other programs. I have just checked another program and it works correctly. Only this one program fails.
The program structure (same structure as other programs, just different data):
I thought that the reason of the strange behaviour of the debugger might be in your code (its purpose , for example ). Don't you test some anti-debugging features, do you?
From GetMemory.pkg
Quote
; Allocate all of memory except for a reserved block for JIT debugger.
vertograd,
I get the problem even before I allocate memory, right at Main entry.
My memory allocation always allocates the largest block possible (VirtualAlloc), and does this 10 times, then de-allocates a block thought to be large enough to bring in the debugger then does an int 3. If the program dies with no messages (instant death), then the reserved block is not large enough, double the size and try again until you get the breakpoint. If you get the breakpoint immediately, then reduce the reserved size by a half and try again until you fail (saving the last working size until failure. Now you have the lowest and highest size for success and failure, so try for the middle of that range and replace the lower limit if failure or the higher limit for success and try again until you get to a size such that that (high - low) is a system page size (4 KB). You now have the minimum size required by the debugger loading. Can that value and allocate all of memory and deallocate the canned reserve size from the smallest allocation (delete that allocation, allocate the reserved size, allocate the rest then delete the4 reserved size).
If there is any interest, I can supply a zip with my procedures, probably in the form of a working program that only starts by allocating all memory, then gets a huge number of large blocks, then takes an int 3 so you can examine the allocations. This is all 32 bit code, I have not tried this with 64 bit code.
I use this method for all of my memory needs.
Dave.
Dave,
It does sound interesting but I must confess I'm a newbie learning MASM basics .
This whole thread confuses the hell out of me,...but, it seems like an interesting problem.
I have several questions:
- Your program is failing for some unknown reason. It is written in assembly language and compiled with a version of MASM. You are using VS 2008 as a JIT debugger, which apparently launches when your program crashes. Why are you using a NET Framework JIT debugger for native code ?
- Why not use OllyDbg (http://www.ollydbg.de/) ??? Assembly programmers seem to prefer it over other inferior debuggers.
- Where exactly is the program failing ? Do you have exception handlers written and added to the exception handler chain ? (Visual Studio does this automatically, whereas MASM does not.)
- Have you tried to compile other versions of your program, using variations (and commenting out lines of code that could possibly contribute to the error) to accurately identify the problem ?
...The reason I'm asking all these annoying questions is that, it is well known that loading an executable into a debugger often alters the sequence of execution of the processor instructions. This is a very subtle error,...and extremely difficult to identify.
Of course, I could be wrong,...I almost always am,... :lol:
Zen,
My program is not failing. I inserted an int 3 at the entry to Main so I could step through the code. What is failing is Visual Studio when it is loaded to process the int 3 Exception.
I am using Visual Studio debugger because I am familiar with it. I tried OllyDbg but liked VS better.
The failure occurs right at the entry to the program. No code has been executed except for the int 3.
I have assembled the code with both MASM 8.0 and MASM 9.0. Both fail the same way.
I have removed all code and data from my program except my memory allocator and the very small data required for it. The stripped down version also has the VS error - brings up the data include file when it is, in fact, executing the first instruction in the code segment - the int 3. The debug window properly shows the code lines intermixed with lines from the DataDef include file.
This same structure with different data works correctly for another of my programs with an inserted int 3 at the start of the program thus invoking VS.
Finally, I am not loading an executable into a debugger, it is loaded normally by the system and the debugger only gets involved when the int 3 is executed. There are no sequences of processor instructions that could be altered.
I am not positive, but I am now thinking that the problem may be that the code segment following the INCLUDE of the DataDef may be so small that it is causing problems??? All of the other programs that I use and that run correctly have large code segments.
Dave.
Quote from: KeepingRealBusy on July 31, 2013, 10:06:04 AM
I have assembled the code with both MASM 8.0 and MASM 9.0. Both fail the same way.
You could try Masm v6.15. It writes the old COFF line-number debug info, unlike Masm 8 or 9, which write the new CV8 format. Actually, the new format is better organized and easier to handle, so one would think that if the new format fails, the probability for the old forma to fail is 150%, but you never know.
In case you need to use Masm 8+: I just uploaded the most up-to-date PE version - this tool allows to display CV8 debug info, including line-number info. The output is probably way too technical for most people, but it should at least give a hint if something is "corrupted". Of course, IF something is corrupted the question will arise what to do then - AFAIK MS is not going to fix errors in Masm ASAP.
DAVE,
Ahh Hah !!! It all makes sense to me now. I wondered what the int 3 instruction was there for.
So, you're looking at an exception, and Visual Studio is your registered debugger.
...I googled the phrase: int 3 exception,...and, found this discussion at StackOverflow: Catching Opcode 0xCC as an Exception (http://stackoverflow.com/questions/6034783/catching-opcode-0xcc-as-an-exception)
The person that answers the inquiry states: "Please note that a code running with a debugger attached will not see the exception, because the debugger handles it and clears it before passing the hand back to the code."
Here is the MSDN webpage about: Structured Exception Handling (http://msdn.microsoft.com/en-us/library/windows/desktop/ms680657(v=vs.85).aspx)
...I seem to remember that EDGAR was involved in writing an assembly program to output the information from an exception handler, about a year ago,...and the question was so complicated that no one understood what he was talking about,...except EDGAR, of course.
Here is the link to that Old UK Forum thread: AddVectoredExceptionHandler Single Stepping, Mar 2011 (http://www.masmforum.com/board/index.php?topic=16242.0)
If you search the Old UK MASM Forum (using the advanced search feature, check all) for the search term: exception,...and by user: donkey,...you will find a number of Forum posts (EDGAR often writes in the GoAsm forum) that deal with exception handling in MASM.
Here is an example of typical EDGAR genius: Monitoring a Memory Address, May 2011 (http://www.masmforum.com/board/index.php?topic=16599.0),...
or: Breakpoint problem, Aug 2009 (http://www.masmforum.com/board/index.php?topic=12069.0)
...or, the mind-boggling EDGARism: Reading Raw Debug Symbol Data, Mar 2010 (http://www.masmforum.com/board/index.php?topic=13686.0)
...Thank GOD all the Canadians aren't as smart as EDGAR,...or, they would have taken over the world long ago,...
...What do you think ?
Quote from: Zen on August 01, 2013, 03:20:35 AM
code running with a debugger attached will not see the exception, because the debugger handles it and clears it before passing the hand back to the code.
What happens if Olly is your JIT is that Windows greets you with a fat MsgBox; if you click Cancel or hit Escape, Olly takes over and says "INT 3 command at ... ". You can press F8 to continue stepping through the code.
Olly shields you from going through KiUserExceptionDispatcher, that's all.
P.S.: EDGAR has recently been a bit silent, therefore here an example of SEH in Genuine MASM
TM:
include \masm32\MasmBasic\MasmBasic.inc ; download (http://masm32.com/board/index.php?topic=94.0)
Init tc ; initialise MasmBasic with Try/Catch (aka SEH)
xor ecx, ecx
Try
div ecx ; do something illegal
MsgBox 0, Str$("div succeeded, eax is %i\n", eax), "SEH Test:", MB_OK ; you won't see that box
Catch Only
MsgBox 0, LastEx(info), "SEH in Masm32: div ecx failed", MB_OK
Finally
Exit
TryCatchEnd
end start
Output:
{ERRORE DI EXCEPTION}
Divisione intera per zero.
EIP 0040104B
Code C0000094And don't blame
me for translating "integer division" with "entire division", it's our friends in Redmond who can't afford professional translators.
Zen,
I have pealed this stinking onion to it core. This is what sequence makes it fail or work:
;
; This runtime code fails, i.e. VS takes the breakpoint but displays the data segment.
;
Main PROTO
GetSplitBuffers PROTO
.code
ALIGN OWORD
WXYZ PROC
WXYZ ENDP
;
; This runtime code works, i.e. VS properly takes the breakpoint and displays the code segment.
;
Main PROTO
GetSplitBuffers PROTO
.code
I have attached this stripped down process. You will need to assemble it but you will have to change the ML and LINK lines in makeit.bat (probably to just ML and LINK) because I selectively execute either ML80 and LINK80 or ML90 and LINK90. It is set up to fail, but to make it run good, copy .\pkg\runtime.good to .\pkg\runtime.pkg and re-assemble.
If you or anyone else here has VS 10.0, assemble it with MASM 10.0 and see if it still fails and check if runtime.good makes it work correctly.
In the failing mode, VS displays the data segment as if it were the code segment. To see the int 3, bring up the debugger window and you will see the int 3 code. Then step into GetMemory where there is another int 3. You will still be in the debugger window, but the GetMemory code will be present. If you switch to the GetMemory code window, then you will see the actual code. If you then step out of GetMemory, you will be back at the debugger window following the int 3 that was executed. If you open up the VSTest file and attempt to set a breakpoint anywhere, you will get a disabled breakpoint (a round red circle instead of a red dot with a warning yellow triangle with an '!' in it, if you hover over that triangle, the message comes up saying that there is no code associated with that location). You can just kill the process or just give it a run (F5) and it will terminate normally. As I said, the program works, it is just VS that is confused.
In the good mode, VS will come up with VSTest displayed and work normally.
Dave.
Thanks for the test case!
It's a Masm bug, not a VS or MS debug engine bug.
Here's the line-number info generated by ML:
Source filenames at 4, size D6:
1: name=C:\TEMP\VSTest\VSTest.asm
1B: name=C:\TEMP\VSTest\pkg\VSTest.pkg
39: name=C:\TEMP\VSTest\inc\VSTest.inc
57: name=C:\TEMP\VSTest\pkg\Win32api.pkg
77: name=C:\TEMP\VSTest\pkg\Runtime.pkg
96: name=C:\TEMP\VSTest\pkg\DataDef.pkg
B5: name=C:\TEMP\VSTest\pkg\GetMemory.pkg
Files at E4, size A8
1: ofs= 1 type= 110, chksum=3360C5AD-FC496EF4-E0F70BF1-D06E73DC
2: ofs= 1B type= 110, chksum=1D918322-BE51DA2D-88362C32-DFA921F6
3: ofs= 39 type= 110, chksum=B617C07C-31206FCE-71B9A6DE-C3D71097
4: ofs= 57 type= 110, chksum=D80EE68A-C9F561AE-11C7B6AE-C6BD7878
5: ofs= 77 type= 110, chksum=865F4701-6C93B025-0647E748-5A10F574
6: ofs= 96 type= 110, chksum=7EA41EED-0F512590-49A8123C-3B46D037
7: ofs= B5 type= 110, chksum=F035DFC7-5C4DDD3D-8A4A113E-1F043A33
Line numbers at 194, size 80:
section: start=0, idx=0, length=12
file srcfile=60(=5) items=2 length=1C [Runtime.pkg]
# Offset Line
-----------------------
1: 0 5
2: 0 6
file srcfile=78(=6) items=5 length=34 [DataDef.pkg]
# Offset Line
-----------------------
1: 1 5
2: 1 6
3: 2 7
4: 7 8
5: 9 10
file srcfile=90(=7) items=3 length=24 [GetMemory.pkg]
# Offset Line
-----------------------
1: 10 2
2: 10 3
3: 11 4
Masm is confused and writes line-number info for file DataDef.pkg instead of file VSTest.pkg.
Japeth,
Thank you for the information.
Now, let me display my ignorance. What program did you use to make this determination? I assume JWASM does not have this problem, but if I use JWASM to assemble, will I still be able to use VS as my debugger? Is the latest JWASM available for download at SourceForge?
Dave.
Japeth,
One more item. Visual Studio has a bug as well because MASM 9.0 is packaged with Visual Studio 2008 and it fails just like MASM 8.0. Just sayin.
Dave.
Japeth,
Finding a working solution or placing blame on any program (MASM or VS) is still not an adequate answer. As I have explained, I have many programs with the same structure AND running with the same runtime.pkg that do not fail in this way. There has to be some other thing causing MASM to do such a thing.
In the mean time, I have a workaround. I copied all of the code in VSTest.pkg to a new Start.pkg and moved the trailing includes to Start.pkg, and added one INCLUDE in the VSTest.pkg to includeStart.pkg. VSTest.pkg still has the initial INCLUDES and only a 3 line PROC that just CALLS Start. Now when I start, Main starts and CALLS Start and Start has the initial int 3. I get the int 3 interrupt and VS displays the Start.pkg code and everything is otherwise normal.
Dave.