Strange Visual Studio problem

[KeepingRealBusy]

Zen,

I have pealed this stinking onion to it core. This is what sequence makes it fail or work:

Code Select Expand


;
;   This runtime code fails, i.e. VS takes the breakpoint but displays the data segment.
;
Main                        PROTO
GetSplitBuffers             PROTO
.code
ALIGN   OWORD
WXYZ    PROC
WXYZ    ENDP


Code Select Expand


;
;   This runtime code works, i.e. VS properly takes the breakpoint and displays the code segment.
;
Main                        PROTO
GetSplitBuffers             PROTO
.code


I have attached this stripped down process. You will need to assemble it but you will have to change the ML and LINK lines in makeit.bat (probably to just ML and LINK) because I selectively execute either ML80 and LINK80 or ML90 and LINK90. It is set up to fail, but to make it run good, copy .\pkg\runtime.good to .\pkg\runtime.pkg and re-assemble.

If you or anyone else here has VS 10.0, assemble it with MASM 10.0 and see if it still fails and check if runtime.good makes it work correctly.

In the failing mode, VS displays the data segment as if it were the code segment. To see the int 3, bring up the debugger window and you will see the int 3 code. Then step into GetMemory where there is another int 3. You will still be in the debugger window, but the GetMemory code will be present. If you switch to the GetMemory code window, then you will see the actual code. If you then step out of GetMemory, you will be back at the debugger window following the int 3 that was executed. If you open up the VSTest file and attempt to set a breakpoint anywhere, you will get a disabled breakpoint  (a round red circle instead of a red dot with a warning yellow triangle with an '!' in it, if you hover over that triangle, the message comes up saying that there is no code associated with that location). You can just kill the process or just give it a run (F5) and it will terminate normally. As I said, the program works, it is just VS that is confused.

In the good mode, VS will come up with VSTest displayed and work normally.

Dave.

[japheth]

Thanks for the test case!

It's a Masm bug, not a VS or MS debug engine bug.

Here's the line-number info generated by ML:

Code Select Expand


Source filenames at 4, size D6:
  1: name=C:\TEMP\VSTest\VSTest.asm
1B: name=C:\TEMP\VSTest\pkg\VSTest.pkg
39: name=C:\TEMP\VSTest\inc\VSTest.inc
57: name=C:\TEMP\VSTest\pkg\Win32api.pkg
77: name=C:\TEMP\VSTest\pkg\Runtime.pkg
96: name=C:\TEMP\VSTest\pkg\DataDef.pkg
B5: name=C:\TEMP\VSTest\pkg\GetMemory.pkg

Files at E4, size A8
1: ofs=  1 type= 110, chksum=3360C5AD-FC496EF4-E0F70BF1-D06E73DC
2: ofs= 1B type= 110, chksum=1D918322-BE51DA2D-88362C32-DFA921F6
3: ofs= 39 type= 110, chksum=B617C07C-31206FCE-71B9A6DE-C3D71097
4: ofs= 57 type= 110, chksum=D80EE68A-C9F561AE-11C7B6AE-C6BD7878
5: ofs= 77 type= 110, chksum=865F4701-6C93B025-0647E748-5A10F574
6: ofs= 96 type= 110, chksum=7EA41EED-0F512590-49A8123C-3B46D037
7: ofs= B5 type= 110, chksum=F035DFC7-5C4DDD3D-8A4A113E-1F043A33

Line numbers at 194, size 80:
section: start=0, idx=0, length=12

file srcfile=60(=5) items=2 length=1C    [Runtime.pkg]
   #    Offset     Line
-----------------------
   1:        0        5
   2:        0        6

file srcfile=78(=6) items=5 length=34    [DataDef.pkg]
   #    Offset     Line
-----------------------
   1:        1        5
   2:        1        6
   3:        2        7
   4:        7        8
   5:        9       10

file srcfile=90(=7) items=3 length=24    [GetMemory.pkg]
   #    Offset     Line
-----------------------
   1:       10        2
   2:       10        3
   3:       11        4


Masm is confused and writes line-number info for file DataDef.pkg instead of file VSTest.pkg.

[KeepingRealBusy]

Japeth,

Thank you for the information.

Now, let me display my ignorance. What program did you use to make this determination? I assume JWASM does not have this problem, but if I use JWASM to assemble, will I still be able to use VS as my debugger? Is the latest JWASM available for download at SourceForge?

Dave.

[KeepingRealBusy]

Japeth,

One more item. Visual Studio has a bug as well because MASM 9.0 is packaged with Visual Studio 2008 and it fails just like MASM 8.0.  Just sayin.

Dave.

[KeepingRealBusy]

Japeth,

Finding a working solution or placing blame on any program (MASM or VS) is still not an adequate answer. As I have explained, I have many programs with the same structure AND running with the same runtime.pkg that do not fail in this way. There has to be some other thing causing MASM to do such a thing.

In the mean time, I have a workaround. I copied all of the code in VSTest.pkg to a new Start.pkg and moved the trailing includes to Start.pkg, and added one INCLUDE in the VSTest.pkg to includeStart.pkg. VSTest.pkg still has the initial INCLUDES and only a 3 line PROC that just CALLS Start. Now when I start, Main starts and CALLS Start and Start has the initial int 3. I get the int 3 interrupt and VS displays the Start.pkg code and everything is otherwise normal.

Dave.