64 bit debuggers for masm 64

[jj2007]

jj2007:  I am hunting for x64dbg.ini.  Haven't found it yet.

\Masm32\x64Dbg\release\x64\x64dbg.ini

[markallyn]

Hello sinsi,

The link.exe version I'm running doesn't have a /DEBUGTYPE switch.  Has only /DEBUG.  I'm assembling with ml64.exe with /Zi and /Zf options.  When I link I use the command line:  "link.exe /DEBUG /SUBSYSTEM:console /ENTRY:main /OUT:mystart.exe mystart.obj".    Then I get the error message that MSPDB100.DLL is the wrong version.

Mark

[markallyn]

JJ:

Thank you.  Very kind.  I'll let you know what happened.

Mark

[aw27]

Then I get the error message that MSPDB100.DLL is the wrong version.

Each link.exe requires a matched version of MSPDB in the same folder (or in system32).
Were you using the link.exe from VS2017 you would need mspdb140.dll in the same folder.
Sounds complicated?

[fearless]

I use the microsoft linker to output pdb files for debugging with x64dbg:

LINK.EXE /SUBSYSTEM:WINDOWS /DEBUG /DEBUGTYPE:CV /PDB

and also the /Zi /Zd switches when assembling to help with showing other debug info

[markallyn]

AW27 and Fearless,

AW27:  I have mspd100.dll and mspd140.dll in the same folder along with link.exe.  Moreover, this folder is reachable via the PATH environmental variable.

Fearless:  Thanks for the info on the assembler\linker switches.  I'll try them forthwith.

Thanks to both of you for helping me with this blasted issue.

Markl

[markallyn]

Hi Fearless,

Would love to report that your link.exe switches solved my problem, but they didn't.  Still getting the cryptic message "incorrect MSPDB100.DLL version; recheck installation of this product".

Regards,
Mark

[fearless]

Here is my stuff im using for the linker, from one of the vs studio installations (v12.x in the file manifests):
https://www.dropbox.com/s/wozllpfru8cbttf/vs2012tools.zip?dl=0
probably will require the visual c runtime for 12.x (MSVCR120.dll) installed in your system somewhere
I have all those files in my masm\bin folder

[aw27]

OK, make sure they are the same bitness than link.exe

[markallyn]

aw27,

I checked for bitness and came away completely confused by what I saw.  All three of ml64.exe, link.exe, and mspdb100.dll appear to be 32-bit PE files.  I say this because that is what I got from dumpbin /HEADERS and also from looking at the symbol following PE in the binary files using notepad. 

Perhaps you can explain this.  I don't understand how ml64.exe could assemble code which uses 64 bit registers without itself being a 64-bit program. 

Mark

[jj2007]

I don't understand how ml64.exe could assemble code which uses 64 bit registers without itself being a 64-bit program.

The translation of human-readable source code into machine code has nothing to do with the "bitness" of the assembler. You could generate 64-bit machine code with a 16-bit QBasic program.

[aw27]

I checked for bitness and came away completely confused by what I saw.  All three of ml64.exe, link.exe, and mspdb100.dll appear to be 32-bit PE files.  I say this because that is what I got from dumpbin /HEADERS and also from looking at the symbol following PE in the binary files using notepad. 

You are correct. Probably you need to run VCVARS64.BAT (or continue investigating).
If this helps, I can advance that this works for VS 2017, but paths are a bit long:
"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.25017\bin\HostX64\x64\ml64" -c -Zp8 /Zd test64.asm
"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.25017\bin\HostX64\x64\link" /SUBSYSTEM:console /ENTRY:Start /DEBUG:FULL test64.obj

[aw27]

Alternatively, if you copy following files from C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.25017\bin\HostX64\x64
to your \masm32\bin directory it will work as well:
link.exe
msobj140.dll
mspdb140.dll
mspdbcore.dll

For Visual Studio 2010, probably you need to copy some more .DLLs in addition to mspdb100.dll

[markallyn]

JJ2007 and aw27,

Thanks for responding. 

JJ:  I understand your point, I believe, but I was surprised that ml64.exe wasn't itself (apparently) a 64 bit program.  I know JWOW is there to handle 32-bit code, but still, it seems surprising.

aw27.  At the moment, I don't have Visual Studio running.  I have tried to install Visual Studio Express, but so far I haven't been successful.

In any case, ml64.exe, link..exe, and mspdb100.dll are all in the same bit format.

I guess I'm going to have to renew my efforts to get VS Xpress working.

Regards,
Mark

[aw27]

aw27.  At the moment, I don't have Visual Studio running.  I have tried to install Visual Studio Express, but so far I haven't been successful.

VS Express editions are crippled, it is better Community Editions which are almost the same as Pro.