Bug in NTVDM.INC/LIB in MASM 11 and 10

[hutch--]

Mike,

I still give you the same advice, if you have a working library then disassemble it and find out what its format is. ALL of the libraries in MASM32 are import libraries that are purely Win32, not hybrid left overs from much earlier operating system versions. Now if you can accurately determine what the format is, you may need to get some of the tools from the era when this stuff was in use, probably from an antique DDK as Microsoft did some unusual stuff long ago. Internal code in the old VXD device drivers for Win9x had 16 bit code in them.

What you are after is not conventional Win32 PE format code, its hybrid 16/32 bit code from the era when Win9x was still being developed.

[pcMike]

Thanks Hutch,

Assuming that I were to find out what format the old NTVDM.LIB is in, what would be our next step? The old .LIB file works fine for me, and if you don't feel it's worthwhile adding BOP support in your NTVDM.LIB I understand. On the other hand if you want to support it, I would be glad to try to help as I feel it would be a worthwhile feature for the MASM32 project.

I googled for "decompile static library" and l found that IDA will disassemble a .LIB file, but this is a commercial product which I don't own.

I tried using DUMPBIN /disasm ntvdm.lib, but it only produced the following summery:

Code Select Expand


Microsoft (R) COFF/PE Dumper Version 9.00.30729.01
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file ntvdm.lib

File Type: LIBRARY

  Summary

          A8 .debug$S
          14 .idata$2
          14 .idata$3
           4 .idata$4
           4 .idata$5
           A .idata$6


Next I tried "DUMPBIN /all NTVDM.LIB" which produced over 4300 lines of output which I attached to a text file here.
I don't  know what to look for to determine the file format however. Can anyone suggest something?

Regards,  Mike

[MichaelW]

Mike,

The MASM32 ntdll.lib is apparently for ntdll.dll, and judging from the dump of your library, it's for ntdll.exe, which as far as I know is correct. This has come up before, see here and here, and further down where Vortex posted an import library.

[pcMike]

Hi Michael,

That is very interesting. The dump shows that my working NTVDM.LIB is for NTVDM.EXE as you said, but I don't think there is such a thing as an NTVDM.DLL file... at least there is not in Windows 10 or Windows 7.  I saw in one of those threads you mentioned that NTVDM.DLL does not exist in XP either.  There is an NTVDMD.DLL, but I would think that would use a different set of functions.

I am surprised that the NTVDM.INC included with MASM32 would reference the function names found in the EXE version of NTVDM.LIB. Assuming that there was an NTVDM.DLL in some version of Windows, would it use the same function names as the .EXE?

If in fact there is an NTVDM.DLL in some versions of Windows, could it be possible to combine both the DLL and EXE libraries into a single .LIB file?

Regards,  Mike

[MichaelW]

There was no NTVDM.DLL included in Windows 2000, so I suspect that there never was such a DLL. And there is no NTVDM.* included in my Windows 8.1-64, and I suspect not in any 64-bit version of Windows because there is no built-in support for 16-bit code.

[pcMike]

I could not find NTVDM.DLL in any versions of Windows (2000 and later) either. As I recall, the NTVDM.LIB and .INC were not added to MASM32 until version 10, so its very odd as the full dump confirms that it is from NTVDM.DLL. There are dozens of "Archive member name at 38D6: ntvdm.dll/", as well as this:

RAW DATA #2
  00000000: 6E 74 76 64 6D 2E 64 6C 6C 00                    ntvdm.dll.

So I compared the output of "dumpbin /exports ntvdm.lib" and the two versions are nearly identical, except for the following:

The 1999 NTVDM.LIB has this export which does not exist in the MASM32v11 version:
_ExpVdmTib


The MASM32 version has the following exports which do not exist in the 1999 version:
_Dos_Flag_Addr
_DpmiSetIncrementalAlloc@4
_demGetPhysicalDriveType@4

They both have a different count for the following export:

1999 version:                 MASM32 v11 version:
_cpu_createthread@4      _cpu_createthread@8

I suspect the 1999 version was created from a Windows 2000 NTVDM, so it's logical that a later version of Windows may have added the 3 additional functions. I suspect that the "_ExpVdmTib" which is missing from the MASM32 version is a likely cause of the issue.

I have attached both versions of the exports dump here.

Regards,  Mike

[pcMike]

Michael and Hutch,

It turns out that Nidud solved the bug in the link that Michael provided:
http://masm32.com/board/index.php?topic=1086.0

Nidud wrote a program that patches the MASM32 NTVDM.LIB and changes every instance of "NTVDM.EXE" to "NTVDM.DLL" as it should be, and this allows a DOS BOP command to successfully register a Win32 Virtual Device Driver DLL that was compiled using this Library.

Hutch, please confirm that you read this, and it would be great if you could apply the fix in MASM32 version 12. :-)

Michael, thanks for pointing me to the solution. I originally tried the versions that Vortex posted in the same thread, but neither one of those were BOP compatible.

Regards,  Mike

[Vortex]

Hi pcMike,

There is no such a thing like BOP compatible import libraries. You can check the Microsoft Portable Executable and Common Object File Format Specification.

[pcMike]

Hi Vortex,

Yes, the only reason that the MASM32 version of NTVDM.LIB would fail to build a DLL which is compatible with a DOS BOP "register VDD" command, was because the MASM32 version incorrectly references a non-existent NTVDM.DLL instead of NTVDM.EXE as it should.  If you manually hex edit it to reference the .EXE instead, then it works perfectly
(or use the patch program mtvdmp.asm that Nidud wrote).

Regards,  Mike

[MichaelW]

Interesting, my Windows 7-64 system has in system32 the file ntvdm64.dll, with the description "16-bit Emulation on NT64".

[dedndave]

i don't think it's impossible
i think they just didn't want that many layers of virtualization in the registry