The MASM Forum

Projects => ObjAsm => Topic started by: Biterider on June 30, 2018, 06:30:07 PM

Title: h2incX
Post by: Biterider on June 30, 2018, 06:30:07 PM
Today is the time to show what's under the hood in the new h2incX project.
It took me some time to analyse the Japheth’s original code. It became clear to me that this project was in a phase where code was still being tested and some other routines were not finished yet.
I decided not to modify the core of how the program was originally designed, but changed the code where necessary to add some necessary new features, such as parsing of annotations and some bug fixes.

Purpose:
This tool is a Win32 console application, whose purpose is to convert C header files to MASM-style include files. A private profile file is used to fine-tune the include files to be created.
Huge C header sets can be converted in one run, for example the Win32 headers in the "Windows Kits" or "PSDK".
Prototypes can be written so that the include file is suitable for dynamic linking to a DLL, using the IAT entries, as well as for static linking to a library.
Optionally, a .DEF file can be written to generate an import library.

Features and limitations
h2incX is very reliable, but it is not perfect. Without a full-fledged C-precompiler/compiler, manual intervention will always be required.
It should be noted that some C header file declarations simply cannot be translated into ASM. For example, there are no things like inline functions in ASM for example.
On some situations h2incx has to "count" braces. This can interfere with “#if“ pre-processor commands, because h2incX cannot evaluate expressions in these commands. As a result, h2incX can become confused and produces incorrect code.
h2incX has some limited knowledge about C++, but its original purpose was to convert C headers. So some C++ specific keywords can confuse h2incX. This also applies to templates.
"far" and "near" qualifiers are skipped, so this tool does not work for 16bit includes.
The current MASM / UASM implementation treats the conditional compile statement "if" differently than the C equivalent “#if”. Therefore, some problems may occur when symbols are undefined.

Result
I took the latest Kit for Win10 and translated all the .h files following the inclusion tree, starting with Windows.h. Unfortunately, the resulting .zip file containing the generated .inc files exceeds the allowed quota of the forum, so I am forced to put the .zip file on an alternative server.

Final notes
The source code is attached to this post. To avoid confusion, I have restarted the h2incX versioning with "B.01.00".
The original code used some pseudo-OO constructs, which I replaced with ObjAsm32 code. This change allowed much better handling of internal lists. The output of its contents is copied to .lst files for better verification of results.
Memory management has also been streamlined, but the core idea of how the information is stored remains the same.


I checked the result with some x64 projects and quickly found that more .h files needed a translation, for example CommCtrl.h. Based on this experience, if someone notices missing files, please report it here, so it can be added to the distribution.

Regards, Biterider
Title: Re: h2incX
Post by: Vortex on June 30, 2018, 07:16:54 PM
Hi Biterider,

Thanks for your tool. Trying your include file set, I received the following error messages :

Code: [Select]
Assembling: SimpleWnd.asm
H:\inc\crtdefs.inc(22) : error A2045: missing angle bracket or brace in literal
H:\inc\vadefs.inc(9) : error A2045: missing angle bracket or brace in literal
H:\inc\stdarg.inc(9) : error A2045: missing angle bracket or brace in literal
H:\inc\winnt.inc(753) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(754) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(761) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(4260) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(5103) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(5199) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(5353) : error A2008: syntax error : TokenUser
H:\inc\winnt.inc(5516) : error A2008: syntax error : TokenSource
H:\inc\winnt.inc(5588) : error A2041: string or text literal too long
H:\inc\winnt.inc(5599) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(5784) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(5790) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(5803) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(6445) : error A2041: string or text literal too long
H:\inc\winnt.inc(7021) : error A2006: undefined symbol : POINTER
H:\inc\winnt.inc(10552) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(10595) : error A2159: structure cannot be instanced
H:\inc\winnt.inc(11092) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(11164) : error A2008: syntax error : POINTER
H:\inc\winnt.inc(11165) : error A2159: structure cannot be instanced
H:\inc\winnt.inc(11183) : error A2041: string or text literal too long
H:\inc\winnt.inc(11685) : error A2041: string or text literal too long
H:\inc\ktmtypes.inc(84) : error A2179: structure improperly initialized
H:\inc\ktmtypes.inc(84) : error A2008: syntax error : in type
H:\inc\ktmtypes.inc(107) : error A2179: structure improperly initialized
H:\inc\ktmtypes.inc(107) : error A2008: syntax error : in type
H:\inc\ktmtypes.inc(116) : error A2179: structure improperly initialized
H:\inc\ktmtypes.inc(116) : error A2008: syntax error : in type
H:\inc\ktmtypes.inc(138) : error A2179: structure improperly initialized
H:\inc\ktmtypes.inc(138) : error A2008: syntax error : in type
H:\inc\winnt.inc(12088) : error A2179: structure improperly initialized
H:\inc\winnt.inc(12088) : error A2008: syntax error : in type
H:\inc\winnt.inc(12095) : error A2159: structure cannot be instanced
H:\inc\winnt.inc(12182) : error A2008: syntax error : POINTER
H:\inc\minwinbase.inc(251) : error A2008: syntax error : POINTER
H:\inc\libloaderapi.inc(32) : error A2008: syntax error : POINTER
H:\inc\securitybaseapi.inc(35) : error A2041: string or text literal too long
H:\inc\wingdi.inc(863) : error A2085: instruction or register not accepted in cu
rrent CPU mode
H:\inc\wingdi.inc(2954) : error A2008: syntax error : POINTER
H:\inc\wingdi.inc(2968) : error A2008: syntax error : POINTER
H:\inc\wingdi.inc(3106) : error A2008: syntax error : POINTER
H:\inc\wingdi.inc(3107) : error A2008: syntax error : POINTER
H:\inc\wingdi.inc(3108) : error A2008: syntax error : POINTER
H:\inc\wingdi.inc(3120) : error A2008: syntax error : POINTER
H:\inc\wingdi.inc(3121) : error A2008: syntax error : POINTER
H:\inc\wingdi.inc(3122) : error A2008: syntax error : POINTER
H:\inc\winuser.inc(232) : error A2131: VARARG parameter requires C calling conve
ntion
 @DefProto(2): Macro Called From
  H:\inc\winuser.inc(232): Include File
H:\inc\winuser.inc(233) : error A2131: VARARG parameter requires C calling conve
ntion
 @DefProto(2): Macro Called From
  H:\inc\winuser.inc(233): Include File
H:\inc\winuser.inc(570) : error A2008: syntax error : POINTER
H:\inc\winuser.inc(577) : error A2008: syntax error : POINTER
H:\inc\winuser.inc(6026) : error A2006: undefined symbol : _WIN32_WCE
H:\inc\winuser.inc(6133) : error A2006: undefined symbol : _WIN32_WCE
H:\inc\mmsyscom.inc(192) : error A2006: undefined symbol : HDRVR
H:\inc\mmsyscom.inc(192) : error A2195: parameter or local cannot have void type

H:\inc\mmsyscom.inc(192) : error A2129: VARARG parameter must be last parameter
H:\inc\mmiscapi.inc(113) : error A2006: undefined symbol : HDRVR
H:\inc\mmiscapi.inc(113) : error A2195: parameter or local cannot have void type

H:\inc\mmiscapi.inc(113) : error A2129: VARARG parameter must be last parameter
H:\inc\mmiscapi.inc(117) : error A2006: undefined symbol : HDRVR
H:\inc\mmiscapi.inc(117) : error A2195: parameter or local cannot have void type

H:\inc\mmiscapi.inc(117) : error A2129: VARARG parameter must be last parameter
H:\inc\playsoundapi.inc(29) : error A2006: undefined symbol : FALSE_
H:\inc\playsoundapi.inc(29) : error A2195: parameter or local cannot have void t
ype
H:\inc\playsoundapi.inc(29) : error A2131: VARARG parameter requires C calling c
onvention
H:\inc\playsoundapi.inc(71) : error A2006: undefined symbol : FALSE_
H:\inc\playsoundapi.inc(71) : error A2195: parameter or local cannot have void t
ype
H:\inc\playsoundapi.inc(71) : error A2131: VARARG parameter requires C calling c
onvention
H:\inc\mmeapi.inc(54) : error A2008: syntax error : POINTER
H:\inc\mmeapi.inc(720) : error A2008: syntax error : POINTER
H:\inc\nb30.inc(36) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(74) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(154) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(159) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(201) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(587) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(589) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(591) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(602) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(604) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(606) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(646) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(650) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(651) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(657) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(661) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(662) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(669) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(673) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(674) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(675) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(677) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(683) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(687) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(688) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(689) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(691) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(703) : error A2008: syntax error : POINTER
H:\inc\rpcdce.inc(704) : fatal error A1012: error count exceeds 100; stopping assembly

Attached is the example project. It can be assembled with WinInc209 offered by johnsa and habran.
Title: Re: h2incX
Post by: Biterider on June 30, 2018, 09:32:01 PM
Hi Vortex
Thank you for your feedback.  :t
I will take a look at it.

Biterider
Title: Re: h2incX
Post by: LiaoMi on July 01, 2018, 03:16:53 AM
Hi Biterider,

I tried to rebuild an example for x64 from Vortex, somewhere there is a bug in the code for x64, I have not looked at yet where. The executable file was compiled with sdk from ToutEnMasm, in addition there is a file with errors that occurs during build with h2incX incs.

Title: Re: h2incX
Post by: habran on July 02, 2018, 08:39:05 AM
Hi Biterider :biggrin:
You are doing a great job :t
I have looked in ole2.inc and found some functions without @DefProto, EG:
   
Code: [Select]
IsAccelerator proto fastcall :HACCEL, :DWORD, :LPMSG, :ptr WORDwhich are in WinInc expressed as:
   
Code: [Select]
@DefProto WINOLEAPI, IsAccelerator, stdcall, , <:HACCEL, :DWORD, :LPMSG, :ptr WORD>, 16I am afraid that your version will not work for 32bit because of fastcall convention, however, I might be wrong.
Anyway, I understand how much effort you have put in that project and appreciate very much your work :eusa_clap:

Ones when we get it done, we should think about adding C++ compiler to that code ;)
Title: Re: h2incX
Post by: Biterider on July 05, 2018, 05:02:04 AM
Hi
Short feedback: I found some minor issues and the cause of the wrong prototyping. A C-macro interferes with the detection of the import specification.
I will try to stay as close as possible to the original C version by using a masm macro.
I was able to build Vortex Test App and some of mine. For that, I had to translate more header files, which showed me more problems, that I did not see before.
I will need some time to fix this.  :icon_rolleyes:

Biterider
Title: Re: h2incX
Post by: Biterider on September 09, 2018, 05:54:24 PM
Hi
Today I want to show the next h2incX release. I spent a lot of time to add new code to improve the translation. As mentioned above, it is not possible to get a perfect work. C has some features that have no equivalent in asm. There are always workarounds that have to be analysed on a case-by-case basis.
I’m still working with the reduced set of header files for Windows 10 (Windows SDK 10.0.17134.0)  https://developer.microsoft.com/de-de/windows/downloads/windows-10-sdk (https://developer.microsoft.com/de-de/windows/downloads/windows-10-sdk)

Some of the changes are:
Output syntax and spacing management introduced:
Like in the first alpha release, automatic indentation was enhanced for a better human reading. An automatic spacing was also introduced on the parser.

Removal of unnecessary output: ";{", ";}", error & warning count, "#undef", etc.:
Non relevant code was completely removed.

Strategy change of @DefProto for x64:
Whenever possible, an @DefProto macro is generated to prototype an API procedure. The recognition and management of calling conventions has been completely redesigned.

Ini-file completion:
This file is where most of the application intelligence resides. Along with the internal lists, which store the gattered information about macros, types, equates, etc. is stored, it forms the information platform the program uses. After several passes through the .h files, a had enough information to add to the Ini-File to avoid the C forward references.

TypeC detection introduced:
I revised the recognition of native C types to their equivalents.

Conditional sentence evaluation:
This is probably one of the most important improvements. I implemented a conditional evaluator using the Shunting yard algorithm (https://en.wikipedia.org/wiki/Shunting-yard_algorithm (https://en.wikipedia.org/wiki/Shunting-yard_algorithm)) using 3 valued logical tables (https://en.wikipedia.org/wiki/Three-valued_logic (https://en.wikipedia.org/wiki/Three-valued_logic)). The goal was to skip unnecessary code that is not relevant to asm. I had to make assumptions about some symbols that I hard coded in an additional internal table. In a future release, I will move this table to the ini-file. To show where the skipped code was, I use a short comment line in the generated asm code (; [...]).


Since I can’t test all cases in the translation, I’m sure that there will pop up some errors. On my site, I tested all my code, except for the COM stuff, which requires special handling. This will be the next step. In the meantime, when someone wants to check the translation, it will help immensely.

The download of the "teaser" is here available: http://objasm.x10host.com/DwnFiles/180909_Include.zip (http://objasm.x10host.com/DwnFiles/180909_Include.zip)

Thanks, Biterider
Title: Re: h2incX
Post by: LiaoMi on September 09, 2018, 06:46:32 PM
Hi Biterider,

thanks for the updated project, I still have errors ..

Code: [Select]
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio>uasm64 /c -win64 /Cp /nologo SimpleWnd.asm
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\mmsyscom.inc(189) : Error A2102: Symbol not defined : HDRVR
 E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\mmsyscom.inc(189): Included by
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\mmsystem.inc(9): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\windows.inc(93): Included by
    SimpleWnd.asm(15): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\mmiscapi.inc(88) : Error A2102: Symbol not defined : HDRVR
 E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\mmiscapi.inc(88): Included by
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\mmsystem.inc(22): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\windows.inc(93): Included by
    SimpleWnd.asm(15): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\mmiscapi.inc(92) : Error A2102: Symbol not defined : HDRVR
 E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\mmiscapi.inc(92): Included by
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\mmsystem.inc(22): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\windows.inc(93): Included by
    SimpleWnd.asm(15): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(146) : Error A2210: Syntax error: pResource
 PROPSHEETPAGEA_V1_FIELDS(6)[prsht.inc]: Macro called from
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(146): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(42): Included by
    SimpleWnd.asm(18): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(153) : Error A2210: Syntax error: pResource
 PROPSHEETPAGEA_V1_FIELDS(6)[prsht.inc]: Macro called from
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(153): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(42): Included by
    SimpleWnd.asm(18): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(162) : Error A2210: Syntax error: pResource
 PROPSHEETPAGEA_V1_FIELDS(6)[prsht.inc]: Macro called from
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(162): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(42): Included by
    SimpleWnd.asm(18): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(173) : Error A2210: Syntax error: pResource
 PROPSHEETPAGEA_V1_FIELDS(6)[prsht.inc]: Macro called from
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(173): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(42): Included by
    SimpleWnd.asm(18): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(188) : Error A2210: Syntax error: pResource
 PROPSHEETPAGEW_V1_FIELDS(6)[prsht.inc]: Macro called from
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(188): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(42): Included by
    SimpleWnd.asm(18): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(195) : Error A2210: Syntax error: pResource
 PROPSHEETPAGEW_V1_FIELDS(6)[prsht.inc]: Macro called from
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(195): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(42): Included by
    SimpleWnd.asm(18): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(204) : Error A2210: Syntax error: pResource
 PROPSHEETPAGEW_V1_FIELDS(6)[prsht.inc]: Macro called from
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(204): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(42): Included by
    SimpleWnd.asm(18): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(215) : Error A2210: Syntax error: pResource
 PROPSHEETPAGEW_V1_FIELDS(6)[prsht.inc]: Macro called from
  E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\prsht.inc(215): Included by
   E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(42): Included by
    SimpleWnd.asm(18): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(4005) : Error A2210: Syntax error: link
 E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(4005): Included by
  SimpleWnd.asm(18): Main line code
E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(4085) : Error A2102: Symbol not defined : L_MAX_URL_LENGTH
 E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio\Include\CommCtrl.inc(4085): Included by
  SimpleWnd.asm(18): Main line code
SimpleWnd.asm: 105 lines, 1 passes, 665 ms, 0 warnings, 13 errors

E:\DATA\MASM64\HJWasm\H2INCX_New_Renovatio>polink /SUBSYSTEM:WINDOWS SimpleWnd.obj
POLINK: fatal error: File not found: 'SimpleWnd.obj'.
Title: Re: h2incX
Post by: Vortex on September 09, 2018, 07:41:21 PM
Hi Biterider,

I tried to build the same Window example above with your new include file set. Here are the error messages :

Code: [Select]
H:\test>set include=H:\inc

H:\test>\masm32\bin\ml /c /coff SimpleWnd.asm
Microsoft (R) Macro Assembler Version 6.14.8444
Copyright (C) Microsoft Corp 1981-1997.  All rights reserved.

 Assembling: SimpleWnd.asm
H:\inc\crtdefs.inc(391) : error A2006: undefined symbol : SQWORD
H:\inc\crtdefs.inc(754) : error A2005: symbol redefinition : threadlocaleinfostr
uct
H:\inc\minwindef.inc(49) : error A2008: syntax error : IN
H:\inc\minwindef.inc(52) : error A2008: syntax error : OUT
H:\inc\basetsd.inc(56) : error A2006: undefined symbol : SQWORD
H:\inc\winnt.inc(573) : error A2008: syntax error : SQWORD
H:\inc\winnt.inc(574) : error A2008: syntax error : SQWORD
H:\inc\winnt.inc(607) : error A2179: structure improperly initialized
H:\inc\winnt.inc(607) : error A2008: syntax error : in type
H:\inc\winnt.inc(1565) : error A2179: structure improperly initialized
H:\inc\winnt.inc(1565) : error A2008: syntax error : in type
H:\inc\winnt.inc(2628) : error A2005: symbol redefinition : _CONTEXT
H:\inc\winnt.inc(2884) : error A2005: symbol redefinition : _EXCEPTION_RECORD
H:\inc\winnt.inc(4020) : error A2008: syntax error : TokenUser
H:\inc\winnt.inc(4200) : error A2008: syntax error : TokenSource
H:\inc\winnt.inc(4279) : error A2041: string or text literal too long
H:\inc\winnt.inc(5043) : error A2004: symbol type conflict : JOB_OBJECT_NET_RATE
_CONTROL_FLAGS
H:\inc\winnt.inc(5051) : error A2104: invalid type for a data declaration : JOB_
OBJECT_NET_RATE_CONTROL_FLAGS
H:\inc\winnt.inc(5061) : error A2004: symbol type conflict : JOB_OBJECT_IO_RATE_
CONTROL_FLAGS
H:\inc\winnt.inc(5067) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5067) : error A2008: syntax error : in type
H:\inc\winnt.inc(5068) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5068) : error A2008: syntax error : in type
H:\inc\winnt.inc(5069) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5069) : error A2008: syntax error : in type
H:\inc\winnt.inc(5072) : error A2104: invalid type for a data declaration : JOB_
OBJECT_IO_RATE_CONTROL_FLAGS
H:\inc\winnt.inc(5078) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5078) : error A2008: syntax error : in type
H:\inc\winnt.inc(5079) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5079) : error A2008: syntax error : in type
H:\inc\winnt.inc(5080) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5080) : error A2008: syntax error : in type
H:\inc\winnt.inc(5083) : error A2104: invalid type for a data declaration : JOB_
OBJECT_IO_RATE_CONTROL_FLAGS
H:\inc\winnt.inc(5085) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5085) : error A2008: syntax error : in type
H:\inc\winnt.inc(5086) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5086) : error A2008: syntax error : in type
H:\inc\winnt.inc(5087) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5087) : error A2008: syntax error : in type
H:\inc\winnt.inc(5088) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5088) : error A2008: syntax error : in type
H:\inc\winnt.inc(5089) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5089) : error A2008: syntax error : in type
H:\inc\winnt.inc(5090) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5090) : error A2008: syntax error : in type
H:\inc\winnt.inc(5094) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5094) : error A2008: syntax error : in type
H:\inc\winnt.inc(5095) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5095) : error A2008: syntax error : in type
H:\inc\winnt.inc(5096) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5096) : error A2008: syntax error : in type
H:\inc\winnt.inc(5099) : error A2104: invalid type for a data declaration : JOB_
OBJECT_IO_RATE_CONTROL_FLAGS
H:\inc\winnt.inc(5101) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5101) : error A2008: syntax error : in type
H:\inc\winnt.inc(5102) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5102) : error A2008: syntax error : in type
H:\inc\winnt.inc(5103) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5103) : error A2008: syntax error : in type
H:\inc\winnt.inc(5104) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5104) : error A2008: syntax error : in type
H:\inc\winnt.inc(5105) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5105) : error A2008: syntax error : in type
H:\inc\winnt.inc(5106) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5106) : error A2008: syntax error : in type
H:\inc\winnt.inc(5107) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5107) : error A2008: syntax error : in type
H:\inc\winnt.inc(5108) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5108) : error A2008: syntax error : in type
H:\inc\winnt.inc(5109) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5109) : error A2008: syntax error : in type
H:\inc\winnt.inc(5110) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5110) : error A2008: syntax error : in type
H:\inc\winnt.inc(5111) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5111) : error A2008: syntax error : in type
H:\inc\winnt.inc(5112) : error A2179: structure improperly initialized
H:\inc\winnt.inc(5112) : error A2008: syntax error : in type
H:\inc\winnt.inc(5119) : error A2004: symbol type conflict : JOBOBJECT_IO_ATTRIB
UTION_CONTROL_FLAGS
H:\inc\winnt.inc(5183) : error A2041: string or text literal too long
H:\inc\winnt.inc(5711) : error A2004: symbol type conflict : MEM_EXTENDED_PARAME
TER_TYPE
H:\inc\winnt.inc(5717) : error A2089: too many bits in RECORD : Reserved_??19
H:\inc\winnt.inc(6029) : error A2179: structure improperly initialized
H:\inc\winnt.inc(6029) : error A2008: syntax error : in type
H:\inc\winnt.inc(7971) : error A2004: symbol type conflict : IMAGE_AUX_SYMBOL_TY
PE
H:\inc\winnt.inc(9188) : error A2004: symbol type conflict : IMPORT_OBJECT_TYPE
H:\inc\winnt.inc(9196) : error A2004: symbol type conflict : IMPORT_OBJECT_NAME_
TYPE
H:\inc\winnt.inc(9226) : error A2004: symbol type conflict : ReplacesCorHdrNumer
icDefines
H:\inc\winnt.inc(9845) : error A2104: invalid type for a data declaration : INT6
4
H:\inc\winnt.inc(9928) : error A2041: string or text literal too long
H:\inc\winnt.inc(10439) : error A2041: string or text literal too long
H:\inc\ktmtypes.inc(83) : error A2179: structure improperly initialized
H:\inc\ktmtypes.inc(83) : error A2008: syntax error : in type
H:\inc\ktmtypes.inc(109) : error A2179: structure improperly initialized
H:\inc\ktmtypes.inc(109) : error A2008: syntax error : in type
H:\inc\ktmtypes.inc(119) : error A2179: structure improperly initialized
H:\inc\ktmtypes.inc(119) : error A2008: syntax error : in type
H:\inc\ktmtypes.inc(143) : error A2179: structure improperly initialized
H:\inc\ktmtypes.inc(143) : error A2008: syntax error : in type
H:\inc\winnt.inc(10872) : error A2179: structure improperly initialized
H:\inc\winnt.inc(10872) : error A2008: syntax error : in type
H:\inc\winnt.inc(10880) : error A2159: structure cannot be instanced
H:\inc\windef.inc(201) : fatal error A1012: error count exceeds 100; stopping as
sembly
Title: Re: h2incX
Post by: Biterider on September 10, 2018, 01:16:27 AM
Hi
Thanks for testing. I found the issues that caused the troubles.
ML has some limitations and is a bit touchy compared with other assemblers.
I tested SimpleWnd on 32 bit using UASM and I was successful. When using ML or UASM with -win64, many errors occurred.
I'll try to shorten the very long C defs, which are useless anyway.

I will upload a revised version in a few days. Attached my build from SimpleWnd.

Biterider
Title: Re: h2incX
Post by: Vortex on September 10, 2018, 05:13:49 AM
Hi Biterider,

I am afraid you didn't use your version of windows.inc in your attachment. Reading SimpleWnd.asm coming with Test_SimpleWnd.zip :

Code: [Select]
.386
.model flat, stdcall
option casemap:none

include D:\MASM32\Include\windows.inc

Why did you use windows.inc shipped with Masm32?
Title: Re: h2incX
Post by: Vortex on September 10, 2018, 05:19:56 AM
Hello again,

Uasm is reporting a lot of errors :

Code: [Select]
H:\test>C:\UASM32\UASM32 /c /coff SimpleWnd.asm
UASM v2.46, Jan 26 2018, Masm-compatible assembler.
Portions Copyright (c) 1992-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.

SimpleWnd.asm(5) : Error A2106: Cannot open file: "windows.inc" [ENOENT]
SimpleWnd.asm(17) : Error A2210: Syntax error: hInstance
SimpleWnd.asm(23) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(25) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(27) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(31) : Error A2102: Symbol not defined : WNDCLASSEX
SimpleWnd.asm(32) : Error A2102: Symbol not defined : MSG
SimpleWnd.asm(33) : Error A2102: Symbol not defined : HWND
SimpleWnd.asm(36) : Warning A4073: Size not specified, assuming: BYTE
SimpleWnd.asm(45) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(48) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(50) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(54) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(56) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(59) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(61) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(62) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(70) : Error A2102: Symbol not defined : HWND
SimpleWnd.asm(74) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm(77) : Error A2160: INVOKE requires prototype for procedure
SimpleWnd.asm: 87 lines, 1 passes, 8 ms, 1 warnings, 19 errors

H:\test>H:\masm32\bin\polink /SUBSYSTEM:WINDOWS SimpleWnd.obj
POLINK: fatal error: File not found: 'SimpleWnd.obj'.

Build.bat :
Code: [Select]
set include=H:\inc

C:\UASM32\UASM32 /c /coff SimpleWnd.asm
H:\masm32\bin\polink /SUBSYSTEM:WINDOWS SimpleWnd.obj

Attached is the example project.
Title: Re: h2incX
Post by: Biterider on September 10, 2018, 05:36:19 AM
Hi Vortex
Sorry for the confusion. I renamed my original \MASMA32\Include folder and replaced it with the result of h2incX, or in your case, the content of the download.
I did it this way to avoid setting the compilation paths of existing projects.
The Windows.inc version you should use is the one of the  download.


Thanks for helping!
Biterider
Title: Re: h2incX
Post by: Vortex on September 10, 2018, 05:46:17 AM
Hi Biterider,

As I posted above, Uasm is displaying a lot of error messages with your include file set. Please download my SimpleWnd2.zip and try again.
Title: Re: h2incX
Post by: Vortex on September 10, 2018, 05:51:45 AM
Hi Biterider,

Sorry,me too, I had some path problems. Here is the latest error list :

Code: [Select]
H:\test>C:\UASM32\UASM32 /c /coff SimpleWnd.asm
UASM v2.46, Jan 26 2018, Masm-compatible assembler.
Portions Copyright (c) 1992-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.

H:\inc\winuser.inc(5715) : Error A2102: Symbol not defined : _WIN32_WCE
 H:\inc\winuser.inc(5715): Included by
  H:\inc\windows.inc(72): Included by
   SimpleWnd.asm(5): Main line code
H:\inc\winuser.inc(5788) : Error A2102: Symbol not defined : _WIN32_WCE
 H:\inc\winuser.inc(5788): Included by
  H:\inc\windows.inc(72): Included by
   SimpleWnd.asm(5): Main line code
H:\inc\winuser.inc(5892) : Error A2102: Symbol not defined : _WIN32_WCE
 H:\inc\winuser.inc(5892): Included by
  H:\inc\windows.inc(72): Included by
   SimpleWnd.asm(5): Main line code
SimpleWnd.asm: 87 lines, 1 passes, 2555 ms, 0 warnings, 3 errors

H:\test>H:\masm32\bin\polink /SUBSYSTEM:WINDOWS SimpleWnd.obj
POLINK: fatal error: File not found: 'SimpleWnd.obj'.
Title: Re: h2incX
Post by: Biterider on September 10, 2018, 05:59:26 AM
Hi Vortex
This is really weird!
Code: [Select]
UASM v2.46, Jun 23 2018, Masm-compatible assembler.
Portions Copyright (c) 1992-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.

SimpleWnd.asm: 87 lines, 2 passes, 218 ms, 0 warnings, 0 errors

I had to adjust a few paths on your code and it compiled without drama  :P

I'll look into the error list to see where ML has a problem. I have just recognized some of them. The most obvious is the ML6.15 doesn't know what a SQWORD is...

Biterider
Title: Re: h2incX
Post by: Biterider on September 10, 2018, 06:21:37 AM
Hi Vortex
I know what is going on. I'm using the modified UASM version of Habran (see here http://masm32.com/board/index.php?topic=7212.0 (http://masm32.com/board/index.php?topic=7212.0)).
Attached the corrected WinUser.inc file.
Biterider
Title: Re: h2incX
Post by: Biterider on September 10, 2018, 06:26:20 AM
Btw, looking a bit closer into WinNT.h  I found
Code: [Select]
typedef struct _SE_TOKEN_USER {
    union {
        TOKEN_USER TokenUser;
        SID_AND_ATTRIBUTES User;
    } DUMMYUNIONNAME;
    union {
        SID Sid;
        BYTE  Buffer[SECURITY_MAX_SID_SIZE];
    } DUMMYUNIONNAME2;
} SE_TOKEN_USER , PSE_TOKEN_USER;

Should not it be like this?

Code: [Select]
... } SE_TOKEN_USER , *PSE_TOKEN_USER;
Biterider
Title: Re: h2incX
Post by: habran on September 10, 2018, 08:43:27 PM
You are correct Biterider 8)

You have been doing an excellent job :t
I am looking forward to your final work :biggrin:

Title: Re: h2incX
Post by: LiaoMi on September 10, 2018, 09:28:03 PM
Btw, looking a bit closer into WinNT.h  I found
Code: [Select]
typedef struct _SE_TOKEN_USER {
    union {
        TOKEN_USER TokenUser;
        SID_AND_ATTRIBUTES User;
    } DUMMYUNIONNAME;
    union {
        SID Sid;
        BYTE  Buffer[SECURITY_MAX_SID_SIZE];
    } DUMMYUNIONNAME2;
} SE_TOKEN_USER , PSE_TOKEN_USER;

Should not it be like this?

Code: [Select]
... } SE_TOKEN_USER , *PSE_TOKEN_USER;
Biterider

Hi Biterider,

Code: [Select]
;
; Token information class structures
;

TOKEN_USER STRUCT DEFALIGNMASM
User SID_AND_ATTRIBUTES <>
TOKEN_USER ENDS


IFNDEF MIDL_PASS

SE_TOKEN_USER STRUCT DEFALIGNMASM
union DUMMYUNIONNAME
aTokenUser TOKEN_USER <>
User SID_AND_ATTRIBUTES <>
ENDS
union DUMMYUNIONNAME2
Sid SID <>
Buffer BYTE SECURITY_MAX_SID_SIZE dup (?)
ENDS
SE_TOKEN_USER ENDS


PSE_TOKEN_USER EQU <SE_TOKEN_USER>

TOKEN_USER_MAX_SIZE equ < sizeof(TOKEN_USER) + SECURITY_MAX_SID_SIZE>
ENDIF

So its made in sdk from ToutEnMasm  ::)
Title: Re: h2incX
Post by: Vortex on September 11, 2018, 03:15:39 AM
Hi Biterider,

According to the MS documentation :

https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntifs/ns-ntifs-_se_token_user

Code: [Select]
typedef struct _SE_TOKEN_USER {
  union {
    TOKEN_USER         TokenUser;
    SID_AND_ATTRIBUTES User;
  } DUMMYUNIONNAME;
  union {
    SID   Sid;
    UCHAR Buffer[SECURITY_MAX_SID_SIZE];
  } DUMMYUNIONNAME2;
} SE_TOKEN_USER, PSE_TOKEN_USER;

My MinGW setup (ntifs.h ) does not provide this structure. Same result with the header file http://www.acc.umu.se/~bosse/ntifs.h. No any reference to this structure.

Title: Re: h2incX
Post by: Vortex on September 11, 2018, 03:19:45 AM
Hi Biterider,

Using the unmodified Uasm release and the header file supplied with the zip file winuser.zip, I managed to assemble the Window example, thanks.
Title: Re: h2incX
Post by: habran on September 11, 2018, 04:47:30 AM
I found here (https://msdn.microsoft.com/ru-ru/office/mt790565(v=vs.90)) this:
Quote
typedef struct _SE_TOKEN_USER {
  union {
    TOKEN_USER         TokenUser;
    SID_AND_ATTRIBUTES User;
  };
  union {
    SID   Sid;
    UCHAR Buffer[SECURITY_MAX_SID_SIZE];
  };
} SE_TOKEN_USER, *PSE_TOKEN_USER;

It must be $MS error because pointers should have asterisks

I would ask them about that.
 
Title: Re: h2incX
Post by: fearless on September 11, 2018, 05:08:58 AM
Yes looks like an error. I scanned the sdk's and ddk for any reference of usage and it appears PSE_TOKEN_USER isn't used anywhere in any function or other structure - so thats why ms probably didnt notice the typo as PSE_TOKEN_USER is not used anywhere cept at the end of SE_TOKEN_USER structure define, and should be a pointer to that struct.
Title: Re: h2incX
Post by: Biterider on September 11, 2018, 05:34:45 AM
Hi Vortex
At least I know I'm not alone  :biggrin:


Biterider
Title: Re: h2incX
Post by: Biterider on November 14, 2021, 05:45:37 PM
Hi
I found some time to come back to this project and work on some of the found issues.
So far I ported the whole code to BNC (Bitness Neutral Code). h2incX is now a 64 bit app.
I also corrected the multilevel array issue. The produced code looks like

float Matrix [3][4] => Matrix float (4)*(3) dup(?)

Record members are now build in the same order as C++ bitfields.

Function pointer declarations now emit a variable declaration/structure member. Example from OAIdl.h

Code: [Select]
        EXCEPINFO struct
          wCode WORD ?
          wReserved WORD ?
          bstrSource BSTR ?
          bstrDescription BSTR ?
          bstrHelpFile BSTR ?
          dwHelpContext DWORD ?
          pvReserved PVOID ?
          TYPE_pfnDeferredFillIn typedef proto WIN_STD_CALL_CONV :ptr tagEXCEPINFO
          pfnDeferredFillIn typedef ptr TYPE_pfnDeferredFillIn
          pfnDeferredFillIn pfnDeferredFillIn ?
          scode SCODE ?
        EXCEPINFO ends
        tagEXCEPINFO typedef EXCEPINFO
        LPEXCEPINFO typedef ptr EXCEPINFO


Complex structure declarations are handled correctly. e.g.

Code: [Select]
      CRYPT_INTEGER_BLOB struct
        cbData DWORD ?
        pbData POINTER ?
      CRYPT_INTEGER_BLOB ends


Biterider
Title: Re: h2incX
Post by: fearless on November 14, 2021, 10:18:01 PM
I think I posted about this repo before: win32metadata - https://github.com/microsoft/win32metadata (since its related to the generation of api information like h2incX)


But there is an additional repo that generates api info in json format as well: https://github.com/marlersoft/win32json/tree/main/api


Could be useful to generate the api info from the json format perhaps if that is any easier - although you are still relying on the win32meta generation to output correctly the information and it probably also has some edge cases where it cant handle the conversion of the headers.

Thought it useful and interesting to mention it
Title: Re: h2incX
Post by: Biterider on November 15, 2021, 03:48:41 AM
Hi fearless
Thank you, super interesting links.  :thumbsup:
Winmd should be up to date as it is maintained by MS. It's an excellent base from which to build something like MASM Windows.inc, or to get information for an intellisense add-in.

The malersoft repo is easier to read (JSON) because you don't have to deal with the metadata format. The downside is that it is being maintained by a third party and you never know how long it will exist.

IMHO some information is still lost using this approach, e.g. how the .h files are structured and the dependencies. By setting some inclusion flags, you can choose which part of the API you want to use, excluding most of the unused information.

The metadata approach (database) is interesting and for sure a good way to get the asm include files (projections as MS call them  :cool: ).

Biterider
Title: Re: h2incX
Post by: Biterider on November 15, 2021, 05:33:16 AM
Hello fearless
You piqued my interest in the "MASM projection" using the set of JSON files.  :biggrin:
It shouldn't take that much effort to generate some inc files with it. Unfortunately, I'll only see if it works at the end, when everything has been put together.

I think it might be worth it.

Biterider
Title: Re: h2incX
Post by: fearless on November 15, 2021, 06:47:52 AM
Nice, sounds good. Yes a Masm projection would be useful. The point about the third party json generation is good. Although he has a repo for how it is generated, so the third party tool could also be used I imagine if needed, but yes coming straight from MS gives a bit more of a guarantee of stability and longevity - hopefully ;-)