Hi all
I started work on the next version of the C header translator, which I called h2inc+ to denote the extended functionality.
My goals at the moment are:
- Dual bitness (or pure x64)
- Get rid of some dependencies
- Better configurability
- Configurable and persistent database of gattered information
- Much less manual fixes (hard)
- Better management of the preprocessor pass (very hard)
It is a lot of (hard) work, but it is worth it. :biggrin:
In particular, MASM64 could benefit from a fresh set of include files.
If you want to get involved, you are more than welcome to.
Biterider
Excellent :thumbsup:
I suppose you are aware of Gcc's option -E, which allows to get rid of most of the conditional stuff. Is there an equivalent option of MSVC?
Being a while :biggrin:
/E preprocess to stdout /EP preprocess to stdout, no #line
/P preprocess to file
Quote from: Biterider on March 29, 2024, 10:45:35 PMHi all
I started work on the next version of the C header translator, which I called h2inc+ to denote the extended functionality.
My goals at the moment are:
- Dual bitness (or pure x64)
- Get rid of some dependencies
- Better configurability
- Configurable and persistent database of gattered information
- Much less manual fixes (hard)
- Better management of the preprocessor pass (very hard)
It is a lot of (hard) work, but it is worth it. :biggrin:
In particular, MASM64 could benefit from a fresh set of include files.
If you want to get involved, you are more than welcome to.
Biterider
This is useful, nowdays if I need something what is missing, I just search Visual C++ headers to get the pieces to work.
Hi
I have done most of the porting work, implemented some missing switches and cleaned up the code from previous versions.
Now I'm getting closer to the real core of the application.
One thing I need some help on is how to translate this structure to asm. AFAIR, the bits are arranged in reverse order...
From DDE.h
typedef struct {
#ifndef _MAC
unsigned short bAppReturnCode:8,
reserved:6,
fBusy:1,
fAck:1;
#else
unsigned short usFlags;
#endif
} DDEACK;
Biterider
Japheth's h2incx :
DDEACK struct
ifndef _MAC
DDEACK_R0 RECORD bAppReturnCode:8,reserved:6,fBusy:1,fAck:1
DDEACK_R0 <>
else
usFlags WORD ?
endif
DDEACK ends
Quote from: johnsa on June 11, 2012, 06:41:00 PMI've started implementing the RECORD directive, but given that I've got ENUM and I've never used/or seen it used in modern times.. should I bother supporting it?
Hi
It seems more complicated than linear mapping, isn't it?
https://stackoverflow.com/questions/19376426/order-of-fields-when-using-a-bit-field-in-c (https://stackoverflow.com/questions/19376426/order-of-fields-when-using-a-bit-field-in-c)
@JJ: In the latest version of UASM, the RECORD directive is implemented and works as expected (MASM compatible).
Biterider
Hi
The next issue is that all names in the RECORD directive must be unique.
Taking as example
DDEACK_R0 RECORD bAppReturnCode:8,reserved:6,fBusy:1,fAck:1
DDEACK_R1 RECORD bAppReturnCode:2,reserved:1,fBusy:1,fAck:12
This will not work because bAppReturnCode, reserved, fBusy and fAck have already been defined.
To solve this we need a naming convention that includes the usage (record and member name) and the bit field name. Then there are the anonymous members too :undecided:
Biterider
Quote from: Biterider on April 08, 2024, 09:06:13 PM@JJ: In the latest version of UASM, the RECORD directive is implemented and works as expected (MASM compatible).
Hearsay :biggrin:
Jokes apart: that was a long discussion (https://masm32.com/board/index.php?msg=1462) about the right order in RECORDs. Does anybody use them, though? I can't find any in 48k lines of my library code. But of course, they do exist and must be handled correctly.
Hi
@JJ
Quote from: jj2007 on April 08, 2024, 09:49:17 PMJokes apart: that was a long discussion (https://masm32.com/board/index.php?msg=1462) about the right order in RECORDs. Does anybody use them, though? I can't find any in 48k lines of my library code.
Thanks for the link!
The question is how we want to deal with it. What are the best options?
Japheth had a good approach, but it has some flaws. As you can see here https://masm32.com/board/index.php?topic=276.msg1500#msg1500 (https://masm32.com/board/index.php?topic=276.msg1500#msg1500), he uses an incremental index to make the names unique. The problem is that you never know what the index is, and it can change from one translation to the next.
Using a naming scheme like the one I mentioned leads to very long names, which I personally am not happy with.
Biterider
DDEACK_R0 RECORD bAppReturnCodeR0:8,reservedR0:6,fBusyR0:1,fAckR0:1
DDEACK_R1 RECORD bAppReturnCodeR1:2,reservedR1:1,fBusyR1:1,fAckR1:12
You won't need that frequently, but appending first and last letters would work.
IMHO the assemblers should learn how to deal with it. They don't have problems with identical names in structures, right? I am ready to say bye bye to M$ ML.exe and ML64.exe, since UAsm and JWasm are so much better and faster...
I use it in terminal program thusly:
; set up dcb with reversed fbits (BITRECORD in masm32 windows.inc is incorrect)
zBITRECORD RECORD zDummy2:17,zAbortOnError:1,zRtsControl:2,zNull:1,zErrorChar:1,zInX:1,zOutX:1,zTXContinueOnXoff:1,zDsrSensitivity:1,zDtrControl:2,zOutxDsrFlow:1,zOutxCtsFlow:1,zParity:1,zBinary:1
DCBz STRUCT
DCBlength DWORD ?
BaudRate DWORD ?
fbits zBITRECORD <>
wReserved WORD ?
XonLim WORD ?
XoffLim WORD ?
ByteSize BYTE ?
Parity BYTE ?
StopBits BYTE ?
XonChar BYTE ?
XoffChar BYTE ?
ErrorChar BYTE ?
EofChar BYTE ?
EvtChar BYTE ?
wReserved1 WORD ?
DCBz ENDS
Hi jimg
Windows.inc defines the RECORD for the DBC structure as
BITRECORD RECORD fBinary:1,fParity:1,fOutxCtsFlow:1,fOutxDsrFlow:1,fDtrControl:2,fDsrSensitivity:1,fTXContinueOnXoff:1,fOutX:1,fInX:1,fErrorChar:1,fNull:1,fRtsControl:2,fAbortOnError:1,fDummy2:17
which corresponds to the definition sequence in C
typedef struct _DCB {
DWORD DCBlength;
DWORD BaudRate;
DWORD fBinary : 1;
DWORD fParity : 1;
DWORD fOutxCtsFlow : 1;
DWORD fOutxDsrFlow : 1;
DWORD fDtrControl : 2;
DWORD fDsrSensitivity : 1;
DWORD fTXContinueOnXoff : 1;
DWORD fOutX : 1;
DWORD fInX : 1;
DWORD fErrorChar : 1;
DWORD fNull : 1;
DWORD fRtsControl : 2;
DWORD fAbortOnError : 1;
DWORD fDummy2 : 17;
WORD wReserved;
WORD XonLim;
WORD XoffLim;
BYTE ByteSize;
BYTE Parity;
BYTE StopBits;
char XonChar;
char XoffChar;
char ErrorChar;
char EofChar;
char EvtChar;
WORD wReserved1;
} DCB, *LPDCB;
Can you confirm that your reversed RECORD really works?
zBITRECORD RECORD zDummy2:17,zAbortOnError:1,zRtsControl:2,zNull:1,zErrorChar:1,zInX:1,zOutX:1,zTXContinueOnXoff:1,zDsrSensitivity:1,zDtrControl:2,zOutxDsrFlow:1,zOutxCtsFlow:1,zParity:1,zBinary:1
Biterider
I haven't looked at that code in five years, but it was working at the time. There was a short discussion somewhere on this board, or the old one, can't remember which. I'll try the program and look for the posts.
If Siekmanski is still around, he would be the one to ask for sure. I'm pretty sure I stole the code from him (with his permission).
Here's one of the links serial (https://masm32.com/board/index.php?topic=6288.msg67389#msg67389)
Quote from: Biterider on April 09, 2024, 07:14:52 AMCan you confirm that your reversed RECORD really works?
zBITRECORD RECORD zDummy2:17,zAbortOnError:1,zRtsControl:2,zNull:1,zErrorChar:1,zInX:1,zOutX:1,zTXContinueOnXoff:1,zDsrSensitivity:1,zDtrControl:2,zOutxDsrFlow:1,zOutxCtsFlow:1,zParity:1,zBinary:1
Bitfields/Records in assembly must be defined in the reverse order than its definition in C.
For an example I attached usp10.inc that works.
usp10.h for comparison: usp10.h (https://github.com/wine-mirror/wine/blob/master/include/usp10.h)
Quote from: Biterider on April 08, 2024, 09:37:01 PMThe next issue is that all names in the RECORD directive must be unique.
Regarding Bitfields/Records MASM is a mess with a lot of hassle to deal with.
When I was translating my own windows headers with japheth's h2incx tool I had to solve this problem manually. I just set a numeric suffix to the often doubled fieldname "reserved", so like reserved
n:7, where
n is a numeric.
The Problem here is: how to remember which suffix the specific fieldname has.
AFAIK AsmC solved that problem by making fieldnames in records non-unique. But this is incompatible to MASM and it's clones.
Another issue is the alignment, i.e. if a record is defined within a structure, respectively in general.
If a record's size in C is defined as DWORD/int, Japheth solved it like this:
ATTR struct
_ATTR_R0 RECORD \
reserved6 : 6, ;// possible underline/other styles (must be NULL)
eol : 1, ;// when set, prevents cursor from selecting character. only valid for last char in line
ctrl : 1, ;// show as an isolated control-character
sel : 1, ;// selection flag (yes/no)
font_ : 7, ;// font-index
len_ : 16 ;// length of this run (in code-units)
union
DWORD ?
_ATTR_R0 <>
ends
fg COLORREF ?
bg COLORREF ?
ATTR ends
Hi Greenhorn
Looking at this memory arrangement, it seems to me that it is not an alignment problem of the RECORD itself, but rather the alignment of the following members. Am I right?
Biterider
Quote from: Biterider on April 10, 2024, 03:03:56 AMHi Greenhorn
Looking at this memory arrangement, it seems to me that it is not an alignment problem of the RECORD itself, but rather the alignment of the following members. Am I right?
Biterider
I'm not sure about it, it's too long ago I struggled with this.
It could also be that it's because of "type" of the record and allocating memory for it, maybe in case that one defines a record with just 28 bit leaving the remaining 4 bit undefined.
IIRC, h2incx has an option/switch to these union to a record or not.
Maybe @_japheth can tell more about it.
The MASM Programmer's Guide does not mention this "union thing".
Edited last post ...
This is the documentation for the "r" switch for h2incX:
-r: size-safe RECORD definitions. May be required if a C bitfield isn't
fully defined, that is, only some bits are declared. With this option
set the record is enclosed in a union together with its type.
example (excerpt from structure MENUBARINFO in winuser.h):
BOOL fBarFocused:1;
BOOL fFocused:1;
is now translated to:
MENUBARINFO_R0 RECORD fBarFocused:1,fFocused:1
union ;added by -r switch
BOOL ? ;added by -r switch
MENUBARINFO_R0 <>
ends ;added by -r switch
So MASM will reserve space for a BOOL (which is 4 bytes). Without
the -r option MASM would pack the bits in 1 byte only.
So, it's about size as I suspected ...
Hi Greenhorn
Thanks for the info. In the meantime, I'm experimenting with C and Masm to see how they implement RECORD bit manipulation. It's really not that complex once you understand that they only work with registers and not memory. This eliminates all the complexity of endianness.
Biterider
With my little skills in macros I've tinkered by that time this ones.
;-------------------------------------------------------------------
;--- GETFIELD() macro function to retrieve a value from a RECORD
;--- example: GETFIELD(SomeRec.Fieldname)
;-------------------------------------------------------------------
GETFIELD macro FieldDotFieldname:REQ
LOCAL reg, is, dot, field, fieldname, size_field
is INSTR 2, <FieldDotFieldname>, <.> ;-- first char must NOT be a '.'
if is eq 0
.err <GETFIELD: Field.Fieldname required. >
endif
while is gt 0 ;-- Loop through argument and find last '.', if any.
dot = is ;-- Store last position of '.'
is INSTR is+1, <FieldDotFieldname>, <.> ;-- Start search at last position of '.'
endm
field SUBSTR <FieldDotFieldname>, 1, dot-1 ;-- get the record
fieldname SUBSTR <FieldDotFieldname>, dot+1 ;-- get the record member
;--- determine the size of the bit field
size_field = (SIZEOF (TYPEOF (field))) ;-- get the size of the record
if (SIZEOF (TYPEOF (field))) EQ 1
reg equ <al> ;-- Record is 1 byte size, set al to store the result.
elseif (SIZEOF (TYPEOF (field))) EQ 2
reg equ <ax> ;-- Record is 2 byte size, set ax to store the result.
elseif (SIZEOF (TYPEOF (field))) EQ 4
reg equ <eax> ;-- Record is 4 byte size, set eax to store the result.
elseif (SIZEOF (TYPEOF (field))) EQ 8
reg equ <rax> ;-- Record is 8 byte size, set rax to store the result.
endif
ifdifi field, reg
mov reg, field ;-- move only if register differs
endif
ifdef __ASMC__
and reg, .MASK fieldname ;-- AND the register by the mask
else
and reg, MASK fieldname ;-- AND the register by the mask
endif
;; if (IS_ARG_SIGNED_PTR(FieldDotFieldname) eq 1)
;; if (.WIDTH fieldname + fieldname) lt size_field * 8
;; shl reg, size_field * 8 - .WIDTH fieldname - fieldname ;-- shift to left (signed)
;; sar reg, size_field * 8 - .WIDTH fieldname ;-- shift to right (signed)
;; else
;; sar reg, fieldname ;-- shift to right (signed)
;; endif
;; else
if (fieldname) gt 0
shr reg, fieldname ;-- shift to right (unsigned)
endif
;; endif
exitm <reg>
endm
;-------------------------------------------------------------------
;--- SETFIELD() macro function to set a value to a RECORD
;--- example: SETFIELD(SomeRec.Fieldname, 1)
;-------------------------------------------------------------------
SETFIELD macro dest:REQ, value:REQ
LOCAL field_dot_name, val, reg, is, dot, field, fieldname, size_field, size_val, movxx
is INSTR 1, <dest>, <!(> ;-- Exclude leading bracket, if any
if is gt 0
field_dot_name SUBSTR <dest>, is+1
else
field_dot_name EQU <dest>
endif
is INSTR 1, <value>, <!)> ;-- Exclude trailing bracket, if any
if is gt 0
val SUBSTR <value>, 1, is-1
else
val EQU <value>
endif
is INSTR 2, field_dot_name, <.> ;-- first char must NOT be a '.'
if is eq 0
.err <SETFIELD: Field.Fieldname required. >
else
while is gt 0 ;-- Loop through argument and find last '.', if any.
dot = is ;-- Store last position of '.'
is INSTR is+1, field_dot_name, <.> ;-- Start search at last position of '.'
endm
endif
field SUBSTR field_dot_name, 1, dot-1 ;-- get the record
fieldname SUBSTR field_dot_name, dot+1 ;-- get the record member
size_field = (SIZEOF (TYPEOF (field))) ;-- get the size of the record
if ((OPATTR (val)) and 00000100b) ;; argument is constant
size_val = size_field ;-- if the value to set is a constant, set the size equal to the record size.
else
size_val = (SIZEOF (TYPEOF (val))) ;-- the value to set is a memory location or a register
endif ;;(OPATTR (val)) and 00000100b
if size_field EQ 1
reg equ <al> ;-- size of the record is 1 byte
movxx equ <mov> ;-- set instruction to mov
elseif size_field EQ 2
reg equ <ax> ;-- size of the record is 2 byte
if size_val EQ 1
movxx equ <movzx> ;-- set instruction to movzx
else
movxx equ <mov> ;-- set instruction to mov
endif
elseif size_field EQ 4
reg equ <eax> ;-- size of the record is 4 byte
if size_val LE 2
movxx equ <movzx> ;-- set instruction to movzx
else
movxx equ <mov> ;-- set instruction to mov
endif
elseif size_field EQ 8
reg equ <rax> ;-- size of the record is 8 byte
if size_val LE 2
movxx equ <movzx> ;-- set instruction to movzx
elseif size_val EQ 4
movxx equ <movsxd> ;-- set instruction to movsxd
else
movxx equ <mov> ;-- set instruction to mov
endif
else
.err <SETFIELD: wrong size of argument 1.>
endif
ifdifi val, reg
movxx reg, val ;-- mov the value to set into register (only if it differs)
endif
if (.WIDTH fieldname) gt 0
shl reg, fieldname ;-- shift to right
ifdef __ASMC__
and field, not .MASK fieldname ;-- and the the record by the mask
else
and field, not MASK fieldname ;-- and the the record by the mask
endif
else
.err <SETFIELD: fieldname required.>
endif
or field, reg ;-- add the value to set (end of macro SETFIELD)
endm
Example
add attrLen, GETFIELD(dword ptr [rcx+rdx].ATTR.len_)
...
SETFIELD(DWORD ptr [rdx+r8].ATTR.len_, 1)
Hi
What I didn't know was that an unnamed field with zero width would force alignment, which I verified by testing with an MS C compiler.
https://en.wikipedia.org/wiki/C_syntax#:~:text=Unnamed%20fields%20consisting,fit%20in%20it.
(https://en.wikipedia.org/wiki/C_syntax#:~:text=Unnamed%20fields%20consisting,fit%20in%20it.)
Biterider
From your link:
QuoteAs a special exception to the usual C syntax rules, it is implementation-defined whether a bit field declared as type int, without specifying signed or unsigned, is signed or unsigned. Thus, it is recommended to explicitly specify signed or unsigned on all structure members for portability.
When I was writing these GET/SETFIELD macros I debugged with VC a C library using bitfields typed as int (signed).
And in deed the compiler used arithmetic for signed operations and I tried to consider this for my macros.
But after thinking a while about it, it made absolutely no sense to me.
First, for a signed value the width of a fieldname must be greater than 1 to store the signed bit. So, what to do if a bitfield is typed as int signed and the fieldname you want to manipulate or retrieve is just one bit?
Second, for example your fieldname is 3 bits in width and has '101' set. Since it is typed as an int you load it into EAX. This will never give you a signed result.
So it makes no sense to me. That's why I commented the 'signed arithmetic' out in the macro.
Correct me if I'm wrong.
What I'm trying to say is: Bitfields/Records typed as signed makes no sense.
Quote from: Greenhorn on April 10, 2024, 06:23:57 AMGETFIELD() macro function to retrieve a value from a RECORD
QuoteI'm just wondering, though: who on earth would really want to use bit fields in an actual program? Thinking back, the only time I've ever used them was to unpack the old DOS date/time structures--remember those? Even then, I never used the assembler, but just defined my own EQUs and did the necessary shifting and masking by hand.
I am curious if the author recognises his text :bgrin:
Quote from: Greenhorn on April 10, 2024, 07:06:52 AMyour fieldname is 3 bits in width and has '101' set
111=-3
110=-2
101=-1
100=-0
000=0
001=1
010=2
011=3I admit it's a rather "forced" interpretation ;-)
Hi Greenhorn
Quote from: Greenhorn on April 10, 2024, 07:06:52 AMSo it makes no sense to me. That's why I commented the 'signed arithmetic' out in the macro.
Reading about the signed/unsigned bitfields, I found this, which makes sense for a compiler
https://stackoverflow.com/questions/149869/does-ansi-c-support-signed-unsigned-bit-fields
(https://stackoverflow.com/questions/149869/does-ansi-c-support-signed-unsigned-bit-fields#:~:text=It's%20very%20important%20to%20qualify%20your%20variables%20as%20signed%20or%20unsigned.%20The%20compiler%20needs%20to%20know%20how%20to%20treat%20your%20variables%20during%20comparisons%20and%20casting.%20Examine%20the%20output%20of%20this%20code%3A)
Biterider
Quote from: jj2007 on April 10, 2024, 07:20:13 AM111=-3
110=-2
101=-1
100=-0
000=0
001=1
010=2
011=3
I admit it's a rather "forced" interpretation ;-)
So Salomon on SOF confirms my "forced" interpretation. I confess I am unsure about
100=-0, what's your take on that?
Hi
I'm playing around with some ideas and may have another approach to the size problem: a filler field.
Since there can be more than one filler, it has to be appended with e.g. a sequential number. The reversal of the field order is also shown here:
REC16 RECORD REC16_Filler1:(sizeof(WORD)*8-5-2), REC16_Field2:2, REC16_Field1:5
REC32 RECORD REC32_Filler1:(sizeof(DWORD)*8-2-4), REC32_Field2:4, REC32_Field1:2
local rec16:REC16, rec32:REC32
mov ax, MASK REC16_Field1
mov eax, MASK REC32_Field1
A problem arises with :0 (=> new storage unit)
A possible solution could be something like
REC64 RECORD REC64_Filler1:(sizeof(QWORD)*8-4-32), REC64_Field2:4, REC64_Filler2:(@WordSize/2*8-3), REC64_Field1:3
here a second filler (REC64_Filler2) fills the rest of the first DWORD.
Note: The numbers in parentesis could be calculated by the translator. I have written them explicitly to visualise what is going on.
Biterider
Hi
I have scanned the entire Windows API (Win11) for a :0 bitfield without finding a match.
It seems that we can safely leave this deature for later.
Biterider
Hi
I have now made several attempts to get the BitField to be correctly translated to asm.
It is much more complicated than expected. The best explanation I have found is here:
https://stackoverflow.com/questions/4129961/how-is-the-size-of-a-struct-with-bit-fields-determined-measured (https://stackoverflow.com/questions/4129961/how-is-the-size-of-a-struct-with-bit-fields-determined-measured)
Biterider
Hi
After a few hours of experimenting with BitFields using an MS compiler (which we need if we want to translate the Windows header files), I came up with a strategy that might be good enough.
It has 4 steps
- Determine the size of the allocation unit (BYTE, WORD, DWORD, QWORD) that is the largest of all field types (signed or unsigned).
- Fill the allocation unit with the fields in the reverse order in which they appear, until the total size exceeds the size of the allocation unit.
- Fill the rest of the allocation unit with a dummy filler.
- Start a new allocation unit and repeat the process from step 2 to the first field.
The implemantation isn't easy but seems to work.
Biterider
QuoteIf enough space remains, a bit-field that immediately follows another bit-field in a structure shall be packed into adjacent bits of the same unit. If insufficient space remains, whether a bit-field that does not fit is put into the next unit or overlaps adjacent units is implementation-defined. The order of allocation of bit-fields within a unit (high-order to low-order or low-order to high-order) is implementation-defined.
Crystal clear indeed. I love C :thumbsup:
Using the MS compiler as "implementation" is the best idea, I guess. As you wrote already, these headers are being used by that compiler.
Hi
After finishing the bitfield renderer, I looked at the rest of the code and found some problems that require rethinking some parts of the code. The parser is fine, but the renderer has become very large and confusing due to the complexity of the C grammar. To start from scratch, I looked for documents describing the C grammar and came across the MS C documentation, which is not so good for this purpose, but necessary because it is the target grammar.
The BNF of C seems to be a very good and compact reference https://cs.wmich.edu/~gupta/teaching/cs4850/sumII06/The%20syntax%20of%20C%20in%20Backus-Naur%20form.htm (https://cs.wmich.edu/~gupta/teaching/cs4850/sumII06/The%20syntax%20of%20C%20in%20Backus-Naur%20form.htm)
The GNU C Reference Manual is also a good supplement https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html (https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html)
This new version of h2inc is taking longer than expected, but I have a good feeling about it. :biggrin:
Biterider
I took a look at that BNF description. (I'm not pretending that I totally understand it, but I do get what it is, a hierarchical list of all the elements that comprise a language.) I was surprised how relatively short and compact it is.
So now are you going to go ahead and write YACC[n], now that you have the complete C language description?
(Heh, just kidding, but it might not be all that much more difficult than untangling the bitfield mess that you're dealing with.)
Quote from: Biterider on April 20, 2024, 04:05:41 PMthe renderer has become very large and confusing due to the complexity of the C grammar
Quote from: jj2007 on March 29, 2024, 11:36:35 PMI suppose you are aware of Gcc's option -E, which allows to get rid of most of the conditional stuff.
Do you use such an option, or do you handle the conditional stuff yourself?
Hi
@NoCForMe: you can take a look at the code here https://github.com/ObjAsm/ObjAsm-C.2/tree/master/Projects/X/h2inc%2B (https://github.com/ObjAsm/ObjAsm-C.2/tree/master/Projects/X/h2inc%2B)
You can get a feel for it, although it is a work in progress and far from finished. I have moved the project to the public area.
@JJ: I'm translating the header files using some conditional switches fixed in the code, e.g. to get rid of all the C++ stuff, and I'm leaving other well-known switches like WIN32_LEAN_AND_MEAN, INCL_WINSOCK_API_PROTOTYPES, INTERNET_PROTOCOL_VERSION, etc. in place so that the include files can be controlled the same way it is shown in the documentation and in the C examples.
Biterider
Hi
I came across another challenge that makes the compilation more complicated: preprocessor directives inside some MASM declarations.
Example using records:
typedef struct {
unsigned short reserved : 6,
#ifndef VERSION2
fBusy : 1;
#else
fBusy:1,
fAck : 1;
#endif
} Status;
In this fictional example, a new bit has been added due to a new version.
Rendering to MASM as
Status struct
Status_REC record \
ifndef VERSION2
fBusy:1,
else
fAck:1,
fBusy:1,
endif
reserved:6
Status_R0 Status_REC <>
Status ends
did not work.
MASM doesn't like to break the declaration, and the first line (Status_REC record) must have at least one bit declaration. In short, you cannot do it this way.
The same happens with prototypes, of which there are a few in the Windows header.
Biterider
I hate to ask you this, but after all this angst and toil, do you think anyone would even use this capability if you somehow manage to implement it? I mean, who out there actually uses bitfield records in assembly language? (Except for those few instances in Win32, like the time/date fields.)
I use bit fields in my programs all the time, but I manage to do it with simple EQUs and bitmasks. No need here for packed records.
Hi
Translating the Windows header files is not an easy task. It has a lot of challenges that I'm shareing here in order to find a practicable solution.
I've done this job at least 2 times and know what I'm talking about. The aim of this new version is to avoid all the extensive work of manually correcting all the cases not covered by the translator, if you read my last post you will have noticed that I used the record case as an example.
More important are the prototypes, which are much more common.
This is one of the most important projects for the future of MASM.
We all know about Windows.inc and its problems. Not to mention 64-bit compatibility.
By sharing the development of the project, which is btw open source, I hope to find some people to help with some areas of the code or to test the final result.
Besides the translation, there is a significant part at the end of the project that needs agreement on how the include files should be arranged. Technically this is a minor point, but it has a lot of potential for discussion.
If you have a better approach, please feel free to share it. The MASM community will greatly appreciate it.
Regards, Biterider
It sounds like we need a C++ "engine", are there any open source ones? Of course, then we need a C++ programmer to adapt the code.
I see it as three areas
- WinAPI, easy enough to parse .lib files
- constants
- structures
If we can agree on syntax, we could each pick a .h file to convert?
Over the past few years I've been adding to my includes as I go (32- and 64-bit mixed) but they're not MASM64 friendly :biggrin:
Hi sinsi
Thank you for your thoughts.
In the beginning I had the same ideas, but based on the experience I gained with this project, I realised that a translator has to work a bit differently. This is because you don't have a preprocessor that 'simply' inserts or discards part of the code. The translator has to go through all possible PP code paths and arrange the code in a way that a MASM-compatible assembler can understand. One of the challenges we have is that the HL directives such as if, else, endif cannot be placed as flexibly as in C.
Nevertheless, it might be interesting to explore your idea further. Unfortunately, I don't know of any open source C compiler code that could be modified for our purposes.
The results of the current h2incX engine are actually very good (you can have a look here https://github.com/ObjAsm/ObjAsm-C.2/tree/master/Code/Inc/Windows (https://github.com/ObjAsm/ObjAsm-C.2/tree/master/Code/Inc/Windows)). I have been using it for a long time without any problems in 32 and 64 bit.
The reason for a reworked engine is the amount of work needed to make small corrections afterwards. These are exactly the cases I wanted to show with some examples in the previous post.
Regards, Biterider
Just a thought out of left field, but have you seen the Magic Number DataBase (https://www.magnumdb.com/)?
Parsing the Windows headers for constants, maybe it could be extended?
Hi Sinsi
I solved the record problem and some others, but at the same time I introduced some regressions.
Maybe this weekend I can post a very first beta. The application is designed to selectively output protos, constants, structures/unions, etc. using some CL-switches.
The implementation of this selectivity dfeature is not fully implemented yet, as I'm concentrating on the rendering atm.
Regards, Biterider
Hi
I'm finishing the core functionality and leaving the "extern" declarations for the end. In the current state, the application can render most of the header files but has problems with this code:
extern "C" {
typedef BOOL (CALLBACK * ImageAbort)(VOID *);
typedef ImageAbort DrawImageAbort;
typedef ImageAbort GetThumbnailImageAbort;
}
Can anyone fluent in C explain what these lines do and what the MASM equivalent would be?
Biterider
first declare callback function and next two with predefined names of them.
Hi Timo
Thanks for the quick replay :thumbsup:
I'm a bit confused about the "typedef".
As far as I understand it, the first line defines a typedef to a proto and the 2 following lines uses this type to define 2 extern C functions.
Do you know how to translate it to MASM?
Biterider
At a guess, maybe could be written in masm something like this:
ImageAbort_Callback_Proto TYPEDEF PROTO C VoidPtrParam:DWORD
ImageAbort_Callback TYPEDEF PTR ImageAbort_Callback_Proto
.DATA
DrawImageAbort ImageAbort_Callback 0
GetThumbnailImageAbort ImageAbort_Callback 0
The DrawImageAbort and GetThumbnailImageAbort values can be assigned at runtime, before they are used and then can be used in Invoke's like:
Invoke GetThumbnailImageAbort, Addr hImageAbort
But its just a guess based on the code
Hi fearless
Thanks for your hints.
I came now to the following translation
ImageAbort_Proto typedef proto C :ptr
ImageAbort typedef ptr ImageAbort_Proto
DrawImageAbort typedef ImageAbort
GetThumbnailImageAbort typedef ImageAbort
in the same header file I can now use the following translation
GdipGetImageThumbnail_ proto WIN_STD_CALL_CONV :ptr GpImage, :UINT, :UINT, :ptr ptr GpImage, :GetThumbnailImageAbort, :ptr VOIDwhich works fine.
In my opinion, the extern directive in C for a typedef is not relevant for masm.
Biterider
Hi
I have solved most of the problems mentioned above and now it is time for a very first beta release. The attached binary is compiled for debugging (pdb is provided to track a bug). The application options can now be placed in an environment variable or file using the "@" command line option, like in other modern tools.
h2inc+.exe -@".\h2inc+_Options.txt"
I provide all the list files where the collected knowledge is stored.
So far I managed to compile the latest set of header files "um_10.0.22621.0" without errors, but tests with other APIs are welcome to strengthen the renderer engine.
In terms of performance, the application rendered 192 header files in less than 1.9 seconds on my machine, which is very good.
The translated set of .inc files can be seen here
https://github.com/ObjAsm/ObjAsm-C.2/tree/master/Projects/X/h2inc%2B/inc (https://github.com/ObjAsm/ObjAsm-C.2/tree/master/Projects/X/h2inc%2B/inc)
I did not enable exception handling to better catch crashes. It will be enabled in the final version.
Regards Biterider
Compliments, Biterider - right now I am too busy to test it, but I surely will :thumbsup:
Thank you