Exclamation Mark Not Working In Macro Expansions

[dawnraider]

Hi Habran!

It's been a while!

Just upgraded to the latest HJwasm (2.15) to confirm that this is a bug. I also can reproduce it in 2.11 and 2.13.

The following source file compiles perfectly under ml64.exe, but not HJWasm:

Code Select Expand


TestMacro MACRO source:REQ
LOCAL rax
        rax TEXTEQU <!rax>
        mov         rax,    source
ENDM
   
.CODE

TestFunc PROC pData:PTR

        TestMacro   pData
        ret

TestFunc ENDP

END


Copy the above text, save as file macro-test.asm, and build using hjwasm.exe /coff /Zg /e1000 /win64 /c /Sa macro-test.asm

The following errors come out:

Code Select Expand


HJWasm v2.15r2, Oct  1 2016, Masm-compatible assembler.
Portions Copyright (c) 1992-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.

macro-test.asm(11) : Error A2101: Macro nesting level too deep
TestMacro(2)[macro-test.asm]: Macro called from
  macro-test.asm(11): Main line code
macro-test.asm: 16 lines, 1 passes, 0 ms, 0 warnings, 1 errors


To assist you, here are the listing file outputs from both assemblers. First the output from ml64.exe

Code Select Expand


00000000                       .CODE

00000000                       TestFunc PROC pData:PTR

                                                TestMacro   pData
                             1  LOCAL rax
= rax                       1                  ??0000 TEXTEQU <!rax>
00000000  55              *        push   rbp
00000001  48/ 8B EC       *        mov    rbp, rsp
00000004  48/ 8B 45 10      1                  mov         ??0000,    pData
                                                ret
00000008  C9              *        leave
00000009  C3              *        ret    00000h

0000000A                       TestFunc ENDP


Now here is the listing file from hjwasm.exe for the same file:

Code Select Expand


                                .CODE
00000000                    *   _TEXT segment
                            *   assume cs:FLAT

00000000                        TestFunc PROC pData:PTR

                                                TestMacro   pData
= ??0000                    1                  ??0000 TEXTEQU <!??0000>
00000000                     1                  mov         ??0000,    pData
                           Error A2101: Macro nesting level too deep
00000000                    *   push rbp
00000001                    *   mov rbp, rsp
00000004                                        ret
00000004                    *   leave
00000005                    *   retn

00000006                        TestFunc ENDP


Please could you investigate?

As always, Thanks in advance.

[habran]

Thanks dawnraider :t
We will investigate ASAP

[dawnraider]

Thanks, Habran!

If you need me to do anything further then just let me know.

Look forward to hearing your plans regarding any fix. :t

[habran]

Hi dawnraider,
I have examined your macro in this case and realized that you are using RAX as a local variable.
RAX is a reserved word and can not be changed to variable.
Again, HJWasm has thrown an error while ML64 let it go.

I am not gonna scrutinize your programming skills, I am to busy for that.

Please, don't be offended, thank you for challenging HJWasm capabilities :t

 

[johnsa]

Just too add to this one,

I've tried changing the local from rax to another name and the same nesting error results, so I would avoid using register names in general but even so.
Before i can comment further on this one I need to understand what the purpose of the macro is supposed to be?
Right now it makes little sense to me,

TestMacro MACRO source:REQ
LOCAL arax                                               ;<- arax is a "MACRO parser local, not a local variable with an actual memory address"
        arax TEXTEQU <!arax>                     ;<- fair enough.. so now arax is considered to be a text literal in the macro engine with value: "!arax" (literal again.. as the arax can't substitute any real value).
        mov         arax,    source                   ;<- arax is neither a register nor memory location, so I can't imagine what this would attempt to do? ML64 may very well produce a listing file for this.. but I'm pretty sure it
; won't amount to anything runnable... i'm really curious to know what it actually attempts to do in it's disassembly of this?
ENDM

[hutch--]

John,

Make yourself an option to clean up trailing junk in string data and it will solve problems of this type in many different places. Its a simple backscan from the end of the string, you can even do it in MASM macros but at the compiler level it would be a lot cleaner and faster.

[johnsa]

Hutch,

You answer made even less sense to me than the question! :) What are we talking about cleaning junk from the end of string data?

[hutch--]

Sorry John, I plugged the posting in the wrong topic.

http://masm32.com/board/index.php?topic=5766.0

Habran's second posting.

[johnsa]

Haha.. ok that explains why that made no sense to me in this context! :)

[jj2007]

clean up trailing junk in string data

BASIC-style garbage collection? Does it offer any advantage over HeapRealloc or HeapFree?

[hutch--]

Nah, just a tail end trimmer, I have had to have text cleanup tools for years to do some of the work I do. You can write them in basic but an assembler or C compiler will do it just fine.

[dawnraider]

Hi Habran and Johnsa,

I am sorry to hear that you decided not to fix this as ML64.exe assembles this and HJWasm is therefore not compatible with ML64.exe at
a basic level.

Any symbol in MASM can be defined as a macro local variable apart from EXITM and ENDM. This is not a bug and appears to be by design.
For example, the ENDP directive for PROC can be defined as a local variable in a macro without resorting to OPTION NOKEYWORD,
as follows:

Code Select Expand


; define macro.
;
WhatDoesThreeEqual MACRO

LOCAL ENDP

ENDP = 3
%echo And 3 = @CatStr(%ENDP)

ENDM

; and call. Outputs "And 3 = 3"
;
WhatDoesThreeEqual


The reason for this coding practice is simple: I am writing cross-platform macros that work on both 32-bit and 64-bit platforms. It is a very
significant overhead maintaining two sets of macros that utilise general purpose registers when all that changes are the register names and
a few stack instructions (i.e. EDX vs RDX, and PUSHFD vs. PUSHFQ).

The following is a fragment of a real peace of production code that I am currently developing in MASM. It assembles cleanly and
produces the expected results with ML.exe :

Code Select Expand


MyMacro MACRO

    LOCAL eax, edx, pushfd, popfd               ;; define local versions of registers that we can override.

IFDEF R15 ; 64-bit mode.

    eax    TEXTEQU <rax>                        ;; substitute register references for 64-bit registers throughout.
    edx    TEXTEQU <rdx>
    pushfd TEXTEQU <pushfq>                     ;; substitute stack instructions for 64-bit instructions throughout.
    popfd  TEXTEQU <popfq>

ELSE    ; 32-bit mode.

    eax    TEXTEQU <!eax>                       ;; use registers references as they appear.
    edx    TEXTEQU <!edx>
    pushfd TEXTEQU <!pushfd>                    ;; use stack instructions as they appear.
    popfd  TEXTEQU <!popfd>

ENDIF

        push        eax                         ; EAX to be restored after save.
        push        edx                         ; EDX to be restored after save.
        pushfd                                  ; EFLAGS to be restored after save.

        mov         edx,            s_pAdtBufAdr
        test        edx,            000000001h
        jz          check_first
        and         edx,            0FFFFFFFEh

        ...

        popfd                                   ; restore registers used during save.
        pop         edx
        pop         eax

ENDM


Now, using ML64.exe, exactly the same macro assembles to:

Code Select Expand


        push        rax
        push        rdx
        pushfq

        mov         rdx,            s_pAdtBufAdr
        test        rdx,            000000001h
        jz          check_first
        and         rdx,            0FFFFFFFEh

        ...

        popfq                                   ; restore registers used during save.
        pop         rdx
        pop         rax


As you can see, this macro produces proper and solid code on both 32-bit and 64-bit assemblers and it means I can write all my
macros using a single 32-bit based code template. It also means that I can use the actual register names in my macros which
makes the macros themselves a great deal more readable and easier to understand than having to use pseudonyms for my register
names all the time.

I am not gonna scrutinize your programming skills, I am to busy for that.

Please, don't be offended, thank you for challenging HJWasm capabilities :t

Don't worry, I'm not offended. However, I do hope you appreciate the value of being able to significantly reduce duplication of time
and effort where multi-platform code bases have to be supported.

I'll just continue to use MASM until HJWasm is sufficiently compatible to be able to handle these basic, if a little non-standard,
macros.

It's a shame as I would like to be able to do the AVX2/EVEX programming without having to buy Visual Studio 2013/Windows 10
licenses (I currently only have VS 2010 installed on my personal PCs...)

I hope this explanation helps.

Cheers.

[jj2007]

The reason for this coding practice is simple: I am writing cross-platform macros that work on both 32-bit and 64-bit platforms.

You may wish to have a lookl at rrr's macros (don't find them right now) and my own stuff. All "dual" examples in MasmBasic assemble as 64- or 32-bit code, with ML (32), ML64, HJWasm or AsmC. Check in particular the includes for rax equ eax

[johnsa]

Code Select Expand


MyMacro MACRO

    LOCAL r_eax, r_edx, r_pushfd, r_popfd               ;; define local versions of registers that we can override.

IFDEF R15 ; 64-bit mode.

    r_eax    TEXTEQU <rax>                        ;; substitute register references for 64-bit registers throughout.
    r_edx    TEXTEQU <rdx>
    r_pushfd TEXTEQU <pushfq>                     ;; substitute stack instructions for 64-bit instructions throughout.
    r_popfd  TEXTEQU <popfq>

ELSE    ; 32-bit mode.

    r_eax    TEXTEQU <!eax>                       ;; use registers references as they appear.
    r_edx    TEXTEQU <!edx>
    r_pushfd TEXTEQU <!pushfd>                    ;; use stack instructions as they appear.
    r_popfd  TEXTEQU <!popfd>

ENDIF

        push        r_eax                         ; EAX to be restored after save.
        push        r_edx                         ; EDX to be restored after save.
        r_pushfd                                  ; EFLAGS to be restored after save.

        mov         r_edx,            s_pAdtBufAdr
        test        r_edx,            000000001h
        jz          check_first
        and         r_edx,            0FFFFFFFEh

        r_popfd                                   ; restore registers used during save.
        pop         r_edx
        pop         r_eax

ENDM


As a work around the above works fine in both.
Personally i think this is better because i like to know when i say "eax" i mean eax, not something else.. so having them as r_eax, r_edx and so on i know i mean the local variable as opposed to the actual register, i think it's more clear as to the intention.. especially given that debugging macros is a pain as you can't step into them at a source level it leaves less room for silly mistakes.

That said there is a definite difference in how they handle the situation:

MASM expands <!eax> as text first before recursing the macro evaluation, where-as hjwasm is recursing first and thus expanding eax prior to <! >  ... which sort of makes sense except for the fact that <eax> should already be literal text and not force any sort of macro expansion without %  .. the ! shouldn't even be required as it's the same as < >
Changing this behaviour however might break other things, so I would stick to the above work-around for now.

[dawnraider]

Thanks for your suggestion. I had thought of this long before I investigated many other possibilities. It is a question of personal preference
and I prefer to leave the 32-bit code base as is and replace the 64-bit variations with macros, leaving the code readable as 32-bit code.
The biggest problem I also found with a method like the one you suggested is that it plays havoc with editor syntax highlighting; all register
names end up being indistinguishable from local/global variable names. This readability aspect was a particular annoyance.

As it happens I had a workaround a while back which I am using now. It's not great but the volume of code changes were small so reducing
the possibility of new bugs being introduced through large code change volume.

... which sort of makes sense except for the fact that <eax> should already be literal text and not force any sort of macro expansion without %  .. the ! shouldn't even be required as it's the same as < >

When MASM does any kind of evaluation of a command or a function parameter list, all open references to text are automatically expanded
as macros before anything else. This is a core "feature" of MASM but it is MASM and there are many people including myself who have had
no choice but to rely on this behaviour over the years to get a particular task done.

It has some particularly tedious ramifications. For example, this:

Code Select Expand


MyMacro MACRO
    LOCAL temp
    temp = 3
    %echo the value of temp is @CatStr(%temp)
ENDM

MyMacro


produces:

Code Select Expand


the value of ??010F is 3


It appears that you can protect against macro expansion by placing the word to be protected in single or double quotes. E.g.

Code Select Expand


MyMacro MACRO
    LOCAL temp
    temp = 3
    %echo the value of 'temp' is @CatStr(%temp)
ENDM

MyMacro


produces:

Code Select Expand


the value of 'temp' is 3


However, this is useless when passing text variables around as the name of the variable ends up in quotes, not the value of the variable!
This is the problem when trying to maintain compatibility with a tool that itself has more bugs and inconsistencies than you care to mention.

It's a hard pill to swallow, but being compatible is not about having the most elegant or fastest implementation; it is about being compatible;
a one-for-one likeness whether one agrees with the correctness of the original implementation or not.

I do thank you, though, for acknowledging the fact that there is a difference in behaviour and for being kind enough to suggest a workaround.

Cheers.