UASM 2.46 "Register value overwritten by INVOKE" bug

[Biterider]

Hello
Recently I found a problem with the "invoke" error detection algorithm. It seems that when passing subregisters, something went wrong, like in the following test case:

Code Select Expand


.xmm
option casemap:none
option dotname     
option frame:auto   
option win64:8     
option stackbase:rsp




.code


Test1 proc Arg1:QWORD, Arg2: QWORD
    mov r10, Arg1
    mov r11, Arg2
    ret
Test1 endp


start proc uses rbx
    invoke Test1, rdx, rcx
    invoke Test1, rdx, ecx
    invoke Test1, edx, rcx
    invoke Test1, edx, ecx
    ret
start endp


end start


Compiling this code should throw an error on each invoke line, but it doesn't.


Regards, Biterider

[johnsa]

Ok, so this is an interesting spot..

It's not the fact that the overwitten register check isn't correct so much as the fact that we're allowing you to pass a 32bit register, EDX to a 64bit RCX parameter.
The responsible code for this seems to have been added in jwasm 2.11 which allows the register extension as long as the target type is not a pointer.

I'm not sure I fully support that idea, the type should match the specified type, in which case the register overwritten check works perfectly.

Does anyone have any thoughts on this ? I'm inclined to remove that allowance as it takes no notion of whether the 32bit value should be sign extended or zero extended, so it seems a bit hackish.

Further more it seems to be allowed on any size variation, and only 32<->64 would actually zero out the upper part of the register, any other sizes would leave the parent register in a broken state (AX/AL for example). So not only does it force zero extension for 32/64bit promotion it breaks smaller ones potentially.

[johnsa]

Ok.. it seems like it might be a valid feature, if the register is passed with a type like SDWORD PTR etc then the register will be correctly extended, so I've fixed the overwrite warning instead.
Will be in 2.46.6 update soon.