The MASM Forum

64 bit assembler => UASM Assembler Development => Topic started by: Biterider on October 07, 2017, 07:28:04 PM

Title: UASM 2.41 local overlapping bug
Post by: Biterider on October 07, 2017, 07:28:04 PM
Hi
I'm writing a 32 bit program which fails with UASM but runs OK with ML.
I condensed the problem to a test file (code attachment). The situation arises when I use a custom prologue. The variables passed to the prologue are OK, but addresses of the local vars seems to overlap. When debugging the code and I manually write into the first element of cBuffer, pFontFamily is also changed and the program crashes (pic attached).

Regards, Biterider
Title: Re: UASM 2.41 local overlapping bug
Post by: habran on October 07, 2017, 07:42:00 PM
Thanks Biterider :t
Will look it up ASAP
Title: Re: UASM 2.41 local overlapping bug
Post by: aw27 on October 07, 2017, 09:05:42 PM
Quote
I'm writing a 32 bit program which fails with UASM but runs OK with ML.
I wonder why it works in MASM because according to the manual "Your macro function must return the parmbytes parameter".
What I mean is that if the custom prologue ends with exitm %ArgBytes it may work in UASM.
Title: Re: UASM 2.41 local overlapping bug
Post by: Biterider on October 07, 2017, 10:02:22 PM
Hi aw27
You are right. Returning %ArgBytes in UASM solves this problem. I rechecked it using ML and it seems not to be sensitive to the returning value. Neither <0> nor %ArgBytes makes a difference.

Biterider
Title: Re: UASM 2.41 local overlapping bug
Post by: Biterider on October 08, 2017, 01:35:08 AM
Hi
Testing on the current application shows that the issue still persists. I looked into the generated code for 4 situations: using UASM vs ML and "exitm 0" vs "exitm  ArgBytes". The only case where the emitted code is correct is using ML and "exitm 0". All other cases differ by far.
Attached are the disassemblies of the 4 cases.

My Masm Manual says about "User-Defined Prologue and Epilogue Code"
QuoteYour macro function must return the parmbytes parameter. However, if the prologue places other
values on the stack after pushing BP and these values are not referenced by any of the local variables,
the exit value must be the number of bytes for procedure locals plus any space between BP and the
locals. Therefore, parmbytes is not always equal to the bytes occupied by the locals.

The text is a bit confusing because you have to return the parameter byte count or the total local space, but the running code requires zero.

Biterider
Title: Re: UASM 2.41 local overlapping bug
Post by: aw27 on October 08, 2017, 03:05:35 AM
I think you are correct, Biterider. UASM (this error comes from JWASM) does not appear to account for the space taken by the LOCALs and relies 100% on the value returned from the macro and relies as well in the macro to make room for the LOCAL variables. I have not checked well, but if you return from the macro with exitm %LocalBytes it might work. Interestingly neither MASM nor UASM appear to care about "Your macro function must return the parmbytes parameter" - it worked previously simply because the byte count for LOCALS was equal to the byte count for the Arguments.
Title: Re: UASM 2.41 local overlapping bug
Post by: Biterider on October 08, 2017, 04:36:57 AM
Hi
OK, returning %LocalBytes works on both. It seems to be an undocumented feature of ML, that retuning 0 assumes the value of the local byte count.
Looking for other User-Defined Prologue code on the forum shows that in some cases returning 0 was a common practice in the past.

Regards, Biterider
Title: Re: UASM 2.41 local overlapping bug
Post by: jj2007 on October 08, 2017, 04:54:08 AM
Right now I don't have the energy to investigate what exactly happens there, but I can tell you when the problem appeared: in the night from 15 to 16 May, 2017 8)

Testcode:include \Masm32\MasmBasic\Res\JBasic.inc
.code
usedeb=1
LoadTextFont proc <cb> uses rdi rbx pIniFile:SIZE_P, pFontFamily:SIZE_P, pFontSize:DWORD
LOCAL cBuffer[4096]:DWORD
LOCAL local_p0:DWORD, local_p1:DWORD, local_p2:DWORD
  lea rbx, cBuffer
  xor edi, edi
  mov local_p0, 4
  mov local_p1, 5
  mov local_p2, 6
  @@:
mov DWORD ptr [rbx+rdi], 0FFFFFFFFh
add rdi, 4
cmp edi, sizeof cBuffer
  jl @B
  deb 4, "before MsgBox", x:rbx, rdi, pIniFile, pFontFamily, pFontSize, local_p0, local_p1, local_p2
  ; MsgBox 0, "before the ret", "Hi", MB_OK or MB_SETFOREGROUND
  ret
LoadTextFont endp

Init ; OPT_64 1 ; put 0 for 32 bit, 1 for 64 bit assembly
;   int 3
  mov esi, 11111111
  mov edi, 22222222
  mov ebx, 33333333
  PrintLine Chr$("This code was assembled with ", @AsmUsed$(1), " in ", jbit$, "-bit format")
  deb 4, "before", esi, edi, ebx, x:esp
  jinvoke LoadTextFont, 1, 2, 3
  deb 4, "after", esi, edi, ebx, x:esp
  ; MsgBox 0, "Wow, it works!!!!", "Hi", MB_OK or MB_SETFOREGROUND
EndOfCode


Expected output:This code was assembled with ml64 in 64-bit format
before
esi     11111111
edi     22222222
ebx     33333333
x:esp   12ff00h

before MsgBox
x:rbx   12bed0h
rdi     16384
pIniFile        1
pFontFamily     2
pFontSize       3
local_p0        4
local_p1        5
local_p2        6

after
esi     11111111
edi     22222222
ebx     33333333
x:esp   12ff00h


That works great for ML64, AsmC and HJWasm64 of 15.5.17, 11:10 but every UAsm version after that produces this in 64-bit mode:after
esi     11111111
edi     -1
ebx     -1
x:esp   12ff00h
- meaning the "uses" part is not correctly translated. Source attached, needs RichMasm (http://masm32.com/board/index.php?topic=5314.0). For the masochists: Prolog and Epilog macros are in \Masm32\MasmBasic\Res\JBasic.inc, and prologue ends with EXITM %(localbytes++SIZE_P*(alignedUses+2))
:P
Title: Re: UASM 2.41 local overlapping bug
Post by: johnsa on October 10, 2017, 12:40:57 AM
JJ, how do i get that testcode to assemble from the command line easily ? :)
So i can debug it.. thanks!

John
Title: Re: UASM 2.41 local overlapping bug
Post by: jj2007 on October 10, 2017, 02:38:03 AM
John,

Options are \masm32\bin\ml64 /c /Zp8 for the 64-bit version. The easiest way (assuming you do have MasmBasic (http://masm32.com/board/index.php?topic=94.0)):
- open the *.asc in \Masm32\MasmBasic\RichMasm.exe
- uncomment the int 3 under Init (int 3 as lowercase triggers the debugger)
- hit F6 and see RichMasm trying to launch \Masm32\x64Dbg\release\x64\x64dbg.exe
- if that is not your path to the debugger, insert under EndOfCode the following line, with your path, of course:
OPT_DebPath64 \Masm32\x64Dbg\release\x64\x64dbg.exe

For the 32-bit build (OPT_64 0), the option is as follows:
OPT_DebPath \Masm32\OllyDbg\ollydbg.exe (or any other path to Olly; not tested with WinDbg but it should work)
Title: Re: UASM 2.41 local overlapping bug
Post by: johnsa on October 10, 2017, 03:11:11 AM
I mean I'm debugging UASM itself, so I want to provide it with all the relevant paths etc so I need to provide a full UASM command line to step through it and see how it assembles.
Title: Re: UASM 2.41 local overlapping bug
Post by: jj2007 on October 10, 2017, 04:00:51 AM
Ok. There is a plain text source in the archive just posted. Otherwise, RichMasm accepts additional options as follows:

OPT_DebugA -whatever -as -many -as -you -like ; assembler commandline
OPT_DebugL -whatever ; linker commandline
Title: Re: UASM 2.41 local overlapping bug
Post by: Biterider on October 11, 2017, 05:28:26 PM
Hi JJ
I'm trying to compile the above MB source but I get the following error
** Start D:\Masm32\MasmBasic\Res\bldallRM.bat **
**** 64-bit assembly ****


OPT_Res:  LocalOverlappingBug.rc


*** Assemble, link and run LocalOverlappingBug ***


*** Assemble using \masm32\bin\UAsm64 /c /Zp8 -win64 tmp_file.asm ***
UASM v2.42, Oct 10 2017, Masm-compatible assembler.
Portions Copyright (c) 1992-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.


** 64-bit assembly **


***********
ASCII build
***********


\Masm32\MasmBasic\Res\JBasic.inc(1582) : Error A2106: Cannot open file: "\Masm32\MasmBasic\Res\pt.inc" [ENOENT]
\Masm32\MasmBasic\Res\JBasic.inc(1582): Included by
  Tmp_File.asm(1): Main line code
____ LABEL GetStdHandle uses invoke j@GetStdHandle
...

It seems the the pt.inc file is missing. I searched the complete MB folder without luck. I'm using your distro dated 4 Oct 17.


Could you send/post it?


Biterider
Title: Re: UASM 2.41 local overlapping bug
Post by: jj2007 on October 11, 2017, 06:06:02 PM
Hi Biterider,

pt.inc should have been generated when trying to build the code from RichMasm, together with \Masm32\MasmBasic\Res\DualWin.inc.

I have just tested it on a fresh installation, and found two little problems:
- it needs \Masm32\bin\uasm32.exe (which is not correct because my default is now UAsm64...)
- it does indeed complain about the missing pt.inc, but the file is there... and on second try, it builds fine.

Which means that RichMasm should wait a second until the file is created properly. Will be corrected in the next release :icon_redface:

Can you check if ?:\Masm32\MasmBasic\Res\pt.inc and DualWin.inc are present? And simply retry if yes?
Title: Re: UASM 2.41 local overlapping bug
Post by: Biterider on October 11, 2017, 06:37:42 PM
Hi JJ
I found DualWin.inc but no PT.inc.
Now I deleted DualWin.inc and pressed F6. The result was better since I could see that something was generated. I checked for pt and dualwin and both were in the res folder. Now i get a notification that "D:MASM32\MASMBASIC\Res\bldallRM.bvv" was not found. That file is not present in the res folder.
Any clue?


Biterider
Title: Re: UASM 2.41 local overlapping bug
Post by: jj2007 on October 11, 2017, 06:49:39 PM
Congrats, you are almost there. Just quit RichMasm (save if necessary), restart and try again. The "bvv" is a bug that creeps up one every three months or so, maybe one day I'll understand it :bgrin:
Title: Re: UASM 2.41 local overlapping bug
Post by: Biterider on October 11, 2017, 06:51:56 PM
Wow... I got it
I restarted RichMasm and pressed F6. It complained about a missing tmp file. Then I pressed F6 again and voilá, I got the inteded output!
I don't know what is going on but I got it working...
Biterider
Title: Re: UASM 2.41 local overlapping bug
Post by: jj2007 on October 11, 2017, 07:07:08 PM
Congrats, and apologies :icon_redface:

For me, it works always fine, of course, on Win7-64, XP and Win10. But I wish I could test my stuff on other machines... Windows != Windows, unfortunately.
Title: Re: UASM 2.41 local overlapping bug
Post by: Biterider on October 11, 2017, 07:17:31 PM
Hi JJ
If it helps, here is my system configuration:


Windows 10 Home x64 / i7 4770K / 16GB RAM


Biterider
Title: Re: UASM 2.41 local overlapping bug
Post by: johnsa on October 11, 2017, 07:43:00 PM
My machines:

Windows 7 x64 Ultimate, i7 3610qm, 16gb ddr3 (always used it)
Windows 10 Pro x64, AMD Threadripper1950X, 32gb ddr4 (my new toy) :)
Windows 8.1 Pro, some dumb i7 chip, 16gb ddr3 (never use it)
Title: Re: UASM 2.41 local overlapping bug
Post by: jj2007 on October 11, 2017, 07:57:07 PM
Quote from: Biterider on October 11, 2017, 06:51:56 PMI restarted RichMasm and pressed F6. It complained about a missing tmp file.

These are the little mysteries :(

Just read on MSDN Is it necessary to call "FlushFileBuffers" before "CloseHandle" (https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/9e1aea76-8e16-4fa1-a895-476a19df409a/is-it-necessary-to-call-flushfilebuffers-before-closehandle?forum=windowssdk), and the answer is a clear NO. So why should a Tmp_File.asm be missing if the handle was closed ::) ... because your machines are faster than the speed of light :eusa_naughty: