I've tried to upload my source file, but wasn't allowed, so it's included below. I should note this isn't really a source file; rather, it's the output file from our own proprietary compiler. We are seeing differences between what jwasm produces and uasm, given the same command line. So, assembling with the following command line:
jwasm -elf64 -Zp8 -Sg -nologo -c -Sa -FlHelloWorld64.lst -FoHelloWorld64.o Main64.asm
Produces a few differences (if you substitute uasm in the call). The first seems minor; in the header jw produces an 'assume' line that includes gs:error whereas uasm specifies gs:nothing. But then what I believe is the source of our problem is that the jw version starts the PROC off with:
push rbp
mov rbp, rsp
where these lines are not to be found in the uasm output, and also a
leave
So it looks like jwasm sets up a normal stack frame under these conditions where uasm does not? Is this expected? Are there different arguments we should be passing to uasm to make it compatible with how jwasm worked? The external manifestation of this is that the program works as expected when assembled with jwasm, and seg faults immediately with uasm. Below is the text of the file Main64.asm:
;; ***************************************
;; *** @main argc,argv ***
;; *** Source: Main64.txt Line: 1 ***
;; ***************************************
.DATA
ALIGN 1
DynamicString_Main64_1 BYTE "Hello World",0
.CODE
;; MST generated alignment code.
IFDEF MST_PROCALIGN
ALIGN MST_PROCALIGN
ENDIF
main PROC SYSCALL_ argc:QWORD,argv:QWORD
SUB RSP,16
PUSH RBX
PUSH R12
PUSH R13
PUSH R14
PUSH R15
MOV R15,[RBP]
MOV [RBP-16],R15
MOV R15,[RBP+8]
MOV [RBP-16+8],R15
SUB RBP,16
mov [rbp+16],rdi
mov [rbp+24],rsi
;; ***Code: (2) @C.printf{"Hello World"}
printf PROTO SYSCALL_:QWORD
PUSH RSI
PUSH RDI
MOV R12,RSI
MOV R13,RDI
PUSH RSP
PUSH QWORD PTR [RSP]
AND SPL,0F0h
LEA RDI,DynamicString_Main64_1
XOR RAX,RAX
CALL printf
ADD RSP,8
POP RSP
POP RDI
POP RSI
POP R15
POP R14
POP R13
POP R12
POP RBX
ret 16
main ENDP
END
Thanks!
In 64 bit JWASM was not properly Win64 ABI compliant, push / call is the correct method in Win32 but Win64 does not use this technique, it uses the first 4 registers then writes any further arguments to 64 bit aligned stack locations. UASM is properly Win64 ABI compliant and works correctly in 64 bit Windows. I do not use Linux so I am not familiar with how both work in any Linux distro.
UASM defaults for SYSVCALL on 64bits linux mac unix etc, so most of the stack is pre aligned.
For my test I still aint figured out why linker do not like the output produced by uasm wen using -fPIC and -shared, most of my code fails with seg fault wen linking uasm output with this new unix/linux GCC/clang thing's, but some sussed wen removed from linking time, some run's others fails for miss stack alignment prediction.
I'm trying to understand why there's a difference between JWasm and UASM (Linux 64 bit). In my example, JWasm produces code that works, and UASM produces code that seg faults before calling printf. I'm trying to understand why there's a difference, whether it's intentional. If it's unintentional, is UASM "correct" where JWasm was wrong? If so, we may have to make modifications to our compiler that produces the .ASM file. But if not, perhaps there's a bug in UASM.
It seems much more likely that we're doing something incorrect here. Or it could be that there's a command line argument that I could assert to get the "old" (working!) behavior. hutch-- mentioned a difference in how calls are done, but the two assemblers produce identical code around the call to printf. It's the setup and teardown of the traditional x86 stack frame that is present in the JWasm output and missing from the UASM output that appears to me to be the main difference.
I'm going to try attaching a ZIP file (seems to be the only allowed format?) with the source file, and the two listing files.
Arguments passed to the assembler are in both cases;
-elf64 -Zp8 -Sg -nologo -c -Sa -FlHelloWorld64.lst -FoHelloWorld64.o Main64.asm
(I had just renamed the listing files for the purpose of zipping them up together)
Quote from: tastewar on August 28, 2019, 10:51:48 PM
I'm trying to understand why there's a difference between JWasm and UASM (Linux 64 bit). In my example, JWasm produces code that works, and UASM produces code that seg faults before calling printf. I'm trying to understand why there's a difference, whether it's intentional. If it's unintentional, is UASM "correct" where JWasm was wrong? If so, we may have to make modifications to our compiler that produces the .ASM file. But if not, perhaps there's a bug in UASM.
It seems much more likely that we're doing something incorrect here. Or it could be that there's a command line argument that I could assert to get the "old" (working!) behavior. hutch-- mentioned a difference in how calls are done, but the two assemblers produce identical code around the call to printf. It's the setup and teardown of the traditional x86 stack frame that is present in the JWasm output and missing from the UASM output that appears to me to be the main difference.
I'm going to try attaching a ZIP file (seems to be the only allowed format?) with the source file, and the two listing files.
Arguments passed to the assembler are in both cases;
-elf64 -Zp8 -Sg -nologo -c -Sa -FlHelloWorld64.lst -FoHelloWorld64.o Main64.asm
(I had just renamed the listing files for the purpose of zipping them up together)
Hi tastewar,
the code differs only in the frame, the differences are shown in red .. there is no frame in UASM.
(https://i.imgur.com/YCIiSDA.png)
Indeed, that's what I've already pointed out. I'm trying to understand if it's intentional, given that UASM started from the same code base. And assuming for the moment that it is intentional, is there a flag we could assert on the command line to get the old (jwasm/masm) behavior?
Doing more searching on the web & forum, this thread seems relevant:
http://masm32.com/board/index.php?topic=6617.0
But I've tried OPTION FRAME:AUTO with no change in output.
What if you combine it with OPTION WIN64:11 (or OPTION WIN64:15) and/or also specify FRAME keyword after the PROC?
Probably this changes something, but the leave as truncated by pop rbp.
OPTION PROC:DEFAULT
;;;; Default Actually this ain't changing nothing because already the default with uasm 64bit mach unix linux
OPTION FRAME:AUTO
OPTION STACKBASE:RBP ; RSP or RBP are supported options for the stackbase, RBP for SYSV linux frame auto.
OPTION WIN64:7 ; 11-15 for RSP and 1-7 for RBP.
OPTION LANGUAGE:SYSTEMV ;;; this supposed to force call for entire file procedure, only if no one call already declared
;;;; DEFAULT END
;; ***************************************
;; *** @main argc,argv ***
;; *** Source: Main64.txt Line: 1 ***
;; ***************************************
.DATA
ALIGN 1
DynamicString_Main64_1 BYTE "Hello World",0
.CODE
;; MST generated alignment code.
IFDEF MST_PROCALIGN
ALIGN MST_PROCALIGN
ENDIF
main PROC SYSTEMV argc:QWORD,argv:QWORD ;; here not specifiing the call later can be change above or force the call including SYSTEMV
SUB RSP,16
PUSH RBX
PUSH R12
PUSH R13
PUSH R14
PUSH R15
MOV R15,[RBP]
MOV [RBP-16],R15
MOV R15,[RBP+8]
MOV [RBP-16+8],R15
SUB RBP,16
mov [rbp+16],rdi
mov [rbp+24],rsi
;; ***Code: (2) @C.printf{"Hello World"}
printf PROTO SYSTEMV :PTR, :VARARG ;; were force the correct call
PUSH RSI
PUSH RDI
MOV R12,RSI
MOV R13,RDI
PUSH RSP
PUSH QWORD PTR [RSP]
AND SPL,0F0h
LEA RDI,DynamicString_Main64_1
XOR RAX,RAX
CALL printf
ADD RSP,8
POP RSP
POP RDI
POP RSI
POP R15
POP R14
POP R13
POP R12
POP RBX
ADD RSP,16 ;; restore stack alignment
ret 16
main ENDP
END
UASM defaults to RSP stack frame under Linux (simply look and you will see).
So, you must force RBP stack frame with OPTION STACKBASE:RBP if you want an RBP stack frame that should work like in Jwasm
DON'T EVER USE OPTION FRAME: AUTO, it has nothing to do with frame stack pointers, it has to do with exception handling. (Or use it, if you love to use things you don't understand).
Has I said, (64bits), linux, unix, machos, defaults to SYSTEMV or the 64bits syscall wai. (cdecl or c call or syscall or fastcall) ignored in 64 bits (windows, unix, linux, machos).
UASM defaults to generic rsp thing, wen using other convention than systemv. The thing breaks with miss stack convention.
So this is the same as using systemv with proc proto thing.
OPTION PROC:DEFAULT
;; ***************************************
;; *** @main argc,argv ***
;; *** Source: Main64.txt Line: 1 ***
;; ***************************************
.DATA
ALIGN 1
DynamicString_Main64_1 BYTE "Hello World",0
.CODE
;; MST generated alignment code.
IFDEF MST_PROCALIGN
ALIGN MST_PROCALIGN
ENDIF
main PROC argc:QWORD,argv:QWORD ;; here not specifiing the call later can be change above or force the call including SYSTEMV
SUB RSP,16
PUSH RBX
PUSH R12
PUSH R13
PUSH R14
PUSH R15
MOV R15,[RBP]
MOV [RBP-16],R15
MOV R15,[RBP+8]
MOV [RBP-16+8],R15
SUB RBP,16
mov [rbp+16],rdi
mov [rbp+24],rsi
;; ***Code: (2) @C.printf{"Hello World"}
printf PROTO :PTR, :VARARG ;; were force the correct call
PUSH RSI
PUSH RDI
MOV R12,RSI
MOV R13,RDI
PUSH RSP
PUSH QWORD PTR [RSP]
AND SPL,0F0h
LEA RDI,DynamicString_Main64_1
XOR RAX,RAX
CALL printf
ADD RSP,8
POP RSP
POP RDI
POP RSI
POP R15
POP R14
POP R13
POP R12
POP RBX
ADD RSP,16 ;; restore stack alignment
ret 16
main ENDP
END
And with all this, o got it all my code cleaned-up, systemv de facto a standard for all 64bits unix distros.
I've tried all these suggestions, but I don't get the stack frame generation like we do with jwasm with any of them. Perhaps I've misunderstood someone's suggestion. If you have a modification of my source that does produce the equivalent listing as jwasm (included in my .ZIP file), please post it! Apologies if I've misunderstood, and someone has solved this. For instance, I tried both WIN64 OPTIONs that fearless suggested, but din't know exactly what was meant by the FRAME part of the suggestion. Everything I tried resulted in a syntax error.
I *do* appreciate the suggestions!
tastewar,
Find Agner Fog's data on calling conventions and see how a stack frame (if used) works in Linux. It is a bit different to Win64 and you need to understand the details but this will help you with the compiler you mentioned. You can soldier on with JWASM but its old and did not do 64 bit correctly.
Actually RBP based stackframe is the default, I thought not.
So this will build and run properly in UASM (but stack alignment was not done in this example).
;--- "hello world" for 64-bit Linux, using SYSCALL.
;--- assemble: UASM -elf64 -Fo=Lin64_1.o Lin64_1.asm
;--- link: gcc Lin64_1.o -o Lin64_1
stdout equ 1
SYS_WRITE equ 1
SYS_EXIT equ 60
.data
string db 10,"Hello, world!",10
.code
main proc
LOCAL myVar : dword
mov myVar, 12
mov edx, sizeof string
mov rsi, offset string
mov edi, stdout
mov eax, SYS_WRITE
syscall
mov eax, SYS_EXIT
syscall
main endp
end main
00000000004004e0 <main>:
4004e0: 55 push %rbp
4004e1: 48 8b ec mov %rsp,%rbp
4004e4: 48 83 ec 08 sub $0x8,%rsp
4004e8: c7 45 fc 0c 00 00 00 movl $0xc,-0x4(%rbp)
4004ef: ba 0f 00 00 00 mov $0xf,%edx
4004f4: 48 be 30 10 60 00 00 movabs $0x601030,%rsi
4004fb: 00 00 00
4004fe: bf 01 00 00 00 mov $0x1,%edi
400503: b8 01 00 00 00 mov $0x1,%eax
400508: 0f 05 syscall
40050a: b8 3c 00 00 00 mov $0x3c,%eax
40050f: 0f 05 syscall
or
00000000004004e0 <main>:
4004e0: 55 push rbp
4004e1: 48 8b ec mov rbp,rsp
4004e4: 48 83 ec 08 sub rsp,0x8
4004e8: c7 45 fc 0c 00 00 00 mov DWORD PTR [rbp-0x4],0xc
4004ef: ba 0f 00 00 00 mov edx,0xf
4004f4: 48 be 30 10 60 00 00 movabs rsi,0x601030
4004fb: 00 00 00
4004fe: bf 01 00 00 00 mov edi,0x1
400503: b8 01 00 00 00 mov eax,0x1
400508: 0f 05 syscall
40050a: b8 3c 00 00 00 mov eax,0x3c
40050f: 0f 05 syscall
Gonna test force removing the frame for the proc. And see if it removes the rbp thing.
Interesting the
OPTION FRAME:NOAUTO
OPTION STACKBASE:RSP
Error A2258: Missing FRAME in PROC, no unwind code will be generated
https://github.com/hjl-tools/x86-psABI/wiki/X86-psABI (https://github.com/hjl-tools/x86-psABI/wiki/X86-psABI)
reading and researching the code....
Quote
The conventional use of %rbp as a frame pointer for the stack frame may be avoided by using
The 128-byte area beyond the location pointed to by %rsp is considered to
be reserved and shall not be modified by signal or interrupt handlers.10 Therefore,
functions may use this area for temporary data that is not needed across function
calls. In particular, leaf functions may use this area for their entire stack frame,
rather than adjusting the stack pointer in the prologue and epilogue. This area is
known as the red zone.
has syscall instruction leaves in kernel, were kernel (The Linux kernel does not honor the red zone and therefore this
area is not allowed to be used by kernel code)
I also don't like frame with proc, but if not required, the best thing is calling the necessary code outside the main or not, depends programmer heaven, if has I with linux, we can directly use syscal instruction, and remove the extra stack save/push for the printf. Has with windows don't allow use the kernel directly.
@tastewar Dono wath version's You are using, so with uasm if some of the options fails with syntax error, remove it and retry.
More background, if anyone is still interested. We have a large project we are working on. One piece is a language interpreter with a plugin architecture that in Windows uses dll's, and in Linux will hopefully use .so files. Currently, we are able to build and run the interpreter in 64 bit Linux using jwasm. It works fine. I'm not sure how any of it is "wrong" since it's a fairly complex program, and it builds and runs, and interacts with the underlying Linux OS just fine.
We have a trivial sample plugin, written in C and built as a .so that the interpreter can load and call. However, we have a large number of such plugins that are written in our proprietary language, and that language has a compiler that originally produced MASM source as its output. On Linux, we now have it producing jwasm source. Since jwasm is dead, and is incapable of producing .so files, we would definitely like to make the switch to uasm! Since it's supposed to be compatible, we just swapped in the executable, and tried calling it instead. But clearly, it's not quite compatible, because given the same source, it isn't producing the same assembly. Specifically, the "traditional" stack frame has gone missing.
I'm attaching a slightly larger example. This one has the original (proprietary) source in the Main64.txt file, the "compiled" ASM in Main64.asm, and the jwasm produced listing file in HelloWorld64CFJW.lst and the uasm produced listing in HelloWorld64CFU.lst. If you run a diff between them, you will see that the major, functional difference is the lack of the stack frame. jwasm produces a function that includes push rbp; mov rbp,rsp; sub rsp,1200 to make space for the locals. I don't see an adjustment in rsp for local variables in the uasm listing.
I apologize if I'm missing something completely obvious/elementary here, but we thought this would be a drop-in, and it's clearly not. My hope is there is some simple command line switch we can assert to have uasm produce functions with a stack frame. Or change something in the asm our compiler is emitting to get the same result.
Anyone?
tastewar,
It seems that johnsa & habran, the two UAsm developers, are currently on holiday. They certainly will help you, even if that requires an entirely new option (but I don't think so). Few people here are familiar with Linux, so be patient, it's just a matter of a few days.
Thanks -- I am sticking around ;-)
You appear to have disregarded, but I have posted an example where an RBP stackframe is produced. UASM will not make a stackframe, unless it is necessary - that's why I included a LOCAL variable to force it.
You can keep trying to make a stackframe appear with FRAME: AUTO or FRAME:NOAUTO, but you will never see that miracle happen. I told you, FRAME is not for that.
I will not download and look at your code, it will not bring anything new in relationship with the stackframe problem and I can not spend time on lateral issues at the moment.
I apologize, AW, if I didn't reply directly to your post. I certainly didn't try anything with the FRAME attribute after reading it. It may be that I posted a response that spoke about trying it, but that was before I read your post, even if it might have been after you posted it. I understand you may not have time to help me, and I certainly don't expect you to expend more effort on my issue than you feel is appropriate for you.
The second example I posted, which you are certainly free to continue to ignore, included local variables, and did not use a FRAME attribute, but produced no stack frame. So there must be more to it than that. Either we are asserting something on the command line or in our code that is causing uasm to not produce the frame, or we are failing to assert something that could. It's a difference between jwasm and uasm, and I am trying to learn what (if anything) we can do to get the older behavior, which definitely works with the bulk of our code. Also, for your (and others') information, unlike the first, the second example is not intended to produce a stand-alone runnable executable.
The first example I had posted would seg fault when built with uasm and run fine built with jwasm. I have years of experience debugging in Windows, but am very new to systems level programming on Linux, which makes it just more of a mystery.
Thanks for your contribution to the discussion!
Cheers,
--Tom
@tastewar :biggrin:
I had a look at your code and could not believe that you were using stdcall in 64-bit :skrewy:
It does not exist stdcall for 64-bit, if you see it somewhere, it is aliased for the standard which is:
System V AMD64 ABI
OK, UASM is not aliasing or producing an error - here you are right! :eusa_clap:
Remove the stdcall and you will see the FRAME you want.
Thank you, AW! I had a feeling that if the right person looked at it, they'd see what was wrong!
Amusingly, removing STDCALL causes jwasm to yield an error (Main64.asm(79) : Error A2091: Language type must be specified) and it seems just as unhappy with SYSCALLV.
I think also that KradMoonRa was onto the problem, but I wasn't understanding.
I tried OPTION LANGUAGE:SYSTEMV which works for uasm, but jwasm doesn't like this either.
Can anyone think of a way for me to specify this that's compatible with both jwasm and uasm?
That is probably a JWASM bug that happens when there are parameters in the function. Use FASTCALL for JWASM and see if the end result is not nonsense. I have not tested.
Use UASM for 64-bit, forget JWASM
> Use UASM for 64-bit, forget JWASM
This is pretty much the action, JWASM was reasonably successful in 32 bit but Win64 uses a different ABI (Application Binary Interface) than the Intel ABI that was used in Win32 and JWASM did not do this correctly. You should NOT be using STDCALL push/call notation in 64 bit as the 64 bit system does not support it correctly.
I am familiar with this stuff in Win64 as I use the 64 bit version of MASM and had to construct the macros to do all of this. There is the first 4 stack locations that are termed "shadow space" which are filled from the 4 registers, RCX RDX R8 and R9 then any following arguments are written to the stack locations after the shadow space.
I have tested UASM with near identical code to 64 bit MASM and it performs correctly so it is properly using the Win64 ABI.
On Windows we still use MASM.
If you want to keep compatibility with JWasm during the transition:
ifdef __UASM__
echo ** the following options are for UAsm only **
else
echo ** the following options are for JWasm only **
endif
@tastewar, Yes with JWasm using other convention name like stdcall syscal fastcal, get's ignored and defaults to systemv. With UASM that don't happens, it gets an generic epilogue prologue wen other convention is written on procedure.
Has we probably gonna have another convention for 64bits the Intel regcall, the best thing to make UASM more JWasm friendly, or (better) fix the bug, is if in 64 bit unix* {(if callconv != systemv || callconv != regcall) calconv = systemv;}, hallways defaults to systemv if also declaring other (syscall, fastcall, stdcall).
And get some TODO work wen working with kernel code, because.
User-level applications use as integer registers for passing the sequence
%rdi, %rsi, %rdx, %rcx, %r8 and %r9. The kernel interface uses %rdi,
%rsi, %rdx, %r10, %r8 and %r9.
Better explained in section A.2 AMD64 Linux Kernel Conventions in https://github.com/hjl-tools/x86-psABI/wiki/x86-64-psABI-1.0.pdf (https://github.com/hjl-tools/x86-psABI/wiki/x86-64-psABI-1.0.pdf)
deleted
@nidud
True, it's by the programmer to. In some old kernel linux distributions, any code has capable of running regarding off the caller. Currently we need to be magician to get the code running on recent linux kernel.
@tastewar
Regarding wath I talk earlier. Applied changes in uasm to ignore syscall cdecl stdcall wen use64 segment procedures on 64bits unix linux machos and defaults to SYSTEMV
Tested and built your HelloWorld got the results for mov rbp, rsp.
This is my test development branch, what's implemented were can change later or not be implemented in the release uasm.
64bits uasm build with windows and ubuntu 18.04
https://github.com/eXOAMP/UASM/releases/download/eX_v2.50_master_alpha0/Uasm_master.7z (https://github.com/eXOAMP/UASM/releases/download/eX_v2.50_master_alpha0/Uasm_master.7z)
With this version can be find the regcall for 32bits and 64bits windows, unix, linux, machos. thiscall for 32bits windows, windows vectorcall for bout 64bits and 32bits (invoke needs work on the implementation).
SYSTEMV
REGCALL
VECTORCALL
THISCALL
Info about Intel regcall https://software.intel.com/en-us/cpp-compiler-developer-guide-and-reference-c-c-calling-conventions (https://software.intel.com/en-us/cpp-compiler-developer-guide-and-reference-c-c-calling-conventions)
Obviously, there are no restriction in the way one function can call another function in Assembly language, it should even be possible (but strangely is not) for Assemblers to have some way to make a custom calling convention using rules from a template available to users, unfortunately they don't (Something like IDA has).
I am not sure how ASMC deals in 64-bit with the traditional 32-bit calling conventions, but I don't think JWASM does it the same way.
In addition, I don't think JWASM supports the SystemV calling convention in any way, let alone aliasing traditional 32-bit calling conventions to SystemV or implementing it behind the scenes. If it does, Japheth never noticed that: :badgrin:
https://sourceforge.net/p/jwasm/feature-requests/27/
Quote from: AW on August 31, 2019, 10:00:28 AMit should even be possible (but strangely is not) for Assemblers to have some way to make a custom calling convention using rules from a template available to users
This is what prologue/epilogue macros do.
@jj2007,
I don't think so, although Microsoft used macros for prologue/epilogue in 32-bit MASM ages ago and, of course, Hutch managed to produce a working solution for the Windows 64-bit ABI. However, macros are cumbersome by their own nature. For example, I can't figure out a way to produce the vectorcall convention using macros.
But there is an opportunity for assemblers, like those hosted in this website, to fill in the blanks left behind by Masm and gain popularity doing so. This was actually what gave recognition to Jwasm years ago and lots of people still believe it is worthwhile to build using Jwasm despite it being dead and buried deep since long.
deleted
I was thinking about custom calling conventions providing like in IDA Pro.
http://darmawan-salihun.blogspot.com/2012/04/using-custom-function-calling.html
https://www.hex-rays.com/products/ida/support/idadoc/1361.shtml
In Win64 you can do anything you like as long as you understand what registers to preserve and what alignment is necessary if you make any system calls at all. Get it wrong and it just won't start and you get no analysis about why. With the number of available registers in Win64 I wonder why you would bother to use other than FASTCALL at all. The main reason why I use LOCAL variables is for higher level code, mainly API code and here you need the stack frame.
The risk with non standard calling methods is messing up the stack where the FASTCALL technique comfortably handle BYTE to QWORD in the same stack locations. JWASM did win32 reasonably well but it was not properly Win64 ABI compliant and is best left to the dustbin of history.
Thanks for the help! We are getting closer to some solutions.
The current challenge is that we are looking for a means of defining a proc that would use traditional stack based parameters. I understand, this is not "the" approved calling convention in x64, but we have scads of code that compiles today with jwasm on x64, which you are free to tell me is broken, but which works with our code when defining our internal functions as stdcall