UASM64 Library

[fearless]

https://github.com/mrfearless/UASM64-Library

UASM64 Library is a x64 port of the functions from the MASM32 Library that are included with the MASM32 SDK.

The functions ported to an x64 version in the UASM64 Library aim to match the parameters and features of the original x86 functions from the MASM32 Library where possible. In a few functions that may not be possible, and an alternative approach to achieve the same desired result may be used instead.

The names of the functions and parameter names in the UASM64 Library compared to the MASM32 Library have been changed to increase readability. Equates are provided in the UASM64.inc file to map to the new function names - which also helps when porting x86 projects to x64 ones.

Additionally, new functions have been added to the UASM64 Library to expand and compliment the existing functions.
UASM64 Library is targeted specifically for use with projects that use the UASM assembler (the x64 version), but likely other compilers and assemblers can utilize it as well.

All credit and thanks to all the original authors and code contributors of the functions in the MASM32 Library.

The UASM64 Library and source code are free to use for anyone, and anyone can contribute to the UASM64 Library project.

[Biterider]

Thank you fearless, for the great work you share with us. 

Biterider

[fearless]

Thanks.

Hopefully others will contribute or make suggestions, or even post code/functions here if they don't have a github account.

[fearless]

I made a google forms survey that should help determine the direction of the project and focus on what functions or function categories should be ported over first (based on user rankings of how often they are used in the MASM32 library), plus other useful questions. Feel free to take part, no email addresses are collected, results summary is shared so others can view them as well, and responses can be edited.

https://forms.gle/k6BBFFm71yiPgEVw7

[jj2007]

This looks like a great project, thanks a lot

Could you please give us a hint on the include files used?

[fearless]

Yes, sorry I should have included that information somewhere, but the include files I used where from the WinInc package. But now that I think of it, the only reason windows.inc is included in some functions is for the TRUE and FALSE. So I might just IFNDEF them in the UASM64.inc file to remove that dependency? and/or see if we cant adjust the others functions as well to keep it all self contained perhaps?

[jack]

@fearless
thank you 

[mabdelouahab]

 
But what about Linux users?

[stoo23]

Would be great to see the Results of the Survey, as and when there are enough to be relevant   

[fearless]

@mabdelouahab - Someone who more knowledge of Linux would need to take a look at the code to be able to suggest what system calls or system library functions could be used instead of the windows api calls in some of the functions. A IFDEF could then be placed in the code to allow for those functions to be used instead of the windows ones. I'm unsure if there is any calling convention issues that would need to be looked at as well? But apart from that some of the functions should be able to be assembled I think.

@stoo23 - so far only 2 responses, so not sure if there is any great demand for the UASM64 library.

I also noticed that the masm32 library has 4 functions that do not have a unicode/wide version:

- Cmpi
- szCmpi
- szTrim
- wordreplace

If anyone has time and wants to create those and also 64-ify then as well so that we have unicode versions of those x86 and x64 functions that would be great.

I'm also in the process of creating a UASM32 library to include any backport of new functions from the UASM64 library.

But I also have other projects on the go at the moment, so I will get round to updating the repo for the next batch of functions I have ported over: Array functions, Text functions and still to do but probably Memory functions as well.

[fearless]

Just pushed an update on the UASM64 Library repo: https://github.com/mrfearless/UASM64-Library/releases/tag/1.0.0.1

• removed reliance on include files
• removed unused equates
• added support for assembling UASM64 for Linux x64 (experimental)
• added Array functions (untested)
• added more CPU functions
• added String functions (Ansi and Unicode)
• added Text functions (Ansi and Unicode)
• added File functions (Ansi and Unicode)
• added Memory functions
• updated mapping of older MASM32 library function names to UASM64 function names
• added CPU constants for CPU_Features and other functions
• updated api text files for RadASM and WinASM IDEs
• updated source code documentation to allow for auto-gen of sphinx .rst files for readthedocs.
• updated rst index files to include a table of functions for each category
• added BuildForWindows.bat and BuildForLinux.bat

I stripped all the dependencies for includes down to just the absolute bare amount (uasm64.inc) and reduced everything down as much as possible to the most basic level to allow for win x64 or linux x64 using the inbuilt platform variable: IF platform EQ 1 (Window x64) or 3 (Linux x64)

I managed to get it to assemble for windows fine, but had a few issues with the linux assembly. Registers overwritten by invoke messages mainly. I managed to solve it by attempting a manual call using the calling convention for linux x64. That seemed to solve the assembly errors and the library was able to be packaged with ar.exe from MinGW.

I havent tested UASM64.a and I have no idea if it will actual work at all. I had to use extern refs to malloc and free from glibc library as memory mapping on linux has its own quirks, where it is better to use another memory allocator like the glibc one than to use the basic page mapping or to try and roll my own allocator.

I also took out the windows SysAlloc/SysFree functions used in the array functions and replaced them with a custom emulated version that uses a 64bit length for the BSTR instead of 32bit. Mainly to allow the linux version to use it as the BSTR used in the array function is just a wrapper around a basic memory allocation of +8 for length at start, and +4 for nulls at the end.

Someone who has linux experience might be able to test it, and to look into this to see if it needs any adjustment - its highly possible that I have overlooked some details in regards to register saving and restoring for the linux calls to actually work properly. Consider it experimental at the moment.

[jj2007]

These days I'm busy with other things, so I can't test it. My sincere compliments, fearless 

[Biterider]

Hi fearless
Thanks for this incredible useful work. 
I played a bit with it and it works beautifully.
I'm afraid I can't help you with the Linux procs.

I noticed 2 small cosmetic things. You mix spaces and tabs to align the code. This works well for default tabs (4 spaces), but misaligns the code for other settings.

You are using lines like 

Code Select Expand

IF @Platform EQ 1 ; Win x64
to make this easier to read, the 1 could be replaced by Platform_Win_x64, and

Code Select Expand

Platform_Win_x64 equ 1
somewhere.


The CRLF GitHub repository also works well. I never managed to get it to work properly 

Regards, Biterider

[fearless]

Thanks.

I usually try and remove the tabs and replace them with spaces, but when working with the files occasionally I forget, plus there is a fair few files now. I mostly do most editing in RadASM and even though I set it to use spaces instead of tabs, tabs still creep in somehow. I also use notepad++ for any vertical column editing including copy/pasting vertically as well as this speeds up a lot of the work instead of editing each line.

For the CRLF on github, i make sure to include the .gitattributes file with the following:

Code Select Expand

* text eol=crlf
*.exe binary
*.a binary
*.dll binary
*.zip binary
*.obj binary
*.res binary
*.lib binary
*.dlg binary
*.mnu binary
*.ttf binary
*.ico binary
*.bmp binary
*.png binary
*.jpg binary
*.rst linguist-documentation
*.bat linguist-language=Text
*.tpl linguist-language=Text
*.rap linguist-language=Text
*.bat linguist-language=Text
*.asm linguist-language=Assembly
*.inc linguist-language=Assembly
*.rc linguist-language=Text
*.hsl linguist-language=Text
makefile linguist-language=Text
conf.py linguist-language=Python
or similar. Thats the one from the UASM64 library, but other repos might have similar entries to help with that, and to designate some file extensions as assembler. Should maybe also add .txt and .md as 'Text' now that I think of it, but github probably defaults to that I think.

Yes the equate could be adjusted like that are even perhaps like this?:

Code Select Expand

Platform_Win_x64 TEXTEQU <@Platform EQ 1>
Platform_Linux_x64 TEXTEQU <@Platform EQ 3>

IF Platform_Win_x64

IF Platform_Linux_x64

Still more to do to port over other functions, plus still not set on the readability of the conversion functions yet, so havent looked at them properly.

And testing of some function groups still to do: array functions, text functions and more file function testing. And of course someone might have to look at testing the linux x64 library to see if it even works and how to implement some of the functions for it.

[mabdelouahab]

Hi fearless,thank you for your effort.

In a 64-bit Linux environment, we use the System V ABI. Some code modifications are needed to accommodate both Windows and Linux platforms.
Based on previous experience, I suggest a specific approach which I will outline.

This macro helped me a lot in achieving compatibility and organizing the code to make it easier for us to update it later.

Code Select Expand

WIN64    equ 1
ELF64  equ 3

__GetFirstSpacePos macro __@str:req
    local space_pos ,tab_pos
      space_pos INSTR <__@str>,< >
      tab_pos INSTR <__@str>,<    >
        IF space_pos
            IF tab_pos
                IF space_pos GT tab_pos
                    EXITM<tab_pos>
                ENDIF
            ENDIF
            EXITM<space_pos>
        ELSE
            EXITM<tab_pos>
        ENDIF
endm
.proc MACRO Proc__Name0,__Args:VARARG
    local space_pos ,tab_pos,procName,usesreg
    space_pos = __GetFirstSpacePos(Proc__Name0)
    IF space_pos
            procName SubStr <Proc__Name0>,1,space_pos
            usesreg SubStr <Proc__Name0>,space_pos
    ELSE
            procName SubStr <Proc__Name0>,1
            usesreg textequ <>
    ENDIF

    lastprocedureName EQU <procName>
    lastprocedureArgs__ =0
    IFB <__Args>
        .code
            align 16
        procName PROC usesreg
    ELSE
        IF @Platform EQ WIN64
            .code
                align 16
            procName PROC FRAME usesreg __Args
        ELSEIF @Platform EQ ELF64
            strArgs equ <>
            j___=1
            FOR __varg,<__Args>
                IF j___ GE 7
                    strArgs CATSTR strArgs,<,__varg>
                ELSE
                    strArgs CATSTR strArgs,<,@&__varg>
                    j___=j___+1
                ENDIF
                lastprocedureArgs__=lastprocedureArgs__+1
            ENDM
            .code
                align 16
            procName PROC usesreg strArgs   
            j___=1
            FOR __varg,<__Args>
                IF j___ LT 7
                    LOCAL __varg
                    @CatStr(<__@SubArg>,%j___) equ <Private_getdecvarname(__varg)>
                    j___=j___+1
                ENDIF
            ENDM
        ENDIF
    ENDIF
endm
.home macro
    IF @Platform EQ WIN64
        ;    Shadow space
    ELSEIF @Platform EQ ELF64
        n=0
        repeat  lastprocedureArgs__
            n=n+1
            IF n LT 7
        %    mov @CatStr(<__@SubArg>,%n), @&@CatStr(<__@SubArg>,%n)
            ENDIF
        endm
    ENDIF
endm
 
.endp macro
local space_pos ,tab_pos
        RET
    lastprocedureName ENDP
endm    Also,the organization of the code by separating the parts that are unlikely to be updated in the future from the parts that are expected to be reviewed and updated.
Example:
IF Platform_Win_x64
    CreateFileMappingA  PROTO hFile:QWORD, lpFileMappingAttributes:QWORD, flProtect:DWORD, dwMaximumSizeHigh:DWORD, dwMaximumSizeLow:DWORD, lpName:QWORD
    MapViewOfFile      PROTO hFileMappingObject:QWORD, dwDesiredAccess:DWORD, dwFileOffsetHigh:DWORD, dwFileOffsetLow:DWORD, dwNumberOfBytesToMap:QWORD
    _CreateFile macro lpName,qwMemorySize
        mov rax, qwMemorySize
        shr rax, 32d
        mov r14, eax
        mov rax, qwMemorySize
        mov r14, eax
        CreateFileMappingA( -1, 0, PAGE_READWRITE or SEC_COMMIT, r14, r15, lpName)
        EXITM<EAX>
    endm
    _MapViewOfFile macro hfile,qwMemorySize
        MapViewOfFile( hfile, FILE_MAP_WRITE, 0, 0, 0)
        EXITM<EAX>
    endm
   
...
IF Platform_Linux_x64
    _CreateFile macro lpName,qwMemorySize
              _open(lpName,O_RDWR or O_CREAT,0)
              EXITM<EAX>         
    endm
    _MapViewOfFile macro hfile,qwMemorySize
        mov r14, qwMemorySize
        add r14, 8 ; alloc 8 extra to store length of mapping
        _mmap(0, r14, PROT_READ or PROT_WRITE ,MAP_SHARED ,hfile, 0    )   
        EXITM<EAX>
    endm
...

The code will be written as follows:
   
.proc File_MapSharedMemory USES RBX ,lpszSharedMemoryName:QWORD, qwMemorySize:QWORD, lpqwMemoryMapHandle:QWORD, lpqwMemoryViewPointer:QWORD
    LOCAL hFile:QWORD
    LOCAL MMHandle:QWORD
    LOCAL MVPointer:QWORD
    LOCAL dwMemorySizeHI:DWORD
    LOCAL dwMemorySizeLO:DWORD
.home
    mov MMHandle, 0
    mov MVPointer, 0
    mov MMHandle, _CreateFile(lpszSharedMemoryName,qwMemorySize)
    .if MMHandle==0 || MMHandle==-1
        ...
    .else
        mov MVPointer,_MapViewOfFile(MMHandle,qwMemorySize)
        .if  MVPointer==0 || MVPointer==-1
          ...
        .else
...
.endp

If I had enough time, I would review the entire library. I will try to find some time if I can.
Thank you anyway.