UASM 2.37 Release

[johnsa]

1) Listing generation bug fixed (as per other thread with a binary 0 being left in the text output).

2) New 32bit code-generation features (implict DS segment override support and assumed size from register):

Code: [Select]


; OLD Style that worked with jwasm/uasm etc up to 2.36.
mov ds:[0C2100Dh],eax
mov ds:[0C2100Dh],ax
mov ds:[0C2100Dh],al
add ds:[0x1234],ax
add ds:[0x12341f25],eax
mov eax,ds:[0C2100Dh]
mov ax ,ds:[0C2100Dh]
mov al ,ds:[0C2100Dh]

; NEW Style for 2.37, implicit DS and size from register.
mov [0C2100Dh],eax
mov [0C2100Dh],ax
mov [0C2100Dh],al
add [0x1234],ax
add [0x12341f25],eax
mov eax,[0C2100Dh]
mov ax ,[0C2100Dh]
mov al ,[0C2100Dh]
imul eax,[12345]
add eax,[0x8c888880]


For 64bit code generation the new absolute immediate addressing (moffs64* from Intel Manuals) mode is now supported :
In addition fixes have been applied to allow un-sized immediate only addressing modes which jwasm/uasm through to 2.36 didn't support correctly.
So the below are now all valid options:

Code: [Select]


add eax,[0x0c888880]
imul rax,[1234]
sub eax,[0x12535]
sub rax,[0x14242054]
mov rax,[0C2100Dh]

mov rax,[SOME_ADDR]
mov rax,[10+(100*256)]

mov [0C2100Dh],rax
mov [0000788880888880h],al
mov rax,[0000788880888880h]
mov rax,7FF8807FF660h
mov eax,[00007FF6601D1010h]
mov ax,[00007FF6601D1010h]
mov al,[00007FF6601D1010h]
mov rax,[0C2100Dh]
mov eax,[0C2100Dh]
mov ax ,[0C2100Dh]
mov al ,[0C2100Dh]
mov [0C2100Dh],rax
mov [0C2100Dh],ax
mov [0C2100Dh],al
mov [0C2100Dh],rax
mov [0C2100Dh],ax
mov [0C2100Dh],al
mov [0000788880888880h],rax
mov [00007FF6601D1010h],al
mov [00007FF6601D1010h],ax
mov al ,[00007FF6601D1010h]
mov ax ,[00007FF6601D1010h]
mov eax,[00007FF6601D1010h]
mov rax,[00007FF8807FF660h]           
mov rax,7FF8807FF660h
mov qword ptr [0C2100Dh],rax
mov [0C2100Dh],eax
mov [0C2100Dh],eax
mov [00007FF6601D1010h],eax
mov QWORD PTR [00007FF8807FF660h],rax



[jj2007]

As usual, works like a charm for my big sources :t

[habran]

Just to add some more info:
For instructions MOV moffs64 only AL, AX, EAX and RAX are allowed, using other registers will cause invalid instruction operands error
and only MOV instruction is allowed, not ADD, SUB, IMUL...
so    MOV RCX,1122334455667788H  works

but   MOV RCX,[1122334455667788H] doesn't
here is a legal encoding:

Code: [Select]

    mov [00007FF6601D1010h],rax
    mov [00007FF8807FF660h],eax
    mov [00007FF8807FF660h],ax
    mov [00007FF8807FF660h],al
    mov rax,[00007FF6601D1010h]
    mov eax,[00007FF6601D1010h]
    mov ax,[00007FF6601D1010h]
    mov al,[00007FF6601D1010h]

INC and DEC are also working now but only 32 bit addresses are legal:

Code: [Select]

    inc qword ptr [0C2100Dh]
    inc dword ptr [0C2100Dh]
    inc word ptr [0C2100Dh]
    inc byte ptr [0C2100Dh]
    dec qword ptr [0C2100Dh]
    dec dword ptr [0C2100Dh]
    dec word ptr [0C2100Dh]
    dec byte ptr [0C2100Dh]


[LiaoMi]

Holla!

Code: [Select]

.code
start:
lea rcx, [rip+6]
lea rcx, [rip-6]


Error in encoding on the EIP RIP  :(

Are there any "assembler" tables to test the whole range of possible assembler variations?  :redface: Then you can compare the addressing with masm64  ..

[habran]

Are you kidding LiaoMi
ML64 doesn't know RIP ::)
However, 0x67 was already fixed in next release, now it is assembled:

Code: [Select]

    19:     lea rcx, [rip+6]
00007ff691621014 48 8D 0D 06 00 00 00             lea rcx, ptr [rip+0x6]


[LiaoMi]

Are you kidding LiaoMi
ML64 doesn't know RIP ::)
However, 0x67 was already fixed in next release, now it is assembled:

Code: [Select]

    19:     lea rcx, [rip+6]
00007ff691621014 48 8D 0D 06 00 00 00             lea rcx, ptr [rip+0x6]


Thanks!

I meant for other cases  Of course I read about it  :icon_exclaim: Here http://www.codegurus.be/Programming/riprelativeaddressing_en.htm#Mode64

Here for example there are such tests for disassembler:
N-version Disassembly Differential Testing of x86 Disassembler.pdf http://security.di.unimi.it/~gianz/pubs/issta10-nversion.pdf
source https://github.com/hlide/differential_testing_of_x86_disassemblers

Differential Testing of x86 Disassemblers

x86 Disassemblers translate a stream of machine code into a sequence of assembly instructions which can produce completely unreliable results due to bad decoding of an instruction. This project is about the correctness of the instruction decoder, the component of disassemblers that is responsible for the decoding of machine instructions using a n-version disassembly methodology based on differential analysis which is specific for Intel x86 architecture.

Given an arbitrary string of bytes, we use multiple (n−1) disassemblers to decode the instruction potentially starting with the first byte of the string, and then we compare the output of the various disassemblers to detect discrepancies. Any discrepancy is a symptom that the instruction decoder of at least one of the tested disassemblers is buggy.

The nth instruction decoder is a CPU-assisted instruction decoder which does not perform itself the decoding, but instead delegates the duty to the perfect instruction decoder implemented in the CPU. While it does not output a disassembly code, it can infer some information about the instruction the string encodes (e.g., whether the instruction is valid, the length of the instruction, and the type of operands) to check whether the output produced by the tested disassemblers is compliant with the format of the instruction.

The output of each disassemblers is normalized to allow a comparison between them and computing a coefficient of agreement, the rationale being to be more confident in the output with the highest agreement among disassemblers.

[LiaoMi]

Are there any "assembler" tables to test the whole range of possible assembler variations?  :redface: Then you can compare the addressing with masm64  ..

About what I said, the description was copied from the site. We can try to test uasm with a large testfile.asm 4.4 mb  :icon_eek:

Code: [Select]

x86 instruction generator

Here's something amusing. I spent the first half of the day writing a short Haskell program which generates x86 instructions in MASM syntax. The program generates all variants of the non-privileged instructions from the opcodes.chm file of the MASM32 package. This means that the instruction generator is not complete at all. FPU, MMX, SSE and other newer-than-x486 instructions are not covered. Nevertheless the generator already generates nearly 150,000 different x86 instructions.

When assembled with MASM32 the resulting file is more than 600 KB big. Trying to disassemble this thing with a few standard disassemblers turns out to be a problem. IDA fails to disassemble an instruction after maybe 5% of the executable and never manages to recover afterwards. Lots of manual help is necessary to convince IDA to go on. OllyDBG manages to disassemble that instruction but has huge gaps at many, many other points of the disassembly. The created file is an interesting test file for x86 disassemblers I'd say.

The Haskell program is just about 300 lines long. 280 of those lines are the definitions of  the instructions and what operands they can take. The generation of the instructions from the instruction definitions is just 20 lines and all but 8 lines are not even strictly necessary. I love Haskell's expressiveness.

Anyway, click here to see the Haskell source or click here to download the whole package including the Haskell program (source + EXE), the generated output of the Haskell program, a MASM32 source file that can be used to assemble the test file, and the test file EXE itself.
http://www.the-interweb.com/serendipity/exit.php?url_id=753&entry_id=115

Code: [Select]

-- Generates lots and lots of x86 instructions
--
-- Visit http://www.the-interweb.com :)

import Data.List

-- Defines the mnemonics and their possible operands
mnemonics = [
("jcxz", jcc_targets),    -- Start with the jcxz and other instructions which only support relative jumps and can not be used later
("jecxz", jcc_targets),
("loop", [[["start"]]]),
("loope", [[["start"]]]),
("loopne", [[["start"]]]),
("aaa", no_operands),
("aad", no_operands),
("aam", no_operands),
("aas", no_operands),
("adc", typical_binary),
("add", typical_binary),
("and", typical_binary),
("bound", [[r16, "someDword" : mem32], [r32, "someDword" : mem64]]),
("bsf", [[r16, r16], [r32, r32], [r16, mem16], [r32, mem32]]),
("bsr", [[r16, r16], [r32, r32], [r16, mem16], [r32, mem32]]),
("bsr", [[r16, r16], [r32, r32], [r16, mem16], [r32, mem32]]),
("bswap", [[r32]]),
("bt", [[mem16, imm8], [mem32, imm8], [r16, imm8], [r32, imm8], [mem16, r16], [mem32, r32]]),
("btc", [[mem16, imm8], [mem32, imm8], [r16, imm8], [r32, imm8], [mem16, r16], [mem32, r32]]),
("btr", [[mem16, imm8], [mem32, imm8], [r16, imm8], [r32, imm8], [mem16, r16], [mem32, r32]]),
("bts", [[mem16, imm8], [mem32, imm8], [r16, imm8], [r32, imm8], [mem16, r16], [mem32, r32]]),
("call", [[r32], [mem32], [["start"]]]),
("cbw", no_operands),
("cdq", no_operands),
("clc", no_operands),
("cld", no_operands),
("cli", no_operands),
("cmc", no_operands),
("cmp", typical_binary),
("cmpsb", no_operands),
("cmpsw", no_operands),
("cmpsd", no_operands),
("cmpxchg", [[r32, r32], [mem32, r32]]),
("cwd", no_operands),
("cwde", no_operands),
("daa", no_operands),
("das", no_operands),
("dec", rvalues),
("div", rvalues),
("enter", [[["2"], ["2"]]]),
("hlt", no_operands),
("idiv", rvalues),
("imul", imul),
("inc", rvalues),
("int", [[["3"]]]),
("ja", jcc_targets),
("jae", jcc_targets),
("jb", jcc_targets),
("jbe", jcc_targets),
("jc", jcc_targets),
("je", jcc_targets),
("jg", jcc_targets),
("jge", jcc_targets),
("jl", jcc_targets),
("jle", jcc_targets),
("jmp", jcc_targets),
("jna", jcc_targets),
("jnae", jcc_targets),
("jnb", jcc_targets),
("jnbe", jcc_targets),
("jnc", jcc_targets),
("jne", jcc_targets),
("jng", jcc_targets),
("jnge", jcc_targets),
("jnl", jcc_targets),
("jnle", jcc_targets),
("jno", jcc_targets),
("jnp", jcc_targets),
("jns", jcc_targets),
("jnz", jcc_targets),
("jo", jcc_targets),
("jp", jcc_targets),
("jpe", jcc_targets),
("jpo", jcc_targets),
("js", jcc_targets),
("jz", jcc_targets),
("lahf", no_operands),
("lds", [[r16, mem32]]),
("lea", [[r16, mem16]]),
("lea", [[r32, mem32]]),
("leave", no_operands),
("les", [[r16, mem32]]),
("lfs", [[r16, mem32]]),
("lgs", [[r16, mem32]]),
("lodsb", no_operands),
("lodsw", no_operands),
("lodsd", no_operands),
("lss", [[r16, mem32]]),
("mov", typical_binary),
("movsx", extend),
("movzx", extend),
("movsb", no_operands),
("movsw", no_operands),
("movsd", no_operands),
("mul", rvalues),
("neg", rvalues),
("nop", no_operands),
("not", rvalues),
("or", typical_binary),
("pop", [[segments_writable], [r16], [mem16], [r32], [mem32]]),
("popa", no_operands),
("popad", no_operands),
("push", [segments] : [[r16], [mem16], [imm16]] ++ [[r32], [mem32], [imm32]]),
("pusha", no_operands),
("pushad", no_operands),
("rcl", rotate),
("rcr", rotate),
("rep", [[["movsb"]], [["movsw"]], [["movsd"]], [["lodsb"]], [["lodsw"]], [["lodsd"]], [["stosb"]], [["stosw"]], [["stosd"]]]),
("repe", [[["cmpsb"]], [["cmpsw"]], [["cmpsd"]], [["scasb"]], [["scasw"]], [["scasd"]]]),
("repne", [[["cmpsb"]], [["cmpsw"]], [["cmpsd"]], [["scasb"]], [["scasw"]], [["scasd"]]]),
("retn", [[[""]], [["4"]]]),
("retf", [[[""]], [["4"]]]),
("rol", rotate),
("ror", rotate),
("sahf", no_operands),
("sal", rotate),
("sar", rotate),
("sbb", typical_binary),
("scasb", no_operands),
("scasw", no_operands),
("scasd", no_operands),
("seta", [[r8], [mem8]]),
("setae", [[r8], [mem8]]),
("setb", [[r8], [mem8]]),
("setbe", [[r8], [mem8]]),
("sete", [[r8], [mem8]]),
("setne", [[r8], [mem8]]),
("setl", [[r8], [mem8]]),
("setge", [[r8], [mem8]]),
("setle", [[r8], [mem8]]),
("setg", [[r8], [mem8]]),
("sets", [[r8], [mem8]]),
("setns", [[r8], [mem8]]),
("setc", [[r8], [mem8]]),
("setnc", [[r8], [mem8]]),
("seto", [[r8], [mem8]]),
("setno", [[r8], [mem8]]),
("setp", [[r8], [mem8]]),
("setnp", [[r8], [mem8]]),
("shl", rotate),
("shr", rotate),
("shld", shift_double),
("shrd", shift_double),
("stc", no_operands),
("std", no_operands),
("sti", no_operands),
("stosb", no_operands),
("stosw", no_operands),
("stosd", no_operands),
("sub", typical_binary),
("test", typical_binary),
("wait", no_operands),
("fwait", no_operands),
("xchg", [[r8, r8], [r16, r16], [r32, r32], [r8, mem8], [r16, mem16], [r32, mem32]]),
("xlatb", [[mem8]]),
("xlatb", no_operands),
("xor", typical_binary)
]

-- Register definitions
r8 = ["al", "bl", "cl", "dl", "ah", "bh", "ch", "dh"]
r16 = ["ax", "bx", "cx", "dx", "si", "di", "sp", "bp"]
r32 = ["eax", "ebx", "ecx", "edx", "esi", "edi", "esp", "ebp"]
segments = "cs" : "ds" : segments_writable
segments_writable = ["es", "fs", "gs", "ss"]

-- Definitions of all possible memory access operand trees
mem8 = ["byte ptr [someDword]"] ++ simple_ptr "byte" r32 ++ ptr_offset "byte" r32 ++ mult_reg "byte" r32 ++ two_regs "byte" r32 ++ mult_two_regs "byte" r32
mem16 = ["word ptr [someDword]"] ++ (simple_ptr "word" r32) ++ (ptr_offset "word" r32) ++ mult_reg "word" r32 ++ two_regs "word" r32 ++ mult_two_regs "word" r32
mem32 = ["dword ptr [someDword]"] ++ (simple_ptr "dword" r32) ++ (ptr_offset "dword" r32) ++ mult_reg "dword" r32 ++ two_regs "dword" r32 ++ mult_two_regs "dword" r32
mem64 = ["qword ptr [someDword]"] ++ (simple_ptr "qword" r32) ++ (ptr_offset "qword" r32) ++ mult_reg "qword" r32 ++ two_regs "qword" r32 ++ mult_two_regs "qword" r32

-- Definitions of a few sample immediates
imm8 = ["12h"]
imm16 = ["512h"]
imm32 = ["6237512h"]

-- Variant for instructions that use no operands
no_operands = [[]]

-- All variants of typical instructions with two operands
-- like ADD, SUB, AND, OR, XOR, ...
typical_binary = [
[r8, imm8], [r16, imm8], [r16, imm16], [r32, imm8], [r32, imm32],
[mem8, imm8], [mem16, imm8], [mem16, imm16], [mem32, imm8], [mem32, imm32],
[r8, r8], [r16, r16], [r32, r32],
[mem8, r8], [mem16, r16], [mem32, r32],
[r8, mem8], [r16, mem16], [r32, mem32]
]

-- Operand variants for conditional jumps
jcc_targets = [[["start"]]]

-- Operand variants for single-operand instructions that write
-- back to the operand
rvalues = [[r8], [mem8], [r16], [mem16], [r32], [mem32]]

-- Operand variants for the IMUL instruction
imul = [
[r8],
[r16],
[r32],
[mem8],
[mem16],
[mem32],
[r16, r16],
[r32, r32],
[r16, mem16],
[r32, mem32],
[r16, imm16],
[r32, imm32],
[r16, r16, imm16],
[r32, r32, imm32],
[r16, mem16, imm16],
[r32, mem32, imm32]
]

-- Operand variants for movsx and movzx
extend = [
[r16, r8],
[r16, mem8],
[r32, r8],
[r32, mem8],
[r32, r16],
[r32, mem16]
]

-- Operand variants for shifts and rotates
rotate = [
[r8, imm8],
[mem8, imm8],
[r16, imm8],
[mem16, imm8],
[r32, imm8],
[mem32, imm8],
[r8, ["cl"]],
[mem8, ["cl"]],
[r16, ["cl"]],
[mem16, ["cl"]],
[r32, ["cl"]],
[mem32, ["cl"]]
]

-- Operand variants for double precision rotates
shift_double = [
[r16, r16, "cl" : imm8],
[mem16, r16, "cl" : imm8],
[r32, r32, "cl" : imm8],
[mem32, r32, "cl" : imm8]
]

-- Flattens a nested list
flatten = foldr (++) []

-- Only memory access operands with at most 1 use of ESP are valid
fmbr = filter (\(x:y:[]) -> x /= "esp" || y /= "esp")

-- Create operands of the form "byte/word/dword ptr [value]"
simple_ptr size = map (\x -> size ++ " ptr [" ++ x ++ "]")

-- Create operands of the form "byte/word/dword ptr [register + 123456h]"
ptr_offset size = map (\x -> size ++ " ptr [" ++ x ++ " + 123456h]")

-- Create operands of the form "byte/word/dword ptr [4 * register + 123456h]"
mult_reg size regs = map (\x -> size ++ " ptr [4 * " ++ x ++ " + 123456h]") (regs \\ ["esp"])

-- Create operands of the form "byte/word/dword ptr [register + register]"
two_regs size regs = map (\(x:y:[]) -> size ++ " ptr [" ++ x ++ " + " ++ y ++ "]") (fmbr $ zip_lists [regs, regs])

-- Create operands of the form "byte/word/dword ptr [4 * register + register]"
mult_two_regs size regs = map (\(x:y:[]) -> size ++ " ptr [4 * " ++ x ++ " + " ++ y ++ "]") (fmbr $ zip_lists [regs \\ ["esp"], regs])

-- Takes a list of the form [["eax", "ebx", ...], ["esi", "edi", ...]] and turns it into
-- [["eax", "esi"], ["eax, "edi"], ["eax", ...], ["ebx", "esi"], ["ebx", "edi"], ["ebx", "..."]]
zip_lists :: [[a]] -> [[a]]
zip_lists x | length x <= 1 = transpose x
zip_lists (x:xs) = [a : b | a <- x, b <- zip_lists(xs)]

-- Converts a list of operands an turns it into a comma-separated string
to_operand_string :: [String] -> String
to_operand_string = concat . intersperse ", "

-- Creates the operand strings for a list of given variants
create_operand_strings :: [[String]] -> [String]
create_operand_strings = map to_operand_string . zip_lists

-- Generates all instruction strings for one variant of an instruction
generate_variant :: String -> [[String]] -> [String]
generate_variant mnemonic =  map (\x -> concat [mnemonic, " ", x]) . create_operand_strings

-- Generates all instruction strings for one mnemonic
gen_instruction_strings :: (String, [[[String]]]) -> [[String]]
gen_instruction_strings (mnemonic, variants) = map (\x -> generate_variant mnemonic x) variants

-- Main function; used to print the generated results
main = putStrLn $ unlines $ flatten $ flatten $ map gen_instruction_strings mnemonics

[habran]

The source has got a lot of errors with the size, like :
mov ax, dword ptr
mov eax qword ptr
bound bx, someDword
bound bx, dword ptr [someDword]
it is annoying that they put that as source that can be assembled
so, my question is: does it suppose to show if assembler would throw errors or assembler should assemble it?