Author Topic: LLVM Macro Assembler - LLVM ML - Masm x64  (Read 1188 times)

LiaoMi

  • Member
  • ****
  • Posts: 711
LLVM Macro Assembler - LLVM ML - Masm x64
« on: January 22, 2020, 09:52:29 PM »
Hi,

llvm will probably support microsoft macro assembler, maybe someone wants to discuss this project .. Clang as an x64 assembler - Masm x64 https://github.com/llvm/llvm-project/issues/99

Code: [Select]
Hi all,

I'm proposing to add MASM support to LLVM's assembler capabilities, which
should nearly complete LLVM's support for cross-platform Windows
compilation.

Goal: Match the functionality of Microsoft's ml.exe and ml64.exe.

I'm currently trying to implement this as a set of extensions to the
assembler; it will correctly handle assembly files containing both GNU and
MASM syntax.

The features would be added to AsmParser and controlled via MCAsmInfo, and
would be available through both llvm-mc and clang. Both tools would remain
able to parse GNU-syntax assembler, but when passed the appropriate flags,
would also handle MASM. My first thought is to trigger MASM support
whenever we're targeting Windows, but we could also make it a discrete
function controlled by a different command-line flag.

We will also define a new driver (llvm-ml) that matches the command-line
interface of ml.exe and/or ml64.exe. This will likely be similar to
clang-cl and llvm-lib in building on top of a previous driver: either
llvm-mc or clang.

Known obstacles:
1. Support for "ifdef <register>": already completed, with a new
tryParseRegister method added to all TargetAsmParsers.
2. Syntax variations: MASM uses infix notation for many directives, and
does not use "."-prefixes on its directives.
3. Macro functions: MASM includes macro functions, which can emit
parameters and not just full instructions. (Probably tricky.)
4. Richer macro language in general: some MASM files rely heavily on
text-substitution for named symbols, as well as resolution of fields in
STRUCTs.

3 & 4 might be best handled by adding a preprocessor stage, but the syntax
is not similar to the C preprocessor... I'm going to try to augmenting
existing macro support first.

Any obstacles I've missed, or critiques of the approaches, would be very
welcome!

Thanks,
- Eric Astor

RFC: MASM support https://lists.llvm.org/pipermail/llvm-dev/2019-November/137112.html

Code: [Select]
Hi all,

I'm working on a project that uses clang-cl & lld-link to build for
Windows, along with some tools out of the Windows SDK... but we're
currently pre-building some pieces of MASM assembly code using Microsoft's
ml.exe & ml64.exe. Unfortunately, it's not all inline assembly, which clang
can already handle, and Microsoft's file-level directives are a bit unusual.

I plan to work on getting llvm-mc to compile (relatively simple) MASM files
when targeting a Windows x86-based platform, with goal of matching the
output of ml.exe and ml64.exe. I've already drafted a proof-of-concept
patch that lets llvm-mc handle MASM's variants of conditional assembly
macros (including the idiomatic use of "ifdef rax" to check if a build is
targeting x86-64)... but macro functions & structs are of course looking a
bit harder.

A few questions:

1. Should all of the changes be locked behind an equivalent to clang's
-fms-compatibility flag, or would it be good if some subset of the
functionality were shared? [e.g., should .ifdef rax be a valid way to check
if the rax register exists?]

2. Is there anyone around who would be willing to answer questions
regarding the intended architecture of llvm-mc and the AsmParser classes?
I'd like to make sure my proposals fit well into the design... and I'm
starting to have trouble finding where these extensions should go. (Also,
I've had some trouble getting used to the recursive-descent parser
conventions being used. For example, how should one handle "try parsing
this identifier as a register, and if that fails, check if it's defined as
a symbol" while not emitting Errors from the first attempt?)

Thanks,
- Eric
llvm-mc & Microsoft's MASM https://lists.llvm.org/pipermail/llvm-dev/2019-November/136879.html

Code: [Select]
Hi all,

Continuing work on llvm-ml (a MASM assembler)... and my latest obstacle is
in enabling MASM's convention that (unless specified) all memory location
references should be RIP-relative. Without it, we emit the wrong
instructions for "call", "jmp", etc., and anything we build fails at the
linking stage.

My best attempt at this so far is a small patch to X86AsmParser.cpp - just
taking any Intel expression with no specified base register and switching
it to use RIP - and this works alright. There's at least one exception: it
breaks the "jcc" instructions, at least "jcc <label>". The issue seems to
be that the "jcc" family exclusively takes a relative offset, never an
absolute reference... so adding a base register causes the operand not to
match. ("jcc" is always RIP-relative anyway.)

I'm not very familiar with the operand-matching logic, and am still pretty
new to LLVM as a whole. Are there more X86 instructions this will interact
badly with? Any thoughts on how this could be handled better?

If this is mostly a valid approach, might there be a way to change the
operand type of "jcc" to accept offset(base) operands, as long as base ==
X86::RIP, then ignore the RIP bit?

Thanks,
- Eric
[llvm-dev] MASM & RIP-relative addressing https://lists.llvm.org/pipermail/llvm-dev/2020-January/138460.html

jj2007

  • Member
  • *****
  • Posts: 10855
  • Assembler is fun ;-)
    • MasmBasic
Re: LLVM Macro Assembler - LLVM ML - Masm x64
« Reply #1 on: January 22, 2020, 10:32:51 PM »
Wishful thinking :cool:
Quote
Known obstacles:
..
3. Macro functions: MASM includes macro functions, which can emit
parameters and not just full instructions. (Probably tricky.)

xanatose

  • Member
  • ***
  • Posts: 400
Re: LLVM Macro Assembler - LLVM ML - Masm x64
« Reply #2 on: March 12, 2020, 05:12:03 AM »
My guess his best option is to look at the code of JWASM
The macros of MASM are very powerful, thus will be difficult to implement from scratch.

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 7800
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: LLVM Macro Assembler - LLVM ML - Masm x64
« Reply #3 on: March 12, 2020, 01:46:43 PM »
Sounds like pipe dreams. I understand John and nidud tweaking the old JWASM to get better results but my view is that a new assembler should not try and ape an old one. The form of MASM dates to just before 1990, TASM about the same yet technology has advanced a very long way since that type of architecture. A new MACRO assembler should produce its own syntax and pre-processor and be free of old assumptions.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :skrewy:

EPAstor

  • Regular Member
  • *
  • Posts: 4
Re: LLVM Macro Assembler - LLVM ML - Masm x64
« Reply #4 on: August 31, 2020, 08:02:35 AM »
Hi all... this is actually my work, so I guess I should chime in. It's absolutely true - there's no reason for a new assembler to ape an old one, right? Except... sometimes, you don't want to rewrite all the old code in a new assembler, but you do want a new feature. In this case? Building projects for Windows that include MASM code - but doing it from Linux. With software that's under a fully free license.

LLVM can already cross-compile very complex C++ applications for Windows - but some of these projects include assembly files. And most of those, when targeting Windows, use MASM.

I absolutely agree that matching MASM perfectly is going to be really, really hard.  The worst bit: due to compliance with the LLVM license, I need to do this without referencing JWASM's implementation; the Open Watcom license isn't compatible. (And no, we can't just use JWASM - my employer, along with many other projects, doesn't use software under the Open Watcom license due to its termination clauses.)

I don't expect to get to 100% compatibility any time soon... or even 90%. I'd estimate that after the next round of patches goes public, I'll be at maybe 90% for everything except macros... and 0% for macros (other than basic TEXTEQU stuff). The next step is to try getting the very basics of macros working. And yep, "Very tricky" is definitely an understatement. The only reason I think I have a hope of pulling it off is through leveraging work already done for macro processing throughout the LLVM project.

Once I've gotten macros working... acceptably... my plan is to come back to this forum and get some more input. If anyone wants to discuss this sooner, let me know and we can start some more detailed conversations.

tl;dr: Yeah, it might be a pipe dream... but it's a reasonably well-researched pipe dream, with a legitimate reason to exist, and someone working on it with employer sponsorship. Let's see if it's possible.

mineiro

  • Member
  • ****
  • Posts: 690
Re: LLVM Macro Assembler - LLVM ML - Masm x64
« Reply #5 on: August 31, 2020, 08:30:38 AM »
I was thinking years ago in modular macros(modules). In linux this can be done easy, in windows a bit more work.
In linux we can create files in memory, so, a modular macro can be a "macro"(assembled/linked) that can be executed by assembler.
Yes, if user write bad code macro will be a bad macro, but this is the difference between free source code from public domain (freedon), the risk.
But we are dealing with strings, just strings, not more than strings, in the end we have a raw assembler with syntax choice. Years ago I was not able to label an address like accents "coração:", imagine languages that we don't know. Today, I can create but I can't  ..., well, I need translate that to... . The first error from begginers persons that like to understand programming languages.
Congrats.
I try all, I choose all, I prefer ideals instead of ideology.
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 7800
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: LLVM Macro Assembler - LLVM ML - Masm x64
« Reply #6 on: September 12, 2020, 03:01:25 PM »
There is something here that I simply don't understand, the GNU assembler (GAS) is a really good tool when used in Intel mode yet I keep seeing the Linux crew trying to use a copy of a proprietary software from the Windows environment, MASM, JWASM, UASM etc which seem to be incompatible with the purist GPL ideology.

If you want purist GPL code, start with a purist GPL assembler, gas is a GAS !
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :skrewy:

jj2007

  • Member
  • *****
  • Posts: 10855
  • Assembler is fun ;-)
    • MasmBasic
Re: LLVM Macro Assembler - LLVM ML - Masm x64
« Reply #7 on: September 12, 2020, 06:31:10 PM »
Does GAS have macros?

Vortex

  • Member
  • *****
  • Posts: 2457
Re: LLVM Macro Assembler - LLVM ML - Masm x64
« Reply #8 on: September 12, 2020, 06:55:45 PM »

Vortex

  • Member
  • *****
  • Posts: 2457
Re: LLVM Macro Assembler - LLVM ML - Masm x64
« Reply #9 on: September 19, 2020, 07:01:20 AM »
Hi Hutch,

Without macros, life with assemblers like Gas is not easy. This is why those Linux coders would prefer macro assemblers.

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 7800
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: LLVM Macro Assembler - LLVM ML - Masm x64
« Reply #10 on: September 19, 2020, 09:19:04 AM »
 :biggrin:

Erol,

With GAS you can tell the men from the boys, its certainly not a soft high level assembler but anyone who is serious about writing true low level assembler will do OK. The macros look a bit primitive but with enough practice they can be civilised enough to use.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :skrewy: