The MASM Forum

Microsoft 64 bit MASM => MASM64 SDK => Topic started by: jj2007 on September 05, 2023, 08:05:25 PM

Title: Should we improve the Masm64 SDK?
Post by: jj2007 on September 05, 2023, 08:05:25 PM
Following a comment from NoCforMe (https://masm32.com/board/index.php?topic=11213.0), I had a look at an old friend, StdOut:

StdOut proc lpszText:DWORD

    LOCAL hOutPut  :DWORD
    LOCAL bWritten :DWORD
    LOCAL sl      :DWORD

    invoke GetStdHandle,STD_OUTPUT_HANDLE
    mov hOutPut, eax

    invoke StrLen,lpszText
    mov sl, eax

    invoke WriteFile,hOutPut,lpszText,sl,ADDR bWritten,NULL

    mov eax, bWritten
    ret

StdOut endp

So I felt tempted to polish it a little bit, and here are two new versions:

include \masm32\include\masm32rt.inc ; purest Masm32 SDK code
include CodeSize.inc
.code
StdOut_s:
StdOut proc lpszText:DWORD

    LOCAL hOutPut  :DWORD
    LOCAL bWritten :DWORD
    LOCAL sl      :DWORD

    invoke GetStdHandle,STD_OUTPUT_HANDLE
    mov hOutPut, eax

    invoke StrLen,lpszText
    mov sl, eax

    invoke WriteFile,hOutPut,lpszText,sl,ADDR bWritten,NULL

    mov eax, bWritten
    ret

StdOut endp
StdOut_endp:
StdOutNew_s:
StdOutNew proc lpszText:DWORD
    LOCAL bWritten :DWORD
    push NULL
    lea eax, bWritten
    push eax
    push rv(StrLen, lpszText)
    push lpszText
    push rv(GetStdHandle, STD_OUTPUT_HANDLE)
    call WriteFile ; invoke WriteFile,hOutPut,lpszText,sl,ADDR bWritten,NULL
    ; mov eax, bWritten ; masmlib.chm: "There is no return value" (not sure if it has ever been used by an insider)
    ret
StdOutNew endp
StdOutNew_endp:

StdOutExtreme_s:
StdOutExtreme proc lpszText:DWORD
    push NULL
    push esp ; we assume that WriteFile writes to lpOverlapped when it's finished
    push rv(StrLen, lpszText)
    push lpszText
    push rv(GetStdHandle, STD_OUTPUT_HANDLE)
    call WriteFile ; invoke WriteFile,hOutPut,lpszText,sl,ADDR bWritten,NULL
    ret
StdOutExtreme endp
StdOutExtreme_endp:

start:
  cls
  invoke StdOut, chr$("Hello World StdOut", 13, 10)   ; print "Hello World" without using the print macro
  invoke StdOutNew, chr$("Hello World StdOutNew", 13, 10)   ; same but with modified library routine
  invoke StdOutExtreme, chr$("Hello World StdOutExtreme", 13, 10)   ; same but with dirty tricks
  CodeSize StdOut
  CodeSize StdOutNew
  CodeSize StdOutExtreme
  exit

end start

Output:
Hello World StdOut
Hello World StdOutNew
Hello World StdOutExtreme
56 bytes for StdOut
43 bytes for StdOutNew
35 bytes for StdOutExtreme

StdOutNew is 23% shorter, and certainly not slower.
StdOutExtreme is 37% shorter, not slower but it's a hack (it will always work, though).

Now the question: if we examine the Masm64 SDK, which has been developed by Hutch in a hurry, and find things that could be improved, should we change it, yes or no?

Some will argue, of course, that Hutch starts rotating in his grave if we touch his baby. I doubt that because Hutch had no problem with input from others, and he was always a fan of elegant code.

What's your opinion?
Title: Re: Should we improve the Masm64 SDK?
Post by: NoCforMe on September 06, 2023, 04:10:56 AM
Heh; funny that when I used StdOut() in a console app (32-bit), I had no idea that it was a homegrown MASM32 creation, only that it was a hell of a lot easier than using raw WriteFile() or WriteConsole(). (I thought it was a Win32 thing.) So good going there.

Regarding your proposed change, I don't know if I have standing here, since I'm not a 64-bit user, but my vote would be sure, go ahead: just make sure you test it, at least test it more than I usually do. ("Hey, it assembles and links: ship it!")
Title: Re: Should we improve the Masm64 SDK?
Post by: jj2007 on September 06, 2023, 05:08:19 AM
Quote from: NoCforMe on September 06, 2023, 04:10:56 AMmy vote would be sure, go ahead: just make sure you test it

For testing, 33 bytes instead of 56:

OPTION PROLOGUE:NONE
OPTION EPILOGUE:NONE
StdOutExtreme proc lpszText:DWORD
    push NULL ; we assume that WriteFile writes to bWritten only when
    push esp ; it's finished, so lpOverlapped can be misused here
    push rv(StrLen, [esp+4+8]) ; lpszText
    push [esp+4+12] ; lpszText
    push rv(GetStdHandle, STD_OUTPUT_HANDLE)
    call WriteFile ; invoke WriteFile,hOutPut,lpszText,sl,ADDR bWritten,NULL
    retn 4
StdOutExtreme endp
OPTION PROLOGUE:PrologueDef
OPTION EPILOGUE:EpilogueDef
:smiley:
Title: Re: Should we improve the Masm64 SDK?
Post by: zedd151 on September 06, 2023, 05:16:55 AM
I would hold off until we get any information back from stoo23, regarding hutch's Masm64 SDK files. I know that there are numerous fixes scattered here and there in the forum, but possibly hutch had them all in one place for inclusion in the next beta or even release version of the SDK.

Other members also have customized fixes, or additions, that they have added to their own copy of the SDK. Would take some effort getting everything together (assuming the other members would like to contribute to this effort).

Just my two cents worth on the issue.
Title: Re: Should we improve the Masm64 SDK?
Post by: daydreamer on September 06, 2023, 06:44:44 AM
I would prefer we continue improve masm64 sdk or if it's suppose not be touched, might turn to use and improve alternative 64 bit package?
But that feels like hutch work is invain if we don't use it/ improve it

Title: Re: Should we improve the Masm64 SDK?
Post by: NoCforMe on September 06, 2023, 07:41:53 AM
Can I make a suggestion here? I think what's needed, unfortunately, is a little bureaucratic process to make sure that any changes like this are tested thoroughly, even if it's something as trivial as this example. That way we'll avoid gnashing of teeth and rending of garments by users who find bugs.

My suggestion is to make sure any changes are tested by someone other than whoever made the change. (Like how one should never try to proofread their own writing.) Then the change can be signed off on and published.

Again, I have little standing on this issue since I don't use any of the 64-bit stuff and am not likely to anytime soon. But I still think this is a good suggestion. (Let's keep it simple, though. No need for excess red tape around here!)
Title: Re: Should we improve the Masm64 SDK?
Post by: Caché GB on September 06, 2023, 09:54:08 AM
A word to the wize.

If you are going to build a MASM 64 bit sdk with the hopes of makeing it
one size fits all you are going to end up playing whack a mole

This is the why Microsoft did not include ml's built in preprocessor
macros like invoke, .if .else, .repeat .until, etc in ml64.

Also to do so you will cripple your code's runtime to a crawl, may as well use 32 bit.

Here is a 64 bit not so extreme version of StdOut.

Remember RAM is cheap, time, not so much.

include \masm64\include64\masm64rt.inc

option prologue:none  ;; <- Undo MASM64 SDK's STACKFRAME macro in masm64rt.inc
option epilogue:none

StdOut proc ;; lpszText:ptr byte  <- no rbp basesd stack so leave off

            add  rsp, -38h

            mov  [rsp+40h], rcx    ; using caller's shadow space

        invoke  GetStdHandle, STD_OUTPUT_HANDLE

            mov  rcx, [rsp+40h]
            sub  rcx, 1
        @SO00A:
            add  rcx, 1
            cmp  byte ptr[rcx], 0
            jnz  @SO00A
            sub  rcx, [rsp+40h]

        invoke  WriteFile, rax, [rsp+40h], rcx, &[rsp+30h] , null  ;; [rsp+30h] = qWritten:qword
            mov  rax, [rsp+30h]

            add  rsp, 38h
            ret

StdOut endp

Test at our own risk.
Title: Re: Should we improve the Masm64 SDK?
Post by: Caché GB on September 06, 2023, 07:55:30 PM
I made StdOut more extreme

option prologue:none
option epilogue:none

StdOut proc ;; lpszText:ptr byte  <- no rbp basesd stack so leave off
 
            add  rsp, -38h

            mov  [rsp+40h], rcx             ; using caller's shadow space

            mov  ecx, STD_OUTPUT_HANDLE
           call  GetStdHandle

            mov   r8, [rsp+40h]             ; inline StrLen
            sub   r8, 1
        @SO00A:
            add   r8, 1
            cmp   byte ptr[r8], 0
            jnz   @SO00A
            sub   r8, [rsp+40h]

            and  qword ptr[rsp+20h], 0
            lea   r9, [rsp+30h]             ; [rsp+30h] = qWritten:qword
            mov  rdx, [rsp+40h]
            mov  rcx, rax
           call  WriteFile

            mov  rax, [rsp+30h]

            add  rsp, 38h
            ret

StdOut endp

If you wish to bench press this against another version.

        counter  equ  100000000

            mov  r14, counter
         @Loop1:

            lea  rcx, MyString
           call  StdOut_Old

            dec  r14
            jnz  @Loop1


            mov  r14, counter
         @Loop2:

            lea  rcx, MyString
           call  StdOut_New

            dec  r14
            jnz  @Loop2

Replace the API call WriteFile with WriteFileEmpty in the two StdOut functions

WriteFileEmpty proc

            ret

WriteFileEmpty endp

Test at our own risk.
Title: Re: Should we improve the Masm64 SDK?
Post by: jj2007 on September 06, 2023, 08:59:51 PM
Quote from: zedd151 on September 06, 2023, 05:16:55 AMI would hold off until we get any information back from stoo23, regarding hutch's Masm64 SDK files. I know that there are numerous fixes scattered here and there in the forum, but possibly hutch had them all in one place for inclusion in the next beta or even release version of the SDK.

Right, we are not in a hurry. I doubt that new stuff will pop up from Hutch' files, but we can certainly wait.

I am bit worried about too much parallel work. The success of the Masm32 SDK was based on a) quality and b) no competition. I know this sounds odd, as competition is usually good, but here we have a very small niche that survived only because members of this forum, Hutch in the first place, worked together instead of competing. I wish this could happen with the Masm64 SDK, too.

P.S., just for fun (source & exe attached, pure Masm32 SDK code):
OPTION PROLOGUE:NONE
OPTION EPILOGUE:NONE
StdOutExtreme proc lpszText:DWORD
  pop eax ; ret addr
  pop ecx ; string to print
  push eax ; ret addr
  push NULL ; we assume that WriteFile writes to bWritten only when
  push esp ; it's finished, so lpOverlapped can be misused here
  push rv(szLen, ecx) ; lpszText
  push ecx  ; lpszText
  push rv(GetStdHandle, STD_OUTPUT_HANDLE)
  call WriteFile ; invoke WriteFile,hOutPut,lpszText,sl,ADDR bWritten,NULL
  retn
StdOutExtreme endp
OPTION PROLOGUE:PrologueDef
OPTION EPILOGUE:EpilogueDef

Result: 50% less size...
Hello World StdOut
Hello World StdOutExtreme
56 bytes for StdOut
28 bytes for StdOutExtreme
Title: Re: Should we improve the Masm64 SDK?
Post by: daydreamer on September 07, 2023, 01:39:22 PM
Improve masm64 sdk,need to be done with good example code thats helpful like Iczelion's example code

Masm32 SDK has lots of example code,would port those to 64bit be good idea?
Multithreading examples, would be useful when you can access 64GB in 64bit mode
Or make examples in an area where you only know?
Title: Re: Should we improve the Masm64 SDK?
Post by: jj2007 on September 07, 2023, 07:30:52 PM
Quote from: daydreamer on September 07, 2023, 01:39:22 PMImprove masm64 sdk,need to be done with good example code

Check \Masm64\Examples, there are over 100 *.asm files
Title: Re: Should we improve the Masm64 SDK?
Post by: jj2007 on September 08, 2023, 07:46:23 AM
In the Lab (https://masm32.com/board/index.php?topic=11235.0) you'll find an example how to speed up the frequently used library function szLen by a factor 4 :biggrin:
Title: Re: Should we improve the Masm64 SDK?
Post by: Caché GB on September 09, 2023, 01:35:47 PM
Quote from: daydreamer on September 07, 2023, 01:39:22 PMImprove masm64 sdk,need to be done with good example code thats helpful like Iczelion's example code

Hi daydreamer

A friend of mine, a c# coder, was curious about MASM. To cut a long story short I
ported Iczelion's tutorial 10.1 to both C++ and MASM 64 bit for him to checkout.

I am dropping them here for your, and any one elses, pleasure. They are Visual Sudio projects. The MASM
version does not require the MASM64 SDK as it was made for a friend of mine and is cut from my homegrown SDK.

Iczelion_x64_10_1 = MASM x64 version
Iczelion_x64_10_2 = C++ version

if you need to ask any questions, go right ahead.

Disclaimer:
1) I am not a technical writer.
2) My PA has not proofread this.
3) There may be errors.

Enjoy.
Title: Re: Should we improve the Masm64 SDK?
Post by: Vortex on September 23, 2023, 08:36:32 PM
Hi Caché,

QuoteThis is the why Microsoft did not include ml's built in preprocessor
macros like invoke, .if .else, .repeat .until, etc in ml64.

The reason of removing those high level language elements was to discourage people from using their own assembler. M$ is rather promoting the C\C++\C# compiler set. The decision is based on commercial reasons.
Title: Re: Should we improve the Masm64 SDK?
Post by: Caché GB on September 24, 2023, 10:42:59 AM
Hi Vortex

Thanks for the correction. Can we at least agree to disagree that if Microsoft did include ml's
built in preprocessor macros, ml64 would be more like a mini compiler than an assembler. Yes, no?

Should we improve the Masm64 SDK? No. We should rebuild it. Refactor it. Make it second to none.
Title: Re: Should we improve the Masm64 SDK?
Post by: Vortex on September 24, 2023, 07:51:13 PM
Hi Caché,

Thanks for your reply. While an assembler and a compiler are different tools, your description is nice. ml.exe with a powerful macro engine is similar to a mini compiler.

We have to respect Hutch's legacy. Some parts of the Masm64 package can be rebuilt. We should keep alive Hutch's projects, that will be our mission.
Title: Re: Should we improve the Masm64 SDK?
Post by: zedd151 on September 24, 2023, 11:47:49 PM
Quote from: Vortex on September 24, 2023, 07:51:13 PMWe have to respect Hutch's legacy. Some parts of the Masm64 package can be rebuilt. We should keep alive Hutch's projects, that will be our mission.
Hi Vortex, that is or should be the main objective. Some folks here keep implying that the SDK should be made compatible with Uasm. That should not be the case. Hutch never wanted that, and he loathed just the suggestion of making either of the SDK's compatible with the Watcom based assemblers. This goes back years, that he has felt this way. All one needs to do is search "Watcom" and use "hutch--" in the Member field of advanced search here on the forum.

The same goes for the "open source", or as hutch put it, "open sauce", crowd here, keep wanting the 64 bit SDK to be placed on github, or sourceforge, or otherwise be put under the GPL licensing. Hutch had purposely written the licenses for the SDK's so that it prevents that from ever happening. That is my read on that part.

The include files need a bit of work, and the macros in the SDK as well. There are some leftovers also that need to be fixed, from the time when Masm64 was originally installed inside of the Masm32 directory, there are some batch files and sources in the examples folder that still reference the Masm32 directory path, for example.

zedd
Title: Re: Should we improve the Masm64 SDK?
Post by: jj2007 on September 25, 2023, 12:29:29 AM
Quote from: zedd151 on September 24, 2023, 11:47:49 PMSome folks here keep implying that the SDK should be made compatible with Uasm.

Indeed, being able to use an assembler that is 5 times as fast might be useful for those who have real projects.

QuoteThe include files need a bit of work, and the macros in the SDK as well. There are some leftovers also that need to be fixed

If you start working right now, maybe we'll see some results for Christmas?
Title: Re: Should we improve the Masm64 SDK?
Post by: HSE on September 25, 2023, 12:47:09 AM
Quote from: jj2007 on September 25, 2023, 12:29:29 AMthat is 5 times as fast might be useful for those who have real projects.

Real projects are build with modules and libraries. Anyway, velocity is perhaps most irrelevant assembler feature.
Title: Re: Should we improve the Masm64 SDK?
Post by: jj2007 on September 25, 2023, 01:57:13 AM
Quote from: HSE on September 25, 2023, 12:47:09 AM
Quote from: jj2007 on September 25, 2023, 12:29:29 AMthat is 5 times as fast might be useful for those who have real projects.

Real projects are build with modules and libraries. Anyway, velocity is perhaps most irrelevant assembler feature.

My experience is different. From building a 20 lines source (which obviously uses "modules and libraries"):

Quote*** Assemble using mlv15  ***
 Assembling: tmp_file.asm
*** MasmBasic version 23.09.2023 ***
## MasmBasic GUI build ##
## creating 1 controls ##

*** Link  IdaDlg.obj rsrc.res  using polink, sub WINDOWS ***
*** IdaDlg.exe: 61952 bytes ***

*** build all took 3520 ms ***
« OK »

Quote*** Assemble using UAsm64  ***
*** MasmBasic version 23.09.2023 ***
## MasmBasic GUI build ##
## creating 1 controls ##
Tmp_File.asm: 20 lines, 3 passes, 129 ms, 0 warnings, 0 errors

*** Link  IdaDlg.obj rsrc.res  using polink, sub WINDOWS ***
*** IdaDlg.exe: 61440 bytes ***

*** build all took 347 ms ***
--------------- OK ---------------
Title: Re: Should we improve the Masm64 SDK?
Post by: daydreamer on September 25, 2023, 04:03:05 AM
Thanks cache gb

Hope this doesnt end in old discussion assemble vs compile
You parse text and output .exe with both
I prefer macro assembler,cpp compiler auto align data breaks my simd code

Jochen and hector,i like uasm,but those millis timings vary much depending on fast ssd,slower mechanical drive,usb drive and older computer= older slower hd

Title: Re: Should we improve the Masm64 SDK?
Post by: Caché GB on September 25, 2023, 04:45:54 AM

daydreamer
I programme assembly using MASM in Visual Studio. To start a MASM project I need to add:
Project -> Build Customizations and click checkbox "masm(.targets, .props)"
So I'am pure MASM. I do not like compilers, and I know nothing about Watcom.

Hutch's legacy will always be respected by me.

I say we should first concentrate on an Include File Base like the one in Windows Kits (10.0.14393.0).
Once the Include File Base has a good grounding, work can begin on the runtime dll and stactic lib.

So all a MASM coder would have to do:

include windows.inc      ; shoot a'n skoot

If in MASM the coder wanted to goof around with some DirectX D2D

include windows.inc
include d2d1_2.inc       ; add Windows Core -> now do D2D stuff

Later, Frameworks can be developed around the Include File Base and the SDK's runtime and stactic libs.

include windows.inc
include d3d11_1.inc      ; add Windows Core -> now do D3D stuff
include ObjAsm.inc       ; add ObjAsm Framework. (By Biterider @ http://objasm.x10host.com/ )

And if the coder wants to be real cool

include windows.inc
include wincodec.inc     ; add Windows Core -> WIC
include MasmBasic.inc    ; add MasmBasic Framework. (By jj2007 @ https://masm32.com/board/index.php?topic=94.0 )


Are we going to do this? I,am in, who else is in.

I think I'll start tomorrow.
Title: Re: Should we improve the Masm64 SDK?
Post by: HSE on September 25, 2023, 05:01:12 AM
Quote from: Caché GB on September 25, 2023, 04:45:54 AMI know nothing about Watcom.

Is some effort to ignore Japheth work.

Indeed, talk is about JWasm family, wich is far more than rustic Watcom assembler.
Title: Re: Should we improve the Masm64 SDK?
Post by: Caché GB on September 25, 2023, 05:16:12 AM
O.K. HSE, you talk about JWasm if you want. I am only interested in MASM.
So are you going to help or not?

Anyone else going to put some effort in?
Title: Re: Should we improve the Masm64 SDK?
Post by: HSE on September 25, 2023, 07:48:19 AM
Quote from: Caché GB on September 25, 2023, 05:16:12 AMI am only interested in MASM.
So are you going to help or not?

:biggrin: Some macros are waiting test and eventually correction/improvements, because Hutch was too busy with videos, printing 3D, hardware, bike, drones, politics, etc

minimum problem with invoke macro (https://masm32.com/board/index.php?topic=9277.msg102004#msg102004)

Invoke macro reading prototypes (https://masm32.com/board/index.php?topic=9346.msg102547#msg102547)

Also Correction in str$ macro (https://masm32.com/board/index.php?topic=9683.msg106059#msg106059)
Title: Re: Should we improve the Masm64 SDK?
Post by: Caché GB on September 25, 2023, 09:25:30 AM
Hi HSE

The Macro Engine could be developed as a component of the SDK in parallel with the Include File Base.

include windows.inc
include SDK64MacroEngine.inc  ; add SDK64 Macro Engine

We would need only a small number of Core Macros to help in the constuction of the Include File Base.

So as it stands, the Core components

1. Include File Base
2. Macro Engine
3. Runtime / stactic
4. Frameworks
Title: Re: Should we improve the Masm64 SDK?
Post by: LordAdef on January 21, 2025, 03:10:21 AM
hi guys,

I didn't know where to write this, so feel free to delete my post if it's out of place.


I am back after a long winter, so I am still catching up. 

please forgive me if the questions are obvious:

#1:. the 64 sdk is a work in progress, as I read. Is there a plan for someone take over the task Hutch has started?

#2: I use Uasm. I read somewhere the 64 doesn't work with Uasm (I am still on 32 bits, planning to make the jump). Is it still true?

thanks in advance
Alex
Title: Re: Should we improve the Masm64 SDK?
Post by: jj2007 on January 21, 2025, 03:14:32 AM
Quote from: LordAdef on January 21, 2025, 03:10:21 AM64 doesn't work with Uasm

Unfortunately, that's true. There are several competing approaches around: some use Masm64, some use UAsm, and some roll their own (https://masm32.com/board/index.php?topic=9266.0). My advice: stick with 32-bit code :bgrin:
Title: Re: Should we improve the Masm64 SDK?
Post by: LordAdef on January 21, 2025, 03:59:55 AM
Quote from: jj2007 on January 21, 2025, 03:14:32 AM
Quote from: LordAdef on January 21, 2025, 03:10:21 AM64 doesn't work with Uasm

Unfortunately, that's true. There are several competing approaches around: some use Masm64, some use UAsm, and some roll their own (https://masm32.com/board/index.php?topic=9266.0). My advice: stick with 32-bit code :bgrin:

Nice, Johen!
I will check it out.
Concerning question #1, it would important that the work Hutch was doing should be carry on, right? 
Title: Re: Should we improve the Masm64 SDK?
Post by: HSE on January 21, 2025, 12:33:09 PM
Hi Alex!

Masm64 SDK probably is more than enough to begin to learn 64 bit assembly.

I use it mostly to build little test, but you can see in the forum LookingFor, a real tool following Hutch's style.

For complex projects ObjAsm, wich must be assembled with UASM, is the option. To use OOP is just optional, and is a complete package in active development.

Anyway, probably JJ is correct about to stick to 32 bit for your current projects. Is better to approach 64 bit without the pressure to make things to work wright now, I think  :thumbsup:

HSE