The MASM Forum

64 bit assembler => 64 Bit Assembler => Topic started by: HSE on April 28, 2021, 11:57:55 PM

Title: Notes about ML64 and h2incX includes and libraries
Post by: HSE on April 28, 2021, 11:57:55 PM
Hi All !

Building Hutch's examples using the Biterider's includes work well (and slow), with some little modifications :biggrin:.

This allow to make all functions Prototyped. And that is interesting to test an invoke macro that understand some kind of PROTO, wich could be necesary for some third party libraries.


Must be replaced ("_code" in this case):
[/list]
Code: [Select]
      DEBUGHOOKINFO struct
          idThread DWORD ?
          idThreadInstaller DWORD ?
          lParam LPARAM ?
          wParam WPARAM ?
          _code SDWORD ?
        DEBUGHOOKINFO ends
in winuser.inc
 
NMHDR.code is used, then must be corrected.


Code: [Select]
    CONTEXT struct
      ···
    CONTEXT ends
 ;   _CONTEXT typedef CONTEXT ; anulado
    PCONTEXT typedef ptr CONTEXT

   
Code: [Select]
           struct
              DUMMYRECORDNAME_??45 record friendlyNameFromEdid_??45:1, friendlyNameForced_??45:1, edidIdsValid_??45:1, reserved_??45:29
              ;DUMMYRECORDNAME_??45 <>
            ends

And an special problem with a 128 bit record (in winnt.inc):
Code: [Select]
      struct HeaderX64
          HeaderX64_??35 record Depth_??35:16, Sequence_??35:48;
          HeaderX64_??35_ record Reserved_??35:4, NextEntry_??35:60
          ;HeaderX64_??35 <>;anulado
        ends

Code: [Select]
  _crt_va_arg macro ap, t
      exitm <((sizeof(t) > sizeof(__int64) or (sizeof(t) and (sizeof(t) - 1)) != 0) ? * * (t * *) ((ap + sizeof(__int64)) - sizeof(__int64)) : * (t *) ((ap + sizeof(__int64)) - sizeof(__int64)))>
    endm
replaced because doesn't matter in assembly
Code: [Select]
      exitm <t ptr ((ap + sizeof(__int64)) - sizeof(__int64))>
in vadefs.inc
Code: [Select]
  IEnumUnknown_Next_Proxy proto WIN_STD_CALL_CONV :ptr IEnumUnknown, :ULONG, :ptr ptr IUnknown, :ptr ULONG
  IEnumUnknown_Next_Stub proto WIN_STD_CALL_CONV :ptr IEnumUnknown, :ULONG, :ptr ptr IUnknown, :ptr ULONG
  IEnumString_Next_Proxy proto WIN_STD_CALL_CONV :ptr IEnumString, :ULONG, :ptr LPOLESTR, :ptr ULONG
  IEnumString_Next_Stub proto WIN_STD_CALL_CONV :ptr IEnumString, :ULONG, :ptr LPOLESTR, :ptr ULONG
  ISequentialStream_Read_Proxy proto WIN_STD_CALL_CONV :ptr ISequentialStream, :ptr, :ULONG, :ptr ULONG
  ISequentialStream_Read_Stub proto WIN_STD_CALL_CONV :ptr ISequentialStream, :ptr byte, :ULONG, :ptr ULONG
  ISequentialStream_Write_Proxy proto WIN_STD_CALL_CONV :ptr ISequentialStream, :ptr, :ULONG, :ptr ULONG
  ISequentialStream_Write_Stub proto WIN_STD_CALL_CONV :ptr ISequentialStream, :ptr byte, :ULONG, :ptr ULONG
  IStream_Seek_Proxy proto WIN_STD_CALL_CONV :ptr IStream, :LARGE_INTEGER, :DWORD, :ptr ULARGE_INTEGER
  IStream_Seek_Stub proto WIN_STD_CALL_CONV :ptr IStream, :LARGE_INTEGER, :DWORD, :ptr ULARGE_INTEGER
  IStream_CopyTo_Proxy proto WIN_STD_CALL_CONV :ptr IStream, :ptr IStream, :ULARGE_INTEGER, :ptr ULARGE_INTEGER, :ptr ULARGE_INTEGER
  IStream_CopyTo_Stub proto WIN_STD_CALL_CONV :ptr IStream, :ptr IStream, :ULARGE_INTEGER, :ptr ULARGE_INTEGER, :ptr ULARGE_INTEGER

Polink can't find the libraries. I don't know why there is an error if commenting them is Ok.
Clean solution is replace by _WIN64, but at this stage I declared it.
Code: [Select]
ifndef _WIN64                                         ;"type(near)" returns in 16-bit 0FF02h,
    if type(near) eq 0ff08h   ;64-bit enabled?          ;in 32-bit 0FF04h and in 64-bit 0FF08h.
      _WIN64 equ 1
      TARGET_BITNESS = 64
    elseif type(near) eq 0ff04h
      TARGET_BITNESS = 32
    endif
  endif

Still testing examples, but look good.

Regards, HSE.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: Biterider on April 29, 2021, 05:46:06 AM
Hi HSE
Thanks for sharing your findings.
Did you create some kind of test platform where you found this issues? Can you share it?
I mainly tested the code with UASM. It may be possible that ML has some incompatibilities and performance issues, but we can work them out.

Point 1: does not seem to be a problem for UASM
Point 2: the WIN header files define it that way. There are cases where the underscore prepended name are referenced.
Point 3: it is not a repetition. First it is defined and then it referenced. Maybe there are better ways to do it, but keep in mind that the translation is a single pass process
Point 4: known problem. C macros are not translated since there is not simple way to do it without a full-featured C compiler. The macro is left in its original form as reference
Point 5: I don't know
Point 6: as you probably know, it is designed the other way around, but if you want to use the include files in this way, it should be fine.

h2incX is a complex program that deserves more attention.
I'll look at it in the coming days.
Please keep posting all your findings.
At the beginning of the application code I wrote some notes about features and bugs.

Biterider

Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: HSE on April 29, 2021, 09:04:32 AM
Hi Biterider!

I mainly tested the code with UASM. It may be possible that ML has some incompatibilities and performance issues, but we can work them out.

Yes. ML64 have differences but for sure is posible to improve compatibility. I think there is not enough critical mass for 5 totally differents SDK.

> Point 1: does not seem to be a problem for UASM
No.

>Point 2: the WIN header files define it that way. There are cases where the underscore prepended name are referenced.
Yes. Headers are readed by compilers.

>Point 3: it is not a repetition. First it is defined and then it referenced. Maybe there are better ways to do it, but keep in mind that the translation is a single pass process
 :biggrin: Sorry, I was thinking in a repetition, ML64 tell me that.

>Point 4: known problem. C macros are not translated since there is not simple way to do it without a full-featured C compiler. The macro is left in its original form as reference
Only one showed a problem in test. But remain more C macros with conditionals.

>Point 5: I don't know
It's a mistery

>Point 6: as you probably know, it is designed the other way around, but if you want to use the include files in this way, it should be fine.
Yes. A lot of posible ways.

h2incX is a complex program that deserves more attention.
I'll look at it in the coming days.
Until now, some few manual modifications look enough to begin the analysis.

Attached reference Hutch' example and modified includes.
HSE
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: LiaoMi on April 29, 2021, 07:59:37 PM
Hi Biterider,

there is one very interesting project in which we can read how it should work in theory.

Quote
Point 4: known problem. C macros are not translated since there is not simple way to do it without a full-featured C compiler. The macro is left in its original form as reference

Rejuvenating C++ Programs through Demacrofication - https://stroustrup.com/icsm-2012-demacro.pdf
Source Code - https://github.com/hiraditya/cpp2cxx

Code: [Select]
#include "header1.h"

#define A 100

#define _c(i) i
#define _a(i) i*i

#define CRYPTOPP_X86_ASM_AVAILABLE
#define FXY(X,Y,Z) ((X) + (Y))
#define fun() FXY(1,200,3)

int i = fun();
int super(int i)
{

#define INSIDE_FUN_SUPER(X,Y) X+100 + j

  //variable 'j' referenced in macro INSIDE_FUN_SUPER
  //the transformation has to be carefully placed after the declaration of 'int j'
  int j = 10;

  return INSIDE_FUN_SUPER(10000, 4000);
}
int l = FXY((i+i),super(10),100);



#define proc(X,Y) {\
  X = FXY(X,Y,0);\
  }
#define MSB(x) (((x) >> 24) & 0xffU)  /* most  significant byte */

#include<iostream>

int main()
{
  int a1 = A;
  std::cout<<"\na1 = "<<a1;
 


#define INSIDE_FUN(X) X+100

  int infun = INSIDE_FUN(105)+a1;

  int p = _c(100);
  int q = _a(100);

  std::cout<<"\np = "<<p;
  std::cout<<"\nq = "<<q;

  std::cout<<"\nl = "<<l;
  std::cout<<"\ninfun = "<<infun;
  std::cout<<"\nMSB(167816141) = "<<MSB(167816141);

  int lval = 0;
  proc(lval, 2);

  std::cout<<"\nlval = "<<lval<<"\n";
  return 0; 
}

Demacrofication for the macro
Code: [Select]
#include "header1.h"


/** Demacrofication for the macro depth0 with unique identifier USE_depth0_examplecpp_3_9*/
constexpr auto depth0 = 0;

/** Demacrofication for the macro a with unique identifier USE_a_examplecpp_4_9*/
constexpr auto a = 100;

/** Demacrofication for the macro b with unique identifier USE_b_examplecpp_5_9*/
constexpr auto b = 200;


/** Demacrofication for the macro _c with unique identifier USE__c_examplecpp_7_9*/
template <class _T1>
auto _c(_T1 i) -> decltype(i)
{
  return i;
}

/** Demacrofication for the macro _a with unique identifier USE__a_examplecpp_8_9*/
template <class _T1>
auto _a(_T1 i) -> decltype(i*i)
{
  return i*i;
}

#define CRYPTOPP_X86_ASM_AVAILABLE

/** Demacrofication for the macro FXY with unique identifier USE_FXY_examplecpp_11_9*/
template <class _T1, class _T2, class _T3>
auto FXY(_T1 X, _T2 Y, _T3 Z) -> decltype(((X) + (Y)))
{
  return ((X) + (Y));
}

/** Demacrofication for the macro fun with unique identifier USE_fun_examplecpp_12_9*/

auto fun() -> decltype(FXY(1,200,3))
{
  return FXY(1,200,3);
}

int i = fun();
int super(int i)
{


  //variable 'j' referenced in macro INSIDE_FUN_SUPER
  //the transformation has to be carefully placed after the declaration of 'int j'
  int j = 10;


/** Demacrofication for the macro INSIDE_FUN_SUPER with unique identifier USE_INSIDE_FUN_SUPER_examplecpp_18_9*/
auto INSIDE_FUN_SUPER = [&j](decltype(10000) X, decltype( 4000) Y) { return X+100 + j; };
  return INSIDE_FUN_SUPER(10000, 4000);
}
int l = FXY((i+i),super(10),100);

int aPtr[] = {1,2,3,4,5,6,7,8,9,10};
int cPtr[] = {1,2,3,4,5,6,7,8,9,10};



/** Demacrofication for the macro proc with unique identifier USE_proc_examplecpp_32_9*/
template <class _T1, class _T2>
void proc(_T1 && X, _T2 && Y)
{
  FXY(X,Y,0);
  }


/** Demacrofication for the macro MSB with unique identifier USE_MSB_examplecpp_35_9*/
template <class _T1>
auto MSB(_T1 x) -> decltype((((x) >> 24) & 0xffU)  /* most  significant byte */)
{
  return (((x) >> 24) & 0xffU)  /* most  significant byte */;
}


/** Demacrofication for the macro SSB with unique identifier USE_SSB_examplecpp_36_9*/
template <class _T1>
auto SSB(_T1 x) -> decltype((((x) >> 16) & 0xffU)  /* second in significance */)
{
  return (((x) >> 16) & 0xffU)  /* second in significance */;
}


/** Demacrofication for the macro TSB with unique identifier USE_TSB_examplecpp_37_9*/
template <class _T1>
auto TSB(_T1 x) -> decltype((((x) >>  8) & 0xffU)  /* third  in significance */)
{
  return (((x) >>  8) & 0xffU)  /* third  in significance */;
}


/** Demacrofication for the macro LSB with unique identifier USE_LSB_examplecpp_38_9*/
template <class _T1>
auto LSB(_T1 x) -> decltype((((x)      ) & 0xffU)  /* least significant byte */)
{
  return (((x)      ) & 0xffU)  /* least significant byte */;
}


#include<iostream>

int main()
{
  int a1 = a;
  int b1 = b;
  int c1 = 'c';
 
  //std::cout<<MSB(102400)<<"\n";
  //std::cout<<LSB(102400)<<"\n";
  //std::cout<<TSB(102400)<<"\n";
  //std::cout<<SSB(102400)<<"\n";
 
  int p = _c(100);
  int q = _a(100);

/** Demacrofication for the macro INSIDE_FUN with unique identifier USE_INSIDE_FUN_examplecpp_55_9*/
auto INSIDE_FUN = [](decltype(105) X) { return X+100; };
  int infun = INSIDE_FUN(105)+a1+b1+c1;
  //std::cout<<p<<"\t"<<q<<"\n"; 
  return 0; 
}

Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: LiaoMi on April 29, 2021, 08:44:13 PM
Hi Biterider!

I mainly tested the code with UASM. It may be possible that ML has some incompatibilities and performance issues, but we can work them out.

Yes. ML64 have differences but for sure is posible to improve compatibility. I think there is not enough critical mass for 5 totally differents SDK.

> Point 1: does not seem to be a problem for UASM
No.

>Point 2: the WIN header files define it that way. There are cases where the underscore prepended name are referenced.
Yes. Headers are readed by compilers.

>Point 3: it is not a repetition. First it is defined and then it referenced. Maybe there are better ways to do it, but keep in mind that the translation is a single pass process
 :biggrin: Sorry, I was thinking in a repetition, ML64 tell me that.

>Point 4: known problem. C macros are not translated since there is not simple way to do it without a full-featured C compiler. The macro is left in its original form as reference
Only one showed a problem in test. But remain more C macros with conditionals.

>Point 5: I don't know
It's a mistery

>Point 6: as you probably know, it is designed the other way around, but if you want to use the include files in this way, it should be fine.
Yes. A lot of posible ways.

h2incX is a complex program that deserves more attention.
I'll look at it in the coming days.
Until now, some few manual modifications look enough to begin the analysis.

Attached reference Hutch' example and modified includes.
HSE

Hi HSE,

you should take into account that Hutch does not use function prototypes, even if they are taken into account, they often do not coincide with, how the functions are called within a macro, means that the prototypes of the translator are different.  :icon_idea:

Quote
 
5 - I don't found what library require this:

IEnumString_Next_Proxy proto WIN_STD_CALL_CONV :ptr IEnumString, :ULONG, :ptr LPOLESTR, :ptr ULONG
Code: [Select]
//+-------------------------------------------------------------------------
//
//  Microsoft Windows
//  Copyright (C) Microsoft Corporation, 1994 - 1996.
//
//  File:       call_as.c
//
//  Contents:   [call_as] wrapper functions for COMMON\types.
//+-------------------------------------------------------------------------
//
//  Function:   IEnumString_Next_Proxy
//
//  Synopsis:   Client-side [call_as] wrapper function for
//              IEnumString::Next.  This wrapper function handles the
//              case where pceltFetched is NULL.
//
//  Notes:      If pceltFetched != 0, then the number of elements
//              fetched will be returned in *pceltFetched.  If an error
//              occurs, then *pceltFetched is set to zero.
//
//--------------------------------------------------------------------------

Windows Apps Win32 API Component Object Model (COM) Objidl.h IEnumString interface IEnumString::Next method
https://docs.microsoft.com/en-us/windows/win32/api/objidl/nf-objidl-ienumstring-next
https://docs.microsoft.com/en-us/windows/win32/midl/call-as

Remarks
The ability to map a function that cannot be called remotely to a remote function is particularly helpful in interfaces that have numerous parameter types that cannot be transmitted across the network. Rather than using many [represent_as] and [transmit_as] types, you can combine all the conversions using [call_as] routines. You supply the two [call_as] routines (client side and server side) to bind the routine between the application calls and the remote calls.

The [call_as] attribute can be used for object interfaces. In this case, the interface definition can be used for local calls as well as remote calls because [call_as] allows an interface that can't be accessed remotely to be transparently mapped to a remote interface. The [call_as] attribute cannot be used with /osf mode.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: HSE on April 29, 2021, 09:39:33 PM
Hi LioMi!

Rejuvenating C++ Programs through Demacrofication - https://stroustrup.com/icsm-2012-demacro.pdf

First idea is to translate macros, not to erase them.

At some point could be interesting to build an specific package, because macros  take time.

you should take into account that Hutch does not use function prototypes, even if they are taken into account, they often do not coincide with, how the functions are called within a macro, means that the prototypes of the translator are different.  :icon_idea:
 
ML64 don't use prototypes. Ideally all functions in 64 bits must be Unprototyped following x64 ABI. If that is followed, Hutch's invoke macro never will fail (except floating point number, but is very simple to correct that).
With very little modifications of invoke macro you can make more easy programming (making automatic the promotion to follow x64 ABI) and to be capable to manage some other ideas (Prototyped functions are included in x64 ABI).

docs.microsoft.com/en-us/windows/win32/api/objidl/nf-objidl-ienumstring-next

 :thumbsup: Apparently is used for UWP programs. Don't say in wich library that functions are. Perhaps are VS functions.

Thanks, HSE.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: TouEnMasm on April 29, 2021, 10:37:58 PM
Quote
ML64 don't use prototypes. Ideally all functions in 64 bits must be Unprototyped following x64 ABI
?????????????????????????????????????????
ML64 can read protototypes and if it was said that prototypes had no use,c and c++ will not use them.
This isn't a rule " 64 bits must be Unprototyped following x64 ABI".
It's only true for ml64 because microsoft don't want to do it,perhaps it was two much expensive.
Microsoft had designed ML64 to replace the inline assembler suppressed in Cand C++,NO MORE.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: jj2007 on April 29, 2021, 10:42:49 PM
ML64 don't use prototypes. Ideally all functions in 64 bits must be Unprototyped following x64 ABI.

I agree with Yves, that sounds strange. Knowing the type madness of the C/C++ fans, there must be a way to tell MASM which arguments are expected, and how many (my own macros do that already).
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: LiaoMi on April 29, 2021, 10:44:41 PM
you should take into account that Hutch does not use function prototypes, even if they are taken into account, they often do not coincide with, how the functions are called within a macro, means that the prototypes of the translator are different.  :icon_idea:
 
ML64 don't use prototypes. Ideally all functions in 64 bits must be Unprototyped following x64 ABI. If that is followed, Hutch's invoke macro never will fail (except floating point number, but is very simple to correct that).
With very little modifications of invoke macro you can make more easy programming (making automatic the promotion to follow x64 ABI) and to be capable to manage some other ideas (Prototyped functions are included in x64 ABI).

Hi HSE,

I meant that the data flow in the Hutch's technique differs from the data flow in the UASM according to the prototype method. The translator works well for UASM and should not be adapted for ml64, otherwise it will break all logic.

Quote
If that is followed, Hutch's invoke macro never will fail (except floating point number, but is very simple to correct that).
:arrow_down:
Code: [Select]
C:\masm64\sdkrc100\um\wtypesbase.inc(85) : Error A2210: Syntax error: SHORT
 C:\masm64\sdkrc100\um\wtypesbase.inc(85): Included by
  C:\masm64\sdkrc100\um\combaseapi.inc(243): Included by
   C:\masm64\sdkrc100\um\objbase.inc(22): Included by
    C:\masm64\sdkrc100\um\ole2.inc(31): Included by
     C:\masm64\sdkrc100\um\ShlObj.inc(77): Included by
      mcalc.asm(33): Main line code
mcalc.asm(245) : Error A2295: Invalid use of pointer operator
 REGISTER(109)[macros64.inc]: Macro called from
  procedure_call(19)[macros64.inc]: Macro called from
   ml_invoke(1)[macros64.inc]: Macro called from
    savefile(1)[macros64.inc]: Macro called from
     mcalc.asm(245): Main line code
mcalc.asm(245) : Error A2295: Invalid use of pointer operator
 REGISTER(109)[macros64.inc]: Macro called from
  procedure_call(23)[macros64.inc]: Macro called from
   ml_invoke(1)[macros64.inc]: Macro called from
    savefile(1)[macros64.inc]: Macro called from
     mcalc.asm(245): Main line code
mcalc.asm(252) : Error A2295: Invalid use of pointer operator
 REGISTER(109)[macros64.inc]: Macro called from
  procedure_call(19)[macros64.inc]: Macro called from
   ml_invoke(1)[macros64.inc]: Macro called from
    GetFontHandle(1)[macros64.inc]: Macro called from
     mcalc.asm(252): Main line code
mcalc.asm(253) : Error A2295: Invalid use of pointer operator
 REGISTER(109)[macros64.inc]: Macro called from
  procedure_call(19)[macros64.inc]: Macro called from
   ml_invoke(1)[macros64.inc]: Macro called from
    GetFontHandle(1)[macros64.inc]: Macro called from
     mcalc.asm(253): Main line code
mcalc.asm(254) : Error A2295: Invalid use of pointer operator
 REGISTER(109)[macros64.inc]: Macro called from
  procedure_call(19)[macros64.inc]: Macro called from
   ml_invoke(1)[macros64.inc]: Macro called from
    GetFontHandle(1)[macros64.inc]: Macro called from
     mcalc.asm(254): Main line code
mcalc.asm(299) : Error A2048: Operands must be the same size: 8 - 4
mcalc.asm(301) : Error A2048: Operands must be the same size: 4 - 8
mcalc.asm(304) : Error A2048: Operands must be the same size: 8 - 4
mcalc.asm(306) : Error A2048: Operands must be the same size: 4 - 8
mcalc.asm(383) : Error A2160: INVOKE requires prototype for procedure
mcalc.asm(443) : Error A2160: INVOKE requires prototype for procedure
mcalc.asm(457) : Error A2160: INVOKE requires prototype for procedure
mcalc.asm(507) : Error A2160: INVOKE requires prototype for procedure
mcalc.asm(522) : Error A2160: INVOKE requires prototype for procedure
mcalc.asm(605) : Error A2160: INVOKE requires prototype for procedure
mcalc.asm(618) : Error A2160: INVOKE requires prototype for procedure
mcalc.asm(628) : Error A2160: INVOKE requires prototype for procedure
mcalc.asm(687) : Error A2145: INVOKE argument type mismatch: argument 4
mcalc.asm(696) : Error A2160: INVOKE requires prototype for procedure
mcalc.asm(697) : Error A2160: INVOKE requires prototype for procedure
mcalc.asm(798) : Error A2145: INVOKE argument type mismatch: argument 3
mcalc.asm(798) : Error A2048: Operands must be the same size: 4 - 8
mcalc.asm(808) : Error A2102: Symbol not defined : CHARRANGE
mcalc.asm(812) : Warning A4073: Size not specified, assuming: BYTE
mcalc.asm(836) : Error A2295: Invalid use of pointer operator
 REGISTER(109)[macros64.inc]: Macro called from
  procedure_call(19)[macros64.inc]: Macro called from
   rv(1)[macros64.inc]: Macro called from
    mcalc.asm(836): Main line code
mcalc.asm(842) : Error A2295: Invalid use of pointer operator
 REGISTER(109)[macros64.inc]: Macro called from
  procedure_call(19)[macros64.inc]: Macro called from
   rv(1)[macros64.inc]: Macro called from
    mcalc.asm(842): Main line code
mcalc.asm(1013) : Error A2160: INVOKE requires prototype for procedure
 loadfile(1)[macros64.inc]: Macro called from
  mcalc.asm(1013): Main line code
mcalc.asm(1033) : Error A2295: Invalid use of pointer operator
 REGISTER(109)[macros64.inc]: Macro called from
  procedure_call(19)[macros64.inc]: Macro called from
   ml_invoke(1)[macros64.inc]: Macro called from
    savefile(1)[macros64.inc]: Macro called from
     mcalc.asm(1033): Main line code
mcalc.asm(1033) : Error A2295: Invalid use of pointer operator
 REGISTER(109)[macros64.inc]: Macro called from
  procedure_call(23)[macros64.inc]: Macro called from
   ml_invoke(1)[macros64.inc]: Macro called from
    savefile(1)[macros64.inc]: Macro called from
     mcalc.asm(1033): Main line code

Hutch's technique x64:
invoke SendMessage,hEdit,EM_SETMARGINS,EC_LEFTMARGIN or EC_RIGHTMARGIN,eax

SendMessage function (winuser.h) - https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendmessage

Code: [Select]
LRESULT SendMessage(
  HWND   hWnd,
  UINT   Msg,
  WPARAM wParam,
  LPARAM lParam
);

What happens to WPARAM, LPARAM, and LRESULT when they travel between 32-bit and 64-bit windows? - https://devblogs.microsoft.com/oldnewthing/20110629-00/?p=10303 (Raymond Chen)
Quote
typedef UINT_PTR WPARAM;
typedef LONG_PTR LPARAM;
typedef LONG_PTR LRESULT;

 typedef __int3264 LONG_PTR;

__int3264
03/30/2020
2 minutes to read
An alias that is resolved to either:

An __int32 in a 32-bit translation and execution environment, or
An __int64 in a 64-bit translation and execution environment. For backward compatibility, it is 32-bit on the wire. The higher 4 bytes MUST be truncated on the sender side during marshaling and MUST be extended appropriately (signed or unsigned), as specified in [C706] section 14.2.5, on the receiving side during unmarshaling.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: HSE on April 30, 2021, 02:39:24 AM
Hi LioMi!

 :biggrin:  :biggrin: :biggrin:

Code: [Select]
Microsoft (R) Windows (R) Resource Compiler Version 10.0.10011.16384
Copyright (C) Microsoft Corporation.  All rights reserved.

 Assembling: mcalc.asm
Microsoft (R) Incremental Linker Version 14.00.24213.1
Copyright (C) Microsoft Corporation.  All rights reserved.

 El volumen de la unidad C no tiene etiqueta.
 El número de serie del volumen es: 1000-D537

 Directorio de C:\masm32\examples64\mcalc

28/04/2021  08:55            27.265 mcalc.asm
28/04/2021  09:24                16 mcalc.bin
29/04/2021  12:59            32.768 mcalc.exe
11/10/2018  22:19               978 mcalc.inc
29/04/2021  12:59            24.267 mcalc.obj
               5 archivos         85.294 bytes
               0 dirs  374.203.731.968 bytes libres
Presione una tecla para continuar . . .

This is about ML64 and Biterider's includes (they are in ObjAsm SDK), not about UASM logic. Anyway I don't see problems using modified includes with UASM, just that NMHDR.code is very used in ObjAsm examples (but is because I changed the name in the structure).


I don't remember a problem with SHORT, perhaps is other translation.

In Hutch2.zip there is a macros64.asm. I think is Hutch original, but with 4 forced expansions of reparg. You can make a diff with your macros64.asm

If you have time, perhaps you can test the modifications to macros64.asm that are under study.

Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: HSE on April 30, 2021, 03:36:53 AM
if it was said that prototypes had no use,c and c++ will not use them.
I'm not talking about c and c++.

This isn't a rule " 64 bits must be Unprototyped following x64 ABI".
I think that rule exist in M$. Most 64 bit Win API apparently follow that rule. You can see how well work Hutch's masm64 SDK. 

I agree with Yves, that sounds strange.
:thumbsup:    Just is that way. :biggrin:

Knowing the type madness of the C/C++ fans
Compilers take care of that.

there must be a way to tell MASM which arguments are expected, and how many (my own macros do that already).
No. That is the idea: because we don't like compilers very much, macros have to take care of that.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: Biterider on April 30, 2021, 04:41:53 AM
Hi HSE
I finally got Hutch2 running. I had to move things around and modify paths but finally (after 19 secs of compilation time) I got the exe.   :greenclp:
I had to try several times as ML64 crashes with cryptic error messages or no error at all.  :sad:
That it takes so long to compile such a short program is [...]

At least I have something to work with  :thumbsup:

Biterider

Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: jj2007 on April 30, 2021, 05:04:42 AM
ML64 crashes with cryptic error messages

I see them frequently when I test my "dual" stuff with ML64. No problem, just try again.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: hutch-- on April 30, 2021, 06:08:18 AM
Some of this makes me laugh, inflict left over prototyping onto 64 bit MASM and you turn it to crippleware. There are enough people who can produce a prototyping technique but its big, slow and clumsy. About the only thing it inherited from ML.EXE was the ancient pre-processor which made it possible to civilise it enough to be able to write working code with.

While both John and nidud are producing assemblers that are morphing into C compilers, 64 bit MASM has no aspirations in that direction at all and the sheer simplicity of its prototyping made for a clean and easy construction method for a large near complete set.

There is a solution for those who need prototyping, decent documentation and if anyone could be bothered, a post source scanner to test if code is written correctly but the best code scanner is the author and MASM makes no assumptions other that the author should write the code correctly.

Now with h2incX, its assumptions make it a crippled piece of junk, the Microsoft C/C++ headers are deliberately produced in the most obscure method possible to prevent access to other tools and this includes assemblers. If you have ever bothered to run the full set of includes through a C compiler with a full listing, you will see that the C compiler strips all of the junk out of the headers and this was one of the many ways that you got a list of functions with the correct number of arguments.

To compound this, the linker libraries do not include the byte count data that 32 bit STDCALL libraries contained. I see the pursuit of old technology for assemblers to have the 1990 format of prototypes as a fool's errand harking back to a simpler era where the Microsoft h2inc.exe actually worked.

To get something like h2incX to work, you need to map the entire C header system to handle the repeated different equates and be able to track the paths taken by any particular version of the Microsoft headers. The construction of the 32 bit windows.inc was a horrendous task that over time was a massive amount of work and testing, that is why most copied its internals rater than doing their own work to produce all of the data.

externdef __imp_SendMessageA:PPROC
SendMessageA equ <__imp_SendMessageA>
  IFNDEF __UNICODE__
    SendMessage equ <__imp_SendMessageA>
  ENDIF

externdef __imp_SendMessageW:PPROC
SendMessageW equ <__imp_SendMessageW>
  IFDEF __UNICODE__
    SendMessage equ <__imp_SendMessageW>
  ENDIF

This matches the linker libraries, pursuing 1990 prototypes is out of date junk with MASM in 64 bit.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: HSE on April 30, 2021, 09:39:16 AM
Hi Biterider!

 :thumbsup:

That it takes so long to compile such a short program is [...]

Perhaps Lena know what happen with ML64: 
https://www.youtube.com/watch?v=DhLO_ZaWaSg

:biggrin: I don't know what.

Remind me an JJ observation: an objasm32 debug program run crazy taking all core time if there is no DebugCenter.

HSE
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: HSE on April 30, 2021, 10:02:45 AM
Hi Hutch!

About the only thing it inherited from ML.EXE was the ancient pre-processor which made it possible to civilise it enough to be able to write working code with.
ML64 is a tool maked for C/C++ people. Is very interesting that they see things from a totally opposite point of view: ML64 is exactly ML, except HLL flow control and invoke was removed.
We know that what was removed potentially can collide with compiler optimizations, there is no other reason.

Thanks for remind us your opinion, always consistent  :thumbsup:

HSE
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: Vortex on April 30, 2021, 05:35:55 PM
Hi HSE,

Quote
We know that what was removed potentially can collide with compiler optimizations, there is no other reason.

I don't think that the main reason is the collisions with the compiler. Simply, M$ is trying to discourage people from using the assembly language. Their main product to sell is Visual Studio and not an assembler.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: TouEnMasm on April 30, 2021, 05:45:12 PM
Quote
I don't think that the main reason is the collisions with the compiler. Simply, M$ is trying to discourage people from using the assembly language. Their main product to sell is Visual Studio and not an assembler.
I aggree with that

Is this a prototype ?
Quote
externdef __imp_SendMessageW:PPROC
SendMessageW equ <__imp_SendMessageW>
  IFDEF __UNICODE__
    SendMessage equ <__imp_SendMessageW>
  ENDIF

Just a call.

Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: hutch-- on April 30, 2021, 05:53:07 PM
The only real distinction between 32 bit ML and 64 bit ML64 is the target market. When ML was released in 1990, it was pointed at the consumer programming market. ML64 is necessary for assembler in 64 bit as CL does not support inline assembler. To do that it must be able to create object modules and with that capacity, it can produce executable code.

The capacities to emulate high level code is of no use to C/C++ programmers so it was never written into the 64 bit version, C/C++ code optimisation has nothing to do with the guts of an asm module and you can routinely write pure mnemonic code that will do the same thing as the emulated high level code. In needing to access an external asm module, there is nothing to collide, the asm code is self contained.

If you have used the 64 bit version for a long time, you will know that it is different from the 32 bit ML.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: hutch-- on April 30, 2021, 05:58:07 PM
> Just a call.

Yes, no need for something else that is useless, 64 bit MASM doed not use or need prototypes, you are confusing the 32 bit version and the Watcom derivatives.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: jj2007 on April 30, 2021, 06:04:18 PM
no need for something else that is useless, 64 bit MASM doed not use or need prototypes

And compilers that count and type check parameters are meant for idiots :thumbsup:

MessageBox(FP4(1.0), "text", "title", MB_OK, "one more", "who cares", FP8(123.456))
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: hutch-- on April 30, 2021, 08:20:14 PM
But no matter how hard you try, 64 bit MASM ignored prototypes. You can cobble together macros that ape prototypes but it makes the assembler really slow. Wishful thinking, a refusal to let go of the old stuff, incapacity to adapt, they are all part of the desire to attach something useless to an assembler that does not need them.

If you want strong prototypes and data types, use a C/C++ compiler and throw away all of the adevantages of a fast low level assembler.  :thdn:
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: jj2007 on April 30, 2021, 08:56:10 PM
You can cobble together macros that ape prototypes but it makes the assembler really slow.

Right, that could be a problem :thumbsup:

Have you timed it with one of your "serious" 64-bit sources? I am curious. Unfortunately, I am still stuck with my 32-bit sources, for example, the 22+ kLines of the RichMasm editor:

Code: [Select]
ML 10.0 build all took 2185 ms
UAsm64 build all took 1688 ms
AsmC build all took 1259 ms
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: hutch-- on April 30, 2021, 10:10:14 PM
So you are timing your macros, not MASM. Without a cannon ball around its neck, MASM is fast. Not that it matters much, its the code speed that matters.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: TouEnMasm on April 30, 2021, 10:35:54 PM

code speed is the same if there is no ..bug.
The one thing is that the code must exist.
Quote
This matches the linker libraries, pursuing 1990 prototypes is out of date junk with MASM in 64 bit
1990 calls and not protoypes are not enough to made code.
i have counted about 30000 prototypes in the SDK.Multiply your calls by 15 and you are on the good path.

Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: jj2007 on April 30, 2021, 10:39:07 PM
i have counted about 30000 prototypes in the SDK.

That's quite a lot. I arrive at about 21000, using the C:\TDM-GCC-64 SDK. Which one do you use? How do you deal with duplicates btw?
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: TouEnMasm on April 30, 2021, 11:06:00 PM
I only know the windows SDK 10.0.19041.0 + CRT.
Duplicate are in the onecore and it is possible that the real number is about 20000.
That is the Forum ML64  call multiply by 10.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: jj2007 on April 30, 2021, 11:17:10 PM
I made a quick test:

include \masm32\MasmBasic\MasmBasic.inc         ; download (http://masm32.com/board/index.php?topic=94.0)
  Init
  GetFiles \Masm32\include64\*.inc
  xor ecx, ecx
  For_ ct=0 To eax-1
        Print "+"
        add ecx, Count(FileRead$(Files$(ct)), "equ !<__imp_")
  Next
  Inkey Str$("\n%i imports found", ecx)
EndOfCode


Output: 69081 imports found  :rolleyes:

Some look a bit odd (\Masm32\include64\mfc42u.inc):
Code: [Select]
externdef __imp_mfc42u_ordinal1000:PPROC
mfc42u_ordinal1000 equ <__imp_mfc42u_ordinal1000>

externdef __imp_mfc42u_ordinal1001:PPROC
mfc42u_ordinal1001 equ <__imp_mfc42u_ordinal1001>

externdef __imp_mfc42u_ordinal1002:PPROC
mfc42u_ordinal1002 equ <__imp_mfc42u_ordinal1002>

externdef __imp_mfc42u_ordinal1003:PPROC
mfc42u_ordinal1003 equ <__imp_mfc42u_ordinal1003>

externdef __imp_mfc42u_ordinal1004:PPROC
mfc42u_ordinal1004 equ <__imp_mfc42u_ordinal1004>

externdef __imp_mfc42u_ordinal1005:PPROC
mfc42u_ordinal1005 equ <__imp_mfc42u_ordinal1005>

externdef __imp_mfc42u_ordinal1006:PPROC
mfc42u_ordinal1006 equ <__imp_mfc42u_ordinal1006>

externdef __imp_mfc42u_ordinal1007:PPROC
mfc42u_ordinal1007 equ <__imp_mfc42u_ordinal1007>

I admit also that I don't quite understand the "no protos" thing:
Code: [Select]
extern __imp_MAPISendMail:PANYARGS
MAPISendMail TEXTEQU <__imp_MAPISendMail>
...
extern __imp_BMAPISendMail:PANYARGS
BMAPISendMail TEXTEQU <__imp_BMAPISendMail>
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: hutch-- on April 30, 2021, 11:51:23 PM
> 1990 calls and not protoypes are not enough to made code.

Yves,

I think you have missed something here as I know you are not a native English speaker.

In 1990 Microsoft released MASM version 6.0 and with it, the technique for prototyping procedure calls. It did the job for folks who needed a high level layer for the then 16 bit assembler. Current 32 bit ML still uses prototypes IF you choose to use "invoke" with procedure calls. You can write lower level assembler with standard push/call notation in 32 bit with NO PROTOTYPES.

My comments on 1990 techniques has nothing to do with code generation, it has to do with an additional high level layer imposed on MASM to make it easier for high level programmers to understand it.

There are a number of other assemblers that don't use ML style prototypes, FASM, GAS among others. The Watcom derivatives aped 32 bit MASM starting from JWASM but Microsoft built the 64 bit version of MASM without the capacity to use prototypes.



Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: TouEnMasm on May 01, 2021, 03:16:24 AM
Quote
Output: 69081 imports found         ....Wo says more ?

1990 must be a manual work.
With automatised procedures ,20000 can be done in a few minutes,perhaps less.

Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: jj2007 on May 01, 2021, 04:45:21 AM
There are a number of other assemblers that don't use ML style prototypes, FASM, GAS among others.

Fascinating:
Code: [Select]
extern __imp_MAPISendMail:PANYARGS
You can just pass any args to SendMail! And it works, most of the time! No PROTOs! Why oh why do the so-called "professional" C/C++ coders waste their precious time with studying their stupid compilers' type error and/or arg count messages?
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: hutch-- on May 01, 2021, 08:21:54 AM
> Why oh why do the so-called "professional" C/C++

Easy, they are writing C/C++.

Now here is a question, why oh why do you time your macros and blame it on MASM. Why oh why don't you impose the same macros on the Watcom derivatives.

Why oh why are you so desperately trying to find  the characteristics of a visual garbage generator when you could easily find something that could generate a 1 gig Hello World /
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: jj2007 on May 01, 2021, 08:24:53 AM
Now here is a question, why oh why do you time your macros and blame it on MASM. Why oh why don't you impose the same macros on the Watcom derivatives.

I do.

Have you timed it with one of your "serious" 64-bit sources? I am curious. Unfortunately, I am still stuck with my 32-bit sources, for example, the 22+ kLines of the RichMasm editor:

Code: [Select]
ML 10.0 build all took 2185 ms
UAsm64 build all took 1688 ms
AsmC build all took 1259 ms
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: hutch-- on May 01, 2021, 09:19:10 AM
 :biggrin:

So you are saying you use your version of invoke, not the native Watcom derivative invoke ?
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: jj2007 on May 01, 2021, 09:20:11 AM
In 64-bit code, yes. But the timings above are obviously for 32-bit code.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: LiaoMi on May 03, 2021, 07:50:43 PM
Hi LioMi!

 :biggrin:  :biggrin: :biggrin:

Code: [Select]
Microsoft (R) Windows (R) Resource Compiler Version 10.0.10011.16384
Copyright (C) Microsoft Corporation.  All rights reserved.

 Assembling: mcalc.asm
Microsoft (R) Incremental Linker Version 14.00.24213.1
Copyright (C) Microsoft Corporation.  All rights reserved.

 El volumen de la unidad C no tiene etiqueta.
 El número de serie del volumen es: 1000-D537

 Directorio de C:\masm32\examples64\mcalc

28/04/2021  08:55            27.265 mcalc.asm
28/04/2021  09:24                16 mcalc.bin
29/04/2021  12:59            32.768 mcalc.exe
11/10/2018  22:19               978 mcalc.inc
29/04/2021  12:59            24.267 mcalc.obj
               5 archivos         85.294 bytes
               0 dirs  374.203.731.968 bytes libres
Presione una tecla para continuar . . .

This is about ML64 and Biterider's includes (they are in ObjAsm SDK), not about UASM logic. Anyway I don't see problems using modified includes with UASM, just that NMHDR.code is very used in ObjAsm examples (but is because I changed the name in the structure).


I don't remember a problem with SHORT, perhaps is other translation.

In Hutch2.zip there is a macros64.asm. I think is Hutch original, but with 4 forced expansions of reparg. You can make a diff with your macros64.asm

If you have time, perhaps you can test the modifications to macros64.asm that are under study.

Hi HSE,

what has been said is true of course, but previously there was a question of compatibility. My trials did not always go smoothly, and I often had syntax problems. It would be nice to test on both versions at once, that's what I wanted to do, and where I ran into a lot of inconsistencies  :azn:

I will try new macros, there are two whole versions, your modification and an update from Hutch  :tongue:
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: hutch-- on May 03, 2021, 08:15:54 PM
Funny part is the calculator was an exercise in interface design, the arithmetic was simple SSE2, the rest was trying to make it look presentable. It came out OK but I have no doubt it could be done a lot better.
Title: Re: Notes about ML64 and h2incX includes and libraries
Post by: HSE on May 03, 2021, 11:32:59 PM
and where I ran into a lot of inconsistencies  :azn:
Others than those enumerated before?

I will try new macros, there are two whole versions, your modification and an update from Hutch  :tongue:
Fantastic  :thumbsup: