The MASM Forum

Projects => ObjAsm => Topic started by: Biterider on January 01, 2018, 05:39:01 AM

Title: ObjAsm64
Post by: Biterider on January 01, 2018, 05:39:01 AM
Hi
The last weeks I started to work on the ObjAsm framework to support x64 using UASM.
It is a work in progress, since not all features are implemented yet. I choose using UASM, because it offers at least the same flexibility we are used to from ML 6.15 for x86.
The currently supported features are:
    ·  Object Oriented Programming (single inheritance)
    ·  Dual method tables:  simultaneous support of virtual method tables for "open interfaces" like COM and "not exposed" methods.
    ·  Native COM support (x64 virtual method table call and Dispatch)
    ·  Some prebuild objects like Collections, Linked Lists, Streams, Controls, Applications, etc.
    ·  Precompiled objects
    ·  Object templates, sub- and  super-classing capabilities
    ·  Virtual, dynamic, static, private methods
    ·  VARARG support
    ·  ANSI and UNICODE support
    ·  ObjMem64 library that covers most of the daily coding needs
    ·  Examples and projects
    ·  Automated compilation and build control
    ·  Etc.

Since ObjAsm64 is in beta stage, I want to ask if somebody is interested in testing it. I don't have an installer or a help file yet, but people coming from ObjAsm32 know how to use this model or you can read the old documentation. Since x64 import libraries and include files are spread around, there will be some path adjustments to do. Like ObjAsm32, ObjAsm64 has a single file that must be modified to specify your local configuration. I also added 2 new Environment Variables (OA64_PATH, MSVS_PATH) to allow the tool chain to work smoothly.
In addition to the last UASM version, you will need a proper linker and maybe some debug facilities. My choose fall on Visual Studio -Community Edition-, which comes with all necessary tools. As development editor I'm still using RadAsm (2), but you can use whatever you prefer. Each project comes with batch files to compile for release or debug targets (RLS, DBG).
Send me a PM if you are interested to participate!
Attached is one of the new translated projects.

Best regards, Biterider
Title: Re: ObjAsm64
Post by: Biterider on January 07, 2018, 02:34:56 AM
Hi
I finished the rework and migration of the OCX Container project to x64.
It proofs how well the new ObjAsm64 framework works and interacts with the x64 COM world.

At the moment, the application supports the following interfaces:
All these interfaces work with several tested OCX controls. The supplied example uses a Flash Player OCX Component, which is almost present in every Windows system. The Container was tested on Win7-64 bit and Win10-64 bit systems.
Other OCX-Components like GUI-Controls (Checkbox, Combobox, etc.) work seamlessly together with the container. Also a Web-Browser Component and the Windows media Player react correctly when they are embedded.
Attached to this post are Flash Player Container and a test swf-file.

Best regards, Biterider
Title: Re: ObjAsm64
Post by: Biterider on January 13, 2018, 11:46:31 PM

Hi
I want to present one of the most used and ancient objects implemented in assembler, called "Collection".
Collections are, in first instance, repositories for any type of objects, including other collections. Mixing different type of objects in this repository is also allowed.

Collections are designed for administrative use, where very high performance is not required but a high degree of flexibility and processing capabilities. In this regard, Collections provide forward and backward searching, forward and backward iterators, stream serialization, insertion, deletion, positioning, etc. supported in the background by an automatic memory management.

Multithread synchronization is ensured, like OA32, due to the use of the low level build in synchronization call mechanism (xOCall).

There are specialized descendants of "Collection" like sorted collections, string collections, data collections, and many others for different management tasks. You can derive your own descendant, redefining the methods you want to change.

The interface of a Collection object class looks like:

Object Collection, CollectionID, Streamable

VirtualMethod    Delete,          POINTER
VirtualMethod    DeleteAt,        DWORD
VirtualMethod    DeleteAll
RedefineMethod   Deserialize,     PDESER_INFO
DynamicMethod    DeserializeItem, POINTER, PDESER_INFO
DynamicMethod    DestroyItem,     POINTER
VirtualMethod    Dispose,         POINTER
VirtualMethod    DisposeAt,       DWORD
VirtualMethod    DisposeAll
RedefineMethod   Done
VirtualMethod    FirstThat,       POINTER, QWORD, QWORD
VirtualMethod    FirstThatNot,    POINTER, QWORD, QWORD
VirtualMethod    ForEach,         POINTER, QWORD, QWORD
VirtualMethod    ForEachRev,      POINTER, QWORD, QWORD
DynamicMethod    GetItem,         POB_Stream, PDESER_INFO
VirtualMethod    IndexOf,         POINTER
RedefineMethod   Init,            POINTER, DWORD, DWORD, DWORD
VirtualMethod    Insert,          POINTER
VirtualMethod    InsertAt,        DWORD, POINTER
VirtualMethod    ItemAt,          DWORD
VirtualMethod    LastThat,        POINTER, QWORD, QWORD
VirtualMethod    LastThatNot,     POINTER, QWORD, QWORD
RedefineMethod   Load,            POB_Stream, PDESER_INFO
VirtualMethod    PutAt,           DWORD, POINTER
DynamicMethod    PutItem,         POB_Stream, POINTER
RedefineMethod   Serialize
DynamicMethod    SerializeItem,   POINTER
RedefineMethod   Store,           POB_Stream
Private
StaticMethod     SetLimit,        DWORD


DefineVariable   pItems,          POINTER,      NULL
DefineVariable   dCount,          DWORD,        0
DefineVariable   dLimit,          DWORD,        0
DefineVariable   dDelta,          DWORD,        0
DefineVariable   dMaxCapacity,    DWORD,        0
DefineVariable   ObjLock,         ObjectLock,   {}

ObjectEnd


It can be observed on the first line, started by the keyword "Object", 3 elements, the name of the object class (Collection), its unique ID and finally its ancestor. Deriving "Collection" from "Streamable" means, that it inherits all capabilities of a "Streamable" object. This way, we can store the whole content of the "Collection" on an arbitray "Stream". In most cases this is a disk file (DiskStream) but it can be a memory chunk or whatever is desired.


After the first definition line follow the methods. These define the procedural capabilities of this object. I'll not comment each one, since the names are self-explanatory. You'll notice the different method types supported by the ObjAsm64 framework.

Finally, after the method definitions follow the internal variables that are managed by "Collection" and their initial values when created.
If very high performance is an issue, other objects should be used like Linked-Lists or Red-Black-Trees.

I hope this post give you a brief insight to this important OO assembler element.

Best regards, Biterider

Title: Re: ObjAsm64
Post by: HSE on January 14, 2018, 01:45:57 AM
Fantastic Biterider!!

You are publishing the ObjAsm`s book by parts!  :t

I will expect  the part of "Red-Black-Trees" :biggrin:

Title: Re: ObjAsm64
Post by: Biterider on January 20, 2018, 04:50:49 PM
Hi
I started to work on the Red-Black Tree. I have 80% ported at the moment, but before I release it, I want to test all methods and check the performance against other algorithms. In the meanwhile you can check some places on the web like:


http://www.cs.iupui.edu/~xkzou/teaching/CS580/Red-black_trees.ppt (http://www.cs.iupui.edu/~xkzou/teaching/CS580/Red-black_trees.ppt)
http://en.wikipedia.org/wiki/Red-black_tree (http://en.wikipedia.org/wiki/Red-black_tree)
http://en.wikipedia.org/wiki/Tree_traversal (http://en.wikipedia.org/wiki/Tree_traversal)
http://en.wikipedia.org/wiki/Threaded_binary_tree (http://en.wikipedia.org/wiki/Threaded_binary_tree)
http://www.eternallyconfuzzled.com/tuts/datastructures/jsw_tut_rbtree.aspx (http://www.eternallyconfuzzled.com/tuts/datastructures/jsw_tut_rbtree.aspx)


There are so many other binary trees like AA-Trees, AVL-Trees, B-Trees, etc. to mention only a few.
Biterider
Title: Re: ObjAsm64
Post by: HSE on January 21, 2018, 03:14:06 AM
It's just a funny name, I'm not using complex data structures  :t

But just yesterday I remember a more practical issue: in Debug System the colors are hardcoded. I change everithing to white foreground and black background in the dark machine (not so nice).

For now I only will read the book, because very infrequently I use a 64 bit machine. But it's very good to know that at some point I can update my ObjAsm32 code to ObjAsm64. Thanks.


Title: Re: ObjAsm64
Post by: Biterider on January 21, 2018, 04:43:33 PM
Hi
One of the interesting features about ObjAsm is the build-in ability to emit information for debugging purposes. There are many other ways to achieve this using other tools or OS APIs. The advantage of using the ObjAsm approach is that it is completely transparent and can be simply modified to any new circumstance. The most used information destination on desktops is an application called "DebugCenter". In this case, the debugee communicates with "DebugCenter" using an OS IPC method. The current implementation can transmit colorized text using the known emphasis attributes and bitmaps.

The described feature is not intended as an alternative to an application debugger. It is intended as a complement to get rapidly information about the logic of the debugee and other information of interest.

Additionally, to the output to "DebugCenter", alternative destinations are implemented for a console window or a log file. In these cases, only plain text can be emitted.

If this feature is required, the debugged application needs to switch it on. These way, the macros that emit the requested information are expanded. Since the release version of the application doesn't need to emit the information, the switch can be simply be turned off.
For this purpose, the application needs to include the following line after the general Setup of the ObjAsm framework:

SysSetup OOP, WINDOWS, WIDE_STRING, DEBUG(WND)


SysSetup may contain several switches, but the one we are interested in is DEBUG(WND). This expression activates the Debugging output to WND. In this context, WND means "DebugCenter", CON a console window and LOG a log-file.

All Debugging macros are located in the Debug.inc file. A short view at the file beginning shows a list of all available macros. They are designed to emit warnings, numbers in different formats, memory dumps, bitmaps, API error descriptions, windows messages, GUID numbers, etc. Interesting to know is, that these macros preserve the complete state of the CPU. That means that they are absolutely transparent to the CPU execution.
At the moment, due to some limitations in the x64 world, not all features of ObjAsm32 are implemented. One of the most notorious is the method performance tracking.

The build process can be triggered for release of debug targets. For this purpose, each project comes with 2 batch files. The switch that controls the debug output is located in the "bin\OPT_ASM_RLS.txt file" (-DISABLE_DEBUG).

Best regards, Biterider
Title: Re: ObjAsm64
Post by: Biterider on January 27, 2018, 11:40:59 PM
Hi
A Red-Black Tree is a not perfect self-balancing binary tree. Each node of the binary tree has an attribute that is often interpreted as the color (red or black). These colors are used to ensure the tree remains approximately balanced during insertions and deletions.
Red-Black trees offer worst-case guarantees for insertion time, deletion time, and search time. In big O notation that would be O(log n) for these 3 operations. This make them valuable in time-sensitive applications such as real-time applications.
https://en.wikipedia.org/wiki/Red%E2%80%93black_tree (https://en.wikipedia.org/wiki/Red%E2%80%93black_tree)

The ObjAsm64 object implementation is based on the following paper
http://www.cs.iupui.edu/~xkzou/teaching/CS580/Red-black_trees.ppt (http://www.cs.iupui.edu/~xkzou/teaching/CS580/Red-black_trees.ppt)


Additional functionality for serialization is also provided without adding additional complexity.
The following DebugCenter output shows the structure of a Red-Black Tree of 16 elements.

QuoteObject at address 0000000202C1E700 => Key = 3
  Object at address 0000000202C1E4D0 => Key = 1
    Object at address 0000000202C1E840 => Key = 0
      Null
      Null
    Object at address 0000000202C1EAC0 => Key = 2
      Null
      Null
  Object at address 0000000202C1E890 => Key = 7
    Object at address 0000000202C1E8E0 => Key = 5
      Object at address 0000000202C1E430 => Key = 4
        Null
        Null
      Object at address 0000000202C1E980 => Key = 6
        Null
        Null
    Object at address 0000000202C1E660 => Key = 11
      Object at address 0000000202C1E390 => Key = 9
        Object at address 0000000202C1E610 => Key = 8
          Null
          Null
        Object at address 0000000202C1EA20 => Key = 10
          Null
          Null
      Object at address 0000000202C1E3E0 => Key = 13
        Object at address 0000000202C1E520 => Key = 12
          Null
          Null
        Object at address 0000000202C245A0 => Key = 14
          Null
          Object at address 0000000202C24140 => Key = 15
            Null
            Null

There are many ways to use a Red-Black Tree. In combination with a host object, a node (RBT_NODE ) must be added as variable. The controlling object (RedBlackTree) provides the functionality to insert, search, iterate, test and delete those objects that are linked together using the RBT_NODEs.
The current interface looks like


Object RedBlackTree, RedBlackTreeID, Streamable
  DynamicMethod     Compare,        POINTER, POINTER
  VirtualMethod     Delete,         POINTER
  VirtualMethod     DeleteAll
  VirtualMethod     Dispose,        POINTER
  VirtualMethod     DisposeAll
  VirtualMethod     Find,           POINTER
  VirtualMethod     FirstThat,      POINTER, QWORD, QWORD   ;-> Proc, 2x Parameters
  VirtualMethod     FirstThatNot,   POINTER, QWORD, QWORD   ;-> Proc, 2x Parameters
  VirtualMethod     ForEach,        POINTER, QWORD, QWORD   ;-> Proc, 2x Parameters
  VirtualMethod     ForEachRev,     POINTER, QWORD, QWORD   ;-> Proc, 2x Parameters
  VirtualMethod     GetFirst
  VirtualMethod     GetLast
  VirtualMethod     GetNext,        POINTER
  VirtualMethod     GetPrev,        POINTER
  RedefineMethod    Init,           POINTER, DWORD
  VirtualMethod     Insert,         POINTER
  VirtualMethod     LastThat,       POINTER, QWORD, QWORD   ;-> Proc, 2x Parameters
  VirtualMethod     LastThatNot,    POINTER, QWORD, QWORD   ;-> Proc, 2x Parameters

  DynamicMethod     GetItem,        POB_Stream          ;-> Stream
  DynamicMethod     PutItem,        POB_Stream, POINTER ;-> Stream, -> Item
  RedefineMethod    Load,           POB_Stream, POINTER ;-> Stream, -> Owner
  RedefineMethod    Store,          POB_Stream          ;-> Stream

  DefineVariable    qNodeOffset,    QWORD,        0     ;Offset from host object to node
  DefineVariable    pRootNode,      PRBT_NODE,    NULL
  DefineVariable    dCount,         DWORD,        0

  DefineVariable    ObjLock,        OBJECT_LOCK,  {}    ;Locking structure for multithreaded access

ObjectEnd


It can be seen, that the object derives from Streamable, inheriting all necessary capabilities to save its content onto any type of stream.

Attached is the source file for those that want to look into the x64 code.


Biterider
Title: Re: ObjAsm64
Post by: Biterider on February 03, 2018, 06:12:39 PM

Hi
Over the time applications became more complex and their main interface also grew larger. They used tabbed toolbars and filled them with many graphical buttons and other control elements.
In 2007 Microsoft began to introduce a form of modular ribbon in which controls are grouped by functionality. Such ribbons use tabs to expose various sets of controls, eliminating the need for many parallel toolbars. Contextual tabs are tabs that are only displayed when user needs them.
For a complete historical description, read on here:


https://en.wikipedia.org/wiki/Ribbon_(computing) (https://en.wikipedia.org/wiki/Ribbon_(computing))
https://msdn.microsoft.com/en-us/library/windows/desktop/dd371191(v=vs.85).aspx (https://msdn.microsoft.com/en-us/library/windows/desktop/dd371191(v=vs.85).aspx)


Last week, I translated some Microsoft Ribbon Framework C files to incorporate this functionality into assembly applications. The end result is an x64 assembly ribbon application that can be seen in the attachment.


Basically, you need to establish communication between the Ribbon framework and the application using some COM interfaces. On the application side, you have to implement the IUIApplication COM interface and CoCreateInstance the Ribbon Framework according to these instructions:


https://msdn.microsoft.com/en-us/library/windows/desktop/dd316924(v=vs.85).aspx (https://msdn.microsoft.com/en-us/library/windows/desktop/dd316924(v=vs.85).aspx)


All GUI controls created communicate through a COM application interface called IUICommandHandler. ObjAsm64 creates a COM component and interface for each control and adds them to the hosting component.


At this point, I have to warn of a Ribbon implementation issue that took some time to understand and find the right tools for. It is the transparency of the images used for the visual interface. Win7 handles it differently than Win8 and Win10. The former only understands BMP files in A8R8G8B8 format, while the other one can handle PNG files without problems.


https://msdn.microsoft.com/en-us/library/windows/desktop/dd316921(v=vs.85).aspx (https://msdn.microsoft.com/en-us/library/windows/desktop/dd316921(v=vs.85).aspx)


Using a suitable image saving (BMP) tool in 32-bit ARGB format will save many headaches. I use the IcoFX freeware version to export these pictures in this format.
The ObjAsm64 implementation is extremely easy. If you insert the following code after the window creation code, the Ribbon framework and the connection between the ribbon controls and the application-side interface will automatically be started.

    mov [rsi].pUIApplication, $New(UIApplication)
    OCall rax::UIApplication.Init, rsi, [rsi].hWnd, $OfsCStrW("APPLICATION_RIBBON")



Shutdown is done by calling the following code
    OCall [rsi].pUIApplication::UIApplication.Release


I hope this will inspire someone to take a look into ribbons to modernize the GUI of assembly applications.  8)


Best regards,
Biterider

Title: Re: ObjAsm64
Post by: jj2007 on February 03, 2018, 08:26:02 PM
This looks beautiful, biterider :t

(but functionality-wise, I hate the ribbon interface... it just looks good but you spend too much time guessing where the function you need is hidden; same for tab controls)
Title: Re: ObjAsm64
Post by: habran on February 04, 2018, 03:22:45 PM
Hi Biterider :biggrin:
ObjAsm64 looks very impressive and promising 8) :t
QuoteThe last weeks I started to work on the ObjAsm framework to support x64 using UASM.
It is a work in progress, since not all features are implemented yet. I choose using UASM, because it offers at least the same flexibility we are used to from ML 6.15 for x86.
I appreciate your proclivities for your choice 8)

I am to busy with the UASM to be able to study your libraries, however, it sounds very promising.

You have mentioned :
QuoteSome prebuild objects like Collections, Linked Lists, Streams, Controls, Applications, etc.
·        Precompiled objects
I sounds intersting, which gives me an idea:
It would be great if you could create some simple starting base for different types of APS, which would have precompiled headers in it so that we can just build on it, like a foundation for the building

I can see by number of downloads that a lot of people is interested in your work but they are to shy to give some feedback ::)

It would be maybe good idea to give some more source code examples, so that people get idea how it can be utilised. It would probably extract ObjAsm64 from Rarely Used Projects

Keep up good work

Best Regards


Title: Re: ObjAsm64
Post by: Biterider on February 06, 2018, 07:45:41 AM
Hi Habran
Thanks for your replay.   :biggrin:
ObjAsm64 is basically a rework of ObjAsm32 for x64. The main work is done, but some features like "Performance Tracking" and "Buffer Security Check" (VS /gs switch) are still missing. The reason is mainly the x64 calling convention. I guess I'll need some help from the UASM team with the necessary Prolog and Epilog coding   8)

I'm thinking about writing a PDF describing the ObjAsm64 basics. Many things are already described in the source code, but the overview should also be written. At the moment I have 6 examples, but they do not cover all the basics. I think, I should revise them first.

Biterider
Title: Re: ObjAsm64
Post by: Biterider on February 11, 2018, 05:14:16 AM
Hi
ObjAsm64 is a single inheritance OOP implementation for x64 assembler code. It is based on macros and therefore needs a MASM-compatible assembler such as UASM.
The ObjAsm64 package contains objects, libraries, include files and samples to build a framework that enables the coder to quickly create complete applications.

The OOP core is in the Objects.inc file. There are several macros that only serve to maintain lists of methods, variables, and so on, that are used to compile the code. This seems complicated, but the idea behind it is very simple. We want to cluster data and code together in form of a structure called object. Behind the scenes are a second structure called "object metadata" (OMD). This data holds the information of the object class, the "object template", the "virtual tables" and finally the "event translation table" (ETT).

The object template is an in-memory structure that is used to copy the initial data to a newly created object, the object instance. The object template can be seen as the source of all derived objects.
ObjAsm64 understands 2 virtual tables. The first is called Interface Method Table (IMT) and the second simply Virtual Method Table (VMT). Both together are called Dual Method Table (DMT). The first is a regular table, as known from other implementations, but the second is immediately before the first and therefore has negative offsets from the common entry point. This implementation with positive and negative offsets in the DMT allows the separation of public interfaces such as COM methods from object private virtual methods.

Object metadata (OMD) chain and layout:
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
                          Object Metadata (OMD) #1                #2    ...    #n
  ??1stOMD   ——>   (+00h) POINTER to next OMD sibling      ——>    ...  ——> NULL = last OMD
                   (+08h) sizeof(TPL) (DWORD)                     ...          ...
                   (+0Dh) Alignment filler (DWORD = 0)            ...          ...
                   (+10h) Object Template (TPL)                   ...          ...
                   (+vvh) Object Dual Method Table (DMT)          ...          ...
                   (+xxh) Object class ID (DWORD)                 ...          ...
                   (+yyh) Ancestor class ID (DWORD)               ...          ...
  ——> ETT start:   (+zzh) # of following ??EventEntry (DWORD)     ...          ...
|    Event               ??EventEntry #1                         ...          ...
|    Translation         ??EventEntry #2                         ...          ...
|    Table               ??EventEntry #3                         ...          ...
|                        ...                                     ...          ...
|
|  Object structures:
|  ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
|  Object Template (TPL)
|    (+00h) POINTER to the Dual Method Table (DMT) —————————
|    (+08h) Variables and dynamic methods                   |
|          ...                                              |
|  Dual Method Table (DMT)                                  |  Offset
|    (-20h) Virtual method 3     (...)                      |    -
|    (-18h) Virtual method 2     (...)                      |    |
|    (-10h) Virtual method 1     (Done destructor)          |    |  Virtual Method Table (VMT)
  ——— (-08h) POINTER to Event Translation Table (ETT)        |    |
ENTRY (+00h) Interface method 1   (QueryInterface method) <——'    0 Dual Method Table (DMT)
      (+08h) Interface method 2   (AddRef method)                 |
      (+10h) Interface method 3   (Release method)                |  Interface Method Table (IMT)
             ...                                                  |
                                                                  +


As can be seen, all new instances of an object share the same DMT. This means that calling a virtual method from each instance uses the same code. There are situations where customization of a single instance is required. For this case, dynamic methods were created. The execution pointer is now stored not in the DMT but in the object instance itself. ObjAsm64 handles this complexity transparently for the coder. The definition of the method type is done with keywords in the description of the objects.

Object Sample, SampleID, Primer
  VirtualMethod     Method1
  DynamicMethod     Method2
ObjectEnd


A regular object definition contains an ID to identify the instance at runtime and the name of its ancestor.
If we remember, an object is nothing more than a structure. This structure can be extended by additional fields to create a new structure with more information. The ancestor plays the role of the original structure and the object definition the way this structure needs to be expanded. The same applies to the DMT. In this way, the complete data and functions are passed to the new object.

The object data is defined with the keyword DefineVariable, followed by a name, a type and an initial value.

Object Sample, SampleID, Primer
  VirtualMethod     Method1
  DynamicMethod     Method2

  DefineVariable    MyVar,     DWORD,   123
ObjectEnd


Since objects are structures, complete other objects can be embedded like variables. ObjAsm64 does that with the keyword Embed:

Object Sample, SampleID, Primer
  VirtualMethod     Method1
  DynamicMethod     Method2

  DefineVariable    MyVar,     DWORD,   123
  Embed             MyObject   
ObjectEnd


In this example, MyObject must be previously defined, so that the assembler knows about it.

The method invocation is done using a macro called OCall. It has the following syntax:

OCall <instance>::<object name>.<method name>, <argument list>

OCall detects instance pointers as well as local names. In this case, it performs the necessary transformations for the method call. The target is the method associated with the object, which is declared as follows:

Method <object name>.<method name>, <options>, <argument list>
  SetObject <register>
  ...
MethodEnd


To save typing work, the passed instance pointer called pSelf is hidden by the method macro. Its type matches the type of the object. Most of the cases, a register is required to access an object member. The job of SetObject is to load this register with the value of pSelf and execute an the ASSUME directive.
MethodEnd restores all ASSUME directives and perform a ret instruction. A macro local label called EOM (end of method) is included to perform a clean method exit from elsewhere in the method code.

It is often necessary to call the code of the ancestor object, in particular when an object instance is initialized. In this case ObjAsm64 calls an ancestor using the ancestors DMT or template using using ACall:

ACall <instance>::<object name>.<method name>, <argument list>

A complete bypass of the DMT is also possible using a direct call (DCall) with exactly the same syntax.
ObjAsm64 also knows a sort of invocation for interfaces like those used by COM or ObjAsm64 itself called ICall. The syntax is also identical to OCall, ACall and DCall.

ICall <instance>::<object name>.<method name>, <argument list>

In a multithread environment, sometimes it is necessary grand exclusive access to a specific execution path to only one thread. ObjAsm64 requires 2 elements for this type of synchronization: an opaque object variable called OBJECT_LOCK and a special calling mechanism called xOCall or xACall. The prepended "x" stands for exclusive.

Regards, Biterider
Title: Re: ObjAsm64
Post by: Biterider on February 17, 2018, 10:10:20 PM
Hi
There is an important feature hidden behind the SysInit and SysDone macros, the run-time initialization and shutdown.
In some cases, objects may require a kind of initial setup before we start working with them. In addition to some basic tasks, such as application heap management, the SysInit macro goes through all declared object templates and calls a static method called Startup. Objects that do not define this method are simply skipped.
For example, you are in the situation where you have many instances of a MyWindow object in your application, but you typically register the window class only once. In this case, the MyWindow.Startup method is called at the very beginning when SysInit is called to perform the registration. The method must be declared in the object definition and has no arguments. The same applies to the Shutdown method.

Object MyWindow, MyWindowID, WinPrimer
StaticMethod      Startup
StaticMethod      Shutdown
...
ObjectEnd

...

Method MyWindow.Startup
local WC:WNDCLASSEX

mov WC.cbSize, sizeof(WNDCLASSEX)
...
invoke RegisterClassEx, addr WC
MethodEnd


When the application shuts down, the reverse scenario may occur. In this case, SysDone simply calls MyWindow.Shutdown.
At this point, you're probably wondering which object instance we're working on. The answer, as mentioned above, is the object template. This fact is important because it has consequences. Only objects created after the SysInit call are initialized properly!

Regards, Biterider
Title: Re: ObjAsm64
Post by: Biterider on February 25, 2018, 10:33:11 PM
Hi
I am often in the situation that I have to visualize or export data. In most cases, the tool of my choice is EXCEL.
To facilitate the EXCEL programming, the way to go is using COM. EXCEL uses various standard interfaces and very intensively dispatch interfaces. I used the EXCEL version 10.0 definitions and translated the core elements with ComView. The rest was manually polished and put into Excel.inc and Excelc.inc.
For convenience, I created an object called ExcelHost that handles the most common tasks, such as opening, closing, data exchange, charting, and so on. This object can easily be extended to handle formulas, macros, pivot tables, and so on. It looks this way:

Object ExcelHost, ExcelHostID, Primer
  StaticMethod      CloseBook,                POINTER
  StaticMethod      DeleteSheet,              POINTER
  RedefineMethod    Done
  StaticMethod      GetCell,                  POINTER, DWORD, DWORD, POINTER, DWORD
  StaticMethod      GetRange,                 POINTER, BSTR
  StaticMethod      GetSheet,                 POINTER, BSTR
  StaticMethod      GetSheetName,             POINTER
  RedefineMethod    Init,                     POINTER
  StaticMethod      NewBook
  StaticMethod      NewChart,                 DWORD, DWORD, BSTR
  StaticMethod      NewSheet,                 POINTER, BSTR
  StaticMethod      Open,                     BSTR
  StaticMethod      SaveAs,                   POINTER, BSTR
  StaticMethod      SelectRange,              POINTER
  StaticMethod      SetCell,                  POINTER, DWORD, DWORD, BSTR
  StaticMethod      SetColumnWidth,           POINTER, BSTR, REAL4

  StaticMethod      GetBorder,                POINTER
  StaticMethod      SetBorderAttr,            POINTER, DWORD, DWORD, DWORD

  StaticMethod      GetFont,                  POINTER
  StaticMethod      SetFontAttr,              POINTER, BSTR, DWORD, DWORD, DWORD, DWORD

  StaticMethod      GetInterior,              POINTER
  StaticMethod      SetInteriorAttr,          POINTER, DWORD, DWORD, DWORD

  StaticMethod      SetChartData,             POINTER, POINTER, DWORD
  StaticMethod      HasChartTitle,            POINTER, DWORD
  StaticMethod      GetChartTitle,            POINTER
  StaticMethod      SetChartTitleText,        POINTER, BSTR
  StaticMethod      GetChartAxis,             POINTER, DWORD, DWORD
  StaticMethod      SetChartAxisAttr,         POINTER, DWORD, DWORD, DWORD, DWORD
  StaticMethod      SetChartAxisValues,       POINTER, DWORD, REAL8, DWORD, REAL8
  StaticMethod      SetChartAxisTicks,        POINTER, DWORD, DWORD, REAL8, DWORD, DWORD, REAL8
  StaticMethod      SetChartAxisGridlines,    POINTER, DWORD, DWORD, DWORD, DWORD, DWORD
  StaticMethod      GetChartLegend,           POINTER
  StaticMethod      GetChartArea,             POINTER
  StaticMethod      GetChartPlotArea,         POINTER
  StaticMethod      GetChartSeries,           POINTER, DWORD
  StaticMethod      SetChartSeriesAttr,       POINTER, DWORD, DWORD, DWORD, DWORD, DWORD, DWORD
  StaticMethod      SetChartSeriesData,       POINTER, BSTR, POINTER, POINTER

  StaticMethod      SetRangeAlignHor,         POINTER, DWORD
  StaticMethod      SetRangeAlignVer,         POINTER, DWORD
  StaticMethod      GetRangeBorder,           POINTER, DWORD
  StaticMethod      GetRangeArray,            POINTER, POINTER
  StaticMethod      CreateArray,              DWORD, DWORD, POINTER

  StaticMethod      SetRowHeight,             POINTER, BSTR, REAL4
  StaticMethod      SetSavedFlag,             POINTER, DWORD
  StaticMethod      SetSheetName,             POINTER, BSTR
  StaticMethod      SetVisible,               DWORD
  StaticMethod      Quit

  DefineVariable    pIExcelApp,               POINTER,  NULL
ObjectEnd


As you can see above, the strings are BStrings as used in COM automation. ObjAsm provides all the necessary tools in the ObjMem64 library and BString.inc macros to handle these types of strings.
Attached is an executable that shows some of the possibilities using ExcelHost. There, you will also find the Excel interface definitions.
As a side note, I had to change the name of the interface definition macros to avoid conflict with other macros coming from other packages. The new format looks like this:


COM_INTERFACE_BEG IExcelFont, IDispatch, <0002084D-0001-0000-C000-000000000046>
COM_MTD_STD  get_Application, ptr ptr Application
...
COM_INTERFACE_END


The start of the interface is signaled with the macro COM_INTERFACE_BEG. It requires as argument a name, a predecessor interface and a GUID. It is convenient to include this last one here to have all relevant information together. The following lines specify the methods contained in the order of occurrence. These can be standard (COM_MTD_STD) or dispatch methods (COM_MTD_DSP). The end of the interface definition is signaled with the macro COM_INTERFACE_END.
If you want to use the interface definitions for your own implementation, you can easily change the above macros. It should be a simple task for a MASM coder   :biggrin:

Regards, Biterider
Title: Re: ObjAsm64
Post by: Biterider on March 05, 2018, 06:20:02 AM
Hi
In recent weeks we have seen customization examples for the classic GetOpenFileName, GetSaveFileName and co. APIs.
There is, however, an alternative that MS recommends. COM programmers designed a way to reduce the complexity of the customization. The core idea is to get access to the file dialogs and their customization using COM interfaces.
In the following example, I only examined the IFileOpenDialog interface defined in the ShObjIDL.inc file. There are interfaces for IFileSaveDialog, IFileOperationProgressSink and many more. I will complete the translation in the next few days, but for this example, it contains all the necessary definitions.
The following code snippet shows how to redefine the Open File dialog box.

COMDLG_FILTERSPEC struc
    pszName LPCWSTR     ?
    pszSpec     LPCWSTR ?
COMDLG_FILTERSPEC ends

MultiFileSpec struc
  FS0 COMDLG_FILTERSPEC {}
  FS1 COMDLG_FILTERSPEC {}
  FS2 COMDLG_FILTERSPEC {}
  FS3 COMDLG_FILTERSPEC {}
MultiFileSpec ends

Method Application.OnCommand, uses rbx rsi, wParam:WPARAM, lParam:LPARAM
    local pIFODlg: POINTER, MFS:MultiFileSpec
    local pISIArr:POINTER, dCount:DWORD, pIShellItem:POINTER, pDisplayName:POINTER

    SetObject rsi
    mov rax, wParam
    .if ax == IDM_OPEN
      invoke CoCreateInstance, offset CLSID_FileOpenDialog, NULL, CLSCTX_INPROC_SERVER, \
                               offset IID_IFileOpenDialog, addr pIFODlg
     
      m2m MFS.FS0.pszName, $OfsCStr("All")
      m2m MFS.FS0.pszSpec, $OfsCStr("*.*")
      m2m MFS.FS1.pszName, $OfsCStr("Assembly")
      m2m MFS.FS1.pszSpec, $OfsCStr("*.asm")
      m2m MFS.FS2.pszName, $OfsCStr("Include")
      m2m MFS.FS2.pszSpec, $OfsCStr("*.inc")
      m2m MFS.FS3.pszName, $OfsCStr("Text")
      m2m MFS.FS3.pszSpec, $OfsCStr("*.txt")

      ICall pIFODlg::IFileOpenDialog.SetFileTypes, MultiFileSpec/COMDLG_FILTERSPEC, addr MFS
      ICall pIFODlg::IFileOpenDialog.SetFileTypeIndex, 2
      ICall pIFODlg::IFileOpenDialog.SetOptions, FOS_ALLOWMULTISELECT
      ICall pIFODlg::IFileOpenDialog.SetTitle, $OfsCStr("Select a file...")
      ICall pIFODlg::IFileOpenDialog.SetOkButtonLabel, $OfsCStr("Select")
      ICall pIFODlg::IFileOpenDialog.SetFileNameLabel, $OfsCStr("Selected file name: ")
      ICall pIFODlg::IFileOpenDialog.Show, [rsi].hWnd
      .if SUCCEEDED(eax)
        ICall pIFODlg::IFileOpenDialog.GetResults, addr pISIArr
        ICall pISIArr::IShellItemArray.GetCount, addr dCount
        xor ebx, ebx
        .while ebx < dCount
          ICall pISIArr::IShellItemArray.GetItemAt, ebx, addr pIShellItem
          ICall pIShellItem::IShellItem.GetDisplayName, SIGDN_NORMALDISPLAY, addr pDisplayName
          mov rax, pDisplayName
          DbgStr rax
          inc ebx
        .endw
      .endif

      ...


                                 
The attachment contains the executable version of the demo.

Regards, Biterider
Title: Re: ObjAsm64
Post by: Biterider on March 11, 2018, 09:04:55 PM
Hi

QuoteSoftware that is capable of having a skin applied is referred to as being skinnable, and the process of writing or applying such a skin is known as skinning. Applying a skin changes a piece of software's look and feel—some skins merely make the program more aesthetically pleasing, but others can rearrange elements of the interface, potentially making the program easier to use.
https://en.wikipedia.org/wiki/Skin_(computing) (https://en.wikipedia.org/wiki/Skin_(computing))

One of the best known skinnable software is Windows Media Player.

I would like to introduce a basic approach of a skinned application using ObjAsm64.
I've integrated some elements, such as buttons, dialogs, gifplayer, tooltip, XMenu, etc. to create this application. The largest application payload are of course the bitmap resources.

Note:
So far, most of the old objects in the repository are running as their 64-bit counterparts.
In the coming weeks, I will focus on troubleshooting to release the first OA64 beta.
I hope for feedback when someone finds a bug.  :t


Regards, Biterider
Title: Re: ObjAsm64
Post by: jj2007 on March 11, 2018, 09:57:23 PM
Very, very cute :t
Title: Re: ObjAsm64
Post by: Biterider on March 19, 2018, 03:13:21 AM
Hi

While I was finishing some functional parts of the ObjAsm64 framework, I came to Object Explorer. This application is a very efficient way to quickly show important information about objects. This information is read using PCRE from the source code by using the structured information ahead object, method or procedure definitions. Whole directories are scanned to get this information. Knowing how to read these definitions, a whole tree of relationships is created that allows us to visually browse through parent and descendant objects.

The Object Explorer application is divided into two panels. On the left side, a tree (XTreeView) displays all object dependencies. On the right side, an OCX_Container manages a Web Browser instance that is fed with HTML code to represent the previously collected information of the shown object selected on the left side. This information panel displays first the object header information followed by the object inheritance path. Thereafter, all object related files are displayed. The icons on the left (Actions) can be used to start the File Explorer or the standard editor. To create or update a precompiled object, a special additional button is provided.

Finally, 2 sections are shown: Methods and Variables.

Methods: this section displays all method members of the current object. In gray, you can see the inherited methods, while in black you can see the object new defined and implemented methods. A plus sign in front of each line can be used to popup the method heading information.

Variables: this section displays in a similar way variable members, their type and initial value (template value). In case of an embedded object, the icon in front of the line changes to indicate this and you can navigate to the corresponding object by clicking on "Type".

Since I haven't found the time to port the last PCRE 8.41 to x64, Object Explorer remains at the moment a 32 bit application.

Regards, Biterider
Title: Re: ObjAsm64
Post by: fearless on March 19, 2018, 05:57:59 AM
looks pretty good.

Just in case you didnt notice but the links on ObjAsm64 dont point to valid files:

QuoteNot Found The requested URL /DwnFiles/ObjAsm64.zip was not found on this server.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
Title: Re: ObjAsm64
Post by: HSE on March 19, 2018, 11:03:34 AM
Is posible to change colors?
Title: Re: ObjAsm64
Post by: Biterider on March 19, 2018, 05:56:38 PM
Hi
@ fearless: Thank you. I am aware that the files are missing. They will be available for download in a few days. I'll inform accordingly.

@ HSE: Currently, no such feature is implemented, but I can if necessary.

Biterider

Title: Re: ObjAsm64
Post by: Biterider on March 26, 2018, 02:35:22 AM
Hi
The TreeView is one of the most useful controls of Windows.
But it is also known that the customization is not easy time consuming. Therefore, some time ago, I wrote this control from the ground up with the goal of providing a higher level of flexibility. It has become a difficult task as the code has become very long. But I have used this for debugging the x64 migration.
Fortunately I managed to revive the control and in fact a few bugs showed up.  :redface:

This will be the last code before the ObjAsm64 release in the next few days.  ;)

Regards, Biterider

Title: PCRE - RegEx
Post by: Biterider on March 29, 2018, 06:01:57 PM
Hi
I would like to draw attention to a really great project "PCRE", which stands for "Perl Compatible Regular Expressions". It can be found here https://www.pcre.org/ (https://www.pcre.org/). I've used it in the past to create RegEx objects to parse text. At the time of the ObjAsm32 there was only the ANSI version, but in the current version this restriction has been removed.  :t

I've been able to compile the current version (8.41) of the project and create a static library out of the 3 available character widths (8, 16 and 32 bits). I also translated a complete .inc file to use with the .lib. Unfortunately, this .zip file containing the files is too big to post here (~ 1.3 MB). Therefore it can be downloaded from here http://objasm.x10host.com/DwnFiles/PCRE_8.41.zip (http://objasm.x10host.com/DwnFiles/PCRE_8.41.zip). In this .zip file contains the 3 single .lib, the merged .lib, the .inc file and 2 ObjAsm64 RegEx objects.

Since Unicode characters can now also be processed, I have created two variants of the RegEx object RegExA and RegExW for the ObjAsm64 project. These can be loaded neutral with the character width.

Regards, Biterider

Title: Re: ObjAsm64
Post by: Biterider on April 02, 2018, 04:56:32 PM
Hi
Today I want to announce the first beta release of ObjAsm64   :P

The complete package can be downloaded from the homepage: http://objasm.x10host.com (http://objasm.x10host.com)
I put the installation instructions here: http://objasm.x10host.com/DwnFiles/Instructions.txt (http://objasm.x10host.com/DwnFiles/Instructions.txt)

Since the code certainly contains some bugs, I ask kindly for feedback  :t

Regards, Biterider
Title: ObjAsm64 learns Exception Handling
Post by: Biterider on April 07, 2018, 05:16:30 AM
Hi
Exception handling in x64 world is a little different than in the x86 world.
The topic is described here by Microsoft https://msdn.microsoft.com/en-us/library/1eyas8tf.aspx (https://msdn.microsoft.com/en-us/library/1eyas8tf.aspx). I found an article that discusses the topic very well: http://www.nynaeve.net/? p = 113 (http://www.nynaeve.net/?%20p%20=%20113). There are also some contributions to "Code Project" that are worth reading.

Essentially for the normal assembler user, exception handling is reduced to using some macros and a language-specific handler. The macro names are similar to the C syntax: __TRY, __EXCEPT, & __FINALLY. These macros perform the basic bookkeeping of the protected code areas. The exception handler performs the stack unrolling and forwarding to the code in the save area after an exception. This handler can be customized for any procedure or globally for all procedures. In ObjAsm64, the latter is called EHandler and is placed in the ObjMem64 library.

The exception handler is always specified after the FAME keyword of a procedure.

Basically a protected code area looks like:

start proc FRAME:EHandler
  SysInit
  DbgClear

__TRY
  xor eax, eax
  mov rcx, [rax]

__EXCEPT
  DbgWarning "Exception"
  DbgHex ExceptRecord.ExceptionCode
  DbgHex ExceptRecord.ExceptionAddress

__FINALLY
  DbgLine2

  SysDone
  invoke ExitProcess, 0
start endp

The instruction mov rcx, [rax] raises an exception and causes the execution of the code after the macro __EXCEPT. The code following the __FINALLY macro is always executed.
The code save area can be completely skipped. In this case, after the exception, the code will continue after __FINALLY.

This little exception framework is not limited to ObjAsm64. It can also be used for any normal asm code.  8)

Regards, Biterider
Title: ObjAsm64 quality tool
Post by: Biterider on April 15, 2018, 12:04:51 AM
Hi
I've been working on a revival of an old tool to improve the coding quality.

Windows x64 ABI is very clear and indicates what volatile and non-volatile registers are. Preserving the non-volatile registers is important as it can lead to random crashes that are hard to find when calling routines from the OS. It is a good practice to adopt the same rules for your own code as it will improve the robustness of your application. Because the underlying operating system code has changed from version to version, non-compliant code may have worked for an older operating system version, but it suddenly stops working with a newer version of it.

I worked with this quality tool to check my own code for common mistakes. Using a variation of the same scanning algorithm, we are able to detect the oposit, an unnecessary register preservation. Unused locals can also be detected to unclutter the code. The quality tool works for OA64 methods and also for x64 procedures.

Since the execution is relatively fast, I implemented a batch mode to scan entire libraries or repositories searching for the above mentioned situations.

Now, there are situations where you deliberately do not want to follow the Windows x64 ABI specification. In these cases, to avoid an unnecessary error or warning message, you can insert a hint for the tool immediately after the method or procedure declaration. The hint is a simple macro that does not generate any code. It is a simple annotation in the code read by the quality tool to get additional information about the intention of the coder.
This quality tool is by far not ready, but it can be used in its present form. I'll keep working on it as it is required.

Note: this version is intended only for x64 code, but it is a 32 bit application  :icon_exclaim:

Regards, Biterider
Title: Re: ObjAsm64
Post by: HSE on April 15, 2018, 12:29:26 AM
Hi Biterider!!

Between today and tomorrow I will try to install the package!  :t

But in ObjAsm32 the tool have a little problem because requiere entire procedures in same file, and I have a lot of procedures that result from including several files.

In theory is not very difficult to solve that, but just now I'm making little things with parsers.

Regards.
Title: Re: ObjAsm64
Post by: Biterider on April 15, 2018, 12:59:37 AM
Hi HSE
I am glad to hear that you are also involved with OA64.  :t
Please let me know how it went with the installation.


Regards, Biterider
Title: Re: ObjAsm64
Post by: HSE on April 15, 2018, 03:46:35 AM
Here in 7-64.

No problem with ObjMem64.
Some errors assembling objects.

Title: Re: ObjAsm64
Post by: Biterider on April 15, 2018, 03:58:53 AM
Hi HSE
Please try this new set of files (72 objects).

Biterider
Title: Re: ObjAsm64
Post by: HSE on April 15, 2018, 05:04:33 AM
Some minor problems:

CvtRes is called without path (evidently x32 version  is founded first), but that will requiere to change some .bat

I forget the Perl thing (it's in the book but not in the instructions.txt)


Title: Re: ObjAsm64
Post by: qWord on April 15, 2018, 09:53:21 AM
Great!

did work after some manual corrections in the WinInc1 and Objects2 headers. Only Demo06 can't be build because of missing HtmlHelp.lib. The conflict definition of IUnknown and IClassFactory does IMO show that Interfaces and Objects should be distinguished declaratively ;-)

Kind regards

qWord


[1] conflict with IClassFactory; CONNECTDATA::pUnk; some pointer types removed
[2] ANNOTATE-macro missing; some paths in the D3*-files

Title: Re: ObjAsm64
Post by: Biterider on April 15, 2018, 06:27:06 PM
Hi guys
Thank you very much for your feedback. Without it, I'll never improve the code.  :t

@HSE: first post:
- all references to GDI.inc removed.
- IClassFactory redefinition: solved --> needs a general better solution.
- CONNECTDATA symbol problem: solved. Server needs a cookie management function.
- IShellFolder conflict: solved --> needs a general better solution.
- WordHostmissing files: please install the Code\Inc and Code\Lib directories.

@HSE: second post:
- D3Engine.err "% include &ObjPath&D3Math.inc" resolution: no clue atm.
- Missing ANNOTATE macro: included now. Name changed to ANNOTATION.
- Graph2D is not ready by now. Demo07 will come soon.
- PCRE841S.inc missing: added.

@qWord:
- HtmlHelp.lib issue in Demo06: 64 bit version of HtmlHelp is included Code\Inc and Code\Lib directories. I renamed the old Help\ObjAsm32.chm file to ObjAsm64.chm to have something to display when help is called.

I updated following files on the Homepage (http://objasm.x10host.com):

Note: the most problematic issue are the api include files. There are different works out there, but none of them is consistent and complete. That is the reason of some incompatibilities that I tried to patch manually until I noticed that a complete different approach is needed. I asked the author of h2incX to release the sources and he generously did it. I'm working now on the translator, which I think is the next needed step.

Regards, Biterider
Title: Re: ObjAsm64
Post by: HSE on April 16, 2018, 10:12:55 AM
 :t :t :t

Almost nothing: in ExceptionDemo.asm msvcrt and kernel32  paths are hardcoded.
Title: Re: ObjAsm64
Post by: Biterider on April 17, 2018, 03:48:51 AM
Thanks HSE  :t
I corrected the paths.


Regards, Biterider
Title: h2incX64
Post by: Biterider on April 22, 2018, 04:51:56 PM
Hello
The last few days I've been working on Japheth's h2incX code http://masm32.com/board/index.php?topic=7006.msg75149#msg75149 (http://masm32.com/board/index.php?topic=7006.msg75149#msg75149).
I found some memory management issues that I could solve. Looking at the code, it becomes very clear that it was developed for 32-bit applications, so a mayor revamping of the application seems to be necessary.
The goal is to make something like "Windows.inc" or "WinInc.inc" projects but for 64-bit.

The reason I'm writing here is to ask if anyone has used this tool before and is willing to help with this project, especially when it comes to some difficult C language interpretation.  ;)

Regards, Biterider
Title: Re: ObjAsm64
Post by: Biterider on April 28, 2018, 09:38:29 PM
Hello
I'm making some progress in the h2incX project. I hope it is not pointless work, because so far no much interest was signaled.
Nevertheless, I will continue to work on the tool.  :P

I can confirm that reading a code is much more difficult than writing your own. At this rate, I think I'll need another week to release the first beta.

Regards, Biterider
Title: Re: ObjAsm64
Post by: LiaoMi on April 28, 2018, 11:07:27 PM
Hi Biterider,

probably the best development option would be the testing path on complex files, we can compare the results, finalizing the conversion in stages. You can contact ToutEnMasm, he has a very cool converter, but also not perfect, I attached the last version below, its unfortunate that there is no source code, but at least you can compare the results  :icon_rolleyes:
Title: Re: ObjAsm64
Post by: GoneFishing on April 29, 2018, 01:43:27 AM
Hi Biterider,

It is not pointless work.
As for me, I need H2INCX for Linux.  It's a real lot of work to port it to another platform. I'm not sure if I'll be able to accomplish the task  properly. First I'm going to make it  just "Linux - compilable" 







     







Title: Re: ObjAsm64
Post by: Biterider on May 06, 2018, 03:14:33 PM
Hi
Thank you for your feedback!  :t
@ LiaoMi: I'm starting to work with the header files of Windows Kits\10\Include\10.0.16299.0\. I also looked at the Translatorus. Without the sources, it is hard to say how it works, but from the setup file, I think it's strongly based on table lookups. I found another tool on the web "xlatHinc" by J. Radburn. Unfortunately, none of these tools works perfectly.

@ GoneFishing: I never coded for Linux, but I'm really tempted to do. For the time being, I will stay on Windows to finish this project.  :biggrin:

I've the impression that it will be very difficult to program a perfect tool that can handle all cases of the C/C++ syntax.
I wonder how the WinInc project was done. I'm my opinion, some parts have been manually fixed.


Regards, Biterider
Title: Re: ObjAsm64
Post by: habran on May 07, 2018, 11:00:54 AM
Hi Biterider,

I was using my mdi64.exe to fix headers pointers with asterix "*" to INT_PTR before I would use x2incx.exe
to convert file. here is subroutine for that:

FixPointers proc FRAME USES rsi rdi rbx RawText:LPSTR, FixedText:LPSTR,pszFileName :LPCTSTR, fSize:DWORD
local szName [MAX_PATH] :BYTE
local szVar [MAX_PATH] :BYTE
mov rsi,rcx
mov rdi,rdx
mov rbx,rcx
add rbx,r9

.while rsi < rbx
;search for "proto" and transform data
     xor rcx,rcx
     mov r8,rsi
.while BYTE PTR[r8]!= 10
next: mov     al,[r8]
.if   al =='('
            jmp   transfer
            .elseif     al =='*'
            jmp   found
            .endif
            inc     ecx
            inc     r8
            .if r8>rbx
            jmp finish
            .endif
        .endw
found:
.if (BYTE PTR [r8]== '*' && BYTE PTR [r8-1] == '/')
.repeat
.if (BYTE PTR[r8] == 0)
jmp finish
.endif
mov al,[r8]
inc r8
.until (al == '/' && BYTE PTR[r8-2] == '*')
mov rsi,r8
jmp next
.endif
.if ecx < 32
   .if BYTE PTR [r8-1] == ' ' || BYTE PTR [r8-1] == 9
    mov WORD PTR[rdi],909h
    add rdi,2
    mov rax," RTP_TNI"
    mov [rdi],rax
    add rdi,8
    mov rsi,r8
    inc rsi
   .endif
.endif
transfer:
.repeat
mov al,[rsi]
mov [rdi],al
inc rsi
inc rdi
.until al == 10
.endw
finish:
mov byte ptr[rdi],0
xor rax,rax
ret
FixPointers endp

Than after conversion to .inc with h2incx.exe, I was using another routine to convert to x64:

HeadersTo64bits proc FRAME USES rsi rdi rbx r12 RawText:LPSTR, FixedText:LPSTR,pszFileName :LPCTSTR, fSize:DWORD
local szName [MAX_PATH] :BYTE
local szVar [MAX_PATH] :BYTE
mov rsi,rcx
mov rdi,rdx
mov r12,rcx
add r12,r9

invoke lstrcpy,addr szName, pszFileName
invoke PathRemoveExtension,addr szName
invoke PathStripPath, addr szName
invoke CharUpper,addr szName
invoke lstrcat,addr szName, addr szApi
invoke lstrcpy,ADDR szVar,ADDR szWin
invoke lstrcat,ADDR szVar,ADDR szName
.while rsi < r12
;search for "proto" and transform data

.while BYTE PTR[rsi]!= "p"
next: mov     al,[rsi]
            mov     [rdi],al   
        inc     rsi
            inc     rdi
            .if rsi>r12
            jmp finish
            .endif
        .endw
.if dword ptr[rsi+1]=="otor" && byte ptr[rsi-1]==10
add     rsi,5
invoke wsprintf,rdi,addr szDef
add rdi,rax
invoke wsprintf,rdi,addr szVar
add rdi,rax
.if byte ptr[rsi]=="_"
inc rsi
.endif
.while byte ptr[rsi] != " "
mov al,[rsi]
mov [rdi],al
inc rdi
inc rsi
.endw
mov byte ptr[rdi],","
inc rdi
mov al,[rsi]
mov [rdi],al
inc rsi
inc rdi
invoke wsprintf,rdi,addr szStdcall
add rdi,rax
.while byte ptr[rsi]!= 13
mov al,[rsi]
inc rsi
.if al==":"
jmp vars
.endif
.endw
;dec rsi
jmp endline
vars: invoke wsprintf,rdi,addr szComas
add rdi,rax
dec rsi
xor rdx,rdx
.while byte ptr[rsi]!= 13
mov al,[rsi]
.if al==":"
inc rdx
.endif
mov [rdi],al
inc rsi
inc rdi
.endw
mov byte ptr[rdi],">"
inc rdi
mov byte ptr[rdi],","
inc rdi
shl rdx,2
invoke wsprintf,rdi,CSTR("%i"), rdx
add rdi,rax
mov al,13
endline: mov [rdi],al
inc rdi
inc rsi
mov al,[rsi]
mov [rdi],al
inc rsi
inc rdi
mov rax,"ednretxe"
.if qword ptr[rsi]==rax
.while byte ptr [rsi] != 10
inc rsi
.endw
inc rsi
.while byte ptr [rsi] != 10
inc rsi
.endw
inc rsi
.endif
.else
jmp     next
.endif
.endw
finish:
mov byte ptr[rdi],0
xor rax,rax
ret
HeadersTo64bits endp

where .data is:

szDef db '@DefProto ',0
szStdcall db 'stdcall',0
szComas db ', , <',0
szApi db 'API, ',0
szWin db 'WIN',0


I hope it will give you some clue how to do that
I was planing to work on h2incx source to make it work, as well as use some C/C++ compiler to create perfect .inc files, however, at this moment I have taken some brake from UASM and programming for several months.
I would appreciate if you can upgrade h2incx

best regards

Title: Re: ObjAsm64
Post by: Biterider on May 07, 2018, 09:22:36 PM
Hi Habran
Yesterday I got almost all working again.  :P

The sources we recieved from Japheth don't generate the .inc files from the WinInc project. I assume the author has made some changes or it is a previous version. Nonetheless. I have ported the code from a 16-bit pseudo-object coding style to ObjAsm32. I also implemented code indentation, which makes the translated header file much easier to read.
The translation of COM interfaces is not ready. There I have a problem, because there are many syntax styles for asm. Which one should I take?

I also solved the problem of prototype annotations, at least for the moment.

I need a different approach for known structures, macros, etc. The .ini file is becoming too big!

Prototypes from heapapi.h looks like
HeapCreate proto :DWORD, :SIZE_T, :SIZE_T
HeapDestroy proto :HANDLE
HeapAlloc proto :HANDLE, :DWORD, :SIZE_T
HeapReAlloc proto :HANDLE, :DWORD, :LPVOID, :SIZE_T
HeapFree proto :HANDLE, :DWORD, :LPVOID
HeapSize proto :HANDLE, :DWORD, :LPCVOID
GetProcessHeap proto
HeapCompact proto :HANDLE, :DWORD
HeapSetInformation proto :HANDLE, :HEAP_INFORMATION_CLASS, :PVOID, :SIZE_T


Should they be in this format?

The other problem that is easier to fix is the reserved words from UASM. Do you have a list that you can provide?

Regards, Biterider

PS: If desired, I can post the current sources.
Title: Re: ObjAsm64
Post by: jj2007 on May 08, 2018, 01:05:22 AM
Are they supposed to work with both 32-bit and 64-bit code, with SIZE_T being 32 bits wide for the former and 64 bits for the latter? I am asking because Google spit out a hilarious conversation on what SIZE_T means on SOF (https://stackoverflow.com/questions/918787/whats-sizeofsize-t-on-32-bit-vs-the-various-64-bit-data-models) 8)
Title: Re: ObjAsm64
Post by: Biterider on May 08, 2018, 02:58:18 AM
Hi JJ
don't blame me, I'm just the translator!  ;)

Biterider
Title: Re: ObjAsm64
Post by: habran on May 08, 2018, 04:49:12 AM
If you look in Winbase.h in WinInc you will see how it suppose to be to work in both 32 and 64 bit:

@DefProto WINBASEAPI, HeapCreate, stdcall, ,     <:DWORD, :SIZE_T, :SIZE_T>, 12
@DefProto WINBASEAPI, HeapDestroy, stdcall, ,    <:HANDLE>, 4
@DefProto WINBASEAPI, HeapAlloc, stdcall, ,      <:HANDLE, :DWORD, :SIZE_T>, 12
@DefProto WINBASEAPI, HeapReAlloc, stdcall, ,    <:HANDLE, :DWORD, :LPVOID, :SIZE_T>, 16
@DefProto WINBASEAPI, HeapFree, stdcall, ,       <:HANDLE, :DWORD, :LPVOID>, 12
@DefProto WINBASEAPI, HeapSize, stdcall, ,       <:HANDLE, :DWORD, :LPCVOID>, 12
@DefProto WINBASEAPI, HeapValidate, stdcall, ,   <:HANDLE, :DWORD, :LPCVOID>, 12
@DefProto WINBASEAPI, HeapCompact, stdcall, ,    <:HANDLE, :DWORD>, 8
@DefProto WINBASEAPI, GetProcessHeap, stdcall, , <>, 0
@DefProto WINBASEAPI, GetProcessHeaps, stdcall, , <:DWORD, :PHANDLE>, 8
Title: Re: ObjAsm64
Post by: habran on May 08, 2018, 08:40:20 AM
This is defined in WINASM.INC:

;--- macro to define a prototype, either directly or by using
;--- an IAT entry

ifdef _WIN64
WINSTDCALLCONV equ <fastcall>
@DefProto macro apiqual:REQ, name_:REQ, type_, namesuffix, parms, suffix
;;echo defproto: apiqual
%ifidn <apiqual>,<__declspec ( dllimport )>
  proto_&name_ typedef proto parms
  externdef __imp_&name_: ptr proto_&name_
  name_&namesuffix equ <__imp_&name_>
else
  name_&namesuffix proto parms
endif
endm
else
WINSTDCALLCONV equ <stdcall>
@DefProto macro apiqual:REQ, name_:REQ, type_, namesuffix, parms, suffix
;;echo defproto: apiqual
%ifidn <apiqual>,<__declspec ( dllimport )>
  proto_&name_ typedef proto type_  parms
  ifnb <suffix>
    externdef stdcall _imp__&name_&@&suffix: ptr proto_&name_
    name_&namesuffix equ <_imp__&name_&@&suffix>
  else
    externdef c _imp__&name_: ptr proto_&name_
    name_&namesuffix equ <_imp__&name_>
  endif
else
  name_&namesuffix proto type_ parms
endif
endm
endif

;ifdef COBJMACROS
Title: Re: ObjAsm64
Post by: habran on May 08, 2018, 10:29:57 AM
As you can see WINSTDCALLCONV  is used to create stdcall or fastcall

WinMain proto WINSTDCALLCONV :HINSTANCE, :HINSTANCE, :LPSTR, :DWORD
Title: Re: ObjAsm64
Post by: Biterider on May 08, 2018, 03:52:44 PM
Hi habran
I see the convinience of using a macro for this task. I'll go this way!
What about the return value? As far as I know, UASM is/will be able to evaluate it.
Shouldn't we pass it as an additional parameter?

Biterider





Title: Re: ObjAsm64
Post by: habran on May 08, 2018, 07:52:43 PM
Don't bother ;)
That is up to programmer to decide.
The return value is usually in RAX register and we can chose AL, AX, EAX or RAX, depending on our need.
I am glad you liked the macro :t
I like typed parameters as it is now in WinInc, it is more readable :biggrin:
I am happy that You have undertaken that hard task, because I lake your approach to that task and I
trust in your programming skills 8)
Title: Re: ObjAsm64
Post by: Biterider on June 23, 2018, 10:16:09 PM
Hi
Good news!  :eusa_dance:
I have been able to build the first set of include files with the new h2incX (version 0.10).
I've been working on this project for the last few weeks to better support MASM, UASM and ObjAsm with the latest header files for Win10.
The next step is to set up the right switches in a Windows.inc prepended file, named WinAsm.inc, as it was in the original project.
In the analysis of the converted code, I came to the conclusion that creating a tool that works completely without human interaction will be very difficult, at least using this approach. The most important reason is that C works differently than MASM. In C you can create definitions based on things that come later. To understand this, I've expanded the functionality of h2incX to visualize the "Include Tree". There are many cases of symbols that are used before they are defined. It is easy to understand this using this pseudo graphical tree.

After setting the main switches, I have to work a bit more on the COM interfaces and the matching macros.

Biterider