Upcoming ObjAsm changes to support UEFI

[Biterider]

Hi
To support UEFI I will discuss some necessary changes and their impact. My goals are:
1. Existing projects should require little or no modification
2. UEFI support should leverage existing framework resources as much as possible
3. The complexity of the framework should not become overwhelming

The most efficient way to achieve these goals is to introduce a new target called TARGET_PLATFORM. It can be e.g. WINDOWS, UEFI, LINUX or NONE.
This way there is no need to build a completely separate support library. ObjMem.inc can choose the exposed procedures and aliases according to the selected platform.
The tool chain can be controlled in the same way. For UEFI projects, the target settings would be:
TARGET_PLATFORM = UEFI
TARGET_BIN_FORMAT = DLL
TARGET_BITNESS = 64
The linker would generate a 64-bit DLL and an additional step converts the image to EFI format.
Copying or installing the .efi file is up to the user.

The procedures involving memory management must be duplicated and modified to support UEFI memory management. These new procedures will be prefixed with "UEFI_" and will be referenced in ObjMem.inc by a well-known alias.
Unsolved are precompiled objects with modified memory management. I tend to leave them uncompiled and compile in the main project. Only Windows objects would then be precompiled.

There will also be a 32-bit UEFI version which will probably not be used but can be targeted.

From what I've seen, multiprocessing can be challenging, and synchronization primitives need to be written.

That's all for now. All comments are welcome.

Biterider

[HSE]

Hi Biterider!

TARGET_PLATFORM = UEFI
TARGET_BIN_FORMAT = DLL
TARGET_BITNESS = 64

If you don't change current default also could need :

Code Select Expand

TARGET_USER_INTERFACE = USER_INTERFACE_NONE

These new procedures will be prefixed with "UEFI_"

Prefix is a so bad idea that I think you say that to force participation  .

Personally I prefer postfixed with "E", because "U" remember me Unicode.

Unsolved are precompiled objects with modified memory management.

Anyway objects responsives to STR_TYPE are like that  .

Regards, HSE.

[Biterider]

Hi HSE
Thanks for thinking along.
There is a change under the hood regarding TARGET_USER_INTERFACE, the new options will be NONE, GUI & CLI. For UEFI I think CLI is best.

Prefix is a so bad idea that I think you say that to force participation  .

I'm serious about that 
Posfixing is currently used for A/W selection.
But it is not so bad. Consider the following test code in ObjMem.inc

Code Select Expand

StrAllocA proto :DWORD
StrAllocW proto :DWORD
UEFI_StrAllocA proto :DWORD
UEFI_StrAllocW proto :DWORD
if TARGET_PLATFORM eq PLATFORM_WINDOWS
  if TARGET_STR_TYPE eq STR_TYPE_ANSI
    StrAlloc textequ <StrAllocA>
  elseif TARGET_STR_TYPE eq STR_TYPE_WIDE
    StrAlloc textequ <StrAllocW>
  else
    echo Warning: incompatible TARGET_STR_TYPE
  endif
elseif TARGET_PLATFORM eq PLATFORM_UEFI
  if TARGET_STR_TYPE eq STR_TYPE_ANSI
    StrAlloc textequ <UEFI_StrAllocA>
  elseif TARGET_STR_TYPE eq STR_TYPE_WIDE
    StrAlloc textequ <UEFI_StrAllocW>
  else
    echo Warning: incompatible TARGET_STR_TYPE
  endif
endif

Nothing has changed from a programmer's point of view. The prefix is hidden in the alias.

Code Select Expand


; Procedure:  UEFI_StrAllocW
; Purpose:    Allocate space for a WIDE string with n characters.
; Arguments:  Arg1: Character count without the ZTC.
; Return:     rax -> New allocated WIDE string or NULL if failed.

align ALIGN_CODE
UEFI_StrAllocW proc dChars:DWORD
  local pBuffer:POINTER

  lea edx, [2*ecx + 2]                                  ;Make room for the ZTC
  mov r8, pBootServices
  invoke [r8].EFI_BOOT_SERVICES.AllocatePool, EFI_MEMORY_UC, edx, addr pBuffer    ;Not cacheable
  .if rax == EFI_SUCCESS
    mov rax, pBuffer
    m2z CHRW ptr [rax]                                  ;Set the ZTC
  .else
    xor eax, eax
  .endif
  ret
UEFI_StrAllocW endp


What worries me is that getting UEFI into the model is a lot of work, but nothing too complicated. It's just going to take some time.
Understanding UEFI is complex, but we'll get through it. 

Biterider

[HSE]

Hi Biterider!

Nothing has changed from a programmer's point of view. The prefix is hidden in the alias.

Nothing has changed from a Biterider's point of view . I very frequently have open explorer in ObjMem directory to read the source code of functions to see arguments and outputs.

The documentation is non existing, and we need to understand the use from the code. Very funny but slow.

That was my first post in the forum. Yes, nothing have changed. Prefix is an unnecesary complication, but now I'm thinking that is easy to solve: to change files names (not functions' names) and to create postfixed direct alias  .

What worries me is that getting UEFI into the model is a lot of work, but nothing too complicated. It's just going to take some time.
Understanding UEFI is complex, but we'll get through it. 

The ObjAsm application that originally motivated me to try UEFI work perfectly in CPU0, very easy. Like Johnsa anticipated, is not faster than OS version. To use the other cores have the complication that booting UEFI apparently is running in them with higher priority. That aspect is not so easy.

Regards, HSE.

[Biterider]

Hi HSE
As far as I can tell, a lot of things have changed to help the programmer since the early days of ObjAsm.
You have the \Help\ObjAsm.pdf file that describes the core functionality, debugging, COM, installation, etc. Additionally, there are some examples of increasing level of complexity and an API help file for RadAsm (see attached image). To better understand the object hierarchy and to read the object method documentation quickly, you have the OA_ObjExplorer in the project folder.
Maybe some tools are not perfect or incomplete, but I'm open to discuss changes to improve them. 

As you suggested, I will change the original approach and use UEFI procedures with a posfix. These procs will be aliased like the string procedures. 

Biterider

[HSE]

Hi Biterider!!

As far as I can tell, a lot of things have changed to help the programmer since the early days of ObjAsm.

Yes. There is a very interesting guide But I used a very specific term wich is "documentation". And we were talking about library ObjMem. There is a lot of text in source code files, but nothing in source code files is documentation. Perhaps you can debate with NASA laboratories, but I think that is correct.

Nobody like to write documentation, specially for free. But documentation is independent from code, sometimes follow source code structure, sometimes is similar, never is the same.

Additionally, there are some examples of increasing level of complexity and an API help file for RadAsm (see attached image). To better understand the object hierarchy and to read the object method documentation quickly, you have the OA_ObjExplorer in the project folder.
Maybe some tools are not perfect or incomplete, but I'm open to discuss changes to improve them. 

Nothing of that count like documentation nor help. A help is very close to documentation. Usually you know a help for extension *.hlp or *.chm and documentation for *.doc or *.pdf. Of course there are good and bad documentations and helps.

Some are "Smart things" that are usefull when you already know a lot about the subject. Then they are not so reliable.

The API files tool (they are not a help) depend on editor, and perhaps you have to modify the editor. I don't even know if I have API files in place  I tested only some Fearless API files for his libraries.

Other things that relly on editor are also problematics. I remember some years ago I tried to make a program with MasmBasic, wich don't have a help file but some smart thing in RichMasm, but finally I maked in PowerBasic, wich have a fantastic help!!

As you suggested, I will change the original approach and use UEFI procedures with a posfix. These procs will be aliased like the string procedures. 

  Fantastic

Regards, HSE.

[Biterider]

Hi
I created a new branch in the ObjAsm repo on GitHub called UEFI_Support. When a stable state has been reached, I will merge it.
It contains the current status of modifications to support UEFI.
The ObjAsm project will only contain the basic UEFI samples from HSE and will allow the development of additional packages by third parties.

The most interesting recent addition is the extension of the debug macros to output to the UEFI shell, as shown in the image below.
Currently, the only missing macro is DbgMem, which needs to be completely rewritten and will take a bit longer.

Biterider

[HSE]

Impressive!!

Really another level   

[Biterider]

Hi
DbgMem has been rewritten. 
I removed all windows dependencies and now it also shows the memory contents on the UEFI shell too.
The next step will be to merge into GitHub the main and UEFI_Support branches.

Biterider

[HSE]

Hi Biterider!

I can't found why BenchmarkUEFI in UEFI_Support branch don't work (so many cosmetic changes  )

For sure there is wrong replacements of return macro, ret is missing!!!

Code Select Expand

  invoke StrNew, $OfsCStr("Complete", 13, 10)


So far I understand, UEFI is not going to release memory in any way, then this is an unnecessary leak.

Regards, HSE

(oh, I neither found where happen  "El sistema no puede encontrar el controlador especificado.")

[Biterider]

Hi HSE
Thank you for looking at this. I failed to check the logic of the code and the missing comments make it difficult for me to follow.
I will check the return macro. I ran the code through my code formatter and made some hand modifications to match the overall implementation. It's possible that I made some mistakes.

The UEFI specification explicitly says about EFI_BOOT_SERVICES.Exit that the ExitData must be allocated from a pool, which is what StrNew does https://uefi-d.dpldocs.info/uefi.spec.EFI_EXIT.html#:~:text=The%20ExitData%20buffer%20must%20be%20allocated%20by%20calling%20AllocatePool().

It would be cool if you have some time if you could double check the code. I uploaded a lot of changes to the ObjMem library yesterday, so it would be a good idea to sync the code.

Biterider

[HSE]

Hi Biterider

I failed to check the logic of the code and the missing comments make it difficult for me to follow.

It's Paoloni code ( C logic ) 

I ran the code through my code formatter

That work wrong: everything in upper case  . I use everything posible in lower case to become code readable.

The UEFI specification explicitly says about EFI_BOOT_SERVICES.Exit that the ExitData must be allocated from a pool, which is what StrNew does https://uefi-d.dpldocs.info/uefi.spec.EFI_EXIT.html#:~:text=The%20ExitData%20buffer%20must%20be%20allocated%20by%20calling%20AllocatePool().

Perfect, it's an official leak  . Then we must use:

Code Select Expand

invoke [xbx].EFI_BOOT_SERVICES.Exit, ImageHandle, EFI_SUCCESS, 0, NULL because ExitData is ignored if EFI_SUCCESS.

It would be cool if you have some time if you could double check the code. I uploaded a lot of changes to the ObjMem library yesterday, so it would be a good idea to sync the code.

I readed some changes from GitHub days ago, now I'm building examples.

HSE

[Biterider]

Hi HSE
Rereading the UEFI specification from here https://uefi.org/sites/default/files/resources/UEFI_Spec_2_1_D.pdf it says at page 181:

ImageHandle: Handle that identifies the image. This parameter is passed to the image on entry.
ExitStatus: The image's exit code.
ExitDataSize: The size, in bytes, of ExitData. Ignored if ExitStatus is EFI_SUCCESS.
ExitData: Pointer to a data buffer that includes a Null-terminated Unicode string, optionally followed by additional binary data. The string is a description that the caller may use to further indicate the reason for the image's exit. ExitData is only valid if ExitStatus is something other than EFI_SUCCESS. The ExitData buffer must be allocated by calling AllocatePool().

On page 176 it explains the concept using StartImage:
The caller of this function is responsible for returning the ExitData buffer to the pool by calling FreePool() when the buffer is no longer needed.

Technically, returning a string from the pool is not an error. It's unnecessary if we return EFI_SUCCESS as it will be ignored, but in all cases the responsibility for freeing memory is on the caller's side.
I'll remove the string allocation and pass a zero size and a NULL pointer to match the EFI_SUCCESS case.

Biterider

[HSE]

The caller of this function is responsible for returning the ExitData buffer to the pool by calling FreePool() when the buffer is no longer needed.

They are presuming what is the caller, not who.

Original Benchmark work with UEFI_Support, I don't see where problem is.

Look there is some problem in uqword2dec, but qword2dec work.

[Biterider]

Hi HSE
I will verify that using with your original code. 

uqword2dec and qword2dec are the same proc. In this case, both are synonyms for uqword2decW.
I'll check if there's a bug there, but I've done extensive testing on this proc and it's unlikely there's a bug.
When I get the code working I'll try to figure out what's going wrong.

Biterider