The MASM Forum

Projects => Easy Code IDE 32/64-bit => Topic started by: David BS on July 11, 2014, 01:46:56 PM

Title: EasyCode with NASM/YASM
Post by: David BS on July 11, 2014, 01:46:56 PM
Hi folks,

I had utilized EasyCode with MASM32 successfully - the IDE is really great! Congratulations Ramon!

I would like to know if there is some way to put YASM/NASM within EasyCode. I would like to compile some INTEL source codes (built to YASM/NASM) and adapt them to DLL (they're almost all built to console mode).

If it's not possible, could you recommend any other IDE to work with YASM/NASM, or, any tool to convert source code from NASM to MASM?

Thanks in advance!

Title: Re: EasyCode with NASM/YASM
Post by: jj2007 on July 11, 2014, 04:20:07 PM
any tool to convert source code from NASM to MASM?

The NASM example here (http://en.wikibooks.org/wiki/X86_Assembly/NASM_Syntax#Hello_World_.28Using_C_libraries_and_Linking_with_gcc.29) is hilariously complicated, but it looks feasible. If you can post a source with, say, 500 lines, I could give it a try with RosAsm2Masm (http://masm32.com/board/index.php?topic=3375.msg35680#msg35680).
Title: Re: EasyCode with NASM/YASM
Post by: sinsi on July 11, 2014, 04:35:26 PM
The NASM example here (http://en.wikibooks.org/wiki/X86_Assembly/NASM_Syntax#Hello_World_.28Using_C_libraries_and_Linking_with_gcc.29) is hilariously complicated, but it looks feasible.

I weep for the future of ASM
Code: [Select]
        sub     esp, 4          ; Allocate space on the stack for one 4 byte parameter
 
        lea     eax, [fmtStr]
        mov     [esp], eax      ; Arg1: pointer to format string
Pretty sure that nasm has a push instruction...
Title: Re: EasyCode with NASM/YASM
Post by: jj2007 on July 11, 2014, 05:26:56 PM
I weep for the future of ASM

Just in case the noobs who are reading this don't understand why we are whining here:

include \masm32\include\masm32rt.inc
 
.data
        fmtStr  db 'hello, world',0Ah,0
 
.code
main:
  if 0            ; MASM syntax, "pure" assembler, 14 bytes
      push offset fmtStr
      call crt_printf
      add esp, 4

  elseif 0            ; MASM syntax using invoke, 14 bytes
      invoke crt_printf, chr$("hello, world", 10)

  elseif 1            ; MASM syntax using a macro, 14 bytes
      printf("hello, world\n")

  else            ; NASM example (http://en.wikibooks.org/wiki/X86_Assembly/NASM_Syntax#Hello_World_.28Using_C_libraries_and_Linking_with_gcc.29), 21 bytes
        sub     esp, 4          ; Allocate space on the stack for one 4 byte parameter
        lea     eax, [fmtStr]
        mov     [esp], eax      ; Arg1: pointer to format string
        call    crt_printf         ; Call printf(3):
        add     esp, 4          ; Pop stack once

  endif
        ret
end main


BTW the 50% increase in code size is not the fault of NASM but rather of the Wiki author :(
Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 11, 2014, 08:10:22 PM
Code: [Select]
        sub        esp, 4

should be replaced by
Code: [Select]
        sub        esp, byte 4
It's shorter and faster. Works with NASM as well as with YASM.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: jj2007 on July 11, 2014, 08:25:50 PM
With MASM syntax for the NASM example, sub esp, 4 uses already the short 3-byte encoding.
Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 11, 2014, 08:53:09 PM
Jochen,

my point was the original code from your link.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: dedndave on July 12, 2014, 12:31:46 AM
i've seen things like that in compiled code
i suppose, under the right circumstances, it might be justified
for example, you have several larger arguments to pass

so - the SUB ESP,x construct is a "cookie-cutter" template for more complex calls
they calculate the total space required for parameters - then fill it in
one that i saw was for COM - there were a few VARIANT structures passed

the problem i see there is crashability - lol
if you are near the stack limit - SUB ESP,x will get you a silent termination   :redface:
Title: Re: EasyCode with NASM/YASM
Post by: rsala on July 12, 2014, 02:36:32 AM
Hi David BS,

Thanks for your kind words about Easy Code!

Although the IDE is prepared for working with Masm and/or GoAsm, I guess you could use Nasm/Yasm if you disabled all syntax correction and set the compiler path to the folder where Nasm/Yasm is.

Regards,

Ramon
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 12, 2014, 01:08:17 PM
Although the IDE is prepared for working with Masm and/or GoAsm, I guess you could use Nasm/Yasm if you disabled all syntax correction and set the compiler path to the folder where Nasm/Yasm is.

Hmmm.... I understood everything you wrote guys. Thanks.

But my doubt was really about what Ramon answered: if I could put EasyCode to compile code using Yasm AND having syntax correction and some built-in macros. Of course I would like to get that usefull help...

Thank you Ramon. And congrats for the very nice job.

I really don't know what I'm going to do now: if I'll try to translate INTEL's YASM code to MASM (and have EasyCode working and making easier my life) or try anything else. 

One question: is not clear to me in YASM syntax if I have to define the DLLMain routine/header to build a DLL file - as I do in MASM syntax. It's not clear if it is performed just by the compiler and its parameters or if in addition I must define it is a DLL within the code.  Anyone can help me in that question?

Thanks again to all.
Regards.
Title: Re: EasyCode with NASM/YASM
Post by: jj2007 on July 12, 2014, 05:41:41 PM
if I'll try to translate INTEL's YASM code to MASM (and have EasyCode working and making easier my life) or try anything else.
That is really your decision. Writing a converter is easy, but are you more hooked on YASM, or are you equally comfortable with MASM syntax?

If you can post a source with, say, 500 lines, I could give it a try with RosAsm2Masm (http://masm32.com/board/index.php?topic=3375.msg35680#msg35680).
Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 13, 2014, 08:12:53 PM
Hi Jochen,

if I'll try to translate INTEL's YASM code to MASM (and have EasyCode working and making easier my life) or try anything else.
That is really your decision. Writing a converter is easy, but are you more hooked on YASM, or are you equally comfortable with MASM syntax?

YASM and MASM syntax are not so far apart. That's a fact. AT&T syntax is another world. There's no doubt about that.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: rsala on July 13, 2014, 11:48:45 PM
Hi,

I am afraid I was to optimistic in my first answer. Easy Code, both Masm and GoAsm, internally write some compiler options, so it might not work with Nasm/Yasm. I think the best you can do (not just for using Easy Code, but for making things simple) is trying to translate the code to Masm or GoAsm syntax.

Regards,

Ramon
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 15, 2014, 03:15:21 AM
 :dazzled:

Thanks you all for the answers.

I feel confortable with MASM and, since I'm a Visual Studio programmer, I suppose I can utilize the internal source code debugger to step-by-step my ASM sources during the debug session (for instance, to see if the Visual Basic call and parameters are being correctly sent/received and so forth). I'm not sure if with YASM I have the same environment.

EasyCode really makes the job easier, since it has a lot of functions/macros to help us. So, the union between MASM and EasyCode is preferable to me.

BUT, unfortunetly, all INTEL source codes are made to be YASM/NASM compiled and I believe the most of them are to be linked with C or C++ (argh!  :icon13:). I hate C/C++ since the language is in my view the worst between two worlds: the High Level and Low Level languages.  I can make a whole system utilizing Visual Basic in just ONE DAY, what sould be impossible utilizing C++ or C.  The sources I send now here are being focused into .LIB files, which Visual Basic does not see!

For instance, one of my doubts is about how make a DLL file in YASM!  I cannot see any source example! May I have to define the DLLEntry point there or YASM/NASM will create it automatically?  And the EXPORT directive, must I define in within the code?  Should I have also to create .DEF files?

In MASM I know how to create them and make the MASM/LINK undertstand them to create a DLL file, but with these samples I'm not getting succes with YASM. Or it creates an EXE file or a LIB file - and none of them are my target.

Can anyone light my way?
Thank you very much.
Regards!
Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 15, 2014, 03:37:09 AM
Hi David,

I hope that link (http://stackoverflow.com/questions/5666972/how-can-one-create-dll-files-with-nasm) will help you.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 15, 2014, 03:41:17 AM
Hi David,

I hope that link (http://stackoverflow.com/questions/5666972/how-can-one-create-dll-files-with-nasm) will help you.

Gunther

Hmmm.....  I'll check it now.  Thank you Gunther!
 :biggrin:
Title: Re: EasyCode with NASM/YASM
Post by: rsala on July 15, 2014, 04:39:04 AM
Hi David,

I'd really like to help you but I have no idea how Yasm works. I am sorry!

Ramon
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 15, 2014, 04:57:03 AM
Hi David,
I'd really like to help you but I have no idea how Yasm works. I am sorry!
Ramon


No problem at all Ramon.
I had really appreciated your efforts in help me.
And go on enhancing EasyCode. It's a great software, it saves a lot of time of programmers specially in build MAKE files and macros.
Kindest regards.
Title: Re: EasyCode with NASM/YASM
Post by: rsala on July 15, 2014, 08:25:44 AM
Thanks once again for your kind words! I really appreciate your support!

Ramon
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 15, 2014, 09:31:22 AM
Ramon,

Do you know how can start a classic DLL file but using x64 code?
I'll try the MASM classic DLL file calling the INTEL LIB file...
But I'm not sure if the include files can be those EasyCode put automatically to me when I start a new DLL project.
Could you light my way?

Thanks in advance.

Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 15, 2014, 09:46:47 AM
Hi David,

what about that: YASM will produce an OBJ file for you. The rest is a linker question. You could use GOLINK, which can produce DLL files:
Quote

GoLink.Exe Version 0.27.0.0 - Copyright Jeremy Gordon 2002/12 - JG@JGnet.co.uk

Use one or more command files eg. GoLink @command.fil and/or the command line.
Specify obj,res,dll,ocx,exe or drv input files and/or these switches:-
/base xxxx   = set image base (use hex)
/console     = make console executable (otherwise Windows GUI)
/debug xxx   = coff = embedded coff, dbg = dbg file in subfolder
/dll         = make a dll (otherwise exe)
/e           = empty output file allowed
/entry xxxx  = set program entry point to xxxx
/export xxxx = allow this label to be called from another module
/files       = give full information about input and output files
/fo xxxx     = file output eg. /fo prog.exe, prog.dll or off (no file)
              if no output file is specified, 1st .obj/.res file name is used
/h or /?     = this help
/mix         = link mixed object files or lib code having decorated symbols
/unused      = list unused/unreferenced data and code labels
/uponly      = file is not designed to run on multi-processor machines
/version xxxx= set image version to major minor value
Make a driver using /driver (PE driver) or /wdm (Windows Driver Model driver)
Control stack allocation and commitment using /stacksize and /stackinit
Control message output with /b, /more, /ne, /ni, /no, /nw (see manual)
To make Microsoft Layer for Unicode loader, see manual

Another possibility is POLINK:
Quote

Pelles Linker, Version 7.00.3
Copyright (c) Pelle Orinius 1998-2012

Syntax:
POLINK [ { option | file | @commandfile } ... ]

Options:
/ALIGN:#
/ALLOWBIND[:NO]
/ALLOWISOLATION[:NO]
/ALTERNATENAME:symbol=symbol
/BASE:address
/DEBUG[:NO]
/DEBUGTYPE:{CV|COFF|BOTH}
/DEF:filename
/DEFAULTLIB:filename
/DELAY:{NOBIND|UNLOAD}
/DELAYLOAD:filename
/DLL
/DRIVER[:{UPONLY|WDM}]
/ENTRY:symbol
/EXPORT:symbol[=[module.]symbol][,@ordinal[,NONAME]][,DATA]
/FIXED[:NO]
/FORCE:MULTIPLE
/HEAP:reserve[,commit]
/IMPLIB:filename
/INCLUDE:symbol
/LARGEADDRESSAWARE[:NO]
/LIBPATH:path
/MACHINE:{AMD64|ARM|X64|X86}
/MANIFEST[:NO]
/MANIFESTDEPENDENCY:dependency
/MANIFESTFILE:filename
/MANIFESTUAC[:{NO|execution level}]
/MAP[:filename]
/MAPINFO:{EXPORTS|FIXUPS|LINES|PATHS}
/MERGE:from=to
/NODEFAULTLIB
/NOENTRY
/NXCOMPAT[:NO]
/OPT:{REF|NOREF|WIN98|NOWIN98}
/OSVERSION:#[.##]
/OUT:filename
/RELEASE
/SAFESEH[:NO]
/SECTION:name,[E][R][W][D][K][P]
/STACK:reserve[,commit]
/STUB:filename
/SUBSYSTEM:{CONSOLE|NATIVE|WINDOWS|WINDOWSCE}[,#[.##]]
/SWAPRUN:{CD|NET}
/TSAWARE[:NO]
/VERBOSE
/VERSION:#[.##]
/WS:AGGRESSIVE


HTH,
Gunther
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 15, 2014, 10:17:27 AM
Hi David,

what about that: YASM will produce an OBJ file for you. The rest is a linker question. You could use GOLINK, which can produce DLL files:
HTH,
Gunther

Thank you so much Gunther, but my problem is related to the source file (I'm sending it again).
The file is created 100% but when I try to call it from a x64 Visual Basic application the system hangs.  :icon_redface:
I'm going to be crazy!

The MAIN problem is the fact of the INTEL source code is NOT prepared to be a DLL but a LIB file, and I don't want to get a C++ file ONLY to access it! I'm trying to make it as a DLL and call it directly from Visual Basic.

I had modified the INTEL code adding a DLLEntry procedure as mentioned in NASM manual. Remember: the source code was made to be a LIB of a C/C++ CONSOLE APP.

I compile it successfully using:
vsyasm -f x64 -D WINABI sha512_sse4.asm

I link it successfully using MS LINK and that commandline:
link /entry:DllMain /MACHINE:X64 /NODEFAULTLIB /def:hash.def /DLL /export:sha512_sse4 /subsystem:windows sha512_sse4.obj /out:"hash.dll"

I get the DLL file withou any error from VSYASM and LINK.EXE!  But when I try to call it from Visual Basic the system hangs. I tried two ways to call it, both UNSUCCESSFULLY:

1) To create a STRUCTURE inside VB and call the function via DECLARE (exactly what I done to call CPUID successfully) as shown below:
   
Public Structure GetSHA
        Dim OriginalMsg As String
        Dim SHA_Returned As String
        Dim MsgLen As IntPtr
End Structure

Declare Sub sha512_sse4 Lib "hash.dll" (ByRef sMensagem As String, ByRef ssha As String, ByVal stamanhomensagem As IntPtr)

Public MyRetorno As GetSHA
sha512_sse4(MyRetorno)


2) To create the variables without a structure and point them directly in the function:

Declare Sub sha512_sse4 Lib "hash.dll" (ByRef Message As String, ByRef ReturnedSHA As String, ByVal MsgSize As IntPtr)

 Dim Msg As String = "system test"
 Dim NewMsg As String = Msg.PadRight(64)
 Dim NewMsgLen As UInteger = newMsg.Length

 Dim returned As String = ""
 sha512_sse4(newMsg, returned, NewMsgLen)



WOW!!!  Why INTEL had choosen THIS way to do the things?!?!?  It seems a bad joke with MS programmers....

And I really had appreciated THE WHOLE TEAM efforts to help me. I apologize for any inconvenience but here in Brazil is really too hard to find ANYONE with some ASM knowledge to ask my questions.

Thank you very much for any help.
Kindest regards Gunther and all the guys involved with my questions.

Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 15, 2014, 10:32:28 AM
Ops!  The step (1) is really:

Public Structure GetSHA
        Dim OriginalMsg As String
        Dim SHA_Returned As String
        Dim MsgLen As IntPtr
End Structure

Declare Sub sha512_sse4 Lib "hash.dll" (ByRef Struc as GetSHA)


1) To create a STRUCTURE inside VB and call the function via DECLARE (exactly what I done to call CPUID successfully) as shown below:
   
Public Structure GetSHA
        Dim OriginalMsg As String
        Dim SHA_Returned As String
        Dim MsgLen As IntPtr
End Structure

Declare Sub sha512_sse4 Lib "hash.dll" (ByRef sMensagem As String, ByRef ssha As String, ByVal stamanhomensagem As IntPtr)

Public MyRetorno As GetSHA
sha512_sse4(MyRetorno)

Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 15, 2014, 11:31:19 AM
David,

okay I'll check the source tomorrow.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 15, 2014, 11:34:47 AM

David,
okay I'll check the source tomorrow.
Gunther

Thanks a lot Gunther!  :t
Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 15, 2014, 11:46:07 AM
Hi Dave,

would it help to have the YASM source in - let me say - near masm syntax? I've it in Intel syntax and can post it.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 15, 2014, 12:48:48 PM
Hi Dave,

would it help to have the YASM source in - let me say - near masm syntax? I've it in Intel syntax and can post it.

Gunther

Yes, I believe so...
I'm trying now call ASM code from VB utilizing MARSHAL functions (it pass addresses instead values), but I'm getting error all the time.
I believe the main problem is the fact of the ASM source receive parameters in YASM syntax (directly from stack??) and all of them are VOID type, which is a little bit confuse to set in VB.
But in the point I'm now, any help is valuable.   ;)

Again, thanks a lot and excuse me for any inconvenience.
Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 15, 2014, 08:42:52 PM
Hi Dave,

I've attached a ZIP archive. It's generated by OBJDUMP and contains both, the machine code and the appropriate assembly language instructions in MASM syntax. I hope that helps. The only thing to do is to copy and paste the instructions into a MASM source code with proc and endp statements. NASM and YASM don't have such statements. For both is a procedure just a label. So here is an example. If the yasm syntax looks like:
Code: [Select]
MyProcedure:
        push        rbx
        ...
        ret
you have to write for ml or jWasm:
Code: [Select]
MyProcedure    proc
        push        rbx
        ...
        ret
MyProcedure    endp
The rest is copy and paste, because you've the original YASM source.

I think Intel did it that way, because YASM is available on both platforms (Windows and Linux) producing appropriate output. The Microsoft Assembler runs only under Windows. That might be the reason.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 15, 2014, 10:25:20 PM
Thank you very much Gunther.
I'll try what you recommend still today, as soon as possible.
My job requires my attention right now, but I guess until the middle of day I could try it.

Kindest regards.
Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 16, 2014, 12:00:15 AM
Hi David,

Thank you very much Gunther.
I'll try what you recommend still today, as soon as possible.
My job requires my attention right now, but I guess until the middle of day I could try it.

you're welcome. If you need some more assistance by converting the code, let me know. I'm at home in both worlds, YASM and MASM.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: rsala on July 16, 2014, 08:19:44 AM
Hi David,

I have this DLL 64 test code (attached) but I have never compiled it. However, you will need the MASM64 package.

Regards,

Ramon
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 16, 2014, 08:25:33 AM
Thanks Ramon, I have ML64.

I'll try something with him. I'm looking now the OBJ file created by Gunther from the original INTEL's YASM code.
I'm studing it and analyzing if I will convert it to MASM64 syntax or not.

Thank you.
Kindest regards.
Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 16, 2014, 08:51:49 AM
Hi David,

after reading the entire thread again, I came up with that ideas. Your system hangs, if you're calling the DLL from VB. The reason must not be the DLL code.

The Intel source is designed for a LIB. C and BASIC have very different calling conventions. BASIC passes parameters to procedures by reference (factory default) while C passes by value. BASIC passes from right to left, C passes from left to right. Furthermore, in C cleans the caller the stack, in BASIC has the callee to clean the stack.

That's all a bit tricky and you have to know the details. I'm not very familiar with VB, but I know PowerBASIC very well. You can declare in PB a procedure or function as CDECL and BASIC handles all things like a C compiler. I hope that VB has a similar feature.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: qWord on July 16, 2014, 09:05:29 AM
I'm trying now call ASM code from VB utilizing MARSHAL functions (it pass addresses instead values), but I'm getting error all the time.
I believe the main problem is the fact of the ASM source receive parameters in YASM syntax (directly from stack??) and all of them are VOID type, which is a little bit confuse to set in VB.
The problem for the DLL source from post #21 is that the DllMain follows Stdcall, which is invalid for x64 (fastcall is required). It should be something like:
Code: [Select]
global DllMain:function
DllMain:
mov eax,1
ret

Remakrs that the code only implement the core loop of SHA-512. With the above modification and the following amateurish VB code I was at least able to call sha512_sse4 (VS 2012, Win7 x64). I guess the result is not correct due to wrong preprocessing of the message...
Code: [Select]
Imports System.Runtime.InteropServices.Marshal

Module Module1

    Declare Sub sha512_sse4 Lib "\yasm\hash.dll" (ByVal msg As IntPtr, ByVal sha512 As IntPtr, ByVal N As IntPtr)

    Sub Main()

        Dim str As String = "Hello World"

        Dim zero(127) As Byte

        Dim start_hash As Long() = {&H6A09E667F3BCC908, &HBB67AE8584CAA73B, &H3C6EF372FE94F82B, &HA54FF53A5F1D36F1, &H510E527FADE682D1, &H9B05688C2B3E6C1F, &H1F83D9ABFB41BD6B, &H5BE0CD19137E2179}

        Dim cb_str_len As Integer = str.Length * 1                                  ' or str.Length * SystemDefaultCharSize

        Dim msg_length As Integer = cb_str_len + 1 + 16                             ' 1 + 128 bit
        Dim block_size As Integer = 128                                             ' SHA-512 block size
        Dim hash_size As Integer = 64
        Dim buffer_len As Integer = (msg_length + block_size - 1) And -block_size
        Dim msg_n_blocks As IntPtr = buffer_len / block_size
        Dim pMsg As IntPtr = StringToHGlobalAnsi(str)                               ' or StringToHGlobalAuto
        pMsg = ReAllocHGlobal(pMsg, buffer_len)                                     ' change buffer size

        ' extend message by one bit
        WriteByte(pMsg + cb_str_len, 1)

        ' zero further bytes
        If buffer_len - cb_str_len - 1 Then
            Copy(zero, 0, pMsg + cb_str_len + 1, buffer_len - cb_str_len - 1)
        End If

        ' append message size
        WriteInt32(pMsg + buffer_len - 4, cb_str_len * 8)

        ' create inital hash value
        Dim pSHA512 As IntPtr = AllocHGlobal(hash_size)
        Copy(start_hash, 0, pSHA512, start_hash.Length)

        ' ASM
        sha512_sse4(pMsg, pSHA512, msg_n_blocks)

        '' print message block(s)
        'System.Console.WriteLine("msg bytes: ")
        'Dim b(buffer_len - 1) As Byte
        'Copy(pMsg, b, 0, buffer_len)
        'For i = 0 To b.GetUpperBound(0)
        '    System.Console.Write("{0:X2}", b(i))
        'Next
        'System.Console.WriteLine("\\ end msg bytes \\")

        ' get hash value
        Dim sha512(63) As Byte
        Copy(pSHA512, sha512, 0, 64)

        System.Console.WriteLine("msg = " & str)
        System.Console.Write("hash = ")

        For i = 0 To sha512.GetUpperBound(0)
            System.Console.Write("{0:X2}", sha512(i))
        Next

        System.Console.WriteLine("")

        FreeHGlobal(pMsg)
        FreeHGlobal(pSHA512)

        System.Console.ReadKey()

    End Sub

End Module

Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 16, 2014, 09:46:59 AM
The problem for the DLL source from post #21 is that the DllMain follows Stdcall, which is invalid for x64 (fastcall is required). It should be something like:

Hey qWord!! I had read it in some place but I really hadn't paid attention to that! Hmmm.... For sure, it may impact in the wrong results.... :icon_rolleyes:

The Intel source is designed for a LIB. C and BASIC have very different calling conventions. BASIC passes parameters to procedures by reference (factory default) while C passes by value. BASIC passes from right to left, C passes from left to right. Furthermore, in C cleans the caller the stack, in BASIC has the callee to clean the stack.
Gunther

Hmmm.... I didn't know about THESE languages particularities, mainly with the stack usage. I'm really not familiarized with C/C++ since I had never had interest on them. I had always choosen to know a high-level language AND assembly... I made that decision since Apple ][, joining BASIC (and its Peek/Pode instruction and others I don't remember anymore) with 6502 assembly to make my CP/M card to emulate a decent screen size (wow, 128 text columns on screen!!).  It remind me the 6 lines of 8088 asm code to break the 640kb limit of XT to get more memory to Clipper indexes...  :badgrin:

Thank YOU very much guys. I'll try the code you sent, and the tip you're referencing Gunther.
I really had appreciated the whole efforts of all of you.  :greenclp:


Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 16, 2014, 07:32:42 PM
Hi David,

you should have now enough to note. Let us know the results and if you need more help.

Gunther
Title: SHA512_SSE4
Post by: David BS on July 18, 2014, 07:32:35 AM
 :dazzled:

Thanks to ALL of you for the precious tips and advices, BUT I conclude that the INTEL's code is really wrong OR it's made to inform wrong result in proposital way.

Let me explain:

1- The code is running CLEAR. I made a lot of modifications in ASM and VB2010 - all of them crashes the execution. Just ONE way can provide SOME right results (no crash at all).

2- Gunther, I can inform to VB2010 if I want to pass parameters as POINTER or VALUE. The INTEL code gets both: pointer to VOID* and value for INT. I tried to change the order from left to right in the parameters command-line and ALL tries made the program crash. The sent order is the SAME the source is planned to receive.

3- I have here 2 machines: one Windows 7 x64 (not SSE4x compatible) but another one Windows 8.1 x64 100% SSE4.x compatible. Both gave me the SAME results in the SHA call, so, I conclude that something is wrong. It couldn't happened...

4- I had tried also to pad the original message to SHA as required by FIPS140 (in BITS!! :icon_eek:). None effect.

5- In despite of to make changes in code (eg. pad or not the message), the most interesting thing is that the result are THE SAME, in VALID SIZE, and ramdomly acceptable (any change in the original message caused the SHA change its bytes entirely). But the results are NOT the expected ones if compared to valid SHA512 results (I had utilized two different sources: .NET library and an online SHA service).

6- The tips that all of you gave me were fundamental to compile the ASM and call from VB. Thank YOU all. I couldn't make it alone... It's the main success to me:  the fact of get a original LIB YASM source code and change it to a valid DLL file and call it from Visual Basic 2010/2012. Congratulations to all of you and your efforts were really, really appreciated by me.

Just to help SOMEONE with similar problem, basically the correct procedures to have INTEL code running here as DLL file were (considering it were compiled by YASM/NASM):

a) make both DLL entry and function PUBLIC in the source code:

    Global DllMain
    Global sha512_sse4


b) create the DLL entry in MS WINDOWS format within the source code (__FastCall syntax since it's a x64 code):

    ..start:
    global DllMain:function
    DllMain:
   mov eax,1
   ret


c)  create the .DEF file (Export)
   
     LIBRARY HASH
     EXPORTS sha512_sse4


d) compile it and link utilizing the following commands:

    vsyasm -f x64 -D WINABI sha512_sse4.asm
    link /entry:DllMain /MACHINE:X64 /NODEFAULTLIB /def:hash.def /DLL /export:sha512_sse4 /export:DllMain /subsystem:windows sha512_sse4.obj /out:"hash.dll"


These procedures CAN create a valid x64 DLL using YASM or a compatible compiler.

In the Visual Basic 2010 code I made the following steps:

a) I had declared a SUB call to the ASM function (it's NOT a function call since the ASM code does NOT return a value but changes a pointed entried variable):

    Declare Sub sha512_sse4 Lib "hash.dll" (ByVal sMensagem As IntPtr, ByVal ssha As IntPtr, ByVal stamanhomensagem As UInteger)


    Interesting to note that all parameters are sent BY VALUE since the next commands can get the Memory Pointers of each of them (see IntPtr above).

b) I build all parameters and after, transformed them to POINTERS to be viewed by an original C++ code - a way to make unmanaged code (C++/C) interface with a managed one (C# or VB):

        Dim VarSized64bits As UInteger = (novamensagem.Trim.Length Mod 64) + 1
        Dim Mess2Sha As IntPtr = Marshal.StringToHGlobalAuto(Original_String_Message)
        Dim ReturnSHA(64) As Byte      << an array of 64 bytes = 512 bits
       
        Dim MessFromSha As IntPtr = Marshal.AllocHGlobal(64)     ' pointer to ReturnSHA
        Marshal.Copy(ReturnSHA, 0, MessFromSha, 64)                 ' filled by ...

        sha512_sse4(Mess2Sha, MessFromSha, VarSized64bits)     ' call to the %@&^#% INTEL's code   

        Marshal.Copy(MessFromSha, ReturnSHA, 0, 64)                 ' Get the result and move it to array of bytes.

        ' Free allocated memory.

        Marshal.FreeHGlobal(Mess2Sha)                                       
        Marshal.FreeHGlobal(MessFromSha)



One day I will return to that code... I'm really tired since I'm facing on it around 2 or 3 weeks.
I understand the INTEL code HAS some problem. I can't believe the problem is about parameters order or so forth, since ANY change on the above crashes the IDE.

Again, all of you have been TOO MUCH kind with me. Thank YOU very much. :eusa_clap: :eusa_clap: :eusa_clap:


Title: Re: EasyCode with NASM/YASM
Post by: qWord on July 18, 2014, 08:16:46 AM
Some problems I see:
 - The block size for SHA-512 is 1024 bit => 128 byte
 - I guess the default text encoding in VB is Unicode thus the actual memory size in bytes is (at least) doubled
 - I can't see where you copy the initial hash value

You might break down your code and post a short, runnable example...
Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 18, 2014, 08:37:48 AM
David,

qWords points are important. He's a smart guy and you can learn a lot by attentive reading his posts. Moreover, I would suggest that: Does Intel provide a C code example with appropriate test values? If so, I think it's written for gcc, because that compiler is available in both worlds. I've a native Windows 7-64 and a native 64-bit Linux running here (both with the appropriate gcc). I could try to re-compile the entire code to keep the yasm code running. That would be the first step. If it works, we can see how to call it from a DLL.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 18, 2014, 08:52:30 AM
Dear guys,

INTEL does NOT provide ANY C sample to utilize with that ASM code. It recommends ONLY to utilize it as a LIB.

In my VB code I had tried ALSO to PAD the original message (any lenght) as required to SHA-512 (a lot of zero bits to make a 1024-mod size). The results didn't changed. I believe the ASM code does not PAD the original message as required, and because of it I perform it at my own.

I don't believe VB translate the string to UNICODE unless we require it explicitly. When I debug the string it appears with the ASCII length. I can check it since even appending a LOT of NULL, the string does NOT show it but the its size confirms it 100%.  So, if UNICODE, that lenght should be the double (I agree with qWord).

Yes, I appreciate your efforts in debug the code. I'll send it in attachment as Visual Studio project (ready to load and compile). Please, if you need another format (like pure ASCII for instance), please, let me know.

And once again, THANK YOU very much!

PS: if you have ANY problem to understand the code or the Portuguese variables names, please contact me and I try to change it to English and in a readable way. The main reason to do not make this now is because I'm involved with a hurry support to a client - I can't stop NOW.  But if you have any difficulty, please, let  me know.
Title: INTEL SHA512_SSE4
Post by: David BS on July 18, 2014, 10:15:48 AM
Hi,

Follows in attachment the Visual Basic code IN ENGLISH format and few comments.

Thanks in advance....
Title: Re: EasyCode with NASM/YASM
Post by: qWord on July 18, 2014, 05:24:59 PM
After some time I finally got it run correct. The key points are:
 - the wordsFIPS140 (actual QWORDs) of the returned hash are in Big-endian byte order.
 - the append bit must be add as a byte of the value 0x80 = 128
 - the 128 bit integer that holds the message size in bits, must also be stored in Big-endian byte order
 - the hash must be initialized with H(0)0...7 (see FIPS140) before calling sha512_sse4

In the attachment you can find an example application. Those who want to build it need jWasm (http://www.japheth.de/JWasm.html), Yasm (http://yasm.tortall.net/Download.html), polink and polib - the last two are also included in the MASM32 SDK (otherwise see PellesC).

BTW: Currently I'm a bit confused about the name "sha512_sse4", because (AFAICS) the maximal feature level is SSSE3 due to PSHUFB instruction...



EDIT:
I did modified the attachment. Now it contains a DLL that exports the function sha512(pData,byte_count,pHash), which simplifies usage in VB:
Code: [Select]
Imports System.Runtime.InteropServices.Marshal

Module Module1

    Declare Sub sha512 Lib "hash.dll" (ByVal data As IntPtr, ByVal cb As IntPtr, ByVal hash As IntPtr)


    Sub Sha512Auto(ByRef msg As String, ByRef hash() As Byte)

        Dim raw_size As Integer = msg.Length * SystemDefaultCharSize

        Dim pBuffer As IntPtr
        If raw_size Then
            pBuffer = StringToHGlobalAuto(msg)
        End If

        Dim pHash As IntPtr = AllocHGlobal(64)

        ' call Dll function
        sha512(pBuffer, raw_size, pHash)

        ' get hash value
        Copy(pHash, hash, 0, 64)

        ' free memory
        FreeHGlobal(pHash)
        If raw_size Then
            FreeHGlobal(pBuffer)
        End If

    End Sub

    Sub Sha512Ansi(ByRef msg As String, ByRef hash() As Byte)

        Dim raw_size As Integer = msg.Length * 1

        Dim pBuffer As IntPtr
        If raw_size Then
            pBuffer = StringToHGlobalAnsi(msg)
        End If

        Dim pHash As IntPtr = AllocHGlobal(64)

        ' call Dll function
        sha512(pBuffer, raw_size, pHash)

        ' get hash value
        Copy(pHash, hash, 0, 64)

        ' free memory
        FreeHGlobal(pHash)
        If raw_size Then
            FreeHGlobal(pBuffer)
        End If

    End Sub

    Sub Main()

        Dim str As String = ""
        Dim sha512(63) As Byte


        Sha512Auto(str, sha512)
        Console.WriteLine("msg = '" & str & "'")
        Console.Write("hash = ")
        For i = 0 To sha512.GetUpperBound(0)
            Console.Write("{0:X2}", sha512(i))
        Next
        Console.WriteLine("")
        Console.WriteLine("")

        Console.WriteLine("auto string conversion: ")
        str = "Some Text foo nil ..."
        Sha512Auto(str, sha512)
        Console.WriteLine("msg = '" & str & "'")
        Console.Write("hash = ")
        For i = 0 To sha512.GetUpperBound(0)
            Console.Write("{0:X2}", sha512(i))
        Next
        Console.WriteLine("")
        Console.WriteLine("")

        Console.WriteLine("conversion to ANSI string: ")
        str = "Some Text foo nil ..."
        Sha512Ansi(str, sha512)
        Console.WriteLine("msg = '" & str & "'")
        Console.Write("hash = ")
        For i = 0 To sha512.GetUpperBound(0)
            Console.Write("{0:X2}", sha512(i))
        Next
        Console.WriteLine("")

        Console.ReadKey()

    End Sub

End Module

EDIT: replaced SSE3 with SSSE3
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 18, 2014, 10:25:16 PM
After some time I finally got it run correct. The key points are:
 - the wordsFIPS140 (actual QWORDs) of the returned hash are in Big-endian byte order.
 - the append bit must be add as a byte of the value 0x80 = 128
 - the 128 bit integer that holds the message size in bits, must also be stored in Big-endian byte order
 - the hash must be initialized with H(0)0...7 (see FIPS140) before calling sha512_sse4


Hi qWORD, nice to hear you.

WOW! When we are tired s*** happens!   

It's new to me that the format must be send (and return) in BigEndian syntax (is it the syntax of CPU access into memory, isn't it?). I was used to this in pure ASM code but really I'm not able to suppose that OS (or VB) shouldn't correct this after return from function.

I'll analyse your code (I believe in you, it's just to see with my own eyes).
And thank you very, very much for spend time and efforts in that analysis.
I'll be back.  See you very soon... :t

Title: Re: EasyCode with NASM/YASM
Post by: Gunther on July 18, 2014, 10:31:52 PM
Hi qWord,

good catch.  :t I've overlooked the Big Endian format.

Gunther
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 18, 2014, 11:08:53 PM
qWord, you made a GREAT job really!

Yes, I saw the code is working very nice.
Just to satisfy my curiosity: why are you utilizing H0 const in SHA512.ASM? Which its paper there? I see it is participating of the "initial hash" routine but WHY exactly those bytes (and not 00's for instance)?

I couldn't solve this alone... The need to make it all BIG-ENDIAN format was completely unknown to me: neither INTEL doesn't mention it, nor VB do not provide any internal mechanism to provide it before/after the call.

And yout interface SHA512 is really a joy! I see the procedures there should be really a hard-work to make in pure VB, I mean, in number of lines and CPU load.

THANK YOU VERY MUCH GUY.  :greenclp:
As Gunther said before (tks to you too, you helped me a lot!), you are really bad!!!
And I liked the jWASM syntax, it seems really objective and allows "macroASM" commands.

I have no words to thank all of you.
I hope can properly retribute your efforts sometime.
 :eusa_clap: :eusa_clap: :eusa_clap: :eusa_clap: :eusa_clap:
Thank you all.


Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 18, 2014, 11:14:45 PM
Ops!  Now I see H0 is a kind of SHA initialization (read carefully your message).

I believe the INTEL routine is too poor...  It could preview all these situations and provide the correct mechanism to allow programmers just to call it.

Again, you made a GREAT job qWord!  Thanks!!
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 19, 2014, 01:07:04 AM
qWord, just one more question:

The Sha512Auto does not return the correct value, since it multiplies the original message length for the DefaultCharSize (in VB2010/2012 = 16 bits).  Why that kind of math? I mean, since each character is really 1-byte in VB, why exactly are you doing here?

If I understand you, are you trying to make a routine to get, for instance, UNICODE strings (2-byte)?

Kindest regards and thanks!!

Title: Re: EasyCode with NASM/YASM
Post by: qWord on July 20, 2014, 02:52:26 AM
I mean, since each character is really 1-byte in VB, why exactly are you doing here?
AFAICS Unicode is the default for VB .Net - because the algorithm requires the size in bytes, such calculation must be done.

Regardless that, I add a function that accepts SAFEARRAYs as parameter, such marshaling is simplified (at least I guess that). An other function for testing SSSE3 support is also added.
Code: [Select]
Imports System.Runtime.InteropServices.Marshal
Imports System.Runtime.InteropServices

Module Module1
    Declare Function supports_SSSE3 Lib "\yasm\hash.dll" () As Boolean

    ' The hash-array must have a size of 64 byte.
    Declare Function sha512_from_array Lib "\yasm\hash.dll" (<MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI2)> ByRef data() As Char, <MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI1)> ByRef hash() As Byte) As Boolean
    Declare Function sha512_from_array Lib "\yasm\hash.dll" (<MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI1)> ByRef data() As Byte, <MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI1)> ByRef hash() As Byte) As Boolean
    Declare Function sha512_from_array Lib "\yasm\hash.dll" (<MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_I1)> ByRef data() As SByte, <MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI1)> ByRef hash() As Byte) As Boolean
    Declare Function sha512_from_array Lib "\yasm\hash.dll" (<MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI2)> ByRef data() As UShort, <MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI1)> ByRef hash() As Byte) As Boolean
    Declare Function sha512_from_array Lib "\yasm\hash.dll" (<MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_I2)> ByRef data() As Short, <MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI1)> ByRef hash() As Byte) As Boolean
    Declare Function sha512_from_array Lib "\yasm\hash.dll" (<MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI4)> ByRef data() As UInteger, <MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI1)> ByRef hash() As Byte) As Boolean
    Declare Function sha512_from_array Lib "\yasm\hash.dll" (<MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_I4)> ByRef data() As Integer, <MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI1)> ByRef hash() As Byte) As Boolean
    Declare Function sha512_from_array Lib "\yasm\hash.dll" (<MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI8)> ByRef data() As ULong, <MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI1)> ByRef hash() As Byte) As Boolean
    Declare Function sha512_from_array Lib "\yasm\hash.dll" (<MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_I8)> ByRef data() As Long, <MarshalAs(UnmanagedType.SafeArray, SafeArraySubType:=VarEnum.VT_UI1)> ByRef hash() As Byte) As Boolean

    Sub Main()

        If supports_SSSE3() Then
            Dim str As String = ""
            Dim sha512(63) As Byte

            sha512_from_array(str, sha512)
            Console.WriteLine("msg = '" & str & "'")
            Console.Write("hash = ")
            For i = 0 To sha512.GetUpperBound(0)
                Console.Write("{0:X2}", sha512(i))
            Next
            Console.WriteLine("")
            Console.WriteLine("")

            Console.WriteLine("Unicode string: ")
            str = "Some Text foo nil ..."
            sha512_from_array(str, sha512)
            Console.WriteLine("msg = '" & str & "'")
            Console.Write("hash = ")
            For i = 0 To sha512.GetUpperBound(0)
                Console.Write("{0:X2}", sha512(i))
            Next
            Console.WriteLine("")
            Console.WriteLine("")

            Console.WriteLine("ANSI string: ")
            str = "Some Text foo nil ..."
            sha512_from_array(Text.Encoding.ASCII.GetBytes(str), sha512)
            Console.WriteLine("msg = '" & str & "'")
            Console.Write("hash = ")
            For i = 0 To sha512.GetUpperBound(0)
                Console.Write("{0:X2}", sha512(i))
            Next
            Console.WriteLine("")

        Else
            Console.WriteLine("SSSE3 not supported by CPU")
        End If
        Console.ReadKey()

    End Sub

End Module

regards, qWord
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 20, 2014, 10:37:23 PM
 :biggrin:

Thank you very much qWord!
I'm alerady testing for SS3 thru CPUID (in MASM mode) but certainly your code is really welcome. :greenclp:
Have a terrific weekend.

Kindest regards from Rio. :t
Title: Re: EasyCode with NASM/YASM
Post by: qWord on July 20, 2014, 10:54:18 PM
I'm alerady testing for SS3 thru CPUID
You must test for SSSE3 and not SSE3 as I mistakenly wrote in one of my previous posts.
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 20, 2014, 11:06:57 PM
 :biggrin:
No problem at all... I had already seen it.
Thanks!!
Title: Re: EasyCode with NASM/YASM
Post by: David BS on July 21, 2014, 09:50:07 AM
Hi qWord!   

Could you help me a little bit again?
The DLL I have to check SSE3, SSE4.1 and so forth DOES NOT execute in x64 application.

I had studied your code above (check SSE3 in SHA512) but I'm getting success to convert it (compile).
Can you help me to get the STRUC (eax, ebx,ecd,edx) from CPUID in a single DLL file (may be JWASM or YASM/NASM compatible).

Thank you for any help. Again.
Kindest regards.