The MASM Forum

64 bit assembler => UASM Assembler Development => Topic started by: Gunther on December 26, 2017, 09:37:26 AM

Title: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 26, 2017, 09:37:26 AM
The attached zip archive is.zip contains the C and assembly language sources. The entire application is compiled with the gcc, because I've no other compiler running here. I hope that Habran can help. I used ANSI C, even for the comments. Compilation with VS should therefore not be a problem.

But the main work is done by assembler routines. That's made with the latest release of UASM, an excellent tool that works without complaint, by the way. The detection up to AVX 2 isn't a big deal. The tricky part is the the impressive AVX-512 part together with the existing and scheduled Intel architectures. The program tests a total of 15 different AVX-512 sets and subsets. AVX-512 F is indicated by feature number 14. All other extensions each have their own flag variable, which can be tested separately. I'm trying to explain the reason for that now. AVX-512 at first glance seems to be a big mess. But let's take a closer look; we have, for example:
For more details, please check Intel's manuals, for example here. (https://software.intel.com/sites/default/files/managed/c5/15/architecture-instruction-set-extensions-programming-reference.pdf) There everyone can inform about Galois Field Affine Transformations; these are important in number theory and for cryptology. I'm not saying more about it now.

All in all, a lot is coming up. After all, AMD will have to join in, so as not to lose the technological connection. All these sets and sub-sets are tested and are signaled. A typical output of the program is that:
Code: [Select]
Supported Features by Processor and Operating System
====================================================

Vendor String: GenuineIntel
Brand  String: Intel(R)Core(TM)i7-7820XCPU@3.60GHz

Instruction Sets
----------------

MMX  SSE  SSE2  SSE3  SSSE3  SSE4.1  SSE4.2  AVX  AVX2  AVX-512 F

It's safe to use the following AVX-512 extensions with this machine:
--------------------------------------------------------------------

AVX-512 DQ      : DWORD and QWORD instructions (conversion, transcendental support etc.)
AVX-512 CD      : Conflict Detection instructions offer additional vectorization of loops with possible
                  address conflicts.
AVX-512 BW      : Byte and Word support for 8- and 16-bit integers.
AVX-512 VL      : Vector Length instructions add vector length orthogonality, allowing most AVX-512 instructions
                  to also operate on XMM and YMM registers.

Any processor that implements any portion of the AVX-512 extensions MUST implement AVX-512 F. Some AVX-512 extensions
are currently only planned by Intel. Not every architecture has all the instruction sets built in:

Knights Landing provides      : CD, ER, and PF.
Skylake provides              : CD, BW, DQ, and VL.
For Cannon Lake are scheduled : CD, BW, DQ, VL, IFMA, and VBMI.
For Icelake are scheduled     : CD, BW, DQ, VL, IFMA, and VBMI.
For Knights Mill are scheduled: CD, ER, PF, 4FMAPS, 4VNNIW, and VPOPCNTDQ.

That's what Intel has released so far.

If AVX-512 is not present on a machine, nothing bad happens. Then only the available instruction sets and the bottom part of the architectures are displayed. By the way, CPU-Z displays only AVX-512 F on the same machine.

A lot of tests with different machines and environments and comments would be fine. Thank you.

Gunther


Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: jj2007 on December 26, 2017, 10:29:36 AM
It seems my machine is a bit old...
Code: [Select]
Brand  String: Intel(R)Core(TM)i5-2450MCPU@2.50GHz

        Instruction Sets
        ----------------

MMX  SSE  SSE2  SSE3  SSSE3  SSE4.1  SSE4.2  AVX

It's safe to use the following AVX-512 extensions with this machine:
--------------------------------------------------------------------


Any processor that implements any portion of the AVX-512 extensions
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 26, 2017, 01:59:20 PM
Jochen,

thank you for testing the application.

It seems my machine is a bit old...

But you've AVX which was a big step forward. I should change the logic in the main procedure. The AVX-512 part should only be displayed if the instruction set really exists. That's probably better to avoid irritations.

Gunther
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 26, 2017, 04:02:30 PM
I've changed the logic of the main procedure a bit. The AVX-512 part is only be displayed if the instruction set really exists. That's probably better to avoid irritations.

Gunther
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Biterider on December 26, 2017, 05:28:39 PM
Hi
This is the output on my machine


Biterider
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: habran on December 26, 2017, 06:37:00 PM
Hi Gunther :biggrin:
here is what I get:
Quote
Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

D:\Downloads\is>is
        Supported Features by Processor and Operating System
        ====================================================

Vendor String: GenuineIntel
Brand  String: Intel(R)Core(TM)i7-4700MQCPU@2.40GHz

        Instruction Sets
        ----------------

MMX  SSE  SSE2  SSE3  SSSE3  SSE4.1  SSE4.2  AVX  AVX2


D:\Downloads\is>
It is nicely done and it does the job, however, I would suggest you to take some time to write all in asm using hll
to make it all easy readable and to show capabilities of UASM and we can use that project in Samples project.
There is no rush, take your time, it will also give you chance to get more familiar with UASM capabilities.

regards
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: mabdelouahab on December 26, 2017, 06:45:40 PM
Quote
        Supported Features by Processor and Operating System
        ====================================================

Vendor String: GenuineIntel
Brand  String: Intel(R)Core(TM)i5-4210UCPU@1.70GHz

        Instruction Sets
        ----------------

MMX  SSE  SSE2  SSE3  SSSE3  SSE4.1  SSE4.2  AVX  AVX2


Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 27, 2017, 01:29:48 AM
Thanks to mabdelouahab and Biterider for testing.  :t

Habran,

It is nicely done and it does the job, however, I would suggest you to take some time to write all in asm using hll ...

Are you sure that it justifies the effort? You know that I'm not such a big friend of macros. But we will see; there is a lot of work at the moment.

Gunther
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: nidud on December 27, 2017, 04:29:27 AM
Are you sure that it justifies the effort?

As a learning experience maybe.

Quote
You know that I'm not such a big friend of macros.

The whole point with extending the functionality of the HLL section of the assembler is removal of macros. The main function (without comments) using HLL will then look something like this:
Code: [Select]
include stdio.inc
include stdlib.inc

.code

main proc argc:SINT, argv:ptr

    local pvs[13]:sbyte
    local pbs[48]:sbyte
    local avx512[14]:qword
    local largest:qword
    local largestextended:qword
    local featurenumber:qword

    printf("\tSupported Features by Processor and Operating System\n")
    printf("\t====================================================\n\n")

    mov largest,CpuVs(&pvs)
    printf("Vendor String: %s\n", &pvs)

    mov largestextended,GetPbs()
    .if(largestextended != 0)
        CopyPbs(&pbs)
        printf("Brand  String: %s\n\n", &pbs)

    .else
        printf("Brand  String: Not supported.\n\n")
    .endif

    printf("\tInstruction Sets\n")
    printf("\t----------------\n\n")

    mov featurenumber,Iset()
    PrintIsets(featurenumber)

    GetAVX(&avx512)

    .ifs (featurenumber >= 14)

        PrintAVXAdditional(&avx512)

        printf("\nAny processor that implements any portion of the AVX-512 extensions MUST ")
        printf("implement AVX-512 F. Some AVX-512 extensions\n")
        printf("are currently only planned by Intel. Not every ")
        printf("architecture has all the instruction sets built in:\n\n")
        printf("Knights Landing provides      : CD, ER, and PF.\n")
        printf("Skylake provides              : CD, BW, DQ, and VL.\n")
        printf("For Cannon Lake are scheduled : CD, BW, DQ, VL, IFMA, and VBMI.\n")
        printf("For Icelake are scheduled     : CD, BW, DQ, VL, IFMA, and VBMI.\n")
        printf("For Knights Mill are scheduled: CD, ER, PF, 4FMAPS, 4VNNIW, and VPOPCNTDQ.\n\n")
        printf("That's what Intel has released so far.\n")
    .endif
    xor eax,eax
    ret

main endp

    end

This will assemble with Asmc and (I assume) with some adjustment also with Uasm.

Quote
But we will see; there is a lot of work at the moment.

Given this builds without the need of a C compiler, linker, and external libraries you may actually end up reducing the workload :t

I've made a single-file PE-version of the test but what happen if you actually have AVX-512 installed is untested.
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 27, 2017, 06:29:07 AM
Hi nidud,

thank you for your answer; very interesting, indeed. I think I should take a closer look at ASMC. It could be of great use to me soon. I refer to this post inside of this thread. (http://masm32.com/board/index.php?topic=6776.msg72765#msg72765)

Branislaw (aka Habran), Johnsa and you have done a great deal, hats off. That's exactly Hutch's idea: bring the assembler back to the desktop. Especially young, inexperienced high-level language programmers will benefit from your work. The other side of the coin is: There are cases where these comfortable high-level language elements are even harmful. But maybe we can discuss that in a PM.

Unfortunately, your program crashes. It looks like there is something wrong on the stack.

Gunther
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: habran on December 27, 2017, 08:55:06 AM
Thanks for the flowers Gunther :biggrin:
Quote
There are cases where these comfortable high-level language elements are even harmful.
I wouldn't agree with you here ;)
Assembler programmers must always look the disassembly to see what the assembler produced and if not happy optimise it. HLL is helping users, as well as programmers, to easy understand what are codes for. Proper comments can support it as well.
We have taken care to optimise HLL built in UASM and in most cases they can maybe produce better and safer code than doing it manually.
However, everyone has different opinion and different needs, so it is pretty hard to satisfy all.

John and myself have been working 28 hours a day on UASM and we are both perfectionists, so, the results must show sooner or later 8)
Our purpose is to please ourselves in the first place, so, anyone who find our baby useful, welcome to the club :biggrin:
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: jj2007 on December 27, 2017, 09:43:25 AM
The whole point with extending the functionality of the HLL section of the assembler is removal of macros.

And that is precisely the risk of this approach: macros can be changed by the coder any time, built-in stuff can't, you are stuck with it. The more features you pack into the assembler itself, the more things can go utterly wrong. Why is C++ such a desaster? Because they had to squeeze in twenty different and sophisticated methods ("classes"?) to peel a banana :(
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 27, 2017, 10:05:56 AM
Branislaw,

I wouldn't agree with you here ;)
Assembler programmers must always look the disassembly to see what the assembler produced and if not happy optimise it. HLL is helping users, as well as programmers, to easy understand what are codes for. Proper comments can support it as well.
We have taken care to optimise HLL built in UASM and in most cases they can maybe produce better and safer code than doing it manually.

No offense, please. John and you are hardworking people and have done a great deal. In principle, you are right; in most cases that's true. But there are some rare cases where you need absolute control over every bit in the machine. So, only manual work helps. I think we agree with that. That was my point. Please have a look into this post (http://masm32.com/board/index.php?topic=6776.msg72765#msg72765), and you'll know what I mean. When I change the frame in the midst of an important project, I want to be sure that I will not replace the old overhead with new overhead. These are my concerns and they are justified, right? Every time I look at the assembly of C ++, I throw my hair. Granted, due to the algorithm, the code is pretty nested and twisted. But I have not seen any such botch that the compiler produces for it, even at the highest level of optimization. I think you do not want to trade with me right now. But somehow we will swing the child already.

The more features you pack into the assembler itself, the more things can go utterly wrong. Why is C++ such a desaster? Because they had to squeeze in twenty different and sophisticated methods ("classes"?) to peel a banana :(

Jochen hit the nail on the head. He seems to have had bad experiences with C++, that's life.
 
Gunther
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: hutch-- on December 27, 2017, 11:08:49 AM
I have this problem with the direction that the new Watcom forks are going in and that is trying to do too many things and trying to hold the hot little hand of the programmer instead of pointing them at the hard and complex stuff. At its most basic an assembler is a crude gadget to screw mnemonics together in the right order so it will run on a compatible processor and perform the task it was designed to do. I can see all sorts of good reasons to add familiar capacities for hacking through much of the high level code you need to produce in conjunction with normal mnemonic code, things like .IF, .SWITCH, procedure entry and exit, call automation (INVOKE and similar) and these can be appended on without compromising the crude basics of what an assembler is.

As soon as you go in the direction of complex error checking and the like you are starting to write a high level compiler and in fact this is how languages like early C started. "Just like assembler but easier". The more you hide from the programmer, the less useful the tool is and this would be very unfortunate as I know that a massive amount of work has been done to get these tools up and going. Complex capacity is something that should be done by programmers, not assembler designers in the assembler, if the basic accessories are done properly then a decent set of libraries adds the higher level capacities needed to make the tool address a much wider market. The assumption that the assembler's own internal high level code is better than what the assembler programmer writes is a dangerous one in that while it may be true in some instances, there are enough decent assembler programmers around who can make such assumptions false.

My comments break down to these,

1. Make sure the assembler IS an assembler, not a pseudo compiler.
2. Use modular design so that additional capacities are optional choices for the programmer.
3. Produce a decent PHUKING library or in fact many libraries, that is what made old C the major professional language for so many years.
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 27, 2017, 11:26:11 AM
My comments break down to these,

1. Make sure the assembler IS an assembler, not a pseudo compiler.
2. Use modular design so that additional capacities are optional choices for the programmer.
3. Produce a decent PHUKING library or in fact many libraries, that is what made old C the major professional language for so many years.

Amen to that, it's the way to go.  :t

Gunther
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: AW on December 28, 2017, 01:40:52 AM
The ASM code does follow the 64-bit ABI rules and some of the procedures are running with the stack unaligned.
This is a simple comment, I know you can do much better otherwise would not be producing software to look for the Higs Bosom.  :(
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: johnsa on December 28, 2017, 02:32:33 AM
I have this problem with the direction that the new Watcom forks are going in and that is trying to do too many things and trying to hold the hot little hand of the programmer instead of pointing them at the hard and complex stuff. At its most basic an assembler is a crude gadget to screw mnemonics together in the right order so it will run on a compatible processor and perform the task it was designed to do. I can see all sorts of good reasons to add familiar capacities for hacking through much of the high level code you need to produce in conjunction with normal mnemonic code, things like .IF, .SWITCH, procedure entry and exit, call automation (INVOKE and similar) and these can be appended on without compromising the crude basics of what an assembler is.

As soon as you go in the direction of complex error checking and the like you are starting to write a high level compiler and in fact this is how languages like early C started. "Just like assembler but easier". The more you hide from the programmer, the less useful the tool is and this would be very unfortunate as I know that a massive amount of work has been done to get these tools up and going. Complex capacity is something that should be done by programmers, not assembler designers in the assembler, if the basic accessories are done properly then a decent set of libraries adds the higher level capacities needed to make the tool address a much wider market. The assumption that the assembler's own internal high level code is better than what the assembler programmer writes is a dangerous one in that while it may be true in some instances, there are enough decent assembler programmers around who can make such assumptions false.

My comments break down to these,

1. Make sure the assembler IS an assembler, not a pseudo compiler.
2. Use modular design so that additional capacities are optional choices for the programmer.
3. Produce a decent PHUKING library or in fact many libraries, that is what made old C the major professional language for so many years.

I agree mostly, with the work I'm doing on UASM I try to break it up so that we try to cover the full spectrum:

1) For example adding an OSX version, macho64 support just increases the availability and use of the tool.
2) Calling conventions for Borland register, Vectorcall and SystemV once again extend it's usefulness.
3) Bug fixes which can apply anywhere.
4) Enhancements (and this is where I guess most of the debate comes in), I try to ensure that every new feature in no way compromises or breaks the existing functionality of the assembler
    so some of these are implemented in the core of the assembler with this creed in mind, for example extending the way you can initialise floating point data or unions by specifying which sub-field to use:
Code: [Select]

UnionType union
 _a fourfloats <?>
 _b fourdwords <?>
ends

mything1 UnionType < 1.0, 2.0, 3.0, 4.0 > ; traditionally this is all that was allowed, floating point initializers for floats and only the first member of the union.. but now we have:
mything2 UnionType < 1, 2, 3, 4> ; promote integers to float
mything3 UnionType._a < 1, 2, 3.0, 4.0 > ; specify the field directly and promote integers to floats
mything4 UnionType._b < 1, 2, 3, 4 > ; specify the field, normal dwords


So that is an example of where we try to extend the core assembler in a way that is backwards compatible.

For things that fall into the HLL category the key is that they should never prevent the user from doing things their own way or at the lowest possible level, but provide a mechanism to simplify a lot of the
boiler plate code that even as a veteran assembler programmer starts to become annoying.. In this area I agree with Nidud's approach that borrowing familiar syntax from C is a good option, as long as we don't overdo it.

I'm trying as far as possible to also ensure that all of this HLL work either lives entirely in the built-in macro library thus relying on the existing assembler core or by implementing it in combination with a light pre-pre-processor step. This way the HLL functionality is kept modular, independant from the assembler itself and can be switched on/off or removed if required, the other great thing about stuff in the macro library is you can replace any of the macros with your own at assemble-time, due to the fact that macros can be redefined.

Anyway, thats just my thoughts on how we proceed. There have been a couple of things I've wanted to add but haven't for these reasons, as there is potential for either confusion or obfuscation of implementation, one of them being support for overloading procedures, which can work fine with HLL constructs.. but would be a total disaster with a direct low level CALL, hence it's exclusion from future plans.

We have to accept that as much as we love assembler, it needs to progress in SOME direction to attract more people and even for those of us that don't need to be convinced about the joy of writing assembler it just needs to be made more amenable to working on large projects with less pain and maintenance issues. This is what I have found in my experience at least.. but I agree 100% that nothing new should ever be added that detracts from it's purity or ability to operate at any required level of abstraction.
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: nidud on December 28, 2017, 03:01:03 AM
As for the dark side of C++ this is currently overwhelmed by all the sparks coming out this very concept which is brilliant. The problem here is lack of efficiency and speed.

Another problem with the accumulated complexity of the language is that it's impossible today to write a C++ compiler from scratch. What you get instead is compilers built on top of these to reduce the complexity in order to reduce some of the overhead.

The reason for the accumulated complexity cant really be pinned on one specific idea, person or event but rather a logical extensions of already existing functionality so it sort of make sense, but that doesn't really help.

The problem here is that the we have to interact with these libraries, hence the reason for the COM interface extension (http://masm32.com/board/index.php?topic=5929.msg72173#msg72173) to simplify this interaction.

As for the future I think this will go more towards low energy consumption, higher efficiency and faster execution time rather than mass production of water cooled high energy consuming CPU's. I also think the idea of using this brilliant concept to produce the software, profile out the bottlenecks and convert to assembler will be obsolete in the future. You will need fast and compact libraries to achieve this, optimize from start as oppose to fix it later.
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: nidud on December 28, 2017, 04:04:48 AM
We have to accept that as much as we love assembler, it needs to progress in SOME direction to attract more people and even for those of us that don't need to be convinced about the joy of writing assembler it just needs to be made more amenable to working on large projects with less pain and maintenance issues. This is what I have found in my experience at least.. but I agree 100% that nothing new should ever be added that detracts from it's purity or ability to operate at any required level of abstraction.

I don't believe in compromise with regard to the extension of the HLL section: you either do it or you don't. This means that it have to be linguistically correct, rule-based, simple and logical. This is hard to achieve without using existing syntax. You may try doing it half way, make a few  exceptions here and there. It will work but it wont stick in the long run.

As for taking advice from old farts hanging around this forum that may not be a good idea in this case. We are all typecast in our own habits of doing things and thus a bit anti progressive. So think about the children John: THE CHILDREN  :lol:
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: nidud on December 28, 2017, 04:41:27 AM
Unfortunately, your program crashes. It looks like there is something wrong on the stack.

I removed the include files to simplify the build. They include a win64 stack option so that may be the reason. I added this and a few changes so it works (as before) but it only prints out the supported instruction sets on my end.
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 28, 2017, 05:20:01 AM
Hi nidud,

thank you for providing the new version. It works now without complaint.
Code: [Select]
Supported Features by Processor and Operating System
====================================================

Vendor String: GenuineIntel
Brand  String: Intel(R)Core(TM)i7-7820XCPU@3.60GHz

Instruction Sets
----------------

MMX  SSE  SSE2  SSE3  SSSE3  SSE4.1  SSE4.2  AVX  AVX2  AVX-512 F

It's safe to use the following AVX-512 extensions with this machine:
--------------------------------------------------------------------

AVX-512 DQ      : DWORD and QWORD instructions (conversion, transcendental support etc.)
AVX-512 CD      : Conflict Detection instructions offer additional vectorization of loops with possible
                  address conflicts.
AVX-512 BW      : Byte and Word support for 8- and 16-bit integers.
AVX-512 VL      : Vector Length instructions add vector length orthogonality, allowing most AVX-512 instructions
                  to also operate on XMM and YMM registers.

Any processor that implements any portion of the AVX-512 extensions MUST implement AVX-512 F. Some AVX-512 extensions
are currently only planned by Intel. Not every architecture has all the instruction sets built in:

Knights Landing provides      : CD, ER, and PF.
Skylake provides              : CD, BW, DQ, and VL.
For Cannon Lake are scheduled : CD, BW, DQ, VL, IFMA, and VBMI.
For Icelake are scheduled     : CD, BW, DQ, VL, IFMA, and VBMI.
For Knights Mill are scheduled: CD, ER, PF, 4FMAPS, 4VNNIW, and VPOPCNTDQ.

That's what Intel has released so far.
Solid work.  :t

aw27,

The ASM code does follow the 64-bit ABI rules and some of the procedures are running with the stack unaligned.
This is a simple comment, I know you can do much better otherwise would not be producing software to look for the Higs Bosom.  :(
I love that sort of comments. Allow me, however, the following minimum marginalia:

Gunther
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: AW on December 28, 2017, 06:12:06 AM
Quote
My assembler procedures were directed by the compiler. The compiler takes care of the correct stack alignment for the function call, right?

Yes, for the function call but not for the external function itself.
This has to be done by the ASM routine itself, manually or by the assembler.
The way you declare the procedures will not help the assembler to do it for you. Do you remember that there is a USES clause?

Quote
The ASMC source is not mine
I am not talking about the ASMC part. Your ASM part could well crash the program if there were any API function call or instruction that require alignment.
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 28, 2017, 06:54:07 AM
Yes, for the function call but not for the external function itself.
Fine, but my assembly language procedures are designed for a high-level language call.

Your ASM part could well crash the program if there were any API function call or instruction that require alignment.
Right, but in the assembly language part, not a single API function is called. What the hell is going to crash that? Yes, if, and would and could, but that's hair splitting. All registers used by CPUID are saved in advance on the stack and restored at the end. More is really not required if the API is not used, right?

I already thought something about it. With very few changes (because of the different ABI), the procedure can also be used under SCO UNIX, BSD, Linux and OS X. That's why there is not a single API call in it. There really is not only Windows in this world. Sometimes you have to work in very heterogeneous computer clusters, no matter how they have historically evolved. Whether you like it or not doesn't matter.

Gunther
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: AW on December 28, 2017, 07:15:55 AM
Fine, but my assembly language procedures are designed for a high-level language call.
They are not, and you have no clear idea of what I am talking about; may be we should be talking about chess today.

Quote
I already thought something about it. With very few changes (because of the different ABI), the procedure can also be used under SCO UNIX, BSD, Linux and OS X. That's why there is not a single API call in it. There really is not only Windows in this world. Sometimes you have to work in very heterogeneous computer clusters, no matter how they have historically evolved. Whether you like it or not doesn't matter.
From what I have seen in the last few days all the small programs you brought here have flaws. Instead of fixing you blame the tools and the people.


Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 28, 2017, 09:51:26 AM
aw27,

They are not, and you have no clear idea of what I am talking about;
Is it even worthwhile to go into detail here? First the C prototypes (lines 10 to 14 in the C source code).
Code: [Select]
extern long long int CpuVs(char *p);
extern long long int GetPbs(void);
extern void CopyPbs(char *p);
extern long long int Iset(void);
extern void GetAVX(long long int *p);
Then an example of the correct function call (line 111 in the C source).
Code: [Select]
featurenumber = Iset();
Who can read is clearly in the advantage.

may be we should be talking about chess today.
You can do that in the Colosseum, with who you want, but please don't in my thread.

From what I have seen in the last few days all the small programs you brought here have flaws.
Very nice, then do not use this code, no one is forcing you and certainly not me.

Instead of fixing you blame the tools and the people.

Is it that what you mean with blaming the tools?
That's made with the latest release of UASM, an excellent tool that works without complaint, by the way.

Or did you mean that maybe, with blaming the people?
Branislaw (aka Habran), Johnsa and you have done a great deal, hats off.

In that sense, I plead guilty, but who knows. Dixi et salvavi animam meam.

Gunther







Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: AW on December 28, 2017, 09:10:31 PM
They are not, and you have no clear idea of what I am talking about;
Is it even worthwhile to go into detail here? First the C prototypes (lines 10 to 14 in the C source code).
Code: [Select]
extern long long int CpuVs(char *p);
extern long long int GetPbs(void);
extern void CopyPbs(char *p);
extern long long int Iset(void);
extern void GetAVX(long long int *p);
You don't understand the replies, or skip over them, let me repeat:
Yes, for the function call but not for the external function itself.
This has to be done by the ASM routine itself, manually or by the assembler.
The way you declare the procedures will not help the assembler to do it for you. Do you remember that there is a USES clause?

From what I have seen in the last few days all the small programs you brought here have flaws.
Very nice, then do not use this code, no one is forcing you and certainly not me.
Of course not. CPU Extensions detection code is everywhere and I have software that reports up to avx512 since long. Even so, it is strange you could not make your routines work properly.

That's made with the latest release of UASM, an excellent tool that works without complaint, by the way.
And you also said:
"
That's exactly Hutch's idea: bring the assembler back to the desktop. Especially young, inexperienced high-level language programmers will benefit from your work. The other side of the coin is: There are cases where these comfortable high-level language elements are even harmful.
"

You are looking at UASM as a tool for beginners, when it is not. Beginners learn rough MASM in school through a six month intensive course where they can't use HLL elements or parameters on the PROC line. They follow a book written by Kip Irvine.

You really live in another planet. The planet where companies hire developers and pay them 120000 euro per anum to develop PowerPoint add-ins without even checking their capabilities.




Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: jj2007 on December 28, 2017, 09:23:59 PM
You are looking at UASM as a tool for beginners, when it is not. Beginners learn rough MASM in school through a six month intensive course where they can't use HLL elements or parameters on the PROC line. They follow a book written by Kip Irvine.

No need for strong declarations here: UAsm is in practice 99% compatible with MASM, and 100% for the things that beginners use (there are only a few exotic cases in macros etc where UAsm differs from Masm). And while some beginners are apparently forced into that course, they are not a relevant target group IMHO. However, even these poor students can use UAsm as if it was Masm.

But one problem with the forks (UAsm + AsmC) is indeed that they have new fancy features that can cause trouble, either because the experts who are supposed to use them don't read the FM, or because the experts developing them got lost in too much detail ;)
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: AW on December 28, 2017, 09:36:58 PM
And while some beginners are apparently forced into that course, they are not a relevant target group IMHO. However, even these poor students can use UAsm as if it was Masm.
I am not criticizing the courses, they are good. Students can indeed learn a lot by coding this way:

FastIntegerSQR PROC
   push ebp
   mov ebp, esp
   mov eax, [ebp+8]
   mov edx, [ebp+0ch]
   cmp eax, edx
   jbe @L1
   shr edx,1
   jmp short @L2
@L1:   
   shr eax, 1
@L2:
   add eax, edx
   pop ebp
   ret 8
FastIntegerSQR ENDP
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: johnsa on December 28, 2017, 11:42:11 PM
We have to accept that as much as we love assembler, it needs to progress in SOME direction to attract more people and even for those of us that don't need to be convinced about the joy of writing assembler it just needs to be made more amenable to working on large projects with less pain and maintenance issues. This is what I have found in my experience at least.. but I agree 100% that nothing new should ever be added that detracts from it's purity or ability to operate at any required level of abstraction.

I don't believe in compromise with regard to the extension of the HLL section: you either do it or you don't. This means that it have to be linguistically correct, rule-based, simple and logical. This is hard to achieve without using existing syntax. You may try doing it half way, make a few  exceptions here and there. It will work but it wont stick in the long run.

As for taking advice from old farts hanging around this forum that may not be a good idea in this case. We are all typecast in our own habits of doing things and thus a bit anti progressive. So think about the children John: THE CHILDREN  :lol:

Agreed, it must be correct linguistically and logically, which is yet another good reason to borrow from C, it's familiar and we have a know set of lexing rules and expectations to follow.
I'm all for the children.. I myself am still trying to convince mine to use my Amiga 1200 (which has Devpac 3.18 and all the h/w + rom kernel manuals) to do some funky copper and system take-over code ;) (note they're only 5 and 7 .. so It's hard to convince them to leave the xbox alone to code!)
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: hutch-- on December 29, 2017, 01:02:24 AM
I don't see any problems with using at least some of the C format, it is as John said, don't over do it or you will end up with the clutter and restrictions of C with its strong typing and other irritations. IF blocks are fine, SWITCH blocks work OK and give you clean code without loss but exercise caution with pre-built loop code systems as they are internally obscure and are often hard to translate back to mnemonic code.

In 64 bit having reliable pre-built stackframe procedure entry and exit is sensible as the Microsoft ABI is complex to write manually and with no real gain by doing so, its easy enough to write the simple 4 args or less style of stackframe free leaf procedures directly using registers and if you want this automated, it works fine and with no loss simply as a macro. A word of wisdom here, instead of hard coding "invoke", make the technique with another name and use a macro wrapper to get invoke as this allows you to use the call automation code in a number of other ways such as function form macros with return values.

In a crude pseudo macro,

MyInvoke MACRO args:VARARG
    InternalCallAutomation args
    EXITM <rax>
ENDM

Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 29, 2017, 04:43:14 AM
Jochen,

No need for strong declarations here: UAsm is in practice 99% compatible with MASM, and 100% for the things that beginners use (there are only a few exotic cases in macros etc where UAsm differs from Masm). And while some beginners are apparently forced into that course, they are not a relevant target group IMHO. However, even these poor students can use UAsm as if it was Masm.

But one problem with the forks (UAsm + AsmC) is indeed that they have new fancy features that can cause trouble, either because the experts who are supposed to use them don't read the FM, or because the experts developing them got lost in too much detail ;)

I'm all in your opinion. But you know yourself (do not feed the troll), some people should not even be ignored. The reason is simple: these people are coming before the good Lord. God knows everything, but these people know everything better.

I firmly believe that it would be wiser to discuss these issues in a separate thread in the Soap Box or in the Colosseum. There, of course, you have to expect that is discussed again in a circle. But everyone can then decide for themselves whether he wants to post or not.

Gunther
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: johnsa on December 29, 2017, 07:35:08 AM

"or because the experts developing them got lost in too much detail" ...

That's why we have JJ around :) If we break it, he'll find it!  :t

I think the important point is that although we're maintaining and building uasm.. it's for everyone.. so if you have ideas/suggestions I'm always listening, so possibly unlike other tools like masm or even nasm where really the community has very little say.. we can always accommodate or debate and find solutions that work, and I like to think that we're the easiest going most approachable people ever, who respond to emails and  hopefully give a real quick turnaround in making fixes or updates :)
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: jj2007 on December 29, 2017, 08:57:18 AM
That's why we have JJ around :) If we break it, he'll find it!  :t
I will find it automagically because somewhere in my fat sources a particularly weird little macro will say "oops, that was unexpected" - and then our dream team johnsa & habran have sleepless nights :P

To be fair, others (and in particular José) have put more work into identifying the little bugs. There are not many left anyway, it's fine tuning time :t

UAsm is a great achievement. It beats the original hands down in terms of speed, and that counts a lot when the asm sources grow a little bit bigger. Plus, it is almost bug free, in contrast to the latest ML64 versions. But still, my recommendation would be to concentrate on making UAsm perfect rather than adding new features. Most of them can be implemented with macros anyway.

In 64-bit land, the biggest problem IMHO is not the assembler but the missing equivalent to the Masm32 SDK. Hutch is working hard on a 64-bit SDK, and he is brilliant at that, of course, but he decided to serve only ML64. I couldn't care less because I am not at all interested in 64-bit code, but I believe the success of the Masm32 SDK can't be repeated if coders are forced to use exotic stuff like .if eax}=ecx 8)
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: hutch-- on December 29, 2017, 02:02:39 PM
I commend your rate of persistence but the ugly facts keep raising their ugly head.

1. It is already development/application capable.
2. MASM in various forms has been around for 30 years.
3. It is designed for assembler programmers, not script kiddies.
4. If you can't manage 64 bit MASM, get a visual garbage generator.  :P
5. If ".if rax {} 1234" is too difficult for you to handle, do it with mnemonics like REAL MEN[tm].
6. If you want a compiler, try CL.EXE, its the industry standard.
7. 64 bit MASM is a standard component of Microsoft VS, it has a massive level of distribution.

After waiting for years for Japheth to actually finish JWASM for 64 bit and still not seeing HJWASM/UASM finished after years of work, I use MASM because its here, now, up and running and it produces what code you can write correctly, no hot little hand holding, easily able to create libraries, does not have a pile of specification clutter that you have to negotiate at the front to get it to build 64 bit code and it has the normal manners of most decent assemblers, truly intolerant, terse notation, bites the hand that feeds it and pelts garbage at you if you get it wrong.  :badgrin:

PS : The MASM32 SDK will never be repeated, it came from a world that no longer exists, many clever people worked on it, Iczelion and a number of others on NDAs who will never be identified who contributed code, ideas and testing. What is missing is the collaborative effort as those people no longer exist, I am a dinosaur from that era where people contributed rather than just suck the life out of what was available.
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: jj2007 on December 29, 2017, 07:54:18 PM
I commend your rate of persistence

My apologies - I didn't intend to step on your toes :icon14:
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: johnsa on December 29, 2017, 10:25:01 PM
I commend your rate of persistence but the ugly facts keep raising their ugly head.

1. It is already development/application capable.
2. MASM in various forms has been around for 30 years.
3. It is designed for assembler programmers, not script kiddies.
4. If you can't manage 64 bit MASM, get a visual garbage generator.  :P
5. If ".if rax {} 1234" is too difficult for you to handle, do it with mnemonics like REAL MEN[tm].
6. If you want a compiler, try CL.EXE, its the industry standard.
7. 64 bit MASM is a standard component of Microsoft VS, it has a massive level of distribution.

After waiting for years for Japheth to actually finish JWASM for 64 bit and still not seeing HJWASM/UASM finished after years of work, I use MASM because its here, now, up and running and it produces what code you can write correctly, no hot little hand holding, easily able to create libraries, does not have a pile of specification clutter that you have to negotiate at the front to get it to build 64 bit code and it has the normal manners of most decent assemblers, truly intolerant, terse notation, bites the hand that feeds it and pelts garbage at you if you get it wrong.  :badgrin:

PS : The MASM32 SDK will never be repeated, it came from a world that no longer exists, many clever people worked on it, Iczelion and a number of others on NDAs who will never be identified who contributed code, ideas and testing. What is missing is the collaborative effort as those people no longer exist, I am a dinosaur from that era where people contributed rather than just suck the life out of what was available.

As far as I'm aware there shouldn't much left to complete in UASM especially for Windows based development? If I'm missing something let me know ? :)
My next chunk of work is adding Dwarf debug output for OSX and Linux which should bring those platforms in line with what is possible on Windows tooling wise (apart from the lack of a decent debugger.. I detest GDB). We're also working on a revised Codegen system which will automatically generated opcode maps and instruction tables from the intel/amd documentation, and with a much more flexible extensive instruction table format we're aiming to reduce the codegen size by about 70%, parser by about 15% and would expect 2X performance increase of the codegen portion of the assembler. I've spent a lot of time putting the automated regression testing in place, so that bringing in the new codegen should be flawless in terms of opcode generation.

Once this is in place I will be working on a set of 64bit run-time libraries ( my idea is to create both procedural and OO forms ), including the built-in macro library and converting/updating wininc to include dx, opengl, vulkan, asio etc..
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: hutch-- on December 30, 2017, 02:00:42 AM
You are not the target here John, I just get tired of hearing the same crap over and over again. I don't personally care what anyone uses and I have used some shockers over time myself but I don't foulmouth the GNU as assembler (GAS), Tomas's FASM, Bogdan's assembler or anyone elses tools but I will not be subject to abuse for not kissing the arse of open "sauce" or using Microsoft software.
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: Gunther on December 30, 2017, 02:56:08 AM
You are not the target here John, I just get tired of hearing the same crap over and over again.
I feel the same way.

I don't personally care what anyone uses and I have used some shockers over time myself but I don't foulmouth the GNU as assembler (GAS), Tomas's FASM, Bogdan's assembler or anyone elses tools but I will not be subject to abuse for not kissing the arse of open "sauce" or using Microsoft software.
That's true. GAS does a good job. And Bogdan's Sol ASM (http://oby.ro/sol_asm/) is also a solid tool that you can work with very well. For me, there is no reason at all to look arrogantly at the tools of other programmers.

Gunther
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: johnsa on December 30, 2017, 03:01:22 AM
I had some great times with FASM, it was the only tool that worked for me doing OS development in ASM without a lot of pain, it's where I "borrowed" the idea of flat mode for UASM, which makes writing mixed mode code much easier.. I've got a pretty complete legacy+pxe+uefi bootloader + 64bit smp kernel running now (one of my other little projects) :)
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: hutch-- on December 31, 2017, 01:15:57 PM
Something I needed to add, long ago when I first worked on the MASM32 project, I worked with a number of very talented people to get enough code, ideas and testing done. What I would like to see with UASM is a collection of its users get off their arse and contribute to the endless work that John and Habran have done to get UASM up and running.

Header file are incredibly tedious things to get up and going, I note with some humour that the windows.inc developed over time for the 32 bit MASM32 SDK have contributed to many other projects with different assemblers. Vasily's include file had vast amounts of the masm32 windows.inc in it and over time I have seen many others use bits of it.

The next item on the agenda is writing library modules. This is at least PHUN if you like writing algorithms but it can be just as tedious as writing header files as without an extensive library of among other things, API components for Windows, the assembler is crippled down to the level of console apps. The UNIX guy will also have to do something similar.

Lastly the only thing I hate more than fixing header files is writing CHM help files. It is a soul destroying grindingly irritating task but without it the potential user base has no real way of understanding what the assembler can do.

For those who want to see UASM become a widely accepted programming tool, the time to step up and actually do some work is now, people hanging around sucking the life out of the project and not contributing are not making it any more successful.
Title: Re: Instruction Set Detection including AVX-512 sub-sets
Post by: rsala on January 01, 2018, 12:26:43 AM
The 64-bit version of Easy Code v2 is always compiled with UASM64. I always test the UASM new versions and report any issue to John. Also, all ".inc" and ".lib" files included with EC v2 are always tested with UASM64 and they are fully compatible. All included example projects for UASM, both 32-bit and 64-bit are always tested and work fine.