vpcmpnleub/vpcmpub

[JoeBr]

Either of the lines below will assemble/link fine, but when run generate an Invalid Instruction error (starting with C4 instr encoding) - wondering if you can take a look (or if someone can give me the instruction encoding as a workaround, let me know, unable to find either in X64Dbg - 'Assemble' Right-Click - with the most recent version):

      vpcmpnleub ymm3, ymm3, ymm4
      vpcmpub ymm3, ymm3, ymm4, 6

Thanks

Joe

[jj2007]

Interesting  8)

Code Select Expand

  vpcmpnleub ymm3, ymm3, ymm4
  nops 5
  db 0C4h, 0E3h, 65h, 3Eh, 0DCh 
  nops 5
  vpcmpub ymm3, ymm3, ymm4, 6
  nops 5
  db 0C4h, 0E3h, 65h, 3Eh, 0DCh, 06h 
  nops 5
The second instruction gets the same encoding as the first one, only that a 6 is added. And the code throws an exception.

- wrong encoding?
- right encoding but cpu doesn't support the instruction?

[JoeBr]

Good notes/question - and thanks.  MyBad - looks like these are only included in the AVX512 'set' and newer (albeit they support VEX256 and below as well) per https://en.wikipedia.org/wiki/AVX-512

CPU is an I7 4790K - so I'll go another route.

Thanks Again :)

Joe

[jj2007]

There might also be a problem with the encoding: db 0C4h, 0E3h, 65h, 3Eh, 0DCh, 06h
The trailing 6 is the only difference between the two instructions. How does the CPU know the meaning of the byte that follows 0DCh? It can be 0...7. The instruction add eax, 12345678h, for example, starts with 05.

[habran]

Hi JoeBr
That is good find error.
It is EVEX instruction and UASM should have thrown an error in assembling:
VPCMPUB k1 {k2}, ymm2, ymm3/m256, imm8
as you can see the first register can only be Kn register and not ymm register
with the option EVEX enabled your code produces this:

Code Select Expand


   317:     vpcmpub ymm3, ymm3, ymm4, 6
00007ff7267b1803 62 F3 65 28 3E DC 06             vpcmpub k3, k0, ymm3, ymm4, 0x6 
 
so uasm treats ymm3 as K3 register instead throwing an error

It will be fixed as soon as I get some spare time

[habran]

Thank you again JoeBr :t
I have found out that all VCMPxx and VPCMPxx if they are EVEX instruction first register has to be a musk register (K1 to K7).
They assemble OK if it is written properly but don't report error if it is missused.
The issue has been overseen because AVX and AVX2 are using xmm and ymm registers as a first parameter.
These instructions are specifically EVEX instructions and can be only used with mask registers as a first parameter:  VPCMPB,VPCMPUB,VPCMPD,VPCMPUD,VPCMPQ,VPCMPUQ,VPCMPW,VPCMPUW
the rest can be used as AVX, AVX2 as long as registers are not higher than 15 and are not ZMM registers
UASM will assemble them as AVX instructions.

Will be fixed soon. 

[JoeBr]

Thanks all again for looking into this.

Joe