Author Topic: AVX2 __m256i (ymmword) variable byte shift.  (Read 2864 times)


  • Regular Member
  • *
  • Posts: 18
Re: AVX2 __m256i (ymmword) variable byte shift.
« Reply #15 on: March 02, 2021, 07:54:37 PM »
No. See reply #9.
I tested it. Setting LAA to disabled causes any allocation >= 2Gb to fail.

In the process I discovered 2 bugs in the way c++ works.
(1) For a pointer symbol P, which is identical to &P[0], the c++ doe something stupid and crashes unless the latter is used.
(2) 64-bit integers used for addresses and bytes. Absolute nightmare. 1024*1024*1024*2 equals 2 billion but I must be wrong apparently because the compiler says it equals 18 quadrillion and tries to allocate those bytes.

This is the fastest method to shift right by edx bytes.
Code: [Select]
; Shift Variable Right x bytes (RCX == input ymmword. RDX == count bytes)
ASM_VarShift_Right proc
vmovdqu ymm0, ymmword ptr [rcx] ;extract parameter
vmovd xmm2, edx
mov eax,15
vpbroadcastb ymm2, xmm2
vpaddb ymm1, ymm2, ymmword ptr Shuffle_Order
vmovd xmm3, eax
vpbroadcastb ymm3, xmm3
vpcmpgtb ymm4, ymm1, ymm3
vpand ymm2, ymm1, ymm3
vpshufb ymm0, ymm0, ymm2
vextracti128 xmm3, ymm0, 1
vpblendvb ymm0, ymm0, ymm3, ymm4
Shuffle_Order BYTE 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
ASM_VarShift_Right endp


  • Member
  • *****
  • Posts: 11887
  • Assembler is fun ;-)
    • MasmBasic
Re: AVX2 __m256i (ymmword) variable byte shift.
« Reply #16 on: March 02, 2021, 09:54:27 PM »
LargeAddressAware does nothing anymore. Its a 32-bit thing.

No, it's a 64-bit thing, too:

LAA ON: The address is at 40003018h
LAA OFF: The address is at 403018h

Code: [Select]
include \Masm32\MasmBasic\Res\ ; OPT_64 1 ; for 32 bit, 1 for 64 bit assembly
_number dq 1234567890123456789
  ifidn @Environ(oDebugL), </Largeaddressaware:NO>
Print "LAA OFF: "
Print "LAA ON: "
  Inkey Str$("The address is at %xh\n", offset _number)

OPT_DebugL /Largeaddressaware:NO  ; disable to get LAA ON