Fast Bit Reversal algorithm

[Siekmanski]

    Bit-reversal is needed for a radix 2 FFT so, checked the output again.

    When I was looking for repeating patterns in my earlier bit-reversal routine, I noticed this:
    I could do bit-reversal without reversing bits.

8:   0  4  2  6                   |   1 5 3  7
16: 0  8  4 12 2 10  6 14  |   1 9 5 13 3 11 7 15
32: 0 16 8 24 4 20 12 28 etc. ( every next row is the next power of two )
   
    Second half = first half + 1 and my first thought was, cool, reduce size by 2 and add 1 to get the rest.
    To do it this way, you need a lot of memory read\writes so, not an option.
    Looked over it a few times and, EUREKA !

    I can create a 4*4 matrix to calculate 32 bit-reversals with 1 index ( 16 + 16+1 )
    by transposing 0 8 4 12 ( first quarter of length 16 ) to the first column.
    Then fill the rest of the columns for a 512 size ( 16 * 32 values from 16 indices )
    And it works, the matrix can be used to calculate all powers of two lengths and writes 4 values at once.
   
    Just by only adding the values with 1 and putting them at the right place via the matrix indices I can get all bit-reversals.   

    It's easy to create a new matrix from the BitreversalMatrix for all other power of two sizes.
    By multiplying ( power of two up ) or dividing the BitreversalMatrix ( power of two down ).
    Multiplying is of cource done by shifting ( because we are dealing with power of 2 calculations ).

    For example size 1024:
    Multiply the Matrix by 2 ( 1024 / 512 ) and add 4 for every next 4*4 Matrix step.
    For size 2048 then multiply the Matrix by 4 ( 2048 / 512 ) etc.
    Why dividing by 512 ?, because the BitreversalMatrix represents length 512.

    Next thing to do:
    Initialize the BitreversalMatrix for the size you need.
    Put the complete BitreversalMatrix into xmm4 - xmm7.
    Copy every next row to xmm3 then shuffle each next index to eax ( no memory reads anymore ).
    When you have done all 4 rows, add 4 for to every next 4*4 Matrix step.
    So you can keep all code in registers and use the complete data cache ( size is also a power of two ) to do fast bit-reversals.
    For example if you have a 32 KB L1 data cache, you can do a 8192 size bit-reversal very fast.

    I'll code this soon. Decided to post it at this stage because it is easier to explain the algorithm this way.
   
 Some eaxamples to show the logic of creating the 4*4 matrix:

BitreversalMatrix   dd 00,32,16,48
                           dd 08,40,24,56
                           dd 04,36,20,52
                           dd 12,44,28,60
 
1024 dd 000, 064, 032, 096   ; First  row = BitreversalMatrix * 2 ( 1024 / 512 = 2 )
        dd 016, 080, 048, 112   ; Second row = BitreversalMatrix * 2
        dd 008, 072, 040, 104   ; Third  row = BitreversalMatrix * 2
        dd 024, 088, 056, 120   ; Fourth row = BitreversalMatrix * 2

        Next step refresh the Matrix with these new 16 values:

        dd 004, 068, 036, 100   ; First  row + 4
        dd 020, 084, 052, 116   ; Second row + 4
        dd 012, 076, 044, 108   ; Third  row + 4
        dd 028, 092, 060, 124   ; Fourth row + 4

512     ==  BitreversalMatrix

256     dd 00, 16, 08, 24       ; First  row = BitreversalMatrix / 2 ( 256 / 512 = 0.5 )
          dd 04, 20, 12, 28       ; Second row = BitreversalMatrix / 2

128     dd 00, 08, 04, 12       ; First  row = BitreversalMatrix / 4 ( 128 / 512 = 0.25 )

64       dd 00, 04, 02, 06       ; First  row = BitreversalMatrix / 8 (  64 / 512 = 0.125 )

32       dd 00, 02, 01, 03       ; First  row = BitreversalMatrix / 16 ( 32 / 512 = 0.0625 )

16  and 8 are less than 32 and are therefore special cases, but can be calculated like this,

    lea     esi,BitreversalMatrix
    lea     edi,BitreversalTable

    movdqa  xmm0,oword ptr [esi]
    movdqa  xmm1,oword ptr BitreversalADD

    cmp     BitreversalSize,16
    je      size_16
    cmp     BitreversalSize,8
    jne     done
    psrld   xmm0,3
    movdqa  oword ptr[edi],xmm0
    paddd   xmm0,xmm1
    movdqa  oword ptr[edi+BitreversalSize*2],xmm0
done:
    ret
size_16:
    psrld   xmm0,2
    movdqa  oword ptr[edi],xmm0
    paddd   xmm0,xmm1
    movdqa  oword ptr[edi+BitreversalSize*2],xmm0
    paddd   xmm0,xmm1
    movdqa  oword ptr[edi+BitreversalSize],xmm0
    paddd   xmm0,xmm1
    movdqa  oword ptr[edi+BitreversalSize*2+BitreversalSize],xmm0
    ret

[dedndave]

you may find this post helpful Marinus

http://www.masmforum.com/board/index.php?topic=12722.msg98224#msg98224

[Siekmanski]

Hi Dave,

I'll compare the execution speed of those algorithms with mine this weekend.
Thanks for the link.

[dedndave]

it wasn't my intention that you should use that code, directly
but, if you use that method with SSE code, it should be quite fast   

[Siekmanski]

I'm always in the learning mood so, i'll study those examples.   
My version does 4 bit-reversals in one go. ( in fact 32 with 1 index pointer )
It doesn't do bit-reversal of single values but creates a table for a power of two size.
For example 128 values from 0 - 127 bit-reversed.
I could use a fast one to construct a time decomposition table for my FFT routine.

[Siekmanski]

Hi Dave,

I have studied the bit reversal routines from the thread you posted.
I can not make a fair comparison because you all did a 32 bit bit-reversal of 1 value.

My routine does 128 values and writes them in a memory-buffer.
In my routine it doesn't matter how manny bits you need to reverse.
I still have to implement the Matrix preparation code for every bit-size.
No additional calculations are needed in the main loop, only for matrix shuffles and summations.

The routine in the attachment is fixed for a table of 128 values.
In this example for 128 values it reverses all 7 bits per dword, from value 0 --- > 127

I think it's rather fast including mem-writes. 

Timings on my machine:

Siekmanski bitswap: 82 cycles per 128 values = 0.640625 cycles per value
drizzNew stir-fry bitswap: 3 cycles per value ( the fastest from your posted thread )

Code: [Select]

Fast Bit-reversal using a 4*4 Matrix by Siekmanski 2016.

Please wait 4 seconds before timing begins

82 Cycles for 128 values.

Bitreversed Table results:
000 064 032 096 016 080 048 112 008 072 040 104 024 088 056 120 004 068 036 100
020 084 052 116 012 076 044 108 028 092 060 124 002 066 034 098 018 082 050 114
010 074 042 106 026 090 058 122 006 070 038 102 022 086 054 118 014 078 046 110
030 094 062 126 001 065 033 097 017 081 049 113 009 073 041 105 025 089 057 121
005 069 037 101 021 085 053 117 013 077 045 109 029 093 061 125 003 067 035 099
019 083 051 115 011 075 043 107 027 091 059 123 007 071 039 103 023 087 055 119
015 079 047 111 031 095 063 127

Press any key to continue...
Marinus

[Siekmanski]

If have included "drizzNew stir-fry bitswap routine" to create a 128 size table.
Note: his routine includes memory writes now. ( for creating the table )

Code: [Select]

Fast Bit-reversal using a 4*4 Matrix by Siekmanski 2016.

81 Cycles Siekmanski
3585 Cycles drizzNew stir-fry

Siekmanski's table:
000 064 032 096 016 080 048 112 008 072 040 104 024 088 056 120 004 068 036 100
020 084 052 116 012 076 044 108 028 092 060 124 002 066 034 098 018 082 050 114
010 074 042 106 026 090 058 122 006 070 038 102 022 086 054 118 014 078 046 110
030 094 062 126 001 065 033 097 017 081 049 113 009 073 041 105 025 089 057 121
005 069 037 101 021 085 053 117 013 077 045 109 029 093 061 125 003 067 035 099
019 083 051 115 011 075 043 107 027 091 059 123 007 071 039 103 023 087 055 119
015 079 047 111 031 095 063 127

Drizznew's table:
000 064 032 096 016 080 048 112 008 072 040 104 024 088 056 120 004 068 036 100
020 084 052 116 012 076 044 108 028 092 060 124 002 066 034 098 018 082 050 114
010 074 042 106 026 090 058 122 006 070 038 102 022 086 054 118 014 078 046 110
030 094 062 126 001 065 033 097 017 081 049 113 009 073 041 105 025 089 057 121
005 069 037 101 021 085 053 117 013 077 045 109 029 093 061 125 003 067 035 099
019 083 051 115 011 075 043 107 027 091 059 123 007 071 039 103 023 087 055 119
015 079 047 111 031 095 063 127

Press any key to continue...

[Siekmanski]

Hi ognil,
Replaced it with the faster drizzNew stir-fry 
It is still not a fair comparison, because drizznew's routine is optimised for one 32 bit reversal.

Code: [Select]

Fast Bit-reversal using a 4*4 Matrix by Siekmanski 2016.

76 Cycles Siekmanski
2765 Cycles drizzNew stir-fry

Siekmanski's table:
000 064 032 096 016 080 048 112 008 072 040 104 024 088 056 120 004 068 036 100
020 084 052 116 012 076 044 108 028 092 060 124 002 066 034 098 018 082 050 114
010 074 042 106 026 090 058 122 006 070 038 102 022 086 054 118 014 078 046 110
030 094 062 126 001 065 033 097 017 081 049 113 009 073 041 105 025 089 057 121
005 069 037 101 021 085 053 117 013 077 045 109 029 093 061 125 003 067 035 099
019 083 051 115 011 075 043 107 027 091 059 123 007 071 039 103 023 087 055 119
015 079 047 111 031 095 063 127

Drizznew's table:
000 064 032 096 016 080 048 112 008 072 040 104 024 088 056 120 004 068 036 100
020 084 052 116 012 076 044 108 028 092 060 124 002 066 034 098 018 082 050 114
010 074 042 106 026 090 058 122 006 070 038 102 022 086 054 118 014 078 046 110
030 094 062 126 001 065 033 097 017 081 049 113 009 073 041 105 025 089 057 121
005 069 037 101 021 085 053 117 013 077 045 109 029 093 061 125 003 067 035 099
019 083 051 115 011 075 043 107 027 091 059 123 007 071 039 103 023 087 055 119
015 079 047 111 031 095 063 127

Press any key to continue...

[dedndave]

you missed my intention
examine the stir-fry code - see how it works
then, write SSE code that uses a similar technique   

[Siekmanski]

Hi Dave,

I did understand your intention for me to have a look at Drizz's stir-fry method. 
I have studied it, and it can be done with that very clever technique from Drizz in SSE code to calculate 4 DWORD values in bit-reversed order at once.
For that case the technique of Drizz is unbeatable.

But, my goal was to get an array of all the values from 0 to 255 in a bit-reversed order ( or another power of two length array ).

I found a way to calculate a bit-reversed array without the need of bit mangling or bit manipulation. 

It skips all the calculations needed to reverse the bits of each value.
Because of skipping those calculations it has become a very fast routine which exactly fits my need of a fast calculated bit-reversed array.
In this case it is much faster then using Drizz's stir-fry method in SSE code.

The only math in my routine is, start with the first values from the matrix, then increase those values with 1 to get the next values and place them at the right position in the array and repeat this untill all 256 values are done.
It only needs 256/32 steps to get all the 256 DWORD values in bit-reversed order.
With 1 index pointer from the matrix it puts 8 * 4 values at once at the right position in the array.

Note: This algorithm only works for power of two length arrays !

[Raistlin]

Code: [Select]

Note: This algorithm only works for power of two length arrays !
Which makes perfect sense as data runs are typically 512, 1024, 2048, 4096 and at the extreme 32768 in length
per FFT sub-set. (I have seen 65536 and larger runs in hardware based systems) As far as I understand it the bit-reversal is
calculated once and only once. Then dumped as a lookup table. The data is then dispersed per this lookup-table. The result is fed into
the actual FFT algorithm. Normally a filter or transform is then applied. The result is then reversed (iFFT) and a new
waveform generated that thus incorporates such filters. Power of two use; as per Siekmanski seems intuitive and well conceived.
Please correct me, if I left something major out -  I'am only a novice at this myself.

refs see:
http://www.fftw.org/accuracy/Pentium4-3.60GHz-icc/ (Shows typical data bin sizes for powers of two)
http://www.originlab.com/doc/Origin-Help/FFT-Filter (Filters - general)
http://retouchpro.com/tutorials/?m=show&id=185 (Filters - photographic enhancement)
http://stackoverflow.com/questions/2929401/dsp-filtering-in-the-frequency-domain-via-fft  (Filters - audio)
http://www3.mpifr-bonn.mpg.de/staff/bklein/FFTS/URSI-FFTS.pdf  (hardware based FFT)
 

[hutch--]

i don't claim to have absorbed all of the code present but a 256 item table is one idea I would explore. It means for any give 8 bits you have a 1 memory access to get its inversion. Does this make sense ?

[K_F]

Sorry Hutch, I meant the duplicate entry... I had two duplicate posts appear ?
Anyway back to topic..

If you're creating a single filter to work on many times a single bit reversal would do.
But if you're creating many filters (say a running filter) you have the problem of having to bit reverse every filter.

I've got around this by making an offset address table with 2ExpN sizes and bit reversed this table.
The filter calculations then use this table as an index into the untouched/original filter for it's operations.
On changing the filter, you can just reload a saved filter or make a new one without having to bit reverse anything.

This is what I done so far.. I'll clean it up later.
:)

[Siekmanski]

Hi guys,

We had this discussion before in the fft thread, do we realy need a fast bit-reversal routine for the fft routine ?
Not realy if you are repeating the same "small" fft length over and over again.
But if you need very large fft's it can take some time to create a bit-reversal table.
Especially if you need many fft's of differrent sizes.

My routine doesn't need a lookup table or do actual bit-reversal for any power of 2 size.
In fact it only needs these 4 numbers ( 00, 02, 01, 03 ) to create a matrix on the fly in code for all power of 2 sizes.
It shifts the 4 numbers into the right values for the size you want and transposes the first row to the first collumn and calculates the stride in that column needed for the places in the table.

The only thing it has to do is setting up the matrix values for the power of 2 size you want.
Then it processes the first row of the matrix then the second row and then updates the first and second row to row 3 and 4 if larger sizes are needed etc.

As soon as I have the time to code it, I will post a complete routine which does it all.
Just posted it at this stage to explain how the algoritm works with some fixed examples.

This technique ( with a bit of tweaking ) can be used to create the cos and sin tables.
To create a cos table and a sin table for a 1024 size fft, you only need to calculate 255 sines to construct both the cos and sin tables.

[K_F]

I'm always curious... when you've finished