The MASM Forum

General => The Laboratory => Topic started by: guga on July 18, 2020, 02:22:35 AM

Title: Fast median algorithm
Post by: guga on July 18, 2020, 02:22:35 AM
Hi Guys

Someone knows a fast median algorithm for Integers and also for floats for sorted and unsorted values (without using sorting algorithms, btw) ? The inputed numbers are random and can be repeated. And the algorithm needs to adapt for even and odd lengths of any size.

Ex: A sequence of integers (and also floats/doubles): 1, 18, 0, 0, 0, 0, 2, 2, 5, 99, 600, 785, 758 , 17, 11, 21, 23, 24, 66, 21, 14, 57, 17

The goal is to obey this rules: https://www.mathsisfun.com/median.html

Title: Re: Fast median algorithm
Post by: jj2007 on July 18, 2020, 02:54:31 AM
You googled median algorithm and found Tony Hoare's algo, I suppose?
Title: Re: Fast median algorithm
Post by: guga on July 18, 2020, 03:15:50 AM
Hi JJ.

No i found this one.

https://stackoverflow.com/questions/810657/fastest-code-c-c-to-select-the-median-in-a-set-of-27-floating-point-values

Code: [Select]
// return the median value in a vector of 27 floats pointed to by a
float heapMedian3( float *a )
{
   float left[14], right[14], median, *p;
   unsigned char nLeft, nRight;

   // pick first value as median candidate
   p = a;
   median = *p++;
   nLeft = nRight = 1;

   for(;;)
   {
       // get next value
       float val = *p++;

       // if value is smaller than median, append to left heap
       if( val < median )
       {
           // move biggest value to the heap top
           unsigned char child = nLeft++, parent = (child - 1) / 2;
           while( parent && val > left[parent] )
           {
               left[child] = left[parent];
               child = parent;
               parent = (parent - 1) / 2;
           }
           left[child] = val;

           // if left heap is full
           if( nLeft == 14 )
           {
               // for each remaining value
               for( unsigned char nVal = 27 - (p - a); nVal; --nVal )
               {
                   // get next value
                   val = *p++;

                   // if value is to be inserted in the left heap
                   if( val < median )
                   {
                       child = left[2] > left[1] ? 2 : 1;
                       if( val >= left[child] )
                           median = val;
                       else
                       {
                           median = left[child];
                           parent = child;
                           child = parent*2 + 1;
                           while( child < 14 )
                           {
                               if( child < 13 && left[child+1] > left[child] )
                                   ++child;
                               if( val >= left[child] )
                                   break;
                               left[parent] = left[child];
                               parent = child;
                               child = parent*2 + 1;
                           }
                           left[parent] = val;
                       }
                   }
               }
               return median;
           }
       }

       // else append to right heap
       else
       {
           // move smallest value to the heap top
           unsigned char child = nRight++, parent = (child - 1) / 2;
           while( parent && val < right[parent] )
           {
               right[child] = right[parent];
               child = parent;
               parent = (parent - 1) / 2;
           }
           right[child] = val;

           // if right heap is full
           if( nRight == 14 )
           {
               // for each remaining value
               for( unsigned char nVal = 27 - (p - a); nVal; --nVal )
               {
                   // get next value
                   val = *p++;

                   // if value is to be inserted in the right heap
                   if( val > median )
                   {
                       child = right[2] < right[1] ? 2 : 1;
                       if( val <= right[child] )
                           median = val;
                       else
                       {
                           median = right[child];
                           parent = child;
                           child = parent*2 + 1;
                           while( child < 14 )
                           {
                               if( child < 13 && right[child+1] < right[child] )
                                   ++child;
                               if( val <= right[child] )
                                   break;
                               right[parent] = right[child];
                               parent = child;
                               child = parent*2 + 1;
                           }
                           right[parent] = val;
                       }
                   }
               }
               return median;
           }
       }
   }
}

But i didn´t tested it and also it seems to work only for 27 numbers, and also it seems to be kind of huge to be fast as intended to.
Title: Re: Fast median algorithm
Post by: guga on July 18, 2020, 03:54:35 AM
I found also this oine that seems to be faster, but i´m unable to compile

http://blog.beamng.com/a-faster-selection-algorithm/
Code: [Select]
/* This source code is in the public domain */
#define F_SWAP(a,b) { float temp=(a);(a)=(b);(b)=temp; }
# Note: The code needs more than 2 elements to work
float lefselect(float a[], const int n, const int k) {
    int l=0, m = n-1, i=l, j=m;
    float x;
    while (l<m) {
        if( a[k] < a ) F_SWAP(a,a[k]);
        if( a[j] < a ) F_SWAP(a,a[j]);
        if( a[j] < a[k] ) F_SWAP(a[k],a[j]);
        x=a[k];
        while (j>k & i<k) {
            do i++; while (a<x);
            do j--; while (a[j]>x);
            F_SWAP(a,a[j]);
        }
        i++; j--;
        if (j<k) {
            while (a<x) i++;
            l=i; j=m;
        }
        if (k<i) {
            while (x<a[j]) j--;
            m=j; i=l;
        }
    }
    return a[k];
}

 
Title: Re: Fast median algorithm
Post by: nidud on July 18, 2020, 04:40:16 AM
Here's one, (q)sort and grab the middle.

Code: [Select]
;
; build: asmc64 -q -pe median.asm
;
include stdio.inc
include stdlib.inc
include tchar.inc

type typedef long_t ; item size

compare proto :ptr, :ptr {
    mov eax,[rcx]
    sub eax,[rdx]
    sbb ecx,ecx
    sbb ecx,-1
    and eax,eax
    cmovnz eax,ecx
    }

memxchg proto :ptr, :ptr {
    mov eax,[rcx]
    mov r8d,[rdx]
    mov [rdx],eax
    mov [rcx],r8d
    }

    .data
    array dd 1, 18, 0, 0, 0, 0, 2, 2, 5, 99, 600, 785, 758 , 17, 11, 21, 23, 24, 66, 21, 14, 57, 17

    .code

median proc uses rsi rdi rbx p:ptr, n:size_t

  local result:ptr

    mov rax,rdx

    .if rax > 1
       
        mov rsi,rcx
        dec rax
        mov ecx,type
        mul rcx
        lea rdi,[rsi+rax]
        xor r9d,r9d ; stack level

        lea rax,[rdi+rcx] ; middle from (hi - lo) / 2
        sub rax,rsi
        xor edx,edx
        div rcx
        shr rax,1
        mul rcx
        add rax,rsi
        mov result,rax

        .while 1
           
            mov rcx,type
            lea rax,[rdi+rcx]
            sub rax,rsi
            .ifnz
                xor edx,edx
                div rcx
                shr rax,1
                mul rcx
            .endif
           
            lea rbx,[rsi+rax]

            .ifsd compare(rsi, rbx) > 0
                memxchg(rsi, rbx)
            .endif
            .ifsd compare(rsi, rdi) > 0
                memxchg(rsi, rdi)
            .endif
            .ifsd compare(rbx, rdi) > 0
                memxchg(rbx, rdi)
            .endif

            mov r10,rsi
            mov r11,rdi

            .while 1

                add r10,type
                .if r10 < rdi

                    .continue .ifsd compare(r10, rbx) <= 0
                .endif

                .while 1

                    sub r11,type

                    .break .if r11 <= rbx
                    .break .ifsd compare(r11, rbx) <= 0
                .endw

                mov rcx,r11
                mov rax,r10
                .break .if rcx < rax
                memxchg(rcx, rax)

                .if rbx == r11

                    mov rbx,r10
                .endif
            .endw

            add r11,type

            .while 1

                sub r11,type

                .break .if r11 <= rsi
                .break .ifd compare(r11, rbx)
            .endw

            mov rdx,r10
            mov rax,r11
            sub rax,rsi
            mov rcx,rdi
            sub rcx,rdx

            .ifs rax < rcx

                mov rcx,r11

                .if rdx < rdi

                    push rdx
                    push rdi
                    inc r9d
                .endif

                .if rsi < rcx

                    mov rdi,rcx
                    .continue
                .endif
            .else
                mov rcx,r11

                .if rsi < rcx

                    push rsi
                    push rcx
                    inc r9d
                .endif

                .if rdx < rdi

                    mov rsi,rdx
                    .continue
                .endif
            .endif

            .break .if !r9d

            dec r9d
            pop rdi
            pop rsi
        .endw
        mov rcx,result
        mov eax,[rcx]
    .endif
    ret

median endp

main proc

    printf( "median: %d\n", median( &array, lengthof(array) ) )
    ret

main endp

    end _tstart
Title: Re: Fast median algorithm
Post by: jj2007 on July 18, 2020, 05:36:52 AM
Here's one, (q)sort and grab the middle

Trivial but...
without using sorting algorithms
Title: Re: Fast median algorithm
Post by: guga on July 18, 2020, 05:43:38 AM
Tks Nidud

But, it is in 64 bits and also uses qsort. I´m looking for some that does not need to use sorting algorithms inside.

I found another one made by andy dansby that seems promissor and claims to be even faster then lefselect i posted previously, but i didn´t tested yet

https://www.mmbforums.com//viewtopic.php?f=22&t=4411&start=160

Code: [Select]
float Dansby_kth(float array[], const unsigned int length, const unsigned int kTHvalue)
{
#define F_SWAP(a,b) { float temp=(a);(a)=(b);(b)=temp; }

unsigned int left = 0;
unsigned int left2 = 0;
unsigned int right = length - 1;
unsigned int right2 = length - 1;
unsigned int kthPlus =  kTHvalue;
unsigned int kthMinus = kTHvalue;

while (right > left)
{
//Median of 3 optimization
if( array[right] < array[left] ) F_SWAP(array[left],array[right]);
if( array[right] < array[kTHvalue] ) F_SWAP(array[kTHvalue],array[right]);
if( array[kTHvalue] < array[left] ) F_SWAP(array[left],array[kTHvalue]);
//Median of 3 optimization

const float temp = array[kTHvalue];//this is pivot

kthPlus = kTHvalue + 1;
kthMinus = kTHvalue - 1;

while ((right2 > kTHvalue) && (left2 < kTHvalue))
{
do
{
left2++;
}while (array[left2] < temp);

do
{
right2--;
}while (array[right2] > temp);

F_SWAP(array[left2],array[right2]);
}
left2++;
right2--;
 
if (right2 < kthMinus)
{
while (array[left2] < temp)
{
left2++;
}
left = left2;
right2 = right;
kthMinus --;
}
else
if (kthPlus < left2)
{
while (array[right2] > temp)
{
right2--;
}
right = right2;
left2 = left;
kthPlus ++;
}
else
if( array[left] < array[right] )
{
F_SWAP(array[right],array[left]);
}



}

#undef F_SWAP
return array[kTHvalue];
}



call from

Code: [Select]
//points to the center value (median)
int middleElement = elementsinarray / 2;

Dansby_kth(redArray, elementsinarray,middleElement);
Dansby_kth(greenArray, elementsinarray,middleElement);
Dansby_kth(blueArray, elementsinarray,middleElement);

Title: Re: Fast median algorithm
Post by: daydreamer on July 18, 2020, 05:53:22 AM
what about use MAXPS and MINPS instructions?
Title: Re: Fast median algorithm
Post by: nidud on July 18, 2020, 06:02:36 AM
 :biggrin:

They are all based on sort algorithms (what else) so there's no way around that. You compare, swap and return the middle.

Quote
But, it is in 64 bits and also uses qsort. I´m looking for some that does not need to use sorting algorithms inside.

No, the algo do not use qsort.
Title: Re: Fast median algorithm
Post by: guga on July 18, 2020, 09:02:57 AM
Thank you Nidud, but do you have it for 32 Bits so i can test it ?

Hi Daydreamer. Using maxps/minps ? Sounds interesting. Not sure how to implement it yet, but i´ll give a try.
Title: Re: Fast median algorithm
Post by: nidud on July 18, 2020, 10:17:58 PM
From what I see the heapMedian3 sample have the advantage of not trashing the array but needs a local buffer. The rest of the samples are just variations of the same.

but do you have it for 32 Bits so i can test it ?

You may compile or past the C source to https://gcc.godbolt.org to get the output you need.
Title: Re: Fast median algorithm
Post by: guga on July 19, 2020, 05:34:13 AM
Hi Nidud


Ok, i´ll give a try and test the Heapmedian3 to see if it works as expected. However, as you said, it needs an extra buffer in memory which could be overkilling when we need to calculate the median of a larger array then only 27 bytes longs (say a array of floats, ints etc) with 100 Mb for example.

I[´ll try porting it to see if it works and also check for speed.

I was trying to port Dansby_kth that is very fast and don´t uses extra buffer, but it is producing a wrong result and i can´t fix that :(

Do you have any idea of what could be possibly wrong ?

Code: [Select]

float Dansby_kth(float array[], const unsigned int length, const unsigned int kTHvalue)
{
#define F_SWAP(a,b) { float temp=(a);(a)=(b);(b)=temp; }

unsigned int left = 0;
unsigned int left2 = 0;
unsigned int right = length - 1;
unsigned int right2 = length - 1;
unsigned int kthPlus = kTHvalue;
unsigned int kthMinus = kTHvalue;

while (right > left)
{
//Median of 3 optimization
if (array[right] < array[left]) F_SWAP(array[left], array[right]);
if (array[right] < array[kTHvalue]) F_SWAP(array[kTHvalue], array[right]);
if (array[kTHvalue] < array[left]) F_SWAP(array[left], array[kTHvalue]);
//Median of 3 optimization

const float temp = array[kTHvalue];//this is pivot

kthPlus = kTHvalue + 1;
kthMinus = kTHvalue - 1;

while ((right2 > kTHvalue) && (left2 < kTHvalue))
{
do
{
left2++;
} while (array[left2] < temp);

do
{
right2--;
} while (array[right2] > temp);

F_SWAP(array[left2], array[right2]);
}
left2++;
right2--;

if (right2 < kthMinus)
{
while (array[left2] < temp)
{
left2++;
}
left = left2;
right2 = right;
kthMinus--;
}
else
if (kthPlus < left2)
{
while (array[right2] > temp)
{
right2--;
}
right = right2;
left2 = left;
kthPlus++;
}
else
if (array[left] < array[right])
{
F_SWAP(array[right], array[left]);
}



}

#undef F_SWAP
return array[kTHvalue];
}
Title: Re: Fast median algorithm
Post by: HSE on July 19, 2020, 06:08:51 AM
Hi Guga!

Apparently the algorithm is for uneven data. Obviously not a coincidence, example have 27 observations.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 19, 2020, 06:19:19 AM
Hi guga,

Cooked up this algorithm.

New one: see http://masm32.com/board/index.php?topic=8671.msg94735#msg94735

Title: Re: Fast median algorithm
Post by: jj2007 on July 19, 2020, 06:50:13 AM
Hi Guga,

Just for curiosity: what's the purpose of your question? Do you have a real use for this, or is it just for fun? For most applications, I guess sorting and taking the middle element would be fast enough...
Title: Re: Fast median algorithm
Post by: nidud on July 19, 2020, 06:53:01 AM
Ok, i´ll give a try and test the Heapmedian3 to see if it works as expected. However, as you said, it needs an extra buffer in memory which could be overkilling when we need to calculate the median of a larger array then only 27 bytes longs (say a array of floats, ints etc) with 100 Mb for example.

I[´ll try porting it to see if it works and also check for speed.

You may need to save the array for later use so it depends on the situation. In a benchmark test where the function is called repeatably that's at least the case.

Quote
I was trying to port Dansby_kth that is very fast and don´t uses extra buffer, but it is producing a wrong result and i can´t fix that :(

Do you have any idea of what could be possibly wrong ?

It's difficult to say what the right answer is here. If the array is even (1,2,3,4) the answer is 2 or 3 so you need to make a choice there I guess. The returned result seems to be 18 and I assume the correct answer 17. In that case it overshoots the index by one. Reducing the index by one seems at least on this array to give the right answer.

Code: [Select]
#define F_SWAP(a,b) { float temp=(a);(a)=(b);(b)=temp; }

    float Dansby_kth(float array[], const unsigned int length)
    {

        unsigned int kTHvalue = length / 2 - 1;
        unsigned int left = 0;
        unsigned int left2 = 0;
        unsigned int right = length - 1;
        unsigned int right2 = length - 1;
        unsigned int kthPlus =  kTHvalue;
        unsigned int kthMinus = kTHvalue;

        while (right > left)
        {

            if( array[right] < array[left] ) F_SWAP(array[left],array[right]);
            if( array[right] < array[kTHvalue] ) F_SWAP(array[kTHvalue],array[right]);
            if( array[kTHvalue] < array[left] ) F_SWAP(array[left],array[kTHvalue]);

            const float temp = array[kTHvalue];//this is pivot

            kthPlus = kTHvalue + 1;
            kthMinus = kTHvalue - 1;

            while ((right2 > kTHvalue) && (left2 < kTHvalue))
            {
                do
                {
                    left2++;
                }while (array[left2] < temp);

                do
                {
                    right2--;
                }while (array[right2] > temp);

                F_SWAP(array[left2],array[right2]);
            }
            left2++;
            right2--;

            if (right2 < kthMinus)
            {
                while (array[left2] < temp)
                {
                    left2++;
                }
                left = left2;
                right2 = right;
                kthMinus --;
            }
            else
            if (kthPlus < left2)
            {
                while (array[right2] > temp)
                {
                    right2--;
                }
                right = right2;
                left2 = left;
                kthPlus ++;
            }
            else
            if( array[left] < array[right] )
            {
                F_SWAP(array[right],array[left]);
            }
        }
        return array[kTHvalue];
    }
Title: Re: Fast median algorithm
Post by: Siekmanski on July 19, 2020, 07:00:01 AM
EDIT: Spotted an error -> jb changed to jbe
New attachment: see reply#13
Title: Re: Fast median algorithm
Post by: guga on July 19, 2020, 08:30:41 AM
Hi SiekManski. Tks you a lot. I´ll give a try and see how it works and if i can make it work also for even values.

Hi JJ. This is for a real routine i´m trying to create for image manipulation. I´m trying to port a routine made by google to assembly, but it is a true hell, since it was originally written in python and i´m not being able to convert it to C and neither test it to check if it is working as expected.

The main algorithm claims to remove/hide watermark on images and it is great not only for this purpose, but specially because it can be later adapted  to remove any noise, scratches (and even lens distortions) on any images. If the algorithm works i can try later use it on video as well.

The link of the algorithm in python is here:
https://github.com/rohitrango/automatic-watermark-detection

The algorithm is quite complex, but basically it scans a sequence of 100 images, tries to estimate the watermark location using sobel operator, and later find the median of each one of the images on thee specified pixel locations.

So far i succeeded to port to assembly the 1st steps to the detection routine and ended improving the sobel algo as well. Now it is need to find the median of all pixels on all images and create another image containing only the medians values (for x and y pos). In fact, it creates 2 images, one for sobelX and other for SobelY from which it uses it later to try to identify the alfa values of each one of them (using canny) and later reconstruct the watermark using poisson reconstruction routines.

The algorithm is really amazing and the possibilities of use this are endless (not specific for watermark remover, but for image/video cleaning as well)

I´m not sure if i´ll succeed to port it completely, because when trying to run this stuff in visualstudio, my PC simply freezes all the time after some routines and i have no idea how to properly debug a python file using visualstudio (or even any python editors as well, since i have no idea of what python language syntax do in order to try to reproduce the functions).


About the sobel routine i improved, i suceeded to make the edges more thin and also identify all possible true edges rather then noise. Here is a example of what i suceeded to do so far on sobel algorithm

original image
(https://i.ibb.co/3mHNWfz/Baloons.jpg) (https://ibb.co/3mHNWfz)

Sobel Original
(https://i.ibb.co/nMvQDKv/Sobel.jpg) (https://ibb.co/nMvQDKv)

Sobel Equalized
(https://i.ibb.co/wZk9G43/Sobel-Equalized.jpg) (https://ibb.co/wZk9G43)

Sobel Guga
(https://i.ibb.co/PYXzJ4p/Sobel-Guga.jpg) (https://ibb.co/PYXzJ4p)

Sobel Guga Equalized
(https://i.ibb.co/SvTLcs0/Sobel-Guga-Equalized.jpg) (https://ibb.co/SvTLcs0)



My sobel version is a small improvement to the original sobel algorithm, because it uses percentage of the pixels to compute their magnitudes rather then simply cutoff frequencies as commonly used in sobel. This small update improves the overall quality of the image and find literally all edges, even when they are not bright as the others. So, it tries to stay proportional to the light of the pixels, rather then cutting of when not needed. Doing that it result in thinner edge lines and also a more equalized image. The proof is that when you equalize both images after passing through sobel, mine version has less artifacts then the equivalent sobel image after also passed to equalization.

My version  stays within the limits of 0 to 127 for dark and 128 to 255 to brighter, regardless the values found on the magnitudes of Gx or Gx extrapolates. For example, on the normal sobel, you may find a magnitude of a pixel on a edge whose level is let´s say 300 or 400 and when it find´s it, it simply cut them off to be either black or white (0 or 255). What i did was basically calculate the amount of those extrapolations and adapt them to stay always withing the limits of dark (0 to 127) and bright (128 to 255) without having to cut the extra values off. Those extrapolations btw seems to be associated with the compression of the image that ends to pixelate it. That´s why we can´t simply cut them off and why mine version the edges don´t seems to bend to x and y pos. Each pixel in the sobel corresponds exactly to the luminosity intensity of a pixel on the same position

The only thing i didn´t decided yet is to know what to do with the borders of the image.
Title: Re: Fast median algorithm
Post by: guga on July 19, 2020, 08:35:35 AM
Ok, i´ll give a try and test the Heapmedian3 to see if it works as expected. However, as you said, it needs an extra buffer in memory which could be overkilling when we need to calculate the median of a larger array then only 27 bytes longs (say a array of floats, ints etc) with 100 Mb for example.

I[´ll try porting it to see if it works and also check for speed.

You may need to save the array for later use so it depends on the situation. In a benchmark test where the function is called repeatably that's at least the case.

Quote
I was trying to port Dansby_kth that is very fast and don´t uses extra buffer, but it is producing a wrong result and i can´t fix that :(

Do you have any idea of what could be possibly wrong ?

It's difficult to say what the right answer is here. If the array is even (1,2,3,4) the answer is 2 or 3 so you need to make a choice there I guess. The returned result seems to be 18 and I assume the correct answer 17. In that case it overshoots the index by one. Reducing the index by one seems at least on this array to give the right answer.

Code: [Select]
#define F_SWAP(a,b) { float temp=(a);(a)=(b);(b)=temp; }

    float Dansby_kth(float array[], const unsigned int length)
    {

        unsigned int kTHvalue = length / 2 - 1;
        unsigned int left = 0;
        unsigned int left2 = 0;
        unsigned int right = length - 1;
        unsigned int right2 = length - 1;
        unsigned int kthPlus =  kTHvalue;
        unsigned int kthMinus = kTHvalue;

        while (right > left)
        {

            if( array[right] < array[left] ) F_SWAP(array[left],array[right]);
            if( array[right] < array[kTHvalue] ) F_SWAP(array[kTHvalue],array[right]);
            if( array[kTHvalue] < array[left] ) F_SWAP(array[left],array[kTHvalue]);

            const float temp = array[kTHvalue];//this is pivot

            kthPlus = kTHvalue + 1;
            kthMinus = kTHvalue - 1;

            while ((right2 > kTHvalue) && (left2 < kTHvalue))
            {
                do
                {
                    left2++;
                }while (array[left2] < temp);

                do
                {
                    right2--;
                }while (array[right2] > temp);

                F_SWAP(array[left2],array[right2]);
            }
            left2++;
            right2--;

            if (right2 < kthMinus)
            {
                while (array[left2] < temp)
                {
                    left2++;
                }
                left = left2;
                right2 = right;
                kthMinus --;
            }
            else
            if (kthPlus < left2)
            {
                while (array[right2] > temp)
                {
                    right2--;
                }
                right = right2;
                left2 = left;
                kthPlus ++;
            }
            else
            if( array[left] < array[right] )
            {
                F_SWAP(array[right],array[left]);
            }
        }
        return array[kTHvalue];
    }

Hi NiDud. When the number is even you simply compute the average of the 2 located in the middle. Like this:


Array = (1,2,3,4)
You will calculate the median using 2 and 3
Sum them up and  divide by 2

So. The media for this is: (2+3)/2 = 2.5

So, you are computing the width / 2 and whatever values are in position (width/2) and (Width/2)+1 you simply calculate the average of those 2.
Title: Re: Fast median algorithm
Post by: guga on July 19, 2020, 08:38:33 AM
EDIT: Spotted an error -> jb changed to jbe
New attachment: see reply#13

Thk you a lot, Siekmanski :) I´ll take a look :)
Title: Re: Fast median algorithm
Post by: Siekmanski on July 19, 2020, 09:33:01 AM
Here is the routine for even and uneven population sizes.
No sorting, no memory swaps, no divs.  :cool:
If you want me to explain the algorithm, feel free to ask.

New one: see http://masm32.com/board/index.php?topic=8671.msg94735#msg94735

Title: Re: Fast median algorithm
Post by: guga on July 19, 2020, 09:39:18 AM
Wow  :greenclp: :greenclp: :greenclp:

Tks you Siekmanski. I´ll take a look right now.

Btw. As i explained above to JJ, the development/improvement of this algorithms are really interesting.

I also succeeded yesterday to create a new algorithm that maybe be used to correctly equalize a histogram. In fact, it already does the job stretching the histogram to 128 pixels correctly balanced using a effect similar to gradients and the others 128 pixels missing could be retrieved back perfectly simply equalizing the whole histogram and filling eventual gaps. The problem is that i got stuck with the math, since i don´t know yet how to do with the values i´m finding. So far, the result is this:

Original
(https://i.ibb.co/kKzhVML/frog-1371919.jpg) (https://ibb.co/kKzhVML)

My algo
(https://i.ibb.co/2ym0Hhq/enhance.jpg) (https://ibb.co/2ym0Hhq)
Title: Re: Fast median algorithm
Post by: Siekmanski on July 19, 2020, 09:52:31 AM
I hope it does the job for you. ( and hope it's error free? )  :bgrin:
You have to test it before using it, I have done some testing and found no errors......
Just made it tonight in a hurry when I got the idea for the algorithm.
If it works correct, the routine can be made faster.
Title: Re: Fast median algorithm
Post by: HSE on July 19, 2020, 10:18:06 AM
( and hope it's error free? )  :bgrin:
:biggrin: No, I'm afraid

If the class bigger than half population have more than one element, and one element in excess of half population, then that is the median no matter if population is even.

This is wrong:
Code: [Select]
.data?
MedianBuffer    db 1024 dup (?) ; minimum buffer size must be as large as the biggest number in the data set
Must be:
Code: [Select]
.data
MedianBuffer    db 1024 dup (0) ; minimum buffer size must be as large as the biggest number in the data set

Perhaps:
Code: [Select]
CalculateMedian proc uses ebx esi edi
   
    mov     esi,offset DataSet
    mov     edi,offset MedianBuffer

    mov     ecx,PopulationSize
PopulateLP:
    mov     eax, [esi]
    inc     byte ptr [edi+eax] 
    add     esi,4   
    dec     ecx
    jnz     PopulateLP

    mov     ecx,PopulationSize
    mov     ebx,ecx
    shr     ebx,1
    and     ecx,1                   ; Check even or uneven
    jnz     UnEven
    dec     ebx                     ; Make even
UnEven:

    xor     edx,edx
    mov     eax,-1
FindMedianLP:
    inc     eax
    movzx   ecx,byte ptr[edi+eax]
    add     edx,ecx
    cmp     edx,ebx
    jbe     FindMedianLP

    ;----------------------
    .if edx > ebx         
        mov ecx, edx
        sub ecx, ebx
        .if ecx > 0
            jnz Done
        .endif   
    .endif
    ;-----------------------

    mov     edx,PopulationSize
    and     edx,1                   ; Check even or uneven
    jnz     Done

    mov     edx,eax
FindSecondMiddleNumberLP:
    inc     eax
    movzx   ecx,byte ptr[edi+eax]
    test    ecx,ecx
    jz      FindSecondMiddleNumberLP
    add     eax,edx
    shr     eax,1
Done:                               ; Result in eax
    ret

CalculateMedian endp
Title: Re: Fast median algorithm
Post by: Siekmanski on July 19, 2020, 12:18:07 PM
Hi HSE,
Always thougth Windows initializes the memory in the .rdata section to zero?
Never had a situation in MASM it did not happen.
Maybe other assemblers behave different?

Thank you for finding this bug.  :thumbsup:
Added these 2 lines:

EDIT: New one: see http://masm32.com/board/index.php?topic=8671.msg94735#msg94735


Code: [Select]
    cmp ecx,PopulationSize/2
    jb average

Hope this solves it.
Can you test it?

Code: [Select]
align 4   
CalculateMedian proc uses ebx esi edi
   
    mov     esi,offset DataSet
    mov     edi,offset MedianBuffer

    mov     ecx,PopulationSize
PopulateLP:
    mov     eax,[esi]
    inc     byte ptr [edi+eax] 
    add     esi,4   
    dec     ecx
    jnz     PopulateLP

    mov     ecx,PopulationSize
    mov     ebx,ecx
    shr     ebx,1
    and     ecx,1                   ; Check even or uneven
    jnz     UnEven
    dec     ebx                     ; Make even
UnEven:

    xor     edx,edx
    mov     eax,-1
FindMedianLP:
    inc     eax
    movzx   ecx,byte ptr[edi+eax]
    add     edx,ecx
    cmp     edx,ebx
    jbe     FindMedianLP

    mov     edx,PopulationSize
    and     edx,1                   ; Check even or uneven
    jnz     Done
    mov     edx,eax
FindSecondMiddleNumberLP:
    inc     eax
    movzx   ecx,byte ptr[edi+eax]
   
    cmp     ecx,PopulationSize/2
    jb      Average
   
    test    ecx,ecx
    jz      FindSecondMiddleNumberLP
Average:
    add     eax,edx
    shr     eax,1
Done:                               ; Result in eax
    ret
Title: Re: Fast median algorithm
Post by: guga on July 19, 2020, 03:36:43 PM
I´m not sure i understood what is wrong with the code as suggested by HSE. I´m testing it here and didn´t found incorrect values so far. What i´m missing ?

For a tiny table of pixels (256 integer values only from 0 to 255) the algorithm seems to work fine, so far. I ported it to RosAsm trying to make it a bit faster (not measured to see what happened), using local variables so i could try to understand what the algo is doing. . Also it only needs to preserve ecx, since the data is stored in local vars, so no extra push/pop chains at the start and end of the function. And i set the data in the table to dwords, to replace routines like this:

Code: [Select]
(...)
FindMedianLP:
    inc     eax
    movzx   ecx,byte ptr[edi+eax]
    add     edx,ecx
    cmp     edx,ebx
    jbe     FindMedianLP

To something like:
Code: [Select]
(....)
MedianBuffer    dd 256 dup (0)
(...)

FindMedianLP:
    inc     eax
    add     ecx [edi+eax] ; or add  ecx [MedianBuffer+eax] to avoid using more registers.
    cmp     edx,ebx
    jbe     FindMedianLP

Can you explain to me how does it is working ? I´m not fully understood why.

And also, does it needs to use a table ? This is because even if the algo can be improved for speed, the table should be zeroed whenever the function is called and if we are dealing with huge amount of data like pixels in videos or in my case, hundreds of images to be analysed), then, cleaning the table could slow down the algo a little bit.

And another question,. how to make it work with SSE on Float or Real Numbers ? I mean, the algo is working on a array of integers (gray maps), but it won´t work on another table that is a array of floats to used to compute the median of Sobel data.


I ported like this:

Code: [Select]

[MedianPopulateTable: D$ 0 #256] ; minimum buffer size must be as large as the biggest number in the data set. In my case it is a range of pixels so, 256 values only. In this case, i´m using the table as a 256 dword (D$ in RosAsm syntax) array

Proc CalculateMedian3:
    Arguments @Input, @ArraySize
    Local @IsArraySizeOdd, @MiddleTablePos, @MedianLeftVal, @iCounter
    Uses ecx

    mov ecx D@ArraySize | mov D@iCounter ecx ;  "ArraySize" is the same as PopulationSize. I just renamed it as array so i could better follow what it is doing in the other routines i´m testing it with.
    mov ecx D@Input
    mov D@IsArraySizeOdd &TRUE

    ; Populate Buffer Table
@PopulateLP:
        mov eax D$ecx
        inc D$MedianPopulateTable+eax*4
        add ecx 4
    dec D@iCounter | jne @PopulateLP

    mov ecx D@ArraySize | mov D@MiddleTablePos ecx | shr D@MiddleTablePos 1
    Test_If cl cl ; Is number even ? Set IsArraySizeOdd to FALSE and decrease the position of the array table to be compared with
        dec D@MiddleTablePos
        mov D@IsArraySizeOdd &FALSE ;  set Flag to not odd number (So, it's a even)
    Test_End

    ; FindMedianLP
    xor ecx ecx
    mov eax 0-1
    Do
        inc eax
        add ecx D$MedianPopulateTable+eax*4
    Loop_Until ecx > D@MiddleTablePos

    If D@IsArraySizeOdd = &FALSE ; is Array Size even ?
        ; FindSecondMiddleNumberLP
        mov D@MedianLeftVal eax
        Do
            inc eax
        Loop_Until D$MedianPopulateTable+eax*4 <> 0
        add eax D@MedianLeftVal
        shr eax 1
    End_If

    ; Result in eax

EndP




Title: Re: Fast median algorithm
Post by: jj2007 on July 19, 2020, 05:55:47 PM
Now it is need to find the median of all pixels on all images and create another image containing only the medians values (for x and y pos)

- If we are talking about 32-bit colour images, does it mean you need the routine at byte size level?
- Have you thought of using the arithmetic medium instead? It's definitely much faster, and in practice there might be little difference.

Below an illustration of the difference between median and average. The only little problem is that your data are not sorted :cool:
Title: Re: Fast median algorithm
Post by: daydreamer on July 19, 2020, 07:46:10 PM
I´m not sure i understood what is wrong with the code as suggested by HSE. I´m testing it here and didn´t found incorrect values so far. What i´m missing ?

For a tiny table of pixels (256 integer values only from 0 to 255) the algorithm seems to work fine, so far. I ported it to RosAsm trying to make it a bit faster (not measured to see what happened), using local variables so i could try to understand what the algo is doing. . Also it only needs to preserve ecx, since the data is stored in local vars, so no extra push/pop chains at the start and end of the function. And i set the data in the table to dwords, to replace routines like this:

Code: [Select]
(...)
FindMedianLP:
    inc     eax
    movzx   ecx,byte ptr[edi+eax]
    add     edx,ecx
    cmp     edx,ebx
    jbe     FindMedianLP

To something like:
Code: [Select]
(....)
MedianBuffer    dd 256 dup (0)
(...)

FindMedianLP:
    inc     eax
    add     ecx [edi+eax] ; or add  ecx [MedianBuffer+eax] to avoid using more registers.
    cmp     edx,ebx
    jbe     FindMedianLP

Can you explain to me how does it is working ? I´m not fully understood why.

And also, does it needs to use a table ? This is because even if the algo can be improved for speed, the table should byte zeroed whenever the function is called and if we are dealing with huge amount of data like pixels in videos or in my case, hundreds of images to be analysed), then, cleaning the table could slow down the algo a little bit.

And another question,. how to make it work with SSE on Float or Real Numbers ? I mean, the algo is working on a array of integers (gray maps), but it won´t work on another table that is a array of floats to used to compute the median of Sobel data.

maybe you should have a big dynamic alloc memory buffer already cleared,so that simple move on to next 256 bytes,next time its called?,your going to use dynamic alloc anyway to process big image data
thats what I mean with SSE MAXPS and MINPS is:
if arraya>arrayb swap(arraya,arrayb) :

arrayb=MAXSS(arraya,arrayb)
arraya=MINSS(arraya,arrayb)
so unroll it MAXPS and MINPS does 4 of these conditional swaps in parallel, which could help with big image data,and MAXSS and MINSS to sort the final few numbers
Title: Re: Fast median algorithm
Post by: HSE on July 20, 2020, 04:28:29 AM
Hi Siekmanski!

I have forgotten to say: very smart idea!

Maybe other assemblers behave different?
:thumbsup: Could be that!

Now algo is failling. Something strange:
Code: [Select]
   movzx   ecx,byte ptr[edi+eax]
   
    cmp     ecx,PopulationSize/2    <--- ????
    jb      Average

In previous version I dont write very well, must be (I think):
Code: [Select]
    .if edx > ebx         
        mov ecx, edx
        sub ecx, ebx
        cmp ecx , 0
        jg Done
    .endif
Title: Re: Fast median algorithm
Post by: guga on July 20, 2020, 04:29:40 AM
Now it is need to find the median of all pixels on all images and create another image containing only the medians values (for x and y pos)

- If we are talking about 32-bit colour images, does it mean you need the routine at byte size level?
- Have you thought of using the arithmetic medium instead? It's definitely much faster, and in practice there might be little difference.

Below an illustration of the difference between median and average. The only little problem is that your data are not sorted :cool:

Hi JJ. Yes they are 32 bit color images. About the routines being in byte level, it depends of what part of the routine is being used, i presume. To gain a bit more speed, i converted the images to gray, and insert the data on a huge table to be manipulated later.
even knowing that each gray image has only one color, instead using a table of bytes, i simply convert them to Dwords to gain speed in other parts of the routines, to avoid having to use movzx eax B$ecx+....... So a simple mov eax D$xxx is enough to eliminate the needs of using extra code.

For example, to estimate the watermark, all images to be calculated are converted previously to gray and inserted on a huge table to hold all of the data. So, the 1st thing to do is:

1 - Allocate enough memory to hold all images to be processed. So far it uses 4 Dword Tables per image. One for greymap, ohher for Sobelx, other for SobelY and other for Sobel. The size of each table that stores the pixel data is calculate as: Width*height*4*4. Where 4 is because now seems to be needed them in a Dword format and not byte), and the other 4 because we are using 4 pixel tables (or maps)

The values are stored on a simple Structure followed by a array of structures to manip7ulate the data to be used.
Code: [Select]
[ImgDataHeader:
 ImgDataHeader.Size: D$ 0 ; Total size of our structure with the arrays included
 ImgDataHeader.Count: D$ 0 ; total amount of valid data structures. So, the total of images to be processed that fits to a inputed size. Ex: Count only the images on a directory that have resolution of  960*720
 ImgDataHeader.DataSize: D$ 0 ; Size of each data chunk. Width*Height*4
 ImgDataHeader.MaskWidth: D$ 0 ; Width of the image that will be later converted to a mask
 ImgDataHeader.MaskHeight: D$ 0] ; Width of the image that will be later converted to a mask

Immediately followed by a array of N "FullImageData" structures that are only pointers to the processed data. N is given by  "ImgDataHeader.Count"

[FullImageData:
 FullImageData.pGrayMap: D$ 0 ; Pointer to graymap
 FullImageData.pSobelX: D$ 0 ; Pointer to SobelXMap
 FullImageData.pSobelY: D$ 0 ; Pointer to SobelYMap
 FullImageData.pSobel: D$ 0] ; Pointer to SobelMap
(....)

 FullImageData.pGrayMap100: D$ 0 ; Pointer to graymap of the 100th image. Ex: Points to GreymapImage1 (The 1st greymap)
 FullImageData.pSobelX100: D$ 0 ; Pointer to SobelXMap of the 100th image Ex: Points to SobelXMap1 (The 1st SobelXMap1)
 FullImageData.pSobelY100: D$ 0 ; Pointer to SobelYMap of the 100th image Ex: Points to SobelYMap1 (The 1st SobelYMap1)
 FullImageData.pSobel100: D$ 0] ; Pointer to SobelMap of the 100th image Ex: Points to SobelMap1 (The 1st SobelMap1)
]

Immediatelly followed by the maps, Starting with the greymap
[GreymapImage1: D$ 0#(960*720*4)
(...)
GreymapImage100: D$ 0#(960*720*4)
(...)
[SobelXMap1: D$ 0#(960*720*4)
 SobelYMap1: D$ 0#(960*720*4)
 SobelMap1: D$ 0#(960*720*4)
(...)
SobelXMap100: D$ 0#(960*720*4)
 SobelYMap100: D$ 0#(960*720*4)
 SobelMap100: D$ 0#(960*720*4)

2 - Estimate4 the median value of Sobel

3 - Didn´t reached yet the next parts of the documentation to understand what nees to be done. The next part according to the python version is read the inputed image from whom you wish to remove the watermark, apply a poisson algorithm on the images (don´t know how to do yet, because i didn´t understood what the python version is actually doing.)


All the method is described here:
https://watermark-cvpr17.github.io/supplemental/
https://watermark-cvpr17.github.io/
https://ai.googleblog.com/2017/08/making-visible-watermarks-more-effective.html

And the link to the python version in github i posted.

If you want to read the google documentation that describes the algorithm it can be find here:
http://openaccess.thecvf.com/content_cvpr_2017/papers/Dekel_On_the_Effectiveness_CVPR_2017_paper.pdf


About using average rather the median, well...i agree that it cold improve speed, but i don´t know what the results could be, because the original google article it says to use median to retrieve better results.
Title: Re: Fast median algorithm
Post by: guga on July 20, 2020, 04:59:56 AM
I´m not sure i understood what is wrong with the code as suggested by HSE. I´m testing it here and didn´t found incorrect values so far. What i´m missing ?

For a tiny table of pixels (256 integer values only from 0 to 255) the algorithm seems to work fine, so far. I ported it to RosAsm trying to make it a bit faster (not measured to see what happened), using local variables so i could try to understand what the algo is doing. . Also it only needs to preserve ecx, since the data is stored in local vars, so no extra push/pop chains at the start and end of the function. And i set the data in the table to dwords, to replace routines like this:

Code: [Select]
(...)
FindMedianLP:
    inc     eax
    movzx   ecx,byte ptr[edi+eax]
    add     edx,ecx
    cmp     edx,ebx
    jbe     FindMedianLP

To something like:
Code: [Select]
(....)
MedianBuffer    dd 256 dup (0)
(...)

FindMedianLP:
    inc     eax
    add     ecx [edi+eax] ; or add  ecx [MedianBuffer+eax] to avoid using more registers.
    cmp     edx,ebx
    jbe     FindMedianLP

Can you explain to me how does it is working ? I´m not fully understood why.

And also, does it needs to use a table ? This is because even if the algo can be improved for speed, the table should byte zeroed whenever the function is called and if we are dealing with huge amount of data like pixels in videos or in my case, hundreds of images to be analysed), then, cleaning the table could slow down the algo a little bit.

And another question,. how to make it work with SSE on Float or Real Numbers ? I mean, the algo is working on a array of integers (gray maps), but it won´t work on another table that is a array of floats to used to compute the median of Sobel data.

maybe you should have a big dynamic alloc memory buffer already cleared,so that simple move on to next 256 bytes,next time its called?,your going to use dynamic alloc anyway to process big image data
thats what I mean with SSE MAXPS and MINPS is:
if arraya>arrayb swap(arraya,arrayb) :

arrayb=MAXSS(arraya,arrayb)
arraya=MINSS(arraya,arrayb)
so unroll it MAXPS and MINPS does 4 of these conditional swaps in parallel, which could help with big image data,and MAXSS and MINSS to sort the final few numbers

Hi DayDreamer. yes, it uses a huge dynamic table to work with the data. I commented about this above to JJ. I´m trying to see if the median routine works as expected before go further onto the google documentation and the python version to continue creating this algorithm. The main problem is that i don´t know python language and it is very hard for me to understand exactly what the algorithm is doing precisely. I installed the python version at https://github.com/rohitrango/automatic-watermark-detection but i have no idea how to make it work and neither how to debug it. I tested it on VisualStudio, but the app simply freezes my PC before it reaches here:
Code: [Select]

# W_m is the cropped watermark
W_m = poisson_reconstruct(cropped_gx, cropped_gy)

# Get the number of images in the folder
num_images = len(gxlist)

J, img_paths = get_cropped_images(
    'images/fotolia_processed', num_images, start, end, cropped_gx.shape)

The proper way would be convert this to C, so i could try to see what it is doing and debug it more properly, but i don´t have any idea how to convert python to C. And the worst, the python version uses opencv unnecessarily, because the sobel, canny and even poisson reconstruction can be don on a faster and easier way.
Title: Re: Fast median algorithm
Post by: nidud on July 20, 2020, 06:34:20 AM
Hi NiDud. When the number is even you simply compute the average of the 2 located in the middle. Like this:

Array = (1,2,3,4)
You will calculate the median using 2 and 3
Sum them up and  divide by 2

So. The media for this is: (2+3)/2 = 2.5

So, you are computing the width / 2 and whatever values are in position (width/2) and (Width/2)+1 you simply calculate the average of those 2.

This complicate things given you then will need to know the second value. This means you have to sort the array. A better strategy will be to just choose one. They will most likely be the same (or very close) given the amount of bytes used.

instead using a table of bytes, i simply convert them to Dwords to gain speed in other parts of the routines, to avoid having to use movzx eax B$ecx+....... So a simple mov eax D$xxx is enough to eliminate the needs of using extra code.

 :biggrin:

This is a very bad strategy guga. There is no speed gain there and the 4 times increase in memory will have a negative impact on CPU cache and movzx is not a slow instruction. So use bytes if possible.

I have forgotten to say: very smart idea!

Indeed.

Quote
Maybe other assemblers behave different?
:thumbsup: Could be that!

Dont think the assembler has anything to do with this. Think the expansion of the _BSS segment is done by the OS when loading the image. Anyway, the buffer needs to be cleared on each call. Otherwise it will only work the first time.
Code: [Select]
CalculateMedian proc p:ptr, n:dword

  local count[256]:dword

    xor eax,eax
    mov ecx,256
    mov edx,edi
    lea edi,count
    rep stosd
    mov edi,edx

    mov edx,p
    mov ecx,n
    .repeat
        movzx eax,byte ptr [edx]
        inc count[eax*4]
        add edx,1
    .untilcxz

    mov ecx,n
    shr ecx,1
    xor edx,edx
    mov eax,-1
    .repeat
        inc eax
        add edx,count[eax*4]
    .until edx > ecx
    ret

CalculateMedian endp

Maybe it's possible to just count the value in one loop.

median proc uses esi edi p:ptr, n:dword

    mov esi,p
    mov ecx,n
    mov eax,n
    shr eax,1
    inc eax
    mov edi,eax
    .repeat
        movzx edx,byte ptr [esi]
        cmp edx,edi
        sbb edx,edx
        sbb edx,-1
        add eax,edx
        add esi,1
    .untilcxz
    ret

median endp
Title: Re: Fast median algorithm
Post by: jj2007 on July 20, 2020, 07:23:52 AM
Let's make a test case: test1000.zip are 1,000 random DWORDs between 0 and 125,248. The graph shows their distribution when sorted.
Title: Re: Fast median algorithm
Post by: HSE on July 20, 2020, 08:48:09 AM
Let's make a test case: test1000.zip are 1,000 random DWORDs between 0 and 125,248. The graph shows their distribution when sorted.
Nice. Test fail here  :sad:   :biggrin: My mistake printing result! Perfect here.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 20, 2020, 08:52:34 AM
Had the time and looked at the logic again with a fresh mind and rewrote the routine after a bug report from HSE.
I think it has become very fast because of the use of an index buffer, it uses very few iterations.
The speed and iteration count, depend on how many large number values are in the data set.
If there are less large numbers than half of the data set size, they are discarded and not used in the calculation.

Fully commented source code:
Code: [Select]
CalculateMedian proc uses ebx esi edi
   
    mov     esi,offset DataSet
    mov     edi,offset MedianBuffer ; Zero the MedianBuffer before using this routine.

    mov     ecx,PopulationSize
PopulateLP:
    mov     eax,[esi]
    inc     byte ptr [edi+eax]      ; Populate the Median buffer with the values as index numbers 
    add     esi,4                   ; and keep track for duplicates in one go.
    dec     ecx
    jnz     PopulateLP

    xor     esi,esi                 ; Uneven marker.
    mov     ecx,PopulationSize
    mov     ebx,ecx
    shr     ebx,1                   ; The middle of the PopulationSize.
    and     ecx,1                   ; Check even or uneven.
    jnz     UnEven
    dec     ebx                     ; Make even.
    inc     esi                     ; Set marker true = even.
UnEven:
    xor     edx,edx                 ; Median counter.
    mov     eax,-1                  ; Number counter.
FindMedianLP:
    inc     eax
    movzx   ecx,byte ptr[edi+eax]   ; Count of single and/or duplicate numbers. ( zero is not a number )
    add     edx,ecx                 ; Add it to the Median counter.
    cmp     edx,ebx                 ; Compare if we reached the middle of the PopulationSize.
    jbe     FindMedianLP            ; Repeat until we reached the middle.
    dec     esi                     ; Check even or uneven.
    js      Ready                   ; Ready if uneven.
    cmp     ecx,1                   ; Check if next number is a duplicate. ( no averaging needed )
    ja      Ready                   ; Ready if next number is the same value as the previous number.
    mov     edx,eax                 ; Save the first middle number to calculate the average of 2 middle numbers.
FindSecondMiddleNumberLP:
    inc     eax   
    movzx   ecx,byte ptr[edi+eax]
    test    ecx,ecx                 ; If zero, its not a number.
    jz      FindSecondMiddleNumberLP
    add     eax,edx                 ; Second middle number found, add it to the first middle number.
    shr     eax,1                   ; And get the average.
Ready:
    ret
CalculateMedian endp
Title: Re: Fast median algorithm
Post by: HSE on July 20, 2020, 09:59:07 AM
Now faill JJ test   :cool:
Its jumping to Ready when have to obtain the mean.

No problen with CalculateMedian2.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 20, 2020, 10:30:51 AM
Hi HSE,
You can not divide a number by 1000, because it is the index number.

It fails, because there are numbers higher than 1023 in the file.
You need an index buffer in byte size as large as the highest number+1 in the file.
And no more than 255 duplicate numbers.
You can increase the duplicates if needed by using a word or dword sized index buffer, but then you have to rewrite the code.

It seems a good idea to use a dword sized indexed buffer.
So guga can use it for his graphics and also get the median of a picture containing only 1 or a few different colors.
Then he needs only an index buffer of 256 * 1 dword = 1024 bytes and can handle 4294967295 duplicate colors.

But I think he is only using 256 color indexed pictures which is smarter and faster.


Title: Re: Fast median algorithm
Post by: Siekmanski on July 20, 2020, 11:51:49 AM
Routine for dword sized MedianBuffer ( can handle 4294967295 duplicate numbers/colors )

Waiting for guga how he wants to use this in his graphics routine.....


Code: [Select]
align 4   
CalculateMedian proc uses ebx esi edi
   
    mov     esi,offset DataSet
    mov     edi,offset MedianBuffer ; Zero the MedianBuffer before using this routine.

    mov     ecx,PopulationSize
PopulateLP:
    mov     eax,[esi]
    inc     dword ptr[edi+eax*4]    ; Populate the Median buffer with the values as index numbers 
    add     esi,4                   ; and keep track for duplicates in one go.
    dec     ecx
    jnz     PopulateLP

    xor     esi,esi                 ; Uneven marker.
    mov     ecx,PopulationSize
    mov     ebx,ecx
    shr     ebx,1                   ; The middle of the PopulationSize.
    and     ecx,1                   ; Check even or uneven.
    jnz     UnEven
    dec     ebx                     ; Make even.
    inc     esi                     ; Set marker true = even.
UnEven:
    xor     edx,edx                 ; Median counter.
    mov     eax,-1                  ; Number counter.
FindMedianLP:
    inc     eax
    mov     ecx,[edi+eax*4]         ; Count of single and/or duplicate numbers. ( zero is not a number )
    add     edx,ecx                 ; Add it to the Median counter.
    cmp     edx,ebx                 ; Compare if we reached the middle of the PopulationSize.
    jbe     FindMedianLP            ; Repeat until we reached the middle.
    dec     esi                     ; Check even or uneven.
    js      Ready                   ; Ready if uneven.
    cmp     ecx,1                   ; Check if next number is a duplicate. ( no averaging needed )
    ja      Ready                   ; Ready if next number is the same value as the previous number.
    mov     edx,eax                 ; Save the first middle number to calculate the average of 2 middle numbers.
FindSecondMiddleNumberLP:
    inc     eax   
    mov     ecx,[edi+eax*4]
    test    ecx,ecx                 ; If zero, its not a number.
    jz      FindSecondMiddleNumberLP
    add     eax,edx                 ; Second middle number found, add it to the first middle number.
    shr     eax,1                   ; And get the average.
Ready:
    ret

CalculateMedian endp
Title: Re: Fast median algorithm
Post by: HSE on July 20, 2020, 12:44:03 PM
Hi Siekmanski!

You can not divide a number by 1000, because it is the index number.
Yes, I can. Exactly because the index number is the number.  I'm scaling down data, and loosing precision, nothing else.  :biggrin:
(And fooling JJ's idea of a so big DataBuffer that it's better to sort data  :joking:)


It fails, because there are numbers higher than 1023 in the file.
  :arrow_right: It fails because fails. Maximum number is 125.


Then he needs only an index buffer of 256 * 1 dword = 1024 bytes and can handle 4294967295 duplicate colors.
:thumbsup:
Title: Re: Fast median algorithm
Post by: guga on July 20, 2020, 01:31:27 PM
Tks a lot Siekmanski

I´ll test this new version.

On the previous version, it seems to work fine, but i´ll make more tests on this new one.

So far, yours CalculateMedian i'm using to retrieve the median of a table of grey values on each image.  For example, if i have to scan 100 different images, i create 100 tables of gray representing each image. (each one stored in a Dword rather then a single byte)

For example: The routine is at pos (x/y), and the image has a size of 960*720
X =  0, Y = 0 -->It take the 1st dword (That is equal to the gray value) at pos x/y (0:0) on the 1st image and use it in your routine to compare to the  x,y of the next image until it reaches the 100th image (Which, btw, are stored on a huge linked table to properly points to the correct address). After it retrieve the median at pos x/y, the routine returns and look at the next dword at pos (x+1), y and to the median calculation again untill it reaches the 100th image. Like this:
X = 1, Y = 0. Calculate the median again on this pos from image 1 to image 100

It does this untill it reaches to the last data of he image, so the last Y = 720.

I´ll see if this new version also works ok.


One thing, can you make it work for Real4 and Real8 as well, using SSE ? The functions in the graphic routines contains tables of arrays that uses Floats representing the Sobel Gradient Values. So, in other table we have things like this:
X = 0, Y = 0. Value = F$ 0.565646 (F$ in RosAsm is Real4 in masm, i suppose. So,m the same size as a dword but used in floating point)
X = 1, Y = 0. Value = F$ 0.7.9898
X = 2, Y = 0. Value = F$ 0.11111


I think that perhaps to make it works with Floats, it would be need to handle it using similar calculations as done in Quartiles. I was reading how a quartile deviation works and found out that the quartile2 (Q2) is equal to the median of any value. So, maybe to work with float all is need is a table with 4 Dwords (Real4, in fact), that represents Quartiles 2 and Quartile 3 (each one of them separated by his low and half part).

ex: We can have a table like this:

Quartile2A______Quartile2B_________Quartile3A_________Quartile3B

And we have a sequence of, let´s say 100 Floating values.
[17.2, 12.5, 19.6, 111.88, 99.41, 55, 88, 63.8, 1544.89, 99.8978........]

To retrieve the median, we can simply fill the 1st 8 Dwords of our table and order them and start scanning for the rest of the dwords/floats using SSE, jumping from 4 to 4. Dwords that don´t match the last one.

So, the 1st step could be ordering the 1st 4 value and putting them on the quartile table:
Quartile2A______Quartile2B_______Quartile3A________Quartile3B
12.5____________17.2____________19.6____________111.88

Then it simply store the next 4 dwords in xmm0 and looks fopr conditions like this:
1 - If all next 4 values are smaller or equal to The 1st quartelion2A, the routine jumps over the next 4 dwords and scans again.
2 - If all next 4 values are bigger then the last quartile3B, it will copy all 4 values onto Quartile2A to Quartile3 in order.

or something like this. :greensml: :greensml:

Also, if using a similar way to compute using quartiles (A table containign onlly 2 quartiles - Q2 and Q3 seems to do the trick), perhaps the suggestion of daydreamer to use maxsss and minss could be a good thing  :thumbsup:

https://www.hackmath.net/en/calculator/quartile-deviation
https://www.mathsisfun.com/data/quartiles.html
Title: Re: Fast median algorithm
Post by: Siekmanski on July 20, 2020, 07:56:03 PM
Hi guga,

So, if I did understand it correctly:

1. Convert 100 images to gray values.
2. Get the median of 100 pixels. ( from the same x/y position of all those 100 images )
3. If the image size = 960*720 pixels you'll have at the end 691200 median values for your Sobel function.

Is the above correct?

I asked you this because you use pixel data which is in 4 bytes format per ARGB value.
Every gray pixel value has a range of 0-255 because each color component in the gray RGB value have the same value.
So you can store a gray value as a byte.

My Median routine is based on integer numbers ( used as indexed numbers ) and will not work with Real4 numbers. ( @HSE this is why a data set of Real4 values will fail.  :thumbsup: )
Pixel data are integer data, that's why I came up with this Algorithm using indexed numbers.

But you want the Median values to be in Real4 format?
You can convert and save the calculated Median integer value as a real4 value at the end of this Algorithm and use it for the Sobel function. ( and gain 0.5 value precision for double middle numbers )

And from that point you can do your Quartile Sobel Real4 calculations in fast SIMD instructions SSE SSE2 etc.
Title: Re: Fast median algorithm
Post by: HSE on July 21, 2020, 12:26:14 AM
My Median routine is based on integer numbers ( used as indexed numbers ) and will not work with Real4 numbers. ( @HSE this is why a data set of Real4 values will fail.  :thumbsup: )
Pixel data are integer data, that's why I came up with this Algorithm using indexed numbers.

Sorry Siekmanski! I was thinking you know qWord tricks.

Result of fSlv eax = eax/1000 is an integer. Value in eax is moved to a buffer, FPU load integer from buffer, FPU make calculation, FPU store a rounded to integer value in buffer, and finally integer buffer content is moved to eax.

My corrected patch for version that work:
Code: [Select]
        mov ecx, edx
        sub ecx, ebx
        cmp ecx , 1
        jg Done

CalculateMedian2 work; CalculateMedian3 and 4 fail.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 21, 2020, 12:41:05 AM
 :thumbsup:
Guga is using pixel data thus that trick is not necessary, the Median routine only reads integers.  :cool:
Title: Re: Fast median algorithm
Post by: guga on July 21, 2020, 03:12:16 AM
Hi Siekmanski.

Hi guga,

So, if I did understand it correctly:

1. Convert 100 images to gray values.
2. Get the median of 100 pixels. ( from the same x/y position of all those 100 images )
3. If the image size = 960*720 pixels you'll have at the end 691200 median values for your Sobel function.

Is the above correct?
I asked you this because you use pixel data which is in 4 bytes format per ARGB value.
Every gray pixel value has a range of 0-255 because each color component in the gray RGB value have the same value.
So you can store a gray value as a byte.

My Median routine is based on integer numbers ( used as indexed numbers ) and will not work with Real4 numbers. ( @HSE this is why a data set of Real4 values will fail.  :thumbsup: )
Pixel data are integer data, that's why I came up with this Algorithm using indexed numbers.

Yes, that´s correct, but not in that order. The median calculation is done after sobel magnitudes are found. Basically it does:
1 - Scans a directory of images, select the images that fits to the resolution of 960*720
2 - Create a buffer large enough to store those images that wil be converted onto different tables: grey, sobelx, sobelx and sobel.
3 - Convert all chosen images to gray and put the data on the proper buffer location to be used as a  map. (For now, all 3 channels are preserved, but it can be fixed later to decrease the size of the GrayTable)
4 - Create the Maps for all 3 types of Sobel (Sobelx, Sobely and Sobel itself) from the grey values found in the grey map. The sobel values are in Real4 because they are the fixed normalization of the luminosity (so, the true value - corrected when needed - and then divided by 255)
5 - Calculate the median of each Sobel Maps (Sobelx, SobelY and Sobel). On this step, since i wanted to see the results, i used your median algo to convert the values to integer to create the corresponding bitmap, so i can see what it is going on.
6 - Generate a poisson reconstruction algorithm somehow....
7 - Read the google papers, but have no idea what to do next, yet. :mrgreen: :mrgreen: :mrgreen:
I realize that those steps can be unnecessary since i could simply take the median of greys (in integer) and therefore generate one single sobel map. (or a maximum of 3: sobelX, sobelY and sobel), but since i don´t know what the next steps are doing (in the python version) i choose to keep all the maps so i could later try to understand if the rest of the watermark functions are using those maps or not. One of the parts of the google algorithm uses a possion reconstruction routine and also convert something to canny (I presume it is converting all 100 gray images to canny as well...but don´t know yet)


But you want the Median values to be in Real4 format?
You can convert and save the calculated Median integer value as a real4 value at the end of this Algorithm and use it for the Sobel function. ( and gain 0.5 value precision for double middle numbers )

And from that point you can do your Quartile Sobel Real4 calculations in fast SIMD instructions SSE SSE2 etc.
Yes, please. If you can, could you try a version to work with Real4 ? On this way i could properly use it in step "5" (and others) that are using Real4 data rather then integers to detect the watermark. This is because i´m not sure if calculate the median previously biased only on the integer values of the grey will also works for the rest of the detect watermark algorithm. So i could try later both methods and see which one is getting closer to a result according to the google article.

I presume it could work, but since i don´t know python i can follow/debug and test the resultant values, so i´m basically reading google article, reading the python source and making a trial and error to find if the routines are according to the article.

I´m currently working only on this part of the python routine:
Code: [Select]

def estimate_watermark(foldername):
    """
    Given a folder, estimate the watermark (grad(W) = median(grad(J))) follow Eq.4
    Also, give the list of gradients, so that further processing can be done on it
    """
    if not os.path.exists(foldername):
        warnings.warn("Folder does not exist.", UserWarning)
        return None

    images = []
    for r, dirs, files in os.walk(foldername):
        # Get all the images
        for file in files:
            img = cv2.imread(os.sep.join([r, file]))
            if img is not None:
                images.append(img)
            else:
                print("%s not found." % (file))

    # Compute gradients
    print("Computing gradients.")
    gradx = list(map(lambda x: cv2.Sobel(
        x, cv2.CV_64F, 1, 0, ksize=KERNEL_SIZE), images))
    grady = list(map(lambda x: cv2.Sobel(
        x, cv2.CV_64F, 0, 1, ksize=KERNEL_SIZE), images))

    # Compute median of grads
    print("Computing median gradients.")
    Wm_x = np.median(np.array(gradx), axis=0)
    Wm_y = np.median(np.array(grady), axis=0)

    return (Wm_x, Wm_y, gradx, grady)

Which is called from:
Code: [Select]
gx, gy, gxlist, gylist = estimate_watermark('images/fotolia_processed')

So, if i understood correctly:
gxlist, gylist are all the 100 sobelmaps. The question is: Are they being used again ??? I don´t know yet
gx, gy are the median maps (2 only) of the sobel ones.

I just don´t know if all gxlist will be used or i can use only the median values previously calculate because the next routine is the poisson reconstruction and i didn´t fully understood yet what it is doing. The possion is build as:
Code: [Select]

def poisson_reconstruct(gradx, grady, kernel_size=KERNEL_SIZE, num_iters=11, h=0.1,
                        boundary_image=None, boundary_zero=True):
    """
    Iterative algorithm for Poisson reconstruction.
    Given the gradx and grady values, find laplacian, and solve for image
    Also return the squared difference of every step.
    h = convergence rate
    """
    fxx = cv2.Sobel(gradx, cv2.CV_64F, 1, 0, ksize=kernel_size)
    fyy = cv2.Sobel(grady, cv2.CV_64F, 0, 1, ksize=kernel_size)
    laplacian = fxx + fyy
    m, n, p = laplacian.shape

    if boundary_zero == True:
        est = np.zeros(laplacian.shape)
    else:
        assert(boundary_image is not None)
        assert(boundary_image.shape == laplacian.shape)
        est = boundary_image.copy()

    est[1:-1, 1:-1, :] = np.random.random((m-2, n-2, p))
    loss = []

    for i in range(num_iters):
        old_est = est.copy()
        est[1:-1, 1:-1, :] = 0.25*(est[0:-2, 1:-1, :] + est[1:-1, 0:-2, :] +
                                   est[2:, 1:-1, :] + est[1:-1, 2:, :] - h*h*laplacian[1:-1, 1:-1, :])
        error = np.sum(np.square(est-old_est))
        loss.append(error)

    return (est)

See ? The poisson routine is calculating sobel all over again but this time it seems to be on the target file from which the watermark should be removed, and i have no idea if that is really necessary.
The poisson is called from this part of the code:

Code: [Select]

gx, gy, gxlist, gylist = estimate_watermark('images/fotolia_processed') ; <---- Already did previously (or trying to finish it to see if it ok)

# est = poisson_reconstruct(gx, gy, np.zeros(gx.shape)[:,:,0])
cropped_gx, cropped_gy = crop_watermark(gx, gy) ; <---------- Is really necessary crop the watermark here ??? It can b do0ne previously since the very 1st graymap to we work only on the already cropped tables. (Assuming it is necessary to crop btw)

# random photo
img = cv2.imread('images/fotolia_processed/fotolia_137840668.jpg') ; <--- read the image from where the watermark should be removed
im, start, end = watermark_detector(img, cropped_gx, cropped_gy) ; < ---- try to detect the watermark on this image as well. (This routine is where it will use canny)

# # Save result of watermark detection
# plt.subplot(1, 2, 1)
# plt.imshow(img)
# plt.subplot(1, 2, 2)
# plt.imshow(im)
# plt.savefig('images/results/watermark_detect_result.png')
# We are done with watermark estimation

# W_m is the cropped watermark
W_m = poisson_reconstruct(cropped_gx, cropped_gy) ; <--- Use poisson only on the target image i suppose
(...)

The crop watermark function is like this, and i have n idea what it is actually doing

Code: [Select]

def crop_watermark(gradx, grady, threshold=0.4, boundary_size=2):
    """
    Crops the watermark by taking the edge map of magnitude of grad(W)
    Assumes the gradx and grady to be in 3 channels
    @param: threshold - gives the threshold param
    @param: boundary_size - boundary around cropped image
    """
    W_mod = np.sqrt(np.square(gradx) + np.square(grady))
    # Map image values to [0, 1]
    W_mod = PlotImage(W_mod)
    # Threshold the image with threshold=0.4
    W_gray = image_threshold(np.average(W_mod, axis=2), threshold=threshold)
    x, y = np.where(W_gray == 1)

    # Boundary of cropped image (contain watermark)
    xm, xM = np.min(x) - boundary_size - 1, np.max(x) + boundary_size + 1
    ym, yM = np.min(y) - boundary_size - 1, np.max(y) + boundary_size + 1

    return gradx[xm:xM, ym:yM, :], grady[xm:xM, ym:yM, :]
     # return gradx[xm:xM, ym:yM, 0], grady[xm:xM, ym:yM, 0]



basically, the files in python are these ones i attached here (Some 2 or 3 related to numpy, i didn´t included due to their size), but the full python version  can be downloaded here https://github.com/rohitrango/automatic-watermark-detection
Also, i included the functions used in the source in one single pdf so i could better try to follow the rotines to see if i can understand what it is doing.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 21, 2020, 06:34:19 AM
I thought this is what you wanted to do:
Instead of X and Y convolution matrices you like to use X and Y median matrices to detect the edges in the image?

I do not understand the part, you talk about 100 different pictures.
I'm not really sure what to make of the watermark removal part, or is it only about edge detection?

-> The median calculation is done after sobel magnitudes are found.
Why not directly find the magnitudes from the median matrices. ( don't know if this is a good idea... have to experiment with that thought. )


Title: Re: Fast median algorithm
Post by: guga on July 21, 2020, 07:00:55 AM
I thought this is what you wanted to do:
Instead of X and Y convolution matrices you like to use X and Y median matrices to detect the edges in the image?

I do not understand the part, you talk about 100 different pictures.
I'm not really sure what to make of the watermark removal part, or is it only about edge detection?

No. The median is used to calculate the median of the Sobel value of each pixel on all 100 different images. If i understood the documentation correctly, it calculated the median of the sobel values found on each image.

The part i´m talking about 100 images is from where it compute the median of sobels (Their results are in Real4). i´m using your routine to convert the resultant sobels to integer (rather then Real4), just to actually see what the algo is doing, since i´m not sure if i interpreted correctly what the python functions are actually doing

Ex:
The images have 960*720 pixels. So, it compute the sobelx, sobely and sobel for each image individually. In order to do that, each image must be converted to grey, right ? So, at the end the map generates a 960*720 values of sobel data.

Basically, after the convertion of each image to grey, the algorithm does this:

Calulate the sobel values of each image and take the median of those images like this:

Image1 - X/Y Coords and their sobel values on that locations
(x =0, Y=0. Sobel at this pos. Sobelx = 0.15458, SobelY = 0.564, Sobel = 0.988978)
(x =1, Y=0. Sobel at this pos. Sobelx = 0.14454, SobelY = 0.4565465, Sobel = 0.1978)
(...)
(x = 960, y = 720. Sobel at this pos. Sobelx =  ..................)

Image2 - X/Y Coords and their sobel values on that locations
(x =0, Y=0. Sobel at this pos. Sobelx = 0.458, SobelY = 0.5, Sobel = 0.9978)
(x =1, Y=0. Sobel at this pos. Sobelx = 0.54, SobelY = 0.255, Sobel = 0.78)
(...)
(x = 960, y = 720. Sobel at this pos. Sobelx =  ..................)

Image3 - X/Y Coords and their sobel values on that locations
(x =0, Y=0. Sobel at this pos. Sobelx = 0.18, SobelY = 0.54, Sobel = 0.98)
(x =1, Y=0. Sobel at this pos. Sobelx = 0.454, SobelY = 0.5, Sobel = 0.78)
(...)
(...)
(x = 960, y = 720. Sobel at this pos. Sobelx =  ..................)

Image100
(...)

What it does is compute the median on each location (from image 1 to 100) in order to create one single map data of median sobel values (The  Wm_x /  Wm_y variables)

So, it does:

SobelX median of pos X = 0, Y = 0 = Median of (0.15458,  0.458, 0.18, ....). Of course, what i did here, is convert this floats to integers (multiplying those fractions with 255), so i could be able to actually see what it is doing. (This is not from the article, just something is needed to see the result at this point). That´s why it maybe necessary a algo to compute median using Real4, btw, because in other parts of the watermark algorithm, the median seems to not be calculated from integers whatsoever, but Real4 values.

So, the next location is something like:
SobelX median of pos X = 1, Y = 0 (*255 to convert to integers, so i could see later) = Median of (0.14454*255,  0.54*255, 0.454*255, ....)

So, it generates one single mapped file containing at it's location x,y, the median of all 100 images on that same location x,y.


At least, this is how i interpreted what google algorithm is doing. at "estimate_watermark" function.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 21, 2020, 07:14:41 AM
 :thumbsup:

OK now I understand what you want.
Do you have those 100 images for me?
Then I can try it out, if I have enough memory to store all those values.  :biggrin:
Title: Re: Fast median algorithm
Post by: guga on July 21, 2020, 07:18:25 AM
I´ll upload them for you. I´m testing in some images from my own that are already converted to 960*720. Do you prefer in what format ? Png, jpg..... ???
Title: Re: Fast median algorithm
Post by: Siekmanski on July 21, 2020, 07:27:31 AM
Don't matter, what you think is best.  :thumbsup:
Title: Re: Fast median algorithm
Post by: guga on July 21, 2020, 07:34:44 AM
Ok, here we go  :thumbsup: :thumbsup: :thumbsup:

https://we.tl/t-Ay9zox2FZe
Title: Re: Fast median algorithm
Post by: Siekmanski on July 21, 2020, 08:12:40 AM
Thanks I got them, they are already gray colored.
I need some days to code and test it, don't have much spare time this week.  :sad:
As you know I like challenges like this, I'll be back with some code when ready.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 21, 2020, 08:15:40 AM
Ah, there are more than 256 different gray colors..... need to convert them all.  :biggrin:
Title: Re: Fast median algorithm
Post by: guga on July 21, 2020, 08:25:30 AM
Ah, there are more than 256 different gray colors..... need to convert them all.  :biggrin:

Oops..Sorry, It was not converted to gray when i exported. Here we go all images in grayscale :)

https://we.tl/t-cDUcU7PzGM
Title: Re: Fast median algorithm
Post by: Siekmanski on July 21, 2020, 08:31:12 AM
Thanks, saves us a few lines code.  :biggrin:
Title: Re: Fast median algorithm
Post by: guga on July 21, 2020, 11:04:25 AM
Btw...If you want to test on the new Sobel routine i made, here it is ported to masm. (Sorry, i didn´t used macros here, because i don´t remember how the macros in masm are....but it won´t shouldn´t hard to follow  :smiley:). My variation of Sobel result on thinner edges and a correct equalization of the pixels distribution.

Masm version

Code: [Select]
Float_Two       dq 2.0
Float_One_255   dq 0.00392156862745098 ; (1/255)
Float_SobleVarG dq 180.3122292025696 ; 255/sqrt(2). This is to speed up the calculation rather then using divide.

SobelGetGPLusEx proc near

FractionY       = dword ptr -14h
FractionX       = dword ptr -10h
Divisor         = dword ptr -0Ch
DataCheck       = dword ptr -8
pReturn         = dword ptr -4
pMatrix         = dword ptr  8
pOutSobelX      = dword ptr  0Ch
pOutSobelY      = dword ptr  10h
pOutSobel       = dword ptr  14h

                push    ebp
                mov     ebp, esp
                sub     esp, 14h
                push    edi
                push    edx
                push    ecx
                finit
                mov     edi, [ebp+pMatrix]

                 ; To calculate Gx^2 later. Therefore Gx = M3+M9 + 2*(M6-M4) - (M7+M1)

                fld     dword ptr [edi+20] ; M6
                fsub    dword ptr [edi+12] ; M4
                fmul    Float_Two
                fadd    dword ptr [edi+8] ; M3
                fadd    dword ptr [edi+32] ; M9
                fsub    dword ptr [edi+24] ; M7
                fsub    dword ptr [edi+0] ; M1
                lea     ecx, [ebp+DataCheck]
                fistp   dword ptr [ecx]
                cmp     [ebp+DataCheck], 0
                jnz     short loc_4C7649
                fldz
                fstp    [ebp+FractionX]
                jmp     short loc_4C76AB
; ---------------------------------------------------------------------------

loc_4C7649:
                cmp     [ebp+DataCheck], -255 ; Blacks. Edge too dark. Limit was extrapolated
                jge     short loc_4C7675
                xor     edx, edx
                mov     ecx, 255
                mov     eax, [ebp+DataCheck]
                neg     eax
                div     ecx
                inc     eax
                imul    eax, 255
                mov     [ebp+Divisor], eax
                fild    [ebp+DataCheck]
                fidiv   [ebp+Divisor]
                fstp    [ebp+FractionX]
                jmp     short loc_4C76AB
; ---------------------------------------------------------------------------

loc_4C7675:
                cmp     [ebp+DataCheck], 255 ; Whites. Edge too brigth. Limit was extrapolated
                jle     short loc_4C769F
                xor     edx, edx
                mov     ecx, 255
                mov     eax, [ebp+DataCheck]
                div     ecx
                inc     eax
                imul    eax, 255
                mov     [ebp+Divisor], eax
                fild    [ebp+DataCheck]
                fidiv   [ebp+Divisor]
                fstp    [ebp+FractionX]
                jmp     short loc_4C76AB
; ---------------------------------------------------------------------------

loc_4C769F:
                fild    [ebp+DataCheck]
                fmul    Float_One_255
                fstp    [ebp+FractionX]

loc_4C76AB:
                mov     eax, [ebp+pOutSobelX]
                mov     ecx, [ebp+FractionX]
                mov     [eax], ecx

                 ; To calculate Gy^2 later. Therefore Gy = M7+M9 + 2*(M8-M2) - (M3+M1)
                fld     dword ptr [edi+28] ; M8
                fsub    dword ptr [edi+4] ; M2
                fmul    Float_Two
                fadd    dword ptr [edi+24] ; M7
                fadd    dword ptr [edi+32] ; M9
                fsub    dword ptr [edi+8] ; M3
                fsub    dword ptr [edi+0] ;  M1
                lea     ecx, [ebp+DataCheck]
                fistp   dword ptr [ecx]
                cmp     [ebp+DataCheck], 0
                jnz     short loc_4C76DD
                fldz
                fstp    [ebp+FractionY]
                jmp     short loc_4C773F
; ---------------------------------------------------------------------------

loc_4C76DD:
                cmp     [ebp+DataCheck], -255 ; Blacks. Edge too dark. Limit was extrapolated
                jge     short loc_4C7709
                xor     edx, edx
                mov     ecx, 255
                mov     eax, [ebp+DataCheck]
                neg     eax
                div     ecx
                inc     eax
                imul    eax, 255
                mov     [ebp+Divisor], eax
                fild    [ebp+DataCheck]
                fidiv   [ebp+Divisor]
                fstp    [ebp+FractionY]
                jmp     short loc_4C773F
; ---------------------------------------------------------------------------

loc_4C7709:
                cmp     [ebp+DataCheck], 255 ; Whites. Edge too brigth. Limit was extrapolated
                jle     short loc_4C7733
                xor     edx, edx
                mov     ecx, 255
                mov     eax, [ebp+DataCheck]
                div     ecx
                inc     eax
                imul    eax, 255
                mov     [ebp+Divisor], eax
                fild    [ebp+DataCheck]
                fidiv   [ebp+Divisor]
                fstp    [ebp+FractionY]
                jmp     short loc_4C773F
; ---------------------------------------------------------------------------

loc_4C7733:

                ; Soble = sqrt((255*FractionX)^2+(255*FractionY)^2)) = G = sqrt(Gx^2+Gy^2)
                ; since FractionX and FractionY can have a maximum of 1 and -1, therefore sobleMax = (255/sqrt(2)) * sqrt(FractionX^2+FractionY^2)

                fild    [ebp+DataCheck]
                fmul    Float_One_255
                fstp    [ebp+FractionY]

loc_4C773F:
                mov     eax, [ebp+pOutSobelY]
                mov     ecx, [ebp+FractionY]
                mov     [eax], ecx
                fld     [ebp+FractionX]
                fmul    st, st
                fld     [ebp+FractionY]
                fmul    st, st
                faddp   st(1), st
                fsqrt
                fmul    Float_SobleVarG
                lea     edx, [ebp+pReturn]
                fistp   dword ptr [edx]
                mov     eax, [ebp+pReturn]
                cmp     eax, 255
                jbe     short loc_4C776F
                mov     eax, 255

loc_4C776F:
                mov     ecx, [ebp+pOutSobel]
                mov     [ecx], eax
                pop     ecx
                pop     edx
                pop     edi
                mov     esp, ebp
                pop     ebp
                retn    10h
SobelGetGPLusEx endp


The parameters are:

pMatrix(In):
            Pointer to a 3*3 Matrix of Dwords containing the gray values

pOutSobelX(Out):
            Pointer to a variable that will store the SobelX value. The variable must be is Dword/Float Sized (It exports a float, but, they are the same size ;) ). The saved value is a Real4 from -1 to 1. So they are, actually, the normalized grey. i.e: Color/255
            If the value is negative, it means the pixel is dark. If the value is positive, it means, the pixel is brighter. So, at the end, basically, it goes from 0 to 127 (means dark) and 128 to 255 meaning bright.
           The sign here (negative or positive) was kept because we can use the values of Gx and Gy to compute the direction of the pixel with atan(Gy/Gx)

pOutSobelY(Out):
            Pointer to a variable that will store the SobelY value. The variable must be is Dword/Float Sized (It exports a float, but, they are the same size ;) ). The saved value is a Real4 from -1 to 1. So they are, actually, the normalized grey. i.e: Color/255
            If the value is negative, it means the pixel is dark. If the value is positive, it means, the pixel is brighter. So, at the end, basically, it goes from 0 to 127 (means dark) and 128 to 255 meaning bright.
            The sign here (negative or positive) was kept because we can use the values of Gx and Gy to compute the direction of the pixel with atan(Gy/Gx)

pOutSobel(Out):
            Pointer to a variable that will store the Sobel value. The variable must be a Dword. The saved value is a integer from 0 to 255.


Example of usage:


TmpMatrix   dd 212, 10, 15
            dd 105, 5, 18
            dd 215, 178, 0

            lea eax D$ebp+SobelX
            lea ebx D$ebp+SobelY
            lea ecx D$ebp+Sobel
invoke SobelGetGPlusEx, offset TmpMatrix, ecx, ebx, eax


Note:
The matrix containing the dwords of greys must be previously filled. They correspond basically to the 3 pixels from the width and 3 pixels from the height. Collected like this:

width = 100
height = 200

Pixel Pos in image (x/y) - On this example, to make it simple, i´ll just posting here the grey correspondent values and not the bytes from all channels, ok ?

X0/Y0  212 10 15 59 68 125 656 .....
X0/Y1  105 5 18 25 16 12 56 .....
X0/Y2  215 178 0 9 8 15 65 .....

The matrix is, therefore, always in this order:
M1  M2  M3
M4  M5  M6
M7  M8  M9

I don´t  remember how equates are used in masm, but on the SobelGetGPLusEx  function, when you see [edi+20], it actually refers to M6 position of the matrix.

Therefore:

M1 = 0
M2 = 4
M3 = 8
M4 = 12
M5 = 16
M6 = 20
M7 = 24
M8 = 28
M9 = 32

Size of the matrix = 36 bytes


The Rosasm version below should be a bit better to read. So here is the original version in RosAsm Syntax

Code: [Select]
Proc SobelGetGPLusEx:
    Arguments @pMatrix, @pOutSobelX, @pOutSobelY, @pOutSobel
    Local @pReturn, @DataCheck, @Divisor, @FractionX, @FractionY
    Uses edi, edx, ecx

    finit

    mov edi D@pMatrix

    ; To calculate Gx^2 later. Therfore Gx = M3+M9 + 2*(M6-M4) - (M7+M1)
    fld F$edi+FloatMatricesInt.M6Dis | fsub F$edi+FloatMatricesInt.M4Dis | fmul R$Float_Two
    fadd F$edi+FloatMatricesInt.M3Dis | fadd F$edi+FloatMatricesInt.M9Dis
    fsub F$edi+FloatMatricesInt.M7Dis | fsub F$edi+FloatMatricesInt.M1Dis
    lea ecx D@DataCheck | fistp F$ecx
    If D@DataCheck = 0

        fldz | fstp F@FractionX

    Else_If D@DataCheck <s 0-255 ; Blacks. Edge too dark

        xor edx edx | mov ecx 255 | mov eax D@DataCheck | neg eax | div ecx | inc eax | imul eax 255 | mov D@Divisor eax
        fild D@DataCheck | fidiv F@Divisor |  fstp F@FractionX

    Else_If D@DataCheck >s 255 ; Whites. Edge too brigth

        xor edx edx | mov ecx 255 | mov eax D@DataCheck | div ecx | inc eax | imul eax 255 | mov D@Divisor eax
        fild D@DataCheck | fidiv F@Divisor |  fstp F@FractionX

    Else
        fild D@DataCheck | fmul R$Float_One_255 | fstp F@FractionX
    End_If
    mov eax D@pOutSobelX | mov ecx D@FractionX | mov D$eax ecx

    ; To calculate Gy^2 later. Therefore Gy = M7+M9 + 2*(M8-M2) - (M3+M1)
    fld F$edi+FloatMatricesInt.M8Dis | fsub F$edi+FloatMatricesInt.M2Dis | fmul R$Float_Two
    fadd F$edi+FloatMatricesInt.M7Dis | fadd F$edi+FloatMatricesInt.M9Dis
    fsub F$edi+FloatMatricesInt.M3Dis | fsub F$edi+FloatMatricesInt.M1Dis
    lea ecx D@DataCheck | fistp F$ecx
    If D@DataCheck = 0
        fldz | fstp F@FractionY
    Else_If D@DataCheck <s 0-255 ; Blacks
        xor edx edx | mov ecx 255 | mov eax D@DataCheck | neg eax | div ecx | inc eax | imul eax 255 | mov D@Divisor eax
        fild D@DataCheck | fidiv F@Divisor | fstp F@FractionY; | fmul R$Float255
    Else_If D@DataCheck >s 255 ; Whites
        xor edx edx | mov ecx 255 | mov eax D@DataCheck | div ecx | inc eax | imul eax 255 | mov D@Divisor eax
        fild D@DataCheck | fidiv F@Divisor | fstp F@FractionY; | fmul R$Float255
    Else
        fild D@DataCheck | fmul R$Float_One_255 | fstp F@FractionY
    End_If
    mov eax D@pOutSobelY | mov ecx D@FractionY | mov D$eax ecx


    ; Soble = sqrt((255*FractionX)^2+(255*FractionY)^2)) = G = sqrt(Gx^2+Gy^2)
    ; since FractionX and FractionY can have a maximum of 1 and -1, therefore sobleMax = (255/sqrt(2)) * sqrt(FractionX^2+FractionY^2)

    fld F@FractionX | fmul ST0 ST0 | fld F@FractionY | fmul ST0 ST0 | faddp ST1 ST0 | fsqrt | fmul R$Float_SobleVarG
    lea edx D@pReturn
    fistp F$edx
    mov eax D@pReturn
    If eax > 255 ; Can be replaced with movzx eax ax. But i´m keping the comparition to check for errors while debugging.
        mov eax 255
    End_If
    mov ecx D@pOutSobel | mov D$ecx eax

EndP

And RosAsm equates and variables used are:

Code: [Select]

; variables
[Float_Two: R$ 2]
[FloatOne_255: R$ (1/255)]
[Float_SobleVarG: R$ 180.3122292025696187222153123367365]; 255/sqrt(2))

; equates
[FloatMatricesInt.M1Dis 0
 FloatMatricesInt.M2Dis 4
 FloatMatricesInt.M3Dis 8
 FloatMatricesInt.M4Dis 12
 FloatMatricesInt.M5Dis 16
 FloatMatricesInt.M6Dis 20
 FloatMatricesInt.M7Dis 24
 FloatMatricesInt.M8Dis 28
 FloatMatricesInt.M9Dis 32]

[Size_Of_FloatMatricesInt 36]

Remarks:
1 - Why not using the matrices -1, 0, 1 | -2, 0, 2 | 1, 0, 1 for Gx and 1, 2, 1 | 0, 0, 0 | -1, 2, -1 for Gy ?
 To gain speed. This is why i already calculated the math envolving the matrix arithmetics which results in:

Gx = M3+M9 + 2*(M6-M4) - (M7+M1)
Gy = M7+M9 + 2*(M8-M2) - (M3+M1)

2 - Why did i used fractions and comparitions of DataCheck to 0, -255, 255 ?
Well, because those are the places where normal Sobel algorithm fails. While i was testing the algorithm i saw that some pixels on the edges were excessivelly bright or pure black and also found in some images, those extrapolations anywhere in the image.
Then, i realized that what sobel was producing was, in fact: noise.
But, then i thought, well, where those noises are located after all ? Are they really noise or simply an imperfection of the sobel algorithm itself ? I then, forced the function to export those extrapollations in red so i could actually see where they were located. For my surprise, the vast majority of these extrapolations occours near the true edges resulting in thicker edge line and some random noise here and there along the Gx and Gy edges.
What i saw was that it was generating values in the order of:

-400, -269, -300, -800 too dark
290, 300, 352, 402, etc etc too bright

So, i fixed that, simply calculating how much these values were exceeding and making the following equation:

If the exceeding value is negative i did:

FixedValue = OldValue/(255*(floor(|OldValue|/255)+1))

If the exceeding value is positive:

FixedValue = OldValue/(255*(floor(OldValue/255)+1))

Why is that ? well, i simply deal the excess like an extension of the limit of 255. So, in order for me to keep the values in between 0 and 1 something need to be done.
Therefore, i basically compute how much it is excedding to see which limit it is extrapolating. For example. Say we have a value of 273.

The range is only from 0 to (+/-)255, right ?

So, if the value was found in the range of 273, it means, that it is extrapolating the limit of 255. Therefore i needed to estimate another limit rather then 255 to fix the value. On this case, the new limit is achieved simply dividing the value by 255, and whatever
is the resultant value of the quotient, we simply add 1 to it. Why is that ? Simple...

Let´s say: 273/255 = 1.07058....
It means that it is extrapolating our limit in 0.07058..... Therefore our limit is no longer 255 but. 255*2 = 510. Why is that ? Because it is the next limit (next chunk of 255) that may fits to that remainder.
So, our quotient = 1 , we do (1+1)*255 = 510 as our new limit.

Then i simply divide the value from with this new limit = 273/510 = 0.53294....
So, our new fraction to Gx or Gy is now 0.53294.....inside the limits of 0 to 1

And, sure, i kept the sign intact in both cases. That´s why i needed to take the floor of the modulus of X/255 only when the value was negative.  :azn:

One thing that can also be used on this error cases of Sobel, is properly rebuild the edges. So, we could, simply making the app export those bad locations as bad pixels, and then mark them (as red, for example).
Then after the main sobel operation is over along the whole image, you could simply search for those bad pixels and recalculate the New Edges value (sobel) of them from the "good' sobel values from the neighbour pixels. or do whatever other thing to properly reconstruct those areas, specially because the pixels nearby will also be used in the matrix to compute their own sobels.

So, perhaps, a better techique could be simply flaging this pixel as bad and it´s neighbours ( on a 3x3 matrix) as yellows (warning pixels or something) and avoid computing the sobel from those bad or warned areas.

On this way, you could simply identify all good pixels 1st and create another algorithm to know what to do with the bad pixels and it´s neighbours. But that´s another story. For now, i choose only to calculate their floors to keep the algo as fast and reliable as possible. (Sure, i could also use Scharr operators, but this could be done later with his own algo).

3 - Why using floats to export the values in Gx and Gy and on the other hand, using integer to G ?
This is because how Sobel actually works. The true value in Sobel G will always be a integer from 0 to 255. Extrapolations are not allowed on mine version whatsoever to avoid having to cut off data.

And i´m using floats in Gx (SobelX) and Gy (SobelY), because, since those values are the ones needed to calculate the final integer G (Sobel), i choose to keep them as fractions, so the result of the squared root could be a bit more accurated then simply using integers. G is computed as:
G = sqrt(Gx^2+Gy^2)

So, keeping the values of Gy and Gx as floating point, results on a more accurated result wich always will only varies from a margin of error of 1 depending of the rounding mode of the system.  And also, we can use the values of Gx and Gy to compute the direction (angle) of the pixel with atan(Gy/Gx).

At the very end of the function, you can see that it will only needed to check is G in eax is bigger then 255 (which, btw, may not never happens, or when it does, the extrapolation is at the maximum value of 256 due to the round mode).

Sure...i could, as well, gain a bit more speed simply making movzx eax ax at the end, instead of comparing it with 255. But since i´m testing all of this, i kept the If/EndIf macros, mainly for debugging purposes.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 21, 2020, 06:46:22 PM
Just some ideas:

For example, to get rid of the errors at the image borders:
- image = 100 * 100 pixels
- X and Y matrices = 3 * 3

Allocate a work buffer of 102 * 102 pixels with zeros.
Copy the bitmap data exactly in the middle so we have created a 1 pixel space around the image.
Now we can use the 3 * 3 matrix convolution maps without restoring the image border pixels.
The 1 pixel extra boundary space is only read, not written.

To prevent overshoots in Gx and Gy, we could normalize the matrix members.
If I'm correct, we will have clamped G values between 0-255 after sqrt(Gx^2+Gy^2).

This way we can also experiment with other matrix sizes and/or for example Gaussian ( evenly distributed ) convolution maps.

This is all theoretical, of course, and it has to be proven to work in the real world.  :biggrin:
Title: Re: Fast median algorithm
Post by: jj2007 on July 21, 2020, 08:58:03 PM
Hi Marinus,
Your CalculateMedian algo is blazing fast, and I am trying to understand it :smiley:
However, I get mixed results - any explanation for this?
Code: [Select]
998 µs for MbMedian      M=124998
233 µs for CalcMedian    M=124998
1034 µs for ArraySort    M=124998
#elements=10000, maxY=501499

1662 µs for MbMedian     M=125749
92 µs for CalcMedian     M=110213 <<<<<<<<<<<<<
1567 µs for ArraySort    M=125749
#elements=20000, maxY=501499

3334 µs for MbMedian     M=126000
138 µs for CalcMedian    M=126000
2549 µs for ArraySort    M=126000
#elements=40000, maxY=501499

40,000 dwords attached. MedianBufferSize=4032000
Title: Re: Fast median algorithm
Post by: nidud on July 22, 2020, 02:09:54 AM
 :biggrin:

It's fast but obviously not that fast and it also depends on what you feed it.
Maximum swap:

160,150,140,130,120,110,100,90,80,70,60,50,40,30,20,10,0
total [0 .. 3], 1++
   473640 cycles 1.asm: CalculateMedian
   523550 cycles 0.asm: median

No swap:

500 dup(110)
total [0 .. 3], 1++
   282577 cycles 0.asm: median
   619579 cycles 1.asm: CalculateMedian
Title: Re: Fast median algorithm
Post by: Siekmanski on July 22, 2020, 02:33:43 AM
Hi Nidud,

It is not so fast when there are great gaps between the numbers.
But this routine is special designed for gray color bitmap pixel data.
- they are normally close to each other and there are many duplicate values ( only 1 calculation for all the same duplicate numbers )
- They have a small range 0-255
- Big chance the whole buffer is populated at all offsets.
- it can process many thousands of pixel values very fast.

Hi Jochen,

Don't know... could be a bug...

There is one thing you need to know beforehand, what number in the data set is the highest number.
The MedianBufferSize ( in DWORD Size ) needs to be as large as the highest number + 1

It works like this:

- Start with an empty MedianBuffer
- Read the first item from the data set, the value is also the offset in the buffer, increase the value in the buffer by 1.
  When there are duplicate numbers in the data set, the value at that same offset is again increased by 1.
  Read the next items and process them the same way, till you reach the end of the data set.
- Now you have Populated the buffer with all the numbers from the data set.

- Next, you read the buffer and add all the numbers in the buffer together until the addition is as high ( or higher ) as half of the total numbers in the data set.
  The current offset position in the buffer is equal to the value of the Median. ( Middle number )

- When the total numbers in the data set is even, you need a second Middle number.
- Look at the value at the current buffer offset.
  If it is higher then 1, the second Middle number is equal to the first Middle number and the Median is found.
  If it is 1 then go to the next offset in the buffer until the value is not equal to zero.
  Then the current buffer offset position is your second Middle number.
- Add them together and divide by 2 and you have found the Median.
Title: Re: Fast median algorithm
Post by: HSE on July 22, 2020, 02:34:41 AM
Nidud!!

It's a very bad test. The most important thing is the result!!


Later: I found where theoretically result is validated, but don't work well.
Title: Re: Fast median algorithm
Post by: jj2007 on July 22, 2020, 02:41:03 AM
Hi Jochen,

Don't know... could be a bug...

It's a very nice algo, Marinus, similar to a bucket sort. And I found the culprit: I had zeroed the MedianBuffer using the number of elements instead of the sizeof the buffer. Now it works!

For Guga's application, with 0...255 as value range, it's fine. For other data sets with different ranges, especially with negative numbers, it will be difficult.
Title: Re: Fast median algorithm
Post by: nidud on July 22, 2020, 03:26:48 AM
It is not so fast when there are great gaps between the numbers.
But this routine is special designed for gray color bitmap pixel data.
- they are normally close to each other and there are many duplicate values and have a small range 0-255
- Big chance the whole buffer is populated at all offsets.
- it can process many thousands of pixel values very fast.

Think it's the other way around. The sort algo only have to loop half the array if all are equal. Yours have to loop the whole array for the count and (depending on the value) another half to calculate so that's 1:3 in favor of the sort algo.

500 dup(110)
total [0 .. 3], 1++
   282577 cycles 0.asm: median
   619579 cycles 1.asm: CalculateMedian

The sort algo gets slower as more swaps are needed and yours a fixed overhang by clearing the count buffer.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 22, 2020, 03:34:53 AM
Hi Jochen,

Don't know... could be a bug...

It's a very nice algo, Marinus, similar to a bucket sort. And I found the culprit: I had zeroed the MedianBuffer using the number of elements instead of the sizeof the buffer. Now it works!

For Guga's application, with 0...255 as value range, it's fine. For other data sets with different ranges, especially with negative numbers, it will be difficult.

Phew.. that's a relieve.  :thumbsup:

Yeah, it's not really a substitute for a fast sorting algorithm.
But in guga case, it does very well on gray colored bitmaps and will beat sorting algorithms for this kind of processing.
Title: Re: Fast median algorithm
Post by: guga on July 22, 2020, 03:39:40 AM
Just some ideas:

For example, to get rid of the errors at the image borders:
- image = 100 * 100 pixels
- X and Y matrices = 3 * 3

Allocate a work buffer of 102 * 102 pixels with zeros.
Copy the bitmap data exactly in the middle so we have created a 1 pixel space around the image.
Now we can use the 3 * 3 matrix convolution maps without restoring the image border pixels.
The 1 pixel extra boundary space is only read, not written.

To prevent overshoots in Gx and Gy, we could normalize the matrix members.
If I'm correct, we will have clamped G values between 0-255 after sqrt(Gx^2+Gy^2).

This way we can also experiment with other matrix sizes and/or for example Gaussian ( evenly distributed ) convolution maps.

This is all theoretical, of course, and it has to be proven to work in the real world.  :biggrin:

Hi Siekmanski

Well, maybe could work putting the image in the middle, but then you would need to copy the result to another buffer to the image be saved and processed with other graphic libraries such as FreeImage, SDL, etc, right ?

And counting the filling the borders to zero could result on a cascading effect of incorrect values, i presume. I still don´t know what could be the best choice. Let them be zeros as you said, or calculate the average of the sobel (direction and magnitude) of the previous 3 pixels (in x and y pos) and simply using those averages to fill the missing values of the borders. In theory, doing that could keep the structure (direction and magnitude) of the missing data, by simply knowing what are the values and direction of their neighbours (but, it could slow down a bit the computation, because we would need another routine just to calculate those things near the border). IO´m still thinking on what could be better to do.

Btw...i succedded to make it work trhe 1st part of the whole process (estimating the watermark as in estimate_watermark function from python version) :azn:  The only problem is that my version is slow as hell because in order to convert the gray, i kinda exaggerated the result and forced it to get the media of the grays using all different Working Spaces available (I used 16 different matrices for grey :bgrin: :bgrin: :bgrin: :bgrin: ).
I´ll revert this and use only the one to be chosen, like ADOBE_RGB_1998_D65, or CS_MATRIX_NTSC_RGB_C, CS_MATRIX_SRGB_D65_HDTV etc, instead a average of all of them. Afterall, the average don´t seemss to produce a different result as if i used only one work space  :bgrin: :bgrin: :bgrin: :mrgreen: :mrgreen:

I´m fixing it to see the results, but so far here is what it exported after scanning 170 images :thumbsup: :thumbsup: :thumbsup:

(https://i.ibb.co/gMjLgGY/Result1.jpg) (https://ibb.co/gMjLgGY)

Since the im,ages are very similar to each other you can still see the edges of one and another, but, i guess this is how the algorithm works. Later i´ll compare to the results trying to debug at least this 1st step of the python version to compare the resultant values.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 22, 2020, 03:59:34 AM
It is not so fast when there are great gaps between the numbers.
But this routine is special designed for gray color bitmap pixel data.
- they are normally close to each other and there are many duplicate values and have a small range 0-255
- Big chance the whole buffer is populated at all offsets.
- it can process many thousands of pixel values very fast.

Think it's the other way around. The sort algo only have to loop half the array if all are equal. Yours have to loop the whole array for the count and (depending on the value) another half to calculate so that's 1:3 in favor of the sort algo.

500 dup(110)
total [0 .. 3], 1++
   282577 cycles 0.asm: median
   619579 cycles 1.asm: CalculateMedian

The sort algo gets slower as more swaps are needed and yours a fixed overhang by clearing the count buffer.

Yes, it has to read the whole array.
But it stops processing when it finds the median, which could be somewhere in the beginning of the small 256 items buffer.
The overhang is only clearing a buffer of 256 items put that against thousands of pixels.

The median could be in the last part of the image.

For example:
9,10,11,12,13,1,2,3,4,5,6,8  -> median is 7 ( the average of 6 and 8 )

But you are right, it also depends on what you feed it.
Unfortunately there is no, one fits all solution.

This routine is also a candidate for multithreading.
1 Median Buffer for each thread, when done adding the buffers together and find the median.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 22, 2020, 04:53:06 AM
Hi guga,

COOL!  :cool: :cool: :cool:

We don't need other graphics libraries.
GdiPlus is our friend.
We can load images, manipulate the bitmap data and save the new images with it.
Title: Re: Fast median algorithm
Post by: nidud on July 22, 2020, 05:03:41 AM
The overhang is only clearing a buffer of 256 items put that against thousands of pixels.

I'm not sure how it's going to be used but I assumed the result was the median value of one specific pixel from each bitmap. If that's the case the input is 100 bytes.

Quote
The median could be in the last part of the image.

For example:
9,10,11,12,13,1,2,3,4,5,6,8  -> median is 7 ( the average of 6 and 8 )

The count buffer is sorted in ascending order there so most likely towards the middle. I assumed the next value (in case of even) would have to be fetched from the input but the loop counter is the actual value there so the next will be the first non-zero entry.
Title: Re: Fast median algorithm
Post by: guga on July 22, 2020, 05:20:03 AM
Hi guga,

COOL!  :cool: :cool: :cool:

We don't need other graphics libraries.
GdiPlus is our friend.
We can load images, manipulate the bitmap data and save the new images with it.

Great ! I´ll try to find some examples on how to load (and save to other formats as well, not only bitmap) images with gdiplus and also grab their size and pointers to the pixeldata. I didn´t worked with gdiplus yet.

Btw...Take a look at this result :greenclp: :greenclp: :greenclp: :greenclp: :greenclp: :greenclp:

Scanned on 101 images and the edges of the watermark were found almost perfectly :greenclp: :greenclp: :greenclp: :greenclp:

(https://i.ibb.co/0YmCVxg/result-istock.jpg) (https://ibb.co/0YmCVxg)

The more different the images are, better is to identify the watermark or imperfections. The other test i made i used a sequence of images that i saved from a movie so, the scene contains images too similar to each other and that´s why this new test gave better results. But...the most important is that, it seems to be working :)

I´ll remove that routine i made to convert all working spaces and stay with only one to make the function work as fast as possible. Now it is 16 times slower because i´m using 16 different matrices to calculate an average gray on each pixel per image. Unnecessary, btw.

But..it works  :greenclp: :greenclp: :greenclp: :greenclp:  The next step is try to understand on the python version what a hell it is doing with the "crop_watermark" function before going through the watermark_detector and possion functions.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 22, 2020, 05:47:59 AM
The overhang is only clearing a buffer of 256 items put that against thousands of pixels.

I'm not sure how it's going to be used but I assumed the result was the median value of one specific pixel from each bitmap. If that's the case the input is 100 bytes.


You have a point there, I forgot that part, it's thousands of pixels * ( 100 pixels median calculations )
Then I will use multithreading.
1 Median Buffer for each thread, when done adding the buffers together per index number and find the median, will speed it up.
Title: Re: Fast median algorithm
Post by: guga on July 22, 2020, 11:18:54 AM
Hi Siekmanski

I found an issue with the median calculation. When analyzing the routine in the python version, i saw that it also takes the median for negative values as well.

The values on SobelX and SobelY can result in either positive, negative or zeroes. How to make your algo also works for signed integers ? Ex: say we have a sequence of: -100, -20, -5, 0, 36, 158, -12, 14, 66 etc, etc

Or this: -12, 0, 145 . Median = 0. (limited, therefore from -255 to 255)
-100, -20, -5, 0, 36, 158, -12, 14, 66 => Median = 0
-20, -5, 0, 36, 158, -12, 14, 66 => Median = 7
Also, the Real4 version would also be needed, for what i saw in the others routines
Title: Re: Fast median algorithm
Post by: jj2007 on July 22, 2020, 05:37:46 PM
Also, the Real4 version would also be needed, for what i saw in the others routines

You can forget the Real4 version, Marinus' algo is based on the number itself being an index. That works fine for a byte (0...255), but a Real4 has a range of 0...FFFFFFFFh, so you would need an index buffer of 4GB. Use a sort algo instead.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 22, 2020, 07:43:53 PM
Hi Siekmanski

I found an issue with the median calculation. When analyzing the routine in the python version, i saw that it also takes the median for negative values as well.

The values on SobelX and SobelY can result in either positive, negative or zeroes. How to make your algo also works for signed integers ? Ex: say we have a sequence of: -100, -20, -5, 0, 36, 158, -12, 14, 66 etc, etc

Or this: -12, 0, 145 . Median = 0. (limited, therefore from -255 to 255)
-100, -20, -5, 0, 36, 158, -12, 14, 66 => Median = 0
-20, -5, 0, 36, 158, -12, 14, 66 => Median = 7
Also, the Real4 version would also be needed, for what i saw in the others routines

Hi guga,

G = sqrt(Gx^2+Gy^2) -> Sobel result is always positive, no matter if Gx or Gy is negative.

I thought you needed the median of those G numbers?
Title: Re: Fast median algorithm
Post by: guga on July 22, 2020, 08:36:45 PM
Hi Marinus, the G will always be positive. But Gx and Gy can have either positive or negative and i´m not sure if they are being used in other parts of the python routines i´m trying to port.

Maybe i misinterpreted the results for the debugger when i was analyzing the results on Visual Studio. The data showed on VS contains negative values, but i´m not sure if they were a new median of the Gx or Gy that are still being used or only buffers that the algorithm didn't emptied (most likely, i hope).

I´l try to go further without those Gx and Gy and use only the G to see what happens. :thumbsup:


I´m a bit confused in what it is actually doing at:

Code: [Select]

def crop_watermark(gradx, grady, threshold=0.4, boundary_size=2):
    """
    Crops the watermark by taking the edge map of magnitude of grad(W)
    Assumes the gradx and grady to be in 3 channels
    @param: threshold - gives the threshold param
    @param: boundary_size - boundary around cropped image
    """
    W_mod = np.sqrt(np.square(gradx) + np.square(grady)) <---- This is, actually the G. Always positive.
    # Map image values to [0, 1]
    W_mod = PlotImage(W_mod) ; What is this ?
    # Threshold the image with threshold=0.4
    W_gray = image_threshold(np.average(W_mod, axis=2), threshold=threshold)
    x, y = np.where(W_gray == 1)

    # Boundary of cropped image (contain watermark)
    xm, xM = np.min(x) - boundary_size - 1, np.max(x) + boundary_size + 1
    ym, yM = np.min(y) - boundary_size - 1, np.max(y) + boundary_size + 1

    return gradx[xm:xM, ym:yM, :], grady[xm:xM, ym:yM, :] ; < ????????
     # return gradx[xm:xM, ym:yM, 0], grady[xm:xM, ym:yM, 0]
Code: [Select]
def image_threshold(image, threshold=0.5):
    '''
    Threshold the image to make all its elements greater than threshold*MAX = 1
    '''
    m, M = np.min(image), np.max(image)
    im = PlotImage(image)
    im[im >= threshold] = 1
    im[im < 1] = 0
    return im

I have no idea what  return gradx[xm:xM, ym:yM, :], grady[xm:xM, ym:yM, :] means. What exactly it is returning ? Do you have any idea of what is this ? xm:xM, ym:yM, :
Title: Re: Fast median algorithm
Post by: guga on July 22, 2020, 08:39:00 PM
Also, the Real4 version would also be needed, for what i saw in the others routines

You can forget the Real4 version, Marinus' algo is based on the number itself being an index. That works fine for a byte (0...255), but a Real4 has a range of 0...FFFFFFFFh, so you would need an index buffer of 4GB. Use a sort algo instead.

Hi JJ, yeah, i´m aware that. Unfortunately when i come to the part that the algo uses floats to calculate the median, i´ll be forced to use sorting algos. I just hope it don´t becomes too slow, though.
Title: Re: Fast median algorithm
Post by: jj2007 on July 22, 2020, 09:16:57 PM
If your floats are in a given range, convert them to fixed point integers to save time
Code: [Select]
3571 µs for converting from REAL4 to int32
648 ms for MbMedian      M=249925309
720 ms for ArraySort     M=249925309
#elements=5120000, maxY=767 Mio
Code: [Select]
xor ecx, ecx
.Repeat
fld REAL4 ptr [esi]
fmul FP4(10.0)
fistp DWORD ptr [edi]
inc ecx
.Until ecx>=eax
Title: Re: Fast median algorithm
Post by: Siekmanski on July 22, 2020, 09:39:07 PM
Hi guga,

-> I´l try to go further without those Gx and Gy and use only the G to see what happens. :thumbsup:

Yes, I think that's the logic behind it all.

->  W_mod = np.sqrt(np.square(gradx) + np.square(grady)) <---- This is, actually the G. Always positive.
->  W_mod = PlotImage(W_mod) ; What is this ?

Construct the edge detection image with the G values.


As far as I understand so far, correct me if I'm wrong....

- Perform edge detection on all 100 images with the Sobel formula.
- Find the medians, collecting 100 values at all the same x,y locations from 100 edge detected images.
- Use those median values to construct a new image.
- Detect the location of the watermark.
- Perform some magic, to remove the watermark?
Title: Re: Fast median algorithm
Post by: guga on July 22, 2020, 11:06:38 PM
Hi marinus

As far as I understand so far, correct me if I'm wrong....

- Perform edge detection on all 100 images with the Sobel formula.
- Find the medians, collecting 100 values at all the same x,y locations from 100 edge detected images.
- Use those median values to construct a new image.
- Detect the location of the watermark.
- Perform some magic, to remove the watermark?

Yes...that´s correct. But it can look for way more then 100 images. It always depends on how similar the images are. On different images, scanning among 100 is enough to estimate the watermark, but, on similar images (like sequences of scenes from a video), then i presume it would be necessary way more then that due to the similarity of the scenes.

The worst part is the magic :mrgreen: :mrgreen: :mrgreen:  I could understand how to find the edges of the watermark but, retrieve the alpha channel and reconstruct is another story. I´m not into that yet. And to things get worst, the guy who succeeded to make it work after reading the google articles, made it in python, instead in C or any other language that people could easily give a try and eventually fix it.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 22, 2020, 11:20:45 PM
 :biggrin:

Do you have the links to the Google articles?
Title: Re: Fast median algorithm
Post by: guga on July 22, 2020, 11:28:45 PM
:biggrin:

Do you have the links to the Google articles?

Yes...I posted it here in page 2 of the thread, but here are the links and the pdf

All the method is described here from google:
https://watermark-cvpr17.github.io/supplemental/
https://watermark-cvpr17.github.io/
https://ai.googleblog.com/2017/08/making-visible-watermarks-more-effective.html

And the link to the python version in github i posted.

If you want to read the google documentation that describes the algorithm it can be find here:
http://openaccess.thecvf.com/content_cvpr_2017/papers/Dekel_On_the_Effectiveness_CVPR_2017_paper.pdf     <------ Google Article in pdf
Title: Re: Fast median algorithm
Post by: guga on July 23, 2020, 12:01:26 AM
I found it where VisualStudio generated negative values. It was the way the python version was created. It does not calculate the media of G directly, it calculate the median of Gx and later the median of Gy and only after it, it compute the value of G = sqrt (MedianGx^2 + MedianGy^2)

Code: [Select]
def estimate_watermark(foldername):
(...)
    # Compute median of grads
    print("Computing median gradients.")
    Wm_x = np.median(np.array(gradx), axis=0)
    Wm_y = np.median(np.array(grady), axis=0)
    return (Wm_x, Wm_y, gradx, grady)

And it only calculated the value Of G after getting the median of Gy and Gy separately at:
Code: [Select]
def crop_watermark(gradx, grady, threshold=0.4, boundary_size=2):
    """
    Crops the watermark by taking the edge map of magnitude of grad(W)
    Assumes the gradx and grady to be in 3 channels
    @param: threshold - gives the threshold param
    @param: boundary_size - boundary around cropped image
    """
    W_mod = np.sqrt(np.square(gradx) + np.square(grady)) <-------

I have no idea why it is doing like that, rather then calculate the median directly whenever the values of gx and gy are retrieved. Well...i´ll continue working on the crop_watermark function, but, if you could, maybe  doing a algo for calculating negative values of media (so range from -255 to 255), could be good to we see if (or how) it affects the result.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 23, 2020, 12:57:55 AM
I found it where VisualStudio generated negative values. It was the way the python version was created. It does not calculate the media of G directly, it calculate the median of Gx and later the median of Gy and only after it, it compute the value of G = sqrt (MedianGx^2 + MedianGy^2)

Hmmm... if that's the case, my routine has no use at all, calculating 6 numbers will not be beneficial.
Sorting will be faster.
Title: Re: Fast median algorithm
Post by: hutch-- on July 23, 2020, 04:27:35 AM
If you are talking about sorting small number sequences, -256 - 256 as FP values, you should be able to do that with one of the simpler exchange sorts but the comparison would have to be done in FP in the sort. With such a low range of numbers, a bucket sort of some type would probably be the fastest, simply an array that you increment each member when it occurs in the numbers being sorted. This better suits integers but if the FP precision is not a big deal, rounding the values to integer may be viable. Once the array if filled, just grab the half way mark.
Title: Re: Fast median algorithm
Post by: HSE on July 23, 2020, 04:42:41 AM
To work with range -256 to 256 you can add 256 to values then you range became 0 to 512. Later substract 256 to result.

If you want to make integers from real4 in the range 0-1, you have to multiply previously by 100, 1000 or precision you requiere. Later divide result for that number.
Title: Re: Fast median algorithm
Post by: nidud on July 23, 2020, 06:53:35 AM
To work with range -256 to 256 you can add 256 to values then you range became 0 to 512. Later substract 256 to result.

You may use signed values directly without modifying the input:
Code: [Select]
CalculateMedian proc uses edi p:ptr, n:dword

  local count[256]:byte

    xor eax,eax
    mov ecx,256
    lea edi,count
    rep stosb
    mov edx,p
    mov ecx,n
    lea edi,count[128]
    .repeat
        movsx eax,byte ptr [edx]
        inc byte ptr [edi+eax]
        add edx,1
    .untilcxz

    mov ecx,n
    shr ecx,1
    xor edx,edx
    mov eax,-129
    .repeat
        inc eax
        movzx edi,count[eax+128]
        add edx,edi
    .until edx > ecx
    ret

CalculateMedian endp

However, it will then become faster with negative than positive values.

Negative values:

   333245 cycles 1.asm: CalculateMedian
   485963 cycles 0.asm: median

Positive values:

  432918 cycles 0.asm: median
  938836 cycles 1.asm: CalculateMedian


Quote
If you want to make integers from real4 in the range 0-1, you have to multiply previously by 100, 1000 or precision you requiere. Later divide result for that number.

Comparing and swapping is faster using floats (not sure about FPU thought) than integer values.

Code: [Select]
    .686
    .xmm
    .model flat, stdcall

compare macro a, b
    movss  xmm0,[a]
    comiss xmm0,[b]
    exitm<>
    endm

swapif macro a, b
ifdef _MAXSS
    movss xmm0,[a]
    movss xmm1,[b]
    minss xmm1,[a]
    maxss xmm0,[b]
    movss [b],xmm0
    movss [a],xmm1
else
    compare(a, b)
    .ifa
        movss xmm1,[b]
        movss [b],xmm0
        movss [a],xmm1
    .endif
endif
    exitm<>
    endm

swap macro a, b
    movss xmm0,[a]
    movss xmm1,[b]
    movss [b],xmm0
    movss [a],xmm1
    exitm<>
    endm

    .code

    option loopalign:4

median proc uses esi edi ebx p:ptr, n:dword

  local level:dword

    mov eax,n
    .if eax > 1

        mov esi,p
        lea edi,[esi+eax*4-4]
        mov level,0

        .while 1

            lea eax,[edi+4]
            sub eax,esi
            shr eax,3
            lea ebx,[esi+eax*4]

            swapif(esi, ebx)
            swapif(esi, edi)
            swapif(ebx, edi)

            mov ecx,esi
            mov edx,edi

            .while 1

                add ecx,4
                .if ecx < edi

                    compare(ecx, ebx)
                    .continue .ifna
                .endif

                .while 1

                    sub edx,4

                    .break .if edx <= ebx

                    compare(edx, ebx)
                    .break .ifna
                .endw

                .break .if edx < ecx

                swap(edx, ecx)

                .if ebx == edx

                    mov ebx,ecx
                .endif
            .endw

            add edx,4

            .while 1

                sub edx,4

                .break .if edx <= esi

                compare(edx, ebx)
                .break .ifnz
            .endw

            mov eax,edx
            mov edx,ecx
            mov ecx,edi
            sub ecx,edx
            sub eax,esi

            .if eax < ecx

                .if edx < edi

                    push edx
                    push edi
                    inc level
                .endif

                .if esi < ecx

                    mov edi,ecx
                    .continue
                .endif
            .else
                add eax,esi
                .if esi < eax

                    push esi
                    push eax
                    inc level
                .endif

                .if edx < edi

                    mov esi,edx
                    .continue
                .endif
            .endif

            .break .if !level

            dec level
            pop edi
            pop esi

        .endw

        mov ecx,p
        mov edx,n
        shr edx,1
        movss xmm0,[ecx+edx*4]
        .ifnc
            addss xmm0,[ecx+edx*4-4]
            mov eax,2.0
            movd xmm1,eax
            divss xmm0,xmm1
        .endif
    .endif
    ret

median endp

    end
Title: Re: Fast median algorithm
Post by: guga on July 23, 2020, 08:47:10 AM
Tks a lot Nidud. I´ll give a try.

I suceeded to compile the signed integer version (CalculateMedian ), but when trying to assemble the SSE one (in quickeditor), those are the errors i´ve got.

Code: [Select]
C:\masm32\examples\NidudMedian\3dframes2.asm(135) : error A2070:invalid instruction operands
 compare(1): Macro Called From
  swapif(9): Macro Called From
   C:\masm32\examples\NidudMedian\3dframes2.asm(135): Main Line Code
C:\masm32\examples\NidudMedian\3dframes2.asm(135) : error A2070:invalid instruction operands
 compare(2): Macro Called From
  swapif(9): Macro Called From
   C:\masm32\examples\NidudMedian\3dframes2.asm(135): Main Line Code
C:\masm32\examples\NidudMedian\3dframes2.asm(135) : error A2008:syntax error : .
 swapif(10): Macro Called From
  C:\masm32\examples\NidudMedian\3dframes2.asm(135): Main Line Code
C:\masm32\examples\NidudMedian\3dframes2.asm(135) : error A2070:invalid instruction operands
 swapif(11): Macro Called From
  C:\masm32\examples\NidudMedian\3dframes2.asm(135): Main Line Code
C:\masm32\examples\NidudMedian\3dframes2.asm(135) : error A2070:invalid instruction operands
 swapif(12): Macro Called From
  C:\masm32\examples\NidudMedian\3dframes2.asm(135): Main Line Code
C:\masm32\examples\NidudMedian\3dframes2.asm(135) : error A2070:invalid instruction operands
 swapif(13): Macro Called From
  C:\masm32\examples\NidudMedian\3dframes2.asm(135): Main Line Code
C:\masm32\examples\NidudMedian\3dframes2.asm(135) : fatal error A1011:directive must be in control block
 swapif(14): Macro Called From
  C:\masm32\examples\NidudMedian\3dframes2.asm(135): Main Line Code

I suceeded to dl your AsmC to make it run, but don´t know how to generate the executable file so i can see what are those macros used in 'median' function all about. Can you make a executable file for the SSE/Float version one (32 bits, pls), so i can test ?
Title: Re: Fast median algorithm
Post by: guga on July 23, 2020, 06:18:29 PM
Hi Nidud

Can you please make a small test ? 

Check the median (The integer version: CalculateMedian) for these, pls ? -216,-151,136

The correct value is -151.

I´m not sure if i ported correctly your version, because the result i´ve got is wrong

Title: Re: Fast median algorithm
Post by: Siekmanski on July 24, 2020, 12:53:45 AM
Hi guga,

A branchless floating point sort algorithm, only using registers.
To calculate the Median of the 6 numbers from a Sobel Matrix [ 3 * 3 ]

Code: [Select]
align 4
CalculateMedianFP6 proc
; Fill the first 6 xmm registers with the unsorted 6 values
    movaps  xmm0,oword ptr [GrayPixels]
    movlps  xmm4,qword ptr [GrayPixels+16]
    movaps  xmm1,xmm0
    movaps  xmm2,xmm0
    shufps  xmm1,xmm1,00111001b
    shufps  xmm2,xmm2,01001110b
    movaps  xmm3,xmm0
    movaps  xmm5,xmm4
    shufps  xmm3,xmm3,10010011b
    shufps  xmm5,xmm5,00111001b

; Branchless floating point sort algorithm, only using registers.
; 6 numbers = max 12 swaps

    movss   xmm6,xmm1
    movss   xmm7,xmm4
    minps   xmm1,xmm2
    minps   xmm4,xmm5
    maxps   xmm2,xmm6
    maxps   xmm5,xmm7

    movss   xmm6,xmm0
    movss   xmm7,xmm3
    minps   xmm0,xmm2
    minps   xmm3,xmm5
    maxps   xmm2,xmm6
    maxps   xmm5,xmm7

    movss   xmm6,xmm0
    movss   xmm7,xmm3
    minps   xmm0,xmm1
    minps   xmm3,xmm4
    maxps   xmm1,xmm6
    maxps   xmm4,xmm7

    movss   xmm6,xmm1
    movss   xmm7,xmm0
    minps   xmm1,xmm4
    minps   xmm0,xmm3
    maxps   xmm4,xmm6
    maxps   xmm3,xmm7

    movss   xmm6,xmm2
    movss   xmm7,xmm1
    minps   xmm2,xmm5
    minps   xmm1,xmm3
    maxps   xmm5,xmm6
    maxps   xmm3,xmm7

    movss   xmm6,xmm2
    minps   xmm2,xmm4
    maxps   xmm4,xmm6
    movss   xmm6,xmm2
    minps   xmm2,xmm3
    maxps   xmm3,xmm6
   
    addss   xmm2,xmm3
    mulss   xmm2,real4 ptr [Half] ; Median in xmm2
    ret

CalculateMedianFP6 endp
Title: Re: Fast median algorithm
Post by: guga on July 24, 2020, 02:56:20 AM
Thanks a lot, Marinus, i´ll give a try and check the results :thumbsup: :thumbsup: :thumbsup: :thumbsup:
Title: Re: Fast median algorithm
Post by: guga on July 24, 2020, 03:37:16 AM
Hi guga,

A branchless floating point sort algorithm, only using registers.
To calculate the Median of the 6 numbers from a Sobel Matrix [ 3 * 3 ]

Code: [Select]
align 4
CalculateMedianFP6 proc
; Fill the first 6 xmm registers with the unsorted 6 values
    movaps  xmm0,oword ptr [GrayPixels]
    movlps  xmm4,qword ptr [GrayPixels+16]
    movaps  xmm1,xmm0
    movaps  xmm2,xmm0
    shufps  xmm1,xmm1,00111001b
    shufps  xmm2,xmm2,01001110b
    movaps  xmm3,xmm0
    movaps  xmm5,xmm4
    shufps  xmm3,xmm3,10010011b
    shufps  xmm5,xmm5,00111001b

; Branchless floating point sort algorithm, only using registers.
; 6 numbers = max 12 swaps

    movss   xmm6,xmm1
    movss   xmm7,xmm4
    minps   xmm1,xmm2
    minps   xmm4,xmm5
    maxps   xmm2,xmm6
    maxps   xmm5,xmm7

    movss   xmm6,xmm0
    movss   xmm7,xmm3
    minps   xmm0,xmm2
    minps   xmm3,xmm5
    maxps   xmm2,xmm6
    maxps   xmm5,xmm7

    movss   xmm6,xmm0
    movss   xmm7,xmm3
    minps   xmm0,xmm1
    minps   xmm3,xmm4
    maxps   xmm1,xmm6
    maxps   xmm4,xmm7

    movss   xmm6,xmm1
    movss   xmm7,xmm0
    minps   xmm1,xmm4
    minps   xmm0,xmm3
    maxps   xmm4,xmm6
    maxps   xmm3,xmm7

    movss   xmm6,xmm2
    movss   xmm7,xmm1
    minps   xmm2,xmm5
    minps   xmm1,xmm3
    maxps   xmm5,xmm6
    maxps   xmm3,xmm7

    movss   xmm6,xmm2
    minps   xmm2,xmm4
    maxps   xmm4,xmm6
    movss   xmm6,xmm2
    minps   xmm2,xmm3
    maxps   xmm3,xmm6
   
    addss   xmm2,xmm3
    mulss   xmm2,real4 ptr [Half] ; Median in xmm2
    ret

CalculateMedianFP6 endp

Hi Marinus...got it. The speed is awesome  :greenclp: :greenclp: :greenclp:! But the results are being switched among xmm2 and xmm3. I mean, sometimes it puts the correct result in xmm2 and others in xmm3. Check this values as input, pls:

[FloatDataSelect12: F$ 136, -151, -216, -40, -13, -215, -24, -104, 230] ; Median = -40. The correct result is in xmm3
[FloatDataSelect13: F$ -36, -51, -16, -10, -213, -115, -214, -14, -230] ; Median = -51. The correct result is in  xmm2
[FloatDataSelect14: F$ 0.565856, 0.9898, 0.1112545, 0.55897941, 0.00214548, 0.4564585, 0.6587978, 0.115545, 0.98989] ; Median = 0.55897941. The correct result is in xmm3
[FloatDataSelect15: F$ -0.565856, 0.9898, -0.1112545, -0.55897941, -0.00214548, 0.4564585, -0.6587978, 0.115545, -0.98989] ; Median = -0.1112545. The correct result is in xmm2

Same thing happens when the input is all negative or all positive:
[FloatDataSelect16: F$ 2, F$ 5, F$ 1, F$ 3, F$ 6, F$ 4, F$ 4, F$ 4, F$ 4] ; Median = 4. Result in xmm3
[FloatDataSelect17: F$ -2, F$ -5, F$ -1, F$ -3, F$ -6, F$ -4, F$ -4, F$ -4, F$ -4] ; Median = -4. Result in xmm2

The problem seems to be at the end of the code. Some comparison maybe needed to identify which of the 2 registers (xmm2 or xmm3) contains the correct value. I´m not sure about what is missing to compare, but when we reach at the last "maxps XMM3 XMM6", i´ve got this conditions from xmm2 and xmm3:

Examples of result

Analysis from Result1
[FloatDataSelect15: F$ -0.565856, 0.9898, -0.1112545, -0.55897941, -0.00214548, 0.4564585, -0.6587978, 0.115545, -0.98989] ; Median = -0.1112545. result in xmm2
xmm2 and xmm3 are negative.
xmm2 = -1.112544e-1
xmm3 = -2.145479e-3

Result should be -1.11254e-1 From xmm2

Analysis from Result2
[FloatDataSelect14: F$ 0.565856, 0.9898, 0.1112545, 0.55897941, 0.00214548, 0.4564585, 0.6587978, 0.115545, 0.98989] ; Median = 0.55897941. result in xmm3
xmm2 and xmm3 are positive.
xmm2 = 4.564585e-1
xmm3 = 5.58979e-1

Result should be 5.58979e-1 From xmm3

Analysis from Result3
[FloatDataSelect13: F$ -36, -51, -16, -10, -213, -115, -214, -14, -230] ; Median = -51. result in xmm2
xmm2 and xmm3 are negative.
xmm2 = -51
xmm3 = -36

Result should be -51 From xmm2

Analysis from Result4
[FloatDataSelect12: F$ 136, -151, -216, -40, -13, -215, -24, -104, 230] ; Median = -40. Result in xmm3
xmm2 and xmm3 are negative.
xmm2 = -151
xmm3 = -40

Result should be -40 From xmm3

Can you please check with those values to see what´s happening for it be switching the results ?


Note:

This new algo can be used to calculate the median of sobelX and SobelY of a matrix on one single image, right ?  This is not for N images where we get the Sobel of each pixel at specific coordinates of each image, i presume. In any case, this new algo we can use to calculate the median on the boarders of each image, after getting the median of all N images at specified positions.

So, i presume the algo is handling the matrix 3*3 of image 1. Then, to fix the border issue, all we need to do is:

1- Compute the median of Sobelx and SobelY on all images (not only limited to 100), with other algo similar to this new algo you did. So, the algo will take the pixel values on coord x/y from image 1 until image N, and get the median of them. Save this median related to posx/y to the output at same location x/y. Then go to the next byte at pos x+1/y on all images from 1 to N, get the median of those values and save the result in x+1/y on output...It loops the routine untill ll N images calculated the median of Sobel on all pixels at x/y coords.
The routine maybe restricted to width and height that are divisible by 3, so we can later fill the border with this new algo you did.

2 - After computing the median of sobelx and sobely on all images, we will end up creating two memory buffers containing the median of Sobels x and Y on all images, except the values at their boarders (only on the cases when the width and height are not divisible by 3), and then we simply use this new algo you create to fix the median of Sobelx and SobelY on those 2 created Buffers, whose boarders needs to be fixed.

So, please can you fix this algo (so we can later use it to fix the boarder issues), and later we give a try on other one to get the median of all N images ?
Title: Re: Fast median algorithm
Post by: nidud on July 24, 2020, 04:32:52 AM
I suceeded to dl your AsmC to make it run, but don´t know how to generate the executable file so i can see what are those macros used in 'median' function all about. Can you make a executable file for the SSE/Float version one (32 bits, pls), so i can test ?

Code: [Select]
;
; build: asmc -pe medianf.asm
;
    .686
    .xmm
    .model flat, stdcall

compare macro a, b
    movss  xmm0,[a]
    comiss xmm0,[b]
    exitm<>
    endm

swapif macro a, b
ifdef _MAXSS
    movss xmm0,[a]
    movss xmm1,[b]
    minss xmm1,[a]
    maxss xmm0,[b]
    movss [b],xmm0
    movss [a],xmm1
else
    compare(a, b)
    .ifa
        movss xmm1,[b]
        movss [b],xmm0
        movss [a],xmm1
    .endif
endif
    exitm<>
    endm

swap macro a, b
    movss xmm0,[a]
    movss xmm1,[b]
    movss [b],xmm0
    movss [a],xmm1
    exitm<>
    endm

    .data

     array dd -1.0, 18.0,   0.0,   0.0,   0.0,  0.0,  2.0,  2.0,
               5.0, 99.0, 600.0, 785.0, 758.0, 17.0, 11.0, 21.0,
              23.0, 24.0,  66.0,  21.0,  14.0, 57.0, 17.0

    .code

    ;option loopalign:4

median proc uses esi edi ebx p:ptr, n:dword

  local level:dword

    mov eax,n
    .if eax > 1

        mov esi,p
        lea edi,[esi+eax*4-4]
        mov level,0

        .while 1

            lea eax,[edi+4]
            sub eax,esi
            shr eax,3
            lea ebx,[esi+eax*4]

            swapif(esi, ebx)
            swapif(esi, edi)
            swapif(ebx, edi)

            mov ecx,esi
            mov edx,edi

            .while 1

                add ecx,4
                .if ecx < edi

                    compare(ecx, ebx)
                    .continue .ifna
                .endif

                .while 1

                    sub edx,4

                    .break .if edx <= ebx

                    compare(edx, ebx)
                    .break .ifna
                .endw

                .break .if edx < ecx

                swap(edx, ecx)

                .if ebx == edx

                    mov ebx,ecx
                .endif
            .endw

            add edx,4

            .while 1

                sub edx,4

                .break .if edx <= esi

                compare(edx, ebx)
                .break .ifnz
            .endw

            mov eax,edx
            mov edx,ecx
            mov ecx,edi
            sub ecx,edx
            sub eax,esi

            .if eax < ecx

                .if edx < edi

                    push edx
                    push edi
                    inc level
                .endif

                .if esi < ecx

                    mov edi,ecx
                    .continue
                .endif
            .else
                add eax,esi
                .if esi < eax

                    push esi
                    push eax
                    inc level
                .endif

                .if edx < edi

                    mov esi,edx
                    .continue
                .endif
            .endif

            .break .if !level

            dec level
            pop edi
            pop esi

        .endw

        mov ecx,p
        mov edx,n
        shr edx,1
        movss xmm0,[ecx+edx*4]
        .ifnc
            addss xmm0,[ecx+edx*4-4]
            mov eax,2.0
            movd xmm1,eax
            divss xmm0,xmm1
        .endif
    .endif
    ret

median endp

main proc

    median( &array, 23 )
    ret

main endp

    end main
Title: Re: Fast median algorithm
Post by: nidud on July 24, 2020, 04:40:05 AM
Hi Nidud

Can you please make a small test ? 

Check the median (The integer version: CalculateMedian) for these, pls ? -216,-151,136

The correct value is -151.

I´m not sure if i ported correctly your version, because the result i´ve got is wrong

The sample only handles bytes. Integer values need a larger count buffer (512 words maybe) and adjusted input. It was only an example on how signed values may be used in the algo.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 24, 2020, 05:39:31 AM
Hi guga,

-> I found it where VisualStudio generated negative values. It was the way the python version was created. It does not calculate the media of G directly, it calculate the median of Gx and later the median of Gy and only after it, it compute the value of G = sqrt (MedianGx^2 + MedianGy^2)

This comment from you is the reason I wrote a 6 numbers sorting routine.

The Median Matrix holds only positive numbers ( pixel values are never negative )
Therefore the calculation for the average middle numbers is correct.

And don't forget you only have to sort 6 values per matrix!!!

Code: [Select]
To calculate the edges on the X axis

     [ X X X ]
Gx = [ 0 0 0 ]
     [ X X X ]

To calculate the edges on the Y axis
     
     [ X 0 X ]
Gy = [ X 0 X ]
     [ X 0 X ]

You only need these 6 numbers. ( X's in the matrices )
For the Sobel Matrix convolution as for the Median Matrix calculations.

( You can save the Gray Color pixel conversion calculations as floating point )
( So you need no conversions from integer and floating point between all the calculation )

For the Median Matrix calculation you only read the 6 [X] pixel values ( always positive ) sort them, get the Median and that's the Gx and Gy.
No convolution needed as with the Sobel coefficients.

The G values should always be positive. ( they represent the new pixel color )
Title: Re: Fast median algorithm
Post by: guga on July 24, 2020, 06:41:36 AM
Hi guga,

-> I found it where VisualStudio generated negative values. It was the way the python version was created. It does not calculate the media of G directly, it calculate the median of Gx and later the median of Gy and only after it, it compute the value of G = sqrt (MedianGx^2 + MedianGy^2)

This comment from you is the reason I wrote a 6 numbers sorting routine.

The Median Matrix holds only positive numbers ( pixel values are never negative )
Therefore the calculation for the average middle numbers is correct.

And don't forget you only have to sort 6 values per matrix!!!

Code: [Select]
To calculate the edges on the X axes

     [ X X X ]
Gx = [ 0 0 0 ]
     [ X X X ]

To calculate the edges on the Y axes
     
     [ X 0 X ]
Gy = [ X 0 X ]
     [ X 0 X ]

You only need these 6 numbers. ( X's in the matrices )
For the Sobel Matrix convolution as for the Median Matrix calculations.

( You can save the Gray Color pixel conversion calculations as floating point )
( So you need no conversions from integer and floating point between all the calculation )

For the Median Matrix calculation you only read the 6 [X] pixel values ( always positive ) sort them, get the Median and that's the Gx and Gy.
No convolution needed as with the Sobel coefficients.

The G values should always be positive. ( they represent the new pixel color )

Hi Marinus, ok. But i´m getting a bit confused because this is not what the python version seems to be doing. For what i understood, the sequence of functions are those 2 parts.

1 - Call the function estimate_watermark

This function will export 2 buffers containing the median of Sobelx and SobelY calculated from N images . It will export them in Wm_x and Wm_y like this:

a) Create the sobel x and y on all N images

Code: [Select]
    gradx = list(map(lambda x: cv2.Sobel(
        x, cv2.CV_64F, 1, 0, ksize=KERNEL_SIZE), images))
    grady = list(map(lambda x: cv2.Sobel(
        x, cv2.CV_64F, 0, 1, ksize=KERNEL_SIZE), images))

So, here we have 100 (or N) Memory Buffers where it is stored the SobelX and Sobel Y on each image individually. So:

GradX from image 1 will contain the sobelx from image1. So, we can called it GradX(1)
GradY from image 1 will contain the sobely from image1. So, we can called it GradY(1)

GradX from image 2 will contain the sobelx from image2. So, we can called it GradX(2)
GradY from image 2 will contain the sobely from image2. So, we can called it GradY(2)

GradX from image N will contain the sobelx from image1. So, we can called it GradX(N)
GradY from image N will contain the sobely from image1. So, we can called it GradY(N)

At the end we have N*2 Memory Buffers containing the sobel (Gx and Gy) from all the N images.

b) After it calculate the sobelx and y on all images it then take their median to export on only 2 new buffers (Wm_x and Wm_y), as below:

Code: [Select]
    # Compute median of grads
    print("Computing median gradients.")
    Wm_x = np.median(np.array(gradx), axis=0)
    Wm_y = np.median(np.array(grady), axis=0)

So, this function it is not creating the G on each image, but calculating the median of Gx and Gy from all images to later compute the resultant G from those values as below.

the median calculated from here is at each given position on all N images from where Gradx and GradY were created.

So:
At pos (x = 0, Y = 0), it will get the SobelX and SobelY from images 1 to N and calculate their median and then put the resultant 2 (Gx and Gy) values on Wm_x and Wm_y at the same pos (x = 0, y =  0)
At pos (x = 1, Y = 0), it will get the SobelX and SobelY from images 1 to N and calculate their median and then put the resultant 2 (Gx and Gy) values on Wm_x and Wm_y at the same pos (x = 1, y =  0)

and keep doing it untill all Coordinates were calculated.



2 - Call the function crop_watermark

This function will calculate the median G value from the previously created Wm_x and Wm_y

Code: [Select]
    W_mod = np.sqrt(np.square(gradx) + np.square(grady))
From this part it is taking the 2 memory buffers created to store the median Gx and Gy to calculate the G from only those 2 memory buffers that will be saved in W_mod


So, what is the strategy ? Use your new FastMedianFP to compute median on what ? I really got a bit lost right now. :greensml: :greensml:
Title: Re: Fast median algorithm
Post by: Siekmanski on July 24, 2020, 06:48:49 AM
 :biggrin:

Let's experiment with real image data and see what kind of junk we can create.  :thumbsup:
See if the median matrix method is better or worse than the sobel convolution method.
Title: Re: Fast median algorithm
Post by: guga on July 24, 2020, 06:59:08 AM
 :greensml: :greensml: :greensml:

Ok, i´ll give a try, but what i do next ? Use your new median algo to compute the median from what and where ?


I´ve got lost :bgrin: :bgrin: :bgrin: :bgrin:
Title: Re: Fast median algorithm
Post by: guga on July 24, 2020, 07:14:21 AM
Ok, i´ll try to do this and see what result i got. Maybe inverting the order that was made in python version should also works.

1 - Create the median of greys on all N images and save this on one single memory buffer
2 - Create the Sobel from this new image.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 24, 2020, 07:57:42 AM
Just step for step I think, before we lose track.
This weekend I have some more time.
If you like it ( otherwise you might think I will hijack your project, of course I don't want that ), I can code the image loader, save some bytes in memory etc.
Show the original image on screen and next to it show the different edge detection images routines.
Later we can figure out ( if we succeed... ) how to do the watermark stuff.
Haven't looked into that part yet.
Title: Re: Fast median algorithm
Post by: daydreamer on July 24, 2020, 08:30:35 AM
Just step for step I think, before we lose track.
This weekend I have some more time.
If you like it ( otherwise you might think I will hijack your project, of course I don't want that ), I can code the image loader, save some bytes in memory etc.
Show the original image on screen and next to it show the different edge detection images routines.
Later we can figure out ( if we succeed... ) how to do the watermark stuff.
Haven't looked into that part yet.
That image processing is that similar to search and replace black pixels surrounding lighter pixels with grey ?
postprocessed antialias, compared to line/oval /text outputted with antialias for example gdi+

Title: Re: Fast median algorithm
Post by: Siekmanski on July 24, 2020, 08:34:52 AM
Hi Magnus,

Could be something like that, for now it feels like magic.
Title: Re: Fast median algorithm
Post by: guga on July 24, 2020, 08:45:16 AM
Just step for step I think, before we lose track.
This weekend I have some more time.
If you like it ( otherwise you might think I will hijack your project, of course I don't want that ), I can code the image loader, save some bytes in memory etc.
Show the original image on screen and next to it show the different edge detection images routines.
Later we can figure out ( if we succeed... ) how to do the watermark stuff.
Haven't looked into that part yet.

Sure :thumbsup: :thumbsup:That would be very nice :wink2: Thank you a lot :thumbsup: :thumbsup: :thumbsup:

The part that estimate the watermark don´t seems to be so hard. It´s just a matter of try to find what the exactly the estimate_watermark function is doing in the python version so we can optimize it further. Sobel is a good way to get the edges, but if we succeed to make it work as expected, we can later extend the estimation of the watermark to use other matrices as well, like Scharr operators.

I made one single function to work specifically for sobel called 'SobelGetGPLusEx" (as posted previously), that is able to fix the sobel errors forcing the data to stay within the 0-255 limits (already normalized to 0...1) without cutting, but we can also use as input a matrix on whatever operator we want (and, perhaps on whatever size) and make the same fixes using only one single function.

Currently i was working on the next part that is the crop_watermark function, but i´ll go back now and made the test directly using the median of the gray values to calculate the sobel values and see what the resultant image looks like. Don´t know if changing the order exposed in the python version would improve, but...it´s  worth o give a try. :cool:
Title: Re: Fast median algorithm
Post by: Siekmanski on July 24, 2020, 09:01:23 AM
Cool  :cool:, we are a team now.  :thumbsup:

The goal is to get the location of the watermark with the most suitable edge detection technique.
In a movie the watermark is static and always the same color while the background changes between dark to light.
So when taking the median of multiple frames, the watermark will show up as a brighter color as the surrounding background, I think?
Title: Re: Fast median algorithm
Post by: guga on July 24, 2020, 09:21:54 AM
That image processing is that similar to search and replace black pixels surrounding lighter pixels with grey ?
postprocessed antialias, compared to line/oval /text outputted with antialias for example gdi+

Hi Magnus. For what i understood of the articles and the python code, the very 1st step is a simple edge recognition of all images you feed in. You take the median of the edges on all images in order to create a new one. The newly created image will result on the edges that are present in all images altogether. To identify the edges, the algorithm uses Sobel operator (Which is great for this task, btw),  but, of course, it can be extended to use others that are better like Scharr for example. Of course, no matter what operator we use we will always need to fix them since all of them simply cut off values (frequencies) that are outside the normal range of 0-255, which, btw, is not desired and this is the main reason why you see in sobel images those extremely thick lines on the edges or noise that is generated etc. (I´m quite sure that on Scharr the same fix must be done too). So, the fix and adaptation i made on sobel is a must in whatever operator is used.

What the algorithm does to estimate the edges (watermark) is basically this: For example, if you have a sequence of images that contains a watermark, or a lens defect of the camera, a crack on the lens or whatever other imperfection that are present on all images, the 1st thing to do is identify and enhance only those imperfections (or watermarks, in this case) and create a newly image containing only the enhanced edges found on all of them.

After the detection on all images, it will result on something like this:

(https://i.ibb.co/0YmCVxg/result-istock.jpg) (https://ibb.co/0YmCVxg)

This is the easier part (well.. kind of :greensml: :greensml: :greensml:)

The next step is crop this new image which envolves using a threshold toexclude the pixels that are darker then a specific limit (The python version seems to use a limit of 0.4), then he crop the resultant image and generate some sort of map containing only 1 or 0 (i presume).

This is the final part of the crop_watermark function used in python, but i didn´t understood what it is doing when reach this:
return gradx[xm:xM, ym:yM, :], grady[xm:xM, ym:yM, :]


After it, is time to identify the watermark on the image you want it to be removed.

On this part, the algo do the same thing again, but only in the image from where you want to remove the watermark. So it will detect the edges of one single image (this time using canny), then calculates the chanfer distance between the edges of the loaded image and the edges of the watermark you previously estimated, and crop the image somehow (don´t know how to do it yet). It then returns a rectangle corresponding to the area where the edges were detected (so, a rectangle around the detected watermark).
Perhaps this rectangle is just to visualize the watermark being detected, as it happens when you do a face recognition and those squares shows up all around your face.

The next step is make a rebuild of the watermarked area using poisson_reconstruct algorithm

This is what i understood what the algorithm is doing so far. It´s half the way to remov the watermark, but the rest of the code and other functions i don´t know exactly what are they doing yet

Title: Re: Fast median algorithm
Post by: guga on July 24, 2020, 09:27:42 AM
Cool  :cool:, we are a team now.  :thumbsup:

The goal is to get the location of the watermark with the most suitable edge detection technique.
In a movie the watermark is static and always the same color while the background changes between dark to light.
So when taking the median of multiple frames, the watermark will show up as a brighter color as the surrounding background, I think?
Great ! :thumbsup: :thumbsup: :thumbsup: :thumbsup: :thumbsup:

About the goal..Exactly :eusa_clap: :eusa_clap: :eusa_clap: That´s why on the image i posted previously the watermarks showed up. On movies (or whatever other sequence of images), if the watermark is fixed (or other imperfection, like lens crack, fixed dirt, etc) this will always shows in all images. The 1st (and easier) task is detect.

The next step is as i explained above to magnus. It involves doing a similar thing to the image we want the watermark to be removed. The rest of the techniques we will need to do step by step because i´m not fully understood what the python functions are doing. I choosed to use the python version as a guidance, because the original google article contain too much math symbols i´m not fully understand.
Title: Re: Fast median algorithm
Post by: guga on July 27, 2020, 08:32:49 AM
Hi Marinus

Some results of the new test.

This time i used the median of the grays to compute the resultant sobel and the result is that the image is too noisy

(https://i.ibb.co/GWm9RJT/Fire-Full-Matrix-New.png) (https://ibb.co/GWm9RJT)


On the other method (image below), the watermark were detected without that amount of noises.

So, the better is create the Gy and Gx for all images (separately). So, if we are feeding the algo with 100 images we will need to generate a map for 100 images containing only Gx and 100 images containing only Gy.

And then, take the median of Gy and GY and only after it calculate the Sobel (G) from those median of gy and gx

With the olderr (and better) method, the image is clearer as below

(https://i.ibb.co/Ht44n52/Fire-Full-Matrix424.png) (https://ibb.co/Ht44n52)


So, an algo to calculate the median of floats (negative and positive) is, indeed, necessary
Title: Re: Fast median algorithm
Post by: Siekmanski on July 27, 2020, 09:16:59 AM
Maybe setting a threshold will improve things?

-> So, an algo to calculate the median of floats (negative and positive) is, indeed, necessary

Why? you could do it with byte integers.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 27, 2020, 09:28:55 AM
My idea is:

- load an image.
- convert the colors to gray colors to memory in real4 format.
- use real4 format for the sobel or median sobel.
  and save the real4 G results ( are always positive ) as bytes in memory ( for 100 images )
- then calculate the 100 pixels median of each x/y position in the 100 images.
Title: Re: Fast median algorithm
Post by: guga on July 27, 2020, 02:27:39 PM
A threshold won´t help on this case because we have no idea how much noise will be removed and neither were they could be located. We can have noise with 30 pixels near each other with intensity in between 150 that has the same intensity as the true watermark.


About the median calculation, the negative values are necessary because they do affect the result. Take for example these numbers:

Say we have 7 images and at Pos (x = 50, Y = 75) we have this values on each one of them:

Gy (at pos 50/75) of :-1, 5, -17,100,220,-50, 9 ( So, -1 for image1, 5 for image2, -17 for image 3 and so on...)

The median of these value (on that Pos) is 5

If we set all to positive, the median will result on a different value

Gy (at pos 50/75) = 1,5,9,17,50,100,220 = 17

And the way the algo works, it does not calculates the G (sobel) directly from each Gy and Gx. It computes the Sobel from the median of Gy and Gx. And this median is calculated among different images and not from the same image.

One the previous test (the one with a cleaner result) i made that way. I mean, i simply calculated the Sobel on each image and took the median of the Sobel from them to see the result. Although the image is cleaner, it still have some noise and the algo uses the median separatelly from each to generate a cleaner Sobel image.
Title: Re: Fast median algorithm
Post by: guga on July 27, 2020, 02:33:16 PM
My idea is:

- load an image.
- convert the colors to gray colors to memory in real4 format.
- use real4 format for the sobel or median sobel.
  and save the real4 G results ( are always positive ) as bytes in memory ( for 100 images )
- then calculate the 100 pixels median of each x/y position in the 100 images.

Thats what i did previously, but it´s not how the algo works. Since the resultant values of the median of Gx and Gy using negative and positive values are completely different then simply calculating the G directly from them, the resultant G will not be the same as the result i´ve got (even it is clean on this images i tested). Using separate medians of Gy and Gx to compute the final G is the way the algo is actually working, and it probably will clean the result even more, no matter if we use 100 images or more.

The 100 images it is not a rigid limit, it is set that way just to an approximation but, we can have cases that it will be needed more than that and cases we could need only a few images. It always depend on how much different they are. On video, for example, limiting to 100 images won´t work too good, because 100 images can easily represent the same scene, so we will have 100 images where the algo could fail.


In terms of values, take the values on the example i posted previously.

Imagine we have only 7 images and we are calculating the Sobel Gx and Gy at Pos (x = 50, Y = 75).

The correct way the algo is using is converting them to gray and calculating the Gx and Gy for each image (at that same pos). Like this:

Gy (at pos 50/75) :-1, 5, -17,100,220,-50, 9 ( So, -1 for image1, 5 for image2, -17 for image 3 and so on...)

The median of these value (on that Pos) is 5. (On this example, the result is positive, but, can be negative as well).

Say we have at Gx the values (at the same pos) like this:

Gx (at pos 50/75) = 1,5,9,17,50,100,220 = 17

So, for Pos (50/75) we have the resultant G = sqrt(5^2+17^2) = 17.72.... The correct result.


But...if we did as i did before we will have:

G (at pos 50/75) : sqrt((-1^2)+1^2), sqrt(5^2+5^2), sqrt((-17)^2+9^2) .....

G (at pos 50/75) = 1.41421356, 7.071068, 19.23538, 101.4347, 225.6103, 111.8034, 220.184

And the resultant G median  of the above values is (on the same pos): 101.4347 The incorrect value.


Title: Re: Fast median algorithm
Post by: hutch-- on July 27, 2020, 03:46:35 PM
Guga,

I doubt there is an exhaustively good way to remove a watermark, just for practice I have done it with a few photos and the best after tweaking what you can is done by cloning areas close to where the water mark is. While the human eye is tolerant on many things, field depth is not one of them and cloning from about the same field depth keeps at least that aspect looking OK. On an image it is basically texture that your eye will pick if you get it wrong.

If you can sample the colour(s) used in the water mark you can try reducing that specific set of colours from the masked areas of the water mark. I have recently got a piece of Russian software that appears to be designed for altering images of people (mainly pretty girls) but it can do some useful things, it seems to do some form of AI cloning the close area and filling in the gaps. Its not perfect but it does look OK if you have a bit of practice.
Title: Re: Fast median algorithm
Post by: guga on July 27, 2020, 04:14:45 PM
Guga,

I doubt there is an exhaustively good way to remove a watermark, just for practice I have done it with a few photos and the best after tweaking what you can is done by cloning areas close to where the water mark is. While the human eye is tolerant on many things, field depth is not one of them and cloning from about the same field depth keeps at least that aspect looking OK. On an image it is basically texture that your eye will pick if you get it wrong.

If you can sample the colour(s) used in the water mark you can try reducing that specific set of colours from the masked areas of the water mark. I have recently got a piece of Russian software that appears to be designed for altering images of people (mainly pretty girls) but it can do some useful things, it seems to do some form of AI cloning the close area and filling in the gaps. Its not perfect but it does look OK if you have a bit of practice.

Hi Steve

I thought that too, until i saw the results of the google technique. What it did was, basically finds the areas of an image who have a watermark, then rebuild this watermark, then calculate the alfa (transparency) of them, and apply the inverse results. it works, on the same way as we are doing a layer in Photoshop or Paint Shop pro to insert a watermark. What it did was recover back the layers, including the  value of the alpha channel.

I don´t know which russian software is this, but, generally speaking (for videos), i saw some of those techniques being applied by russian apps (for virtualdub, for example), that they basically estimates where the watermark is and tries to cover it with the surrounded pixels. Some do a good job, others are lousy and the watermark always is blurred.

Those apps that alter people faces seems to use  a technique called "poisson reconstruction" (which, btw, that maybe used on the app you got) that is take an image from some place and putting onto another keeping the texture of the target face intact. (Similar to what deep face does) in videos, like these:

https://www.youtube.com/watch?v=AeRofGJ17Sk

In what concerns deep face (aka deep fake - although i don´t like the term) the technique is amazing, specially for image reconstruction.  With deep face and other techniques we can take an video from 1910 for example, and simply rebuild it completely, making it appears as it was shot today.

The technology evolving image processing is advancing day-by-day and is extraordinary the possibilities of usage.  The same thing goes for audio. Adobe developed a couple of years ago an  research app that could take the voice of someone and simply rebuild it to sounds like another one. The demonstration video i saw, the guy simply took the voice of someone, used it as a database, and then start typing  some text he wanted the app to read.  Then, the algo read the text and used the voice automatically generated to simulate the speech. It is a way advanced text to speech technique with natural results that can mimic the voice of anyone. This, btw, is also interesting for me, because we can recreate lost dubs of old movies easily using this technique bringing to life the sound of  golden actors/dubbers dead long time ago.

Title: Re: Fast median algorithm
Post by: guga on July 27, 2020, 08:29:32 PM
Hi Marinus



I think i´ve got it working as expected. :thumbsup: :thumbsup: :thumbsup:

Take a look at this image and compare with the previous one


(https://i.ibb.co/BGWvjQr/Fire-Full-Matrix43.jpg) (https://ibb.co/BGWvjQr)

This method, indeed, eliminates almost all the noise from the watermark. So, it seems that this is the correct path to go. I´ll move on the next part of the algo that involves cropping the watermark and see what happens.

So, the correct way to reach untill this step is:

1 - Convert N images to gray
2 - Calculate the Sobel Gx and Gy for each one of them. So we will have 2*N new images representing both sobels)
3 - Calculate the median of all Gx found (without changing their signs) , calculating on each pixel at that same position for all images (as explained on previous threads).
4 - Calculate the median of all Gy Found (without changing their signs), calculating on each pixel at that same position for all images (as explained on previous threads).
5 - With the resultant medians in step 4, then we calculate the final Sobel G
Title: Re: Fast median algorithm
Post by: Siekmanski on July 27, 2020, 08:56:21 PM
 :cool:

That looks perfect.  :thumbsup: :thumbsup:
Title: Re: Fast median algorithm
Post by: guga on July 28, 2020, 05:39:39 AM
Hi Marinus

I´m working on the cropping. I analyzed how the cropping routine "crop_watermark" in the python version works.  For what i saw so far, they set a threshold on the resultant image and set the coordinates of the rectangle formed by it.

So, once we found a Sobel of the medians of Gx and Gy, the routine simply normalizes the values of this image (the edges, in fact) and set a threshold. If the pixels are higher then 0.4 then all those pixels are turned onto White (255), otherwise they are settled to black (0). It creates a kind of 'shape' formed by the watermark, and it then gets the coordinates of all pixels settled with white and get the max and min pos to see where they starts and where they ends, calculating the X and Y pos to the 1st  white pixel until the last one, he do this in the width and on the height. Simulating a sort of shape of the area of the watermark (in fact, it creates a rectangle around the watermark).

The problem is that, i fixed the sobel operator to avoid the extrapolating the values, resulting on a image cleaner and with edges more thin. Therefore, when i set the threshold to 0.4, it results on a loss of the edges on the resultant shape. It also happens on the python version, but i cannot evaluate the result of the python because my PC freezes when i go further with it and with visual studio.

I analyzed the resultant image i create in PaintShop Pro and found that, maybe i can use the median again of all pixels on the generated result to set a better threshold biased on the texture of the resultant image itself, rather then guessing like it is done in the python version.

I´ll do this and check the results and maybe, do some fine tuning on the fix i made on sobel to allow enhancing the image on user choice when sobel finds the bad pixels. In theory, adjusting the "threshold" directly through sobel errors may fix the cropping routine as well, since all medians of the whole image should be more equalized.

Using the same threshold as in the python version and with 50 images as a guidance, it generated this shape. Threshold 0.4:
(https://i.ibb.co/1X8VVB6/Fire-Full-Matrix5.png) (https://ibb.co/1X8VVB6)

Setting the threshold to 0.2, the other lines of the shape starts to appear on the real watermark. T 0.2
(https://i.ibb.co/2ctxRhb/Fire-Full-Matrix-T02.png) (https://ibb.co/2ctxRhb)


I can also try later using a sobel of the resultant sobel to see what happens. It also could be more useful, since my sobel adaptation tends to equalize the whole image. The main problem is the fact that maybe passing another sobel filter on the resultant median of sobel could also result in more noise that would demand cropping again.

I´ll see what is better to do.

For now, i have a couple of strategies:

1 - Get the median of the resultant sobel to set the proper threshold.
2 - Adjust my adaptation of the sobel filter to it better handles in what do do with bad pixels
3 - Do strategies 1 and 2 together
4 - Apply a sobel of the sobel and do the steps 1 and 2 above.
5 - Use Scharr operator rather then sobel. (The problem in doing this, is the fact that the python version don´t uses it at all and it could be even harder to follow what they are actually doing.) So, we can exchange the edge operator filter only after we suceed to make this huge white elephant sing :greensml: :greensml: :greensml: :greenclp: :greenclp: :greenclp:
Title: Re: Fast median algorithm
Post by: Siekmanski on July 28, 2020, 07:14:20 AM
Great progress.  :cool:

I'm not that far yet, still working on the graphics tool for these operations.
It's cool to experiment, we can learn a lot from it.  :biggrin:
Title: Re: Fast median algorithm
Post by: guga on July 28, 2020, 07:32:32 AM
Sure...We can learn a lot from this. Those image processing techniques are awesome :smiley: :smiley: :smiley:

I´m doing tests on the results and i can discard one of the strategies.


Step  4 Can be discarded. Applying a sobel over a sobel generates noise (btw...beautiful images :greensml: :greensml: :greensml:). But, it can be used to fix sobel operator even further, since from the resultant images you can see the errors in the borders.

Take a look at this animated blob little 'coronga' monster i created :mrgreen: :mrgreen: :mrgreen:

(https://i.ibb.co/qYc6F81/ezgif-4-32757867a95a.gif) (https://ibb.co/qYc6F81)

The watermark got a severe flu :mrgreen: :mrgreen: :mrgreen:

I applied an iteration of 10 times to achieve this. generated the sobel from the medians and then applied sobel over sobel over sobel....10 times :greenclp: :greenclp:

The 2nd iteration worked to enhance the thinner edges, but added some noise in between areas that were supposed to be black. So, we can discard this step.

I´ll apply step1 and see what happens
Title: Re: Fast median algorithm
Post by: daydreamer on July 28, 2020, 10:12:43 AM
Congrats guga,enough Iterations you can sell it as new Godzilla  :greenclp:

can't it be fixed with some filter out single alone pixels?
Title: Re: Fast median algorithm
Post by: guga on July 28, 2020, 12:37:36 PM
Congrats guga,enough Iterations you can sell it as new Godzilla  :greenclp:

can't it be fixed with some filter out single alone pixels?
:greensml: :greensml: :greensml: :greensml:

Probably it wouldn´t help that much on the way the algo works already. When the final sobel is applied it already removes all isolated pixels. I´m working with different watermarks and trying to see what exactly the algorithm is doing on each one of them. For example this last test i made, it did a quite perfect job in refining the edges and eliminating a lot of noise. The problem is that since i fixed the sobel operator, when it is needed to create a shape for the watermark, it removes the edges that are too dark thinking that it is also noise.

Of course, this happens if the images i´m scanning are not enough (100 is not enough if the images are similar to each other) or have similar textures etc.

If i use the same method as in the python version, it also won´t work, because i can´t actually see the result on python to compare what it is doing with what i´m doing. In python, for example, it uses a threshold that the user chooses to remove the bad pixels, but this won´t work on all images set and i want it to be completely automatic as it is written in the google article.

So, instead using a user made threshold i´ll make a test using the standard deviation of the generated image to calculate a threshold. The STD i´ll most likely use the Minimum Standard Deviation which is closer to error cases in statistics, i presume.

Once i get the value from the Standard Deviation i can then calculate the median of the whole image to establish a threshold (if needed, because if the result is ok, i can simply use the std removal and voilá :mrgreen: :bgrin: :bgrin:).

I need only to see how the algo behave with different images in order to try to see if there is a sort of pattern on all of them.  Because if after using the standard deviation to set a threshold, it still have some issues when creating the shape, i´ll have no alternative, except:
1 - Allow more user control on this.
2 - Review the fine tune i made in sobel operator to it allows some errors  and noise (I really don´t want it to, but...if that could help on the final result of creating the shape, i´ll have to do it)
3 - If all fails or still result in something weird, i´ll use scharr operator and see what happens.

Since i understood how to crop the image and what exactly the python version is doing on this stage, then i can try adjusting the initial analysis of the algo before going further rebuilding the watermark or before loading the image from where the watermark should be detected from.

The main problem is that there´s no magic bullet. We need to do it in try and error, trying to understand how google made it, and also how the python version works for that. The most important is make this beast works somehow. If we succeed, then we can try adjust it, refine, optimize and so on.
Title: Re: Fast median algorithm
Post by: guga on July 30, 2020, 04:42:07 AM
Hi Marinus, here is the code for the Sobel Function i made (Unoptimized yet. I want to guarantee the result is perfect before  do any optimization). And check the results using the variation of your algo JJ did :angelic: :angelic: I have some questins to make to him, but i´ll do it later.

Code: [Select]

; Equates used  as flags to the type of fix we want onto the sobel operator.
[SOBEL_FIX_TRUNCATE 0] ; Normal result. Only truncated all above the limits of -255 and 255
[SOBEL_FIX_SIMPLE 1] ; common adjust of the edges. Much better.

; Equates used on the matrix
[FloatMatricesInt.M1Dis 0
 FloatMatricesInt.M2Dis 4
 FloatMatricesInt.M3Dis 8
 FloatMatricesInt.M4Dis 12
 FloatMatricesInt.M5Dis 16
 FloatMatricesInt.M6Dis 20
 FloatMatricesInt.M7Dis 24
 FloatMatricesInt.M8Dis 28
 FloatMatricesInt.M9Dis 32]

[Size_Of_FloatMatricesInt 36]

[Float_SobleVarG: R$ 180.3122292025696187222153123367365]; 255/sqrt(2)) . This is a variable used to speed up the math when computing the G

;;

    SobelGetGPLusEx
        Calculates the Sobel operator for a given amount of pixels stored opn a 3x3 matrix
   
    Parameters:

        pMatrix(In):    The Inputed pixels stored on a 3x3 matrix where the Sobel edges are calculated. The input values must be in Real4 format from 0 to 255. (No integer yet. I´ll adapt to handle integers later)
                        To make easier understand, the matrix is written using equates and the values must be previously calculated and filled.
                        The format of the matrix is:
                       
                        M1 M2 M3
                        M4 M5 M6
                        M7 M8 M9
                       
                        So, at Point M1, Pixel at pos X0, Y0
                               Point M2, Pixel at pos X1, Y0
                               Point M3, Pixel at pos X2, Y0
                               Point M4, Pixel at pos X0, Y1
                               Point M5, Pixel at pos X1, Y1
                               Point M6, Pixel at pos X2, Y1
                               Point M7, Pixel at pos X0, Y2
                               Point M8, Pixel at pos X1, Y2
                               Point M9, Pixel at pos X2, Y2

        pOutSobelX(out): Pointer to a variable that will Store the SobelX value (Magnitude in X direction. I.e.: Gx).
                         The stored value can be negative, positive or zero. All in Real4 format from -1 to 1

        pOutSobelY(out): Pointer to a variable that will Store the SobelY value (Magnitude in Y direction. I.e.: Gy).
                        The stored value can be negative, positive or zero. All in Real4 format from -1 to 1

        pOutSobel(Out): Pointer to a variable that will Store the Sobel value (Total Magnitude of the pixel. I.e.: G).
                        The output is a positive Integer value from 0 to 255. The magnitude of Sobel operator is given by:
                            G = sqrt(Gx^2+Gy^2)

        FixType(In):    A flag able to fix the Sobel operator when it exceeds the limits of -1 to 1 (Normalized)
                        It´s common to the Sobel Operator extrapolates the resultant limits (-255 to 255. Or -1 to 1, normalized)
                        We can see values (unormalized) in the order of -600, 482, 265, -258, -780 etc etc
                       
                        Currently, in order to fix that 2 Flags can be used:
                       
                        SOBEL_FIX_TRUNCATE (Value = 0). Simply truncates the values of Gx and Gy to -1 (if the result is too dark) or 1 (If the result is too bright)
                        SOBEL_FIX_SIMPLE  (Value = 1). Fix the values adjusting the extrapolation to it stays within the limits.
                                                       The math envolving is this fix is: FixedValue = OldValue/(255*(floor(|OldValue|/255)+1))
                       
        pDegree(Out/Optional): A Pointer to a variable to store the Angle formed by the pixel. Sobel Operator can be used to calculate the angle (direction) of the Pixel.
                               It outputs the angle in Degrees. The format of the output is always in real4 format from 0º to 360º.
                               This parameter can be &NULL if you don´t want to exprt the angle.

        Return Values:
                    The function does not return any value.

    Example of usage:

    [MyMatrix: F$ 25, 200, 30
                  100, 45, 75
                  0, 81, 255]

        call SobelGetGPlusEx MyMatrix, MySobelX, MySobelY, MySobel, SOBEL_FIX_SIMPLE, &NULL
   
;;

Proc SobelGetGPLusEx:
    Arguments @pMatrix, @pOutSobelX, @pOutSobelY, @pOutSobel, @FixType, @pDegree
    Local @pReturn, @DataCheck, @Divisor, @FractionX, @FractionY, @Floor, @YAxis, @XAxis
    Structure @TempAxis 16, @SobelYAxisDis 0, @SobelXAxisDis 8
    Uses edi, edx, ecx, eax

    finit

    mov edi D@pMatrix

    ; To calculate Gx^2 later. Therefore Gx = M3+M9 + 2*(M6-M4) - (M7+M1)
    fld F$edi+FloatMatricesInt.M6Dis | fsub F$edi+FloatMatricesInt.M4Dis | fmul R$Float_Two
    fadd F$edi+FloatMatricesInt.M3Dis | fadd F$edi+FloatMatricesInt.M9Dis
    fsub F$edi+FloatMatricesInt.M7Dis | fsub F$edi+FloatMatricesInt.M1Dis
    lea ecx D@DataCheck | fistp F$ecx
    .If D@DataCheck = 0

        fldz | fstp F@FractionX

    .Else_If D@DataCheck <s 0-255 ; Blacks. Edge too dark.

        If D@FixType = SOBEL_FIX_TRUNCATE ;  Truncate the value to -1. The default Sobel behaviour (It truncates to -255 or 255. But, since we are using normalized version we set the limits to -1 and 1)
            fld R$Float_MinusOne | fstp F@FractionX
        Else ; SOBEL_FIX_SIMPLE. FixedValue = OldValue/(255*(floor(|OldValue|/255)+1))
            xor edx edx | mov ecx 255 | mov eax D@DataCheck | neg eax | div ecx | inc eax | imul eax 255 | mov D@Divisor eax
            fild D@DataCheck | fidiv F@Divisor |  fstp F@FractionX
        End_If

    .Else_If D@DataCheck >s 255 ; Whites. Edge too brigth.

        If D@FixType = SOBEL_FIX_TRUNCATE ;  Truncate the value to -1. The default Sobel behaviour (It truncates to -255 or 255. But, since we are using normalized version we set the limits to -1 and 1)
            fld1 | fstp F@FractionX
        Else ;  SOBEL_FIX_SIMPLE. FixedValue = OldValue/(255*(floor(OldValue/255)+1))
            xor edx edx | mov ecx 255 | mov eax D@DataCheck | div ecx | inc eax | imul eax 255 | mov D@Divisor eax
            fild D@DataCheck | fidiv F@Divisor |  fstp F@FractionX
        End_If

    .Else
        fild D@DataCheck | fmul R$Float_One_255 | fstp F@FractionX
    .End_If
    mov eax D@pOutSobelX | mov ecx D@FractionX | mov D$eax ecx

    ; To calculate Gy^2 later. Therefore Gy = M7+M9 + 2*(M8-M2) - (M3+M1)
    fld F$edi+FloatMatricesInt.M8Dis | fsub F$edi+FloatMatricesInt.M2Dis | fmul R$Float_Two
    fadd F$edi+FloatMatricesInt.M7Dis | fadd F$edi+FloatMatricesInt.M9Dis
    fsub F$edi+FloatMatricesInt.M3Dis | fsub F$edi+FloatMatricesInt.M1Dis
    lea ecx D@DataCheck | fistp F$ecx
    .If D@DataCheck = 0
        fldz | fstp F@FractionY
    .Else_If D@DataCheck <s 0-255 ; Blacks. Edge too dark. FixedValue = OldValue/(255*(floor(|OldValue|/255)+1))
        If D@FixType = SOBEL_FIX_TRUNCATE ;  Truncate the value to -1. The default Sobel behaviour (It truncates to -255 or 255. But, since we are using normalized version we set the limits to -1 and 1)
            fld R$Float_MinusOne | fstp F@FractionY
        Else ; SOBEL_FIX_SIMPLE ;  FixedValue = OldValue/(255*(floor(|OldValue|/255)+1))
            xor edx edx | mov ecx 255 | mov eax D@DataCheck | neg eax | div ecx | inc eax | imul eax 255 | mov D@Divisor eax
            fild D@DataCheck | fidiv F@Divisor |  fstp F@FractionY
        End_If
    .Else_If D@DataCheck >s 255 ; Whites. Edge too brigth. FixedValue = OldValue/(255*(floor(OldValue/255)+1))
        If D@FixType = SOBEL_FIX_TRUNCATE ;  Truncate the value to -1. The default Sobel behaviour (It truncates to -255 or 255. But, since we are using normalized version we set the limits to -1 and 1)
            fld1 | fstp F@FractionY
        Else ; SOBEL_FIX_SIMPLE;  FixedValue = OldValue/(255*(floor(OldValue/255)+1))
            xor edx edx | mov ecx 255 | mov eax D@DataCheck | div ecx | inc eax | imul eax 255 | mov D@Divisor eax
            fild D@DataCheck | fidiv F@Divisor |  fstp F@FractionY
        End_If
    .Else
        fild D@DataCheck | fmul R$Float_One_255 | fstp F@FractionY
    .End_If
    mov eax D@pOutSobelY | mov ecx D@FractionY | mov D$eax ecx

    ; Output the Angle (in degrees) if needed
    If D@pDegree <> 0
        lea eax D@SobelYAxisDis | fld F@FractionY | fstp R$eax
        lea edx D@SobelXAxisDis | fld F@FractionX | fstp R$edx
        call atan2 eax, edx, &True
        mov eax D@pDegree | fstp F$eax
    End_If

    ; And finally create the Sobel G after Gx and Gy are already fixed

    ; Soble = sqrt((255*FractionX)^2+(255*FractionY)^2)) = G = sqrt(Gx^2+Gy^2)
    ; since FractionX and FractionY can have a maximum of 1 and -1, therefore sobleMax = (255/sqrt(2)) * sqrt(FractionX^2+FractionY^2)

    fld F@FractionX | fmul ST0 ST0 | fld F@FractionY | fmul ST0 ST0 | faddp ST1 ST0 | fsqrt | fmul R$Float_SobleVarG
    lea edx D@pReturn
    fistp F$edx
    mov eax D@pReturn
    If eax > 255
        mov eax 255
    End_If
    mov ecx D@pOutSobel | mov D$ecx eax

EndP


Atan function used:
Code: [Select]

[Float_AtanPiFactor: R$ (180/3.1415926535897932384626433832795)]
[Float360: R$ 360]

; Macros used to simulate If/Else/EndIf using Fpu
[Fpu_If | fld #3 | fld #1 | fcompp | fstsw ax | fwait | sahf | jn#2 R0>>]
[Fpu_Else_If | jmp R5>> | R0: | fld #3 | fld #1 | fcompp | fstsw ax | fwait | sahf | jn#2 R0>>]
[Fpu_Else | jmp R5>> | R0:]
[Fpu_End_If | R0: | R5:]


Proc atan2:
    Arguments @pY, @pX, @ConvDegree
    Structure @TempStorage 16, @HueDis 0
    Uses eax, ebx, ecx

    mov ebx D@pY
    mov ecx D@pX

    fld R$ebx
    fld R$ecx
    fpatan
    fstsw ax
    wait
    shr ax 1
    jnb L2>
        fclex | stc | xor eax eax | ExitP
L2:

    .If D@ConvDegree = &TRUE
        fmul R$Float_AtanPiFactor | fst R@HueDis
        Fpu_If R@HueDis < R$FloatZero
            fadd R$Float360
        Fpu_Else_If R@HueDis >= R$Float360
            fsub R$Float360
        Fpu_End_If
    .End_If

    clc
    mov eax &TRUE

EndP



Results:
1 - Using more then 100 different images to create the proper watermark.
(https://i.ibb.co/Tgfk47N/Fire-Full-Matrix-New-Algo-truncate2d.jpg) (https://ibb.co/Tgfk47N)

2 - Using 50 images from where only 3 where actually completely different from each other to create the watermark

(https://i.ibb.co/xjnL1MS/Fire-Full-Matrix-New-Algo-truncate3.jpg) (https://ibb.co/xjnL1MS)


As you can see, the second image have more visible noise because almost all images in the directory was similar to each other. I took from 3 different scenes only, so, at the end we have only 3 different images completely different to compare. But, you can see that even with only 3 real different scenes it properly identified the watermark.

Now it´s time to check if i ported correctly JJ´s version of your algo that can handles float (real4) and negatve values. :azn:
Title: Re: Fast median algorithm
Post by: Siekmanski on July 30, 2020, 07:08:27 AM
 :thumbsup:

Can't wait to see the results on my screen.
Writing the Edge detection Graphics tool takes more time than I thought.  :biggrin:
Title: Re: Fast median algorithm
Post by: guga on July 31, 2020, 06:57:54 AM
Different results using Scharr operator

Applied Scharr without any theshold - Noises all over the place

(https://i.ibb.co/fD5y2SB/Fire-Full-Matrix-New-Algo-truncate2d-Scharr.png) (https://ibb.co/fD5y2SB)

Applied Scharr using a threshold of 254 (so, only pixels with 255 are used)

(https://i.ibb.co/8PTLhXP/Fire-Full-Matrix-New-Algo-truncate2d-Scharr2.png) (https://ibb.co/8PTLhXP)


As you can see, Scharr Operator produces a LOT of noise when the medians are applied (on the same way as we did for Sobel). This is explained because Schar detect/enhances more edges then Sobel. On single images, i mean, for general use, Scharr may result on a more accurate image, but in what concerns identifying differences of several images, it fails miserably because the noises are extrapolated all over the resultant image. The only "good" stuff is that since all pixels marked with 255 are 100% sure that belongs to edges, Scharr could be used to create the shapes of the objects (as showed in the 2nd image) using a threshold to discard all pixels below or equal to 254.

The problem is that it still will have noise, and still won´t be able to identify details in the images that are not edge, in fact. Like the internal parts of the watermark, holes etc. The result of the Medians of Scharr are, basically a rough shape of the watermark or common areas of several images. It did quite a good job creating the shape of the watermark, except the fact that it produced noise all over the true edges.

This cannot be fixed on the normal way. So Scharr operator is out of the scope to this algo we are creating (at least, for now) :greensml: :greensml:

I´ll make a couple of more tests on the math involving the matrix to try identify the relation between the neighbor pixels of the matrix and the values we use in the different operator.s It seems to me that the matrix pos at "M1" can be completelly discarded of the computation, but i´m not sure yet.

I´ll make a couple of more tests today.
Title: Re: Fast median algorithm
Post by: Siekmanski on July 31, 2020, 08:25:53 AM
 :thumbsup:

You could try a lowpass filter to get rid of some noise.

Maybe we can take advantage of atan(Gx/Gy), with a 3*3 matrix we get 6 directions back.
So it should be posible to find the direction of the pixel and eliminate all the pixels that don't point in the same direction.
This is theoretical of course.  :badgrin:
Title: Re: Fast median algorithm
Post by: guga on July 31, 2020, 10:17:17 AM
:thumbsup:

You could try a lowpass filter to get rid of some noise.

Maybe we can take advantage of atan(Gx/Gy), with a 3*3 matrix we get 6 directions back.
So it should be posible to find the direction of the pixel and eliminate all the pixels that don't point in the same direction.
This is theoretical of course.  :badgrin:

Hi Marinus

Lowpass filter is what the crop_watermark routine actually does.  But, this seems to me a bit weird because we can simply revert the formula used in the matrix calculation to best find the numbers we are looking for.

For example, no matter if we are using Sobel, Scharr, Prewitt or whatever, the matrix of a 3x3 algorithm always resumes in:

Gx = A*(M1+M7-M3-M9) + D*(M4-M6) ; Where A and D are the 'magic numbers" used in the operator. For Sobel A = -1, D = -2. For Scharr, A = 47, D = 162 and so on. Those numbers are what forms the matrix and it´s transverse form.
Gy = A*(M1+M3-M7-M9) + D*(M2-M8)

Since the values at Gx and Gy re limited in between -1 to 1, i´m trying to identify the relation (properties) of M1 to M9 according to the values of the magic numbers A and D)

For example. The properties i found so far for Gx and Gy (after solving the math) was:

Gx Equations

M6 > 1/(-D)*((-D)*M4 - 1), M9 < -M1 + M3 - M7
M6 < 1/(-D)*((-D)*M4 - 1), M9 > -M1 + M3 - M7
M6 > 1/(-D)*((-D)*M4 - 1), M9>=1/(-A) (A*M1 + A*M3 + D*M4 + (-D)*M6 + A*M7 + 1)

Gy Equations
M8 > 1/(-D)*((-D)*M2 - 1), M9 > M1 + M3 - M7
M8 < 1/(-D)*((-D)*M2 - 1), M9 <= 1/(-A) (-A*M1 + (-D)*M2 + (-A)*M3 + A*M7 + D*M8 - 1)
M8 > 1/(-D)*((-D)*M2 - 1), M9 >= 1/(-A) (-A*M1 + (-D)*M2 + (-A)*M3 + A*M7 + D*M8 - 1)


If i´m correct, then the M1 value may not be used whatsoever, because for M9 do exist in Gx and Gy, The 1st part of all equations, shows an identify between
"M9 < -M1 + M3 - M7"
"M9 > -M1 + M3 - M7"
"M9 > M1 + M3 - M7"

So, the only logical solution  that M9 exists in all those equations is when M1 = 0. So, M1 may not be used anyway to calculate the final result of Gx and Gy. Which seems to be logical because M1 is the common pixel on all orientations (x, y). So it´s a bit weird we have to include the pixel itself to find how much influence the neighbours pixels are playing with him.

I´m not sure, but, if the logic is correct, then both equations can be resume to:

Gx = A*(0+M7-M3-M9) + D*(M4-M6)
Gy = A*(0+M3-M7-M9) + D*(M2-M8)

Therefore:

Gx = A*(M7-M3-M9) + D*(M4-M6)
Gy = A*(M3-M7-M9) + D*(M2-M8)

This should keep Gx and Gy on their own limits. What i´m trying to find is howM7, M9, M3, M4 etc are related to each other.  See what happens when M9 > or < then a given value or how they behave according to the values of A and D (that are constants) or trying to see if the value at M9 can be equal to M3 or M7 etc etc (so we can fix it directly simply adding or subtracting 1 from a given pixel (or 1/255 normalized), when we find pixels that are equal but they shouldn´t be) etc

This is what i´m currently trying to find if there is some property of the matrix we can use to force both equations to stay within the limits of -1 to 1 without having to be forced to use thresholds or other filters to fix this very same situation.
Title: Re: Fast median algorithm
Post by: daydreamer on August 01, 2020, 03:18:34 AM
:thumbsup:

You could try a lowpass filter to get rid of some noise.

Maybe we can take advantage of atan(Gx/Gy), with a 3*3 matrix we get 6 directions back.
So it should be posible to find the direction of the pixel and eliminate all the pixels that don't point in the same direction.
This is theoretical of course.  :badgrin:

Hi Marinus

Lowpass filter is what the crop_watermark routine actually does.  But, this seems to me a bit weird because we can simply revert the formula used in the matrix calculation to best find the numbers we are looking for.

For example, no matter if we are using Sobel, Scharr, Prewitt or whatever, the matrix of a 3x3 algorithm always resumes in:

Gx = A*(M1+M7-M3-M9) + D*(M4-M6) ; Where A and D are the 'magic numbers" used in the operator. For Sobel A = -1, D = -2. For Scharr, A = 47, D = 162 and so on. Those numbers are what forms the matrix and it´s transverse form.
Gy = A*(M1+M3-M7-M9) + D*(M2-M8)

Since the values at Gx and Gy re limited in between -1 to 1, i´m trying to identify the relation (properties) of M1 to M9 according to the values of the magic numbers A and D)

For example. The properties i found so far for Gx and Gy (after solving the math) was:

Gx Equations

M6 > 1/(-D)*((-D)*M4 - 1), M9 < -M1 + M3 - M7
M6 < 1/(-D)*((-D)*M4 - 1), M9 > -M1 + M3 - M7
M6 > 1/(-D)*((-D)*M4 - 1), M9>=1/(-A) (A*M1 + A*M3 + D*M4 + (-D)*M6 + A*M7 + 1)

Gy Equations
M8 > 1/(-D)*((-D)*M2 - 1), M9 > M1 + M3 - M7
M8 < 1/(-D)*((-D)*M2 - 1), M9 <= 1/(-A) (-A*M1 + (-D)*M2 + (-A)*M3 + A*M7 + D*M8 - 1)
M8 > 1/(-D)*((-D)*M2 - 1), M9 >= 1/(-A) (-A*M1 + (-D)*M2 + (-A)*M3 + A*M7 + D*M8 - 1)


If i´m correct, then the M1 value may not be used whatsoever, because for M9 do exist in Gx and Gy, The 1st part of all equations, shows an identify between
"M9 < -M1 + M3 - M7"
"M9 > -M1 + M3 - M7"
"M9 > M1 + M3 - M7"

So, the only logical solution  that M9 exists in all those equations is when M1 = 0. So, M1 may not be used anyway to calculate the final result of Gx and Gy. Which seems to be logical because M1 is the common pixel on all orientations (x, y). So it´s a bit weird we have to include the pixel itself to find how much influence the neighbours pixels are playing with him.

I´m not sure, but, if the logic is correct, then both equations can be resume to:

Gx = A*(0+M7-M3-M9) + D*(M4-M6)
Gy = A*(0+M3-M7-M9) + D*(M2-M8)

Therefore:

Gx = A*(M7-M3-M9) + D*(M4-M6)
Gy = A*(M3-M7-M9) + D*(M2-M8)

This should keep Gx and Gy on their own limits. What i´m trying to find is howM7, M9, M3, M4 etc are related to each other.  See what happens when M9 > or < then a given value or how they behave according to the values of A and D (that are constants) or trying to see if the value at M9 can be equal to M3 or M7 etc etc (so we can fix it directly simply adding or subtracting 1 from a given pixel (or 1/255 normalized), when we find pixels that are equal but they shouldn´t be) etc

This is what i´m currently trying to find if there is some property of the matrix we can use to force both equations to stay within the limits of -1 to 1 without having to be forced to use thresholds or other filters to fix this very same situation.
If you use float pixels0.0,0.0,0.0,0.0black 1.0,1.0,1.0,1.0 =white while doing math instead of with 0.0-255,255,255,255
0.5,0.5,0.5 grey, keep all constants /variable between 0.0-1.0  maybe help
Because 0.5*0.5 =0.25, while 127.0*127.0 becomes lot bigger than 1.0
 
Title: Re: Fast median algorithm
Post by: guga on August 01, 2020, 09:16:17 AM
Hi Daydreamer. The equations are already in the normalized version, just to make easier to try finding their properties. If i find a common formula to force the values at Gx and Gy be stuck inside -1 to 1, then we can also use a version using integers to try optimizing the routines.

I´m not so good in math, this is why i´m taking so much time to try finding this stuff. What i´m trying to do is finding a way reversing the routines to see if the math fits onto it. This is hard to do because if i succeed to fix a value for let´s say M9, then the value at M3 comes out of his place and vice-versa. That´s why trying to find the proper arithmetic involving this stuff is necessary, because i can see what causes the extrapolations, and what happens if M9 > M4-M3 etc etc.....

Even if i succeed to find it, i still need to see if it will work as expected. For what i´m seeing so far, the values in the imputed matrix can be changed/]updated to it best fits to the limits.  What i´ll also need to see is that if they needs to be changed only locally (so, per operation involving finding the sobel) or if they needs to be physically changed in the others pixels as well, so the next time the function enter on the loop to find the next position, it is already fixed by the previous loop and so on.

I tried to get help on a math group in facebook, but people simply couldn´t find the answer . So, i´m basically trying to do it by hand and also using www.wolframalpha.com to retrieve all possible solutions that can be used.
Title: Re: Fast median algorithm
Post by: mineiro on August 01, 2020, 09:40:20 PM
If you use float pixels0.0,0.0,0.0,0.0black 1.0,1.0,1.0,1.0 =white while doing math instead of with 0.0-255,255,255,255
0.5,0.5,0.5 grey, keep all constants /variable between 0.0-1.0  maybe help
Because 0.5*0.5 =0.25, while 127.0*127.0 becomes lot bigger than 1.0

Hi Daydreamer. The equations are already in the normalized version, just to make easier to try finding their properties. If i find a common formula to force the values at Gx and Gy be stuck inside -1 to 1, then we can also use a version using integers to try optimizing the routines.

I'm not following this topic, when I read Daydreamer answer and guga comment Arithmetic Coding comes to my mind. Maybe can be an answer.
The output is between 0 (low) and 1 (high).
So renormalize/rescaling can be done by binary.
0.5 == 1/2, 0.25 == 1/4 and 0.75 == 3/4
Values get converging between high and low; when a situation like
1/4 and 3/4 happens (the MSB aren't equal) is necessary to readjust (rescaling). If they are the same you output then and rescale so value can fit in a range (register size as an example).
The nice of this is that you can separate model (symbols frequency table as an example) from codification.
Zero is seen as an infinite zeros and 1.0 is seen as infinite ones, so you can do this by using binary only.
So, 1.0 can be the same as 255 or FFFFFFFF... . To avoid overflow you can deal with 7FFFFF... .

The most easy to follow text about AC that I have found and have C source code easy to understand was:
http://www.sable.mcgill.ca/~ebodde/pubs/sable-tr-2007-5.pdf
This text starts by using floating point numbers and in next chapter deals with binary numbers only; and explain why is necessary rescaling.
Well, if this works to you, you can check "range encoding" or "srANS"(assymetrical numeral system, Jarek Duda) thats public domain.
Title: Re: Fast median algorithm
Post by: guga on August 02, 2020, 03:06:26 AM
Hi Mineiro

I´ll read the code, but i still clueless about the proper answer. I´mmpost here in 2 parts, because the text became long. What i found so far is:

I - properties of the matrix involving Sobel and Scharr.


    Scharr Operators works similar to sobel.
   
    In Sobel we get this matrices   
    Sobelx
        -1 0 1
        -2 0 2
        -1 0 1

    Sobely
        -1 -2 -1
        0   0  0
        1   2  1
   
    Scharr Matrices are given by:

    ScharrX
        47  0 -47
        162 0 -162
        47  0 -47

    ScharrY
        47   162   47
        0     0     0
        -47 -162  -47

    Other matrix of scharr is:

    ScharrX
        3  0 -3
        10 0 -10
        3  0 -3

    ScharrY
        3   10   3
        0    0   0
        -3 -10  -3
   

    Laplacian is simple   
        0   -1   0
        -1   4  -1
        0   -1   0


The convolution of the matrices are done like ths:

Matrix Values

A B C
D E F
G H I


Pixels Positions

M1 M2 M3
M4 M5 M6
M7 M8 M9


Convolution is generally done like this:
Convolute = A*M1+D*M4+G*M7+ B*M2+E*M5+H*M8 +C*M3+F*M6+I*M9

In Sobel Operator and Scharr we have one line and one row emptied (with 0) and the opposed side has the exact same values as the other,
but with the sign switched. Thus, to make the math more easy those can be written as:

For Gx
B, E, H = 0
C = -A
F = -D
I = -G

therefore, taking the left side as a reference:

Gx = A*M1+D*M4+G*M7+ B*M2+E*M5+H*M8 +C*M3+F*M6+I*M9

turns onto:

Gx = A*M1+D*M4+G*M7+ 0*M2+0*M5+0*M8 +(-A)*M3+(-D)*M6+(-G)*M9
Gx = A*M1+D*M4+G*M7 -A*M3 -D*M6 -G*M9

We also know that for Gx, A = G (Both are -1 for sobelX and 47 or 3 for Scharr), therefore:

Sobel, A = -1, D = -2
Scharr, A = 47, D = 162


Gx = A*M1+D*M4+A*M7 -A*M3 -D*M6 -A*M9
Gx = A*M1+A*M7-A*M3-A*M9 +D*M4 -D*M6
Gx = A*(M1+M7-M3-M9) + D*(M4-M6)

Since Gx can be a number valid in between -1 to 1 we can define a equation like this:

-1>= A*(M1+M7-M3-M9) + D*(M4-M6) <= 1

Solving it result in:

A*(M1 + M7) + D*M4 + 1 <= A*(M3 + M9) + D*M6

solving again for A, we have:

A*(M1 + M7) - A*(M3 + M9) <=  D*M6 - D*M4 - 1
A*(M1 + M7 - M3 - M9) <=  D*(M6-M4)- 1

A <= (D*(M6-M4) - 1)/ (M1+M7-M3-M9)

If A and D are negative, upon the above equation, we have the following solutions:

M6 = M4 + 1/D, M9 > M1 - M3 + M7
M6 = M4 + 1/D, M9 < M1 - M3 + M7
M6 > M4 + 1/D, M9 > M1 - M3 + M7
M6 > M4 + 1/D, M9 <= (A*(M1+M7-M3) + D*(M4-M6) + 1)/A
M6 < M4 + 1/D, M9 >= (A*(M1+M7-M3) + D*(M4-M6) + 1)/A

Ranaming:

Delta1 = M6-M4
Delta2 = M1+M7-M3

we have:
Delta1 = 1/D, M9 > Delta2
Delta1 = 1/D, M9 < Delta2
Delta1 > 1/D, M9 > Delta2
Delta1 > 1/D, M9 <= (A*Delta2 - D*Delta1 + 1)/A
Delta1 < 1/D, M9 >= (A*Delta2 - D*Delta1 + 1)/A

We now have 2 identities
Delta1 > 1/D, M9 > Delta2
Delta1 > 1/D, M9 <= (A*Delta2 - D*Delta1 + 1)/A

So, for each Delta1 > 1/D, we have the following properties:

M9 > Delta2
A*(M9-Delta2) <= 1 - D*Delta1

Renaming again
K = M9-Delta2, we have:

K > 0
A*K <= 1 - D*Delta1

____________________________________________________________________
____________________________________________________________________

On his turn, Gy, has the following properties:

For Gy
D, E, F = 0
A = -G
B = -H
C = -I

Gy = A*M1+D*M4+G*M7+ B*M2+E*M5+H*M8 +C*M3+F*M6+I*M9
turns onto:


Gy = A*M1+0*M4+(-A)*M7+ B*M2+0*M5+(-B)*M8 +C*M3+0*M6+(-C)*M9
Gy = A*M1-A*M7+ B*M2-B*M8 +C*M3-C*M9
We also know that for Gy, C = A (Both are -1 for sobelY and 47 or 3 for Scharr), therefore:
Gy = A*M1-A*M7+ B*M2-B*M8 +A*M3-A*M9
Gy = A*M1-A*M7 +A*M3-A*M9 +B*M2-B*M8
Gy = A*(M1+M3-M7-M9) + B*(M2-M8)

Since Gy can also be a number valid in between -1 to 1 we can define a equation like this:

-1>= A*(M1+M3-M7-M9) + B*(M2-M8) <= 1

Solving it result in:

A*(M1 + M3) + B*M2 + 1 <= A*(M7 + M9) + B*M8

solving again for A, we have: (Remembering that B = D. I.e: The transversed value of SobelGx)

A <= (D*(M8-M2)-1)/(M1+M3-M7-M9)

Since A and D are negative, upon the above equation, we have these solutions:

1/(-D)*((-D)*M2 - 1)

M8 = M2 + 1/D, M9 > M1 + M3 - M7
M8 = M2 + 1/D, M9 < M1 + M3 - M7
M8 > M2 + 1/D, M9 > M1 + M3 - M7
M8 > M2 + 1/D, M9 <= (A*(M1+M3-M7) + D*(M2-M8) + 1) / A
M8 < M2 + 1/D, M9 >= (A*(M1+M3-M7) + D*(M2-M8) + 1) / A

Ranaming
DeltaY1 = M8-M2
DeltaY2 = M1+M3-M7

we have:
DeltaY1 = 1/D, M9 > DeltaY2
DeltaY1 = 1/D, M9 < DeltaY2
DeltaY1 > 1/D, M9 > DeltaY2
DeltaY1 > 1/D, M9 <= (A*DeltaY2 - D*DeltaY1 + 1) / A
DeltaY1 < 1/D, M9 >= (A*DeltaY2 - D*DeltaY1 + 1) / A

We now have 2 identities
DeltaY1 > 1/D, M9 > DeltaY2
DeltaY1 > 1/D, M9 <= (A*DeltaY2 - D*DeltaY1 + 1) / A

So, for each Delta1 > 1/D, we have the following properties:

M9 > DeltaY2
A*(M9-DeltaY2) <= 1 - D*DeltaY1

Renaming again
Ky = M9-DeltaY2, we have:

Ky > 0
A*Ky <= 1 - D*DeltaY1


Those are the properties i found for the whole matrix operation. It seems correct, but don´t know yet how to apply. For example, M9 or M8 can be negative values ? If they can, then how is it possible since they are, in fact only integers from 0 to 255 (i.e: 0 to 1)?,. Also what happens, for example with the solutions i´ve found like:

M8 = M2 + 1/D, M9 > M1 + M3 - M7

It implies that M9 on this case is always bigger then M1+M3-M7 (also all positive values) only if M8 = M2+1/D. Is that correct ?

Do i need a chunk of Conditional If to evaluate when those conditions are met for each solution ?


Now...Part 2 i got this....(next post)
Title: Re: Fast median algorithm
Post by: guga on August 02, 2020, 03:12:18 AM
II - Directly reverting the equation at G

I tried to revert the final value of G using the equatiosn i found in the previous post. This is what i got.

____________________________________________________________________
____________________________________________________________________

let´s now calculate the total G from those properties and see what we found

Since G^2 = Gx^+Gy^2, we have:

Gx = A*(M1+M7-M3-M9) + D*(M4-M6)
Gy = A*(M1+M3-M7-M9) + D*(M2-M8)


The maximum value of G^2 = 1, and the minimum is 0 therefore:

0<= (A*(M1+M7-M3-M9) + D*(M4-M6))^2 + (A*(M1+M3-M7-M9) + D*(M2-M8))^2 <= 1

This equation is hard to solve. So, let´s rename the variables for easier understanding and use the proper signs
for A and D (in case both are negative) to see what we found.

renaming:
x = M1
y = M7
z = M3
w = M9
k = M4
t = M6
g = M2
h = M8

we end up with this:

0<= (A*(x+y-z-w) + D*(k-t))^2 + (A*(x+z-y-w) + D*(g-h))^2 <= 1

For Sobel:
A = -1, D = -2
For Scharr:
A = 47, D = 162

Let´ use some random values keeping the sign to see if we find a solution:

Using A = -1, D = -2, we have:

0<= (-1*(x+y-z-w) + (-2)*(k-t))^2 + (-1*(x+z-y-w) + (-2)*(g-h))^2 <= 1

And the possible solutions are:
x = 1/2 (-2 g + 2 h - 2 k + 2 t + 2 w - sqrt(2)), z = 1/2 (-2 g + 2 h + 2 k - 2 t + 2 y)
x = 1/2 (-2 g + 2 h - 2 k + 2 t + 2 w + sqrt(2)), z = 1/2 (-2 g + 2 h + 2 k - 2 t + 2 y)


Using A = -17, D = -49, we have:
0<= (-17*(x+y-z-w) + (-49)*(k-t))^2 + (-17*(x+z-y-w) + (-49)*(g-h))^2 <= 1

And the possible solutions are:
x = 1/34 (-49 g + 49 h - 49 k + 49 t + 34 w - sqrt(2)), z = 1/34 (-49 g + 49 h + 49 k - 49 t + 34 y)
x = 1/34 (-49 g + 49 h - 49 k + 49 t + 34 w + sqrt(2)), z = 1/34 (-49 g + 49 h + 49 k - 49 t + 34 y)


Now with positive values:

Using A = 17, D = 49, we have:
0<= (17*(x+y-z-w) + (49)*(k-t))^2 + (17*(x+z-y-w) + (49)*(g-h))^2 <= 1

And the possible solutions are:
x = 1/34 (-49 g + 49 h - 49 k + 49 t + 34 w - sqrt(2)), z = 1/34 (-49 g + 49 h + 49 k - 49 t + 34 y)
x = 1/34 (-49 g + 49 h - 49 k + 49 t + 34 w + sqrt(2)), z = 1/34 (-49 g + 49 h + 49 k - 49 t + 34 y)

Using A = 1, D = 2, we have:
0<= (1*(x+y-z-w) + (2)*(k-t))^2 + (1*(x+z-y-w) + (2)*(g-h))^2 <= 1

x = 1/2 (-2 g + 2 h - 2 k + 2 t + 2 w - sqrt(2)), z = 1/2 (-2 g + 2 h + 2 k - 2 t + 2 y)
x = 1/2 (-2 g + 2 h - 2 k + 2 t + 2 w + sqrt(2)), z = 1/2 (-2 g + 2 h + 2 k - 2 t + 2 y)

Now mixing the signs:

Using A= 59, D = -30
0<= (59*(x+y-z-w) + (-30)*(k-t))^2 + (59*(x+z-y-w) + (-30)*(g-h))^2 <= 1

And the possible solutions are:
x = 1/118 (30 g - 30 h + 30 k - 30 t + 118 w - sqrt(2)), z = 1/118 (30 g - 30 h - 30 k + 30 t + 118 y)
x = 1/118 (30 g - 30 h + 30 k - 30 t + 118 w + sqrt(2)), z = 1/118 (30 g - 30 h - 30 k + 30 t + 118 y)

And switching the signs:
Using A= -59, D = 30
0<= (-59*(x+y-z-w) + (30)*(k-t))^2 + (-59*(x+z-y-w) + (30)*(g-h))^2 <= 1

And the possible solutions are:
x = 1/118 (30 g - 30 h + 30 k - 30 t + 118 w - sqrt(2)), z = 1/118 (30 g - 30 h - 30 k + 30 t + 118 y)
x = 1/118 (30 g - 30 h + 30 k - 30 t + 118 w + sqrt(2)), z = 1/118 (30 g - 30 h - 30 k + 30 t + 118 y)

Ok, we have a standard solution here. Let´s try identify them:

Sign1 = -1 when A and D have the same sign. So, both are positive or both are negative.
Sign1 = 1 when A and D have different signs. So, one is positive and other negative. (or vice-versa)

Sign2 = 1 when A and D have the same sign. So, both are positive or both are negative.
Sign2 = -1 when A and D have different signs. So, one is positive and the other negative. (or vice-versa)

|xxxx|  = Absolute value of a number (all numbers are turned onto positive)

x = |1/(2*A)| * ( (Sign1)*|D|*g + (Sign2)*|D|*h + (Sign1)*|D|*k + (Sign2)*|D|*t + |(2*A)|*w - sqrt(2) ), z = |1/(2*A)| * ( (Sign1)*|D|*g + (Sign2)*|D|*h + (Sign2)*|D|*k + (Sign1)*|D|*t + |(2*A)|*y )
x = |1/(2*A)| * ( (Sign1)*|D|*g + (Sign2)*|D|*h + (Sign1)*|D|*k + (Sign2)*|D|*t + |(2*A)|*w + sqrt(2) ), z = |1/(2*A)| * ( (Sign1)*|D|*g + (Sign2)*|D|*h + (Sign2)*|D|*k + (Sign1)*|D|*t + |(2*A)|*y )


Putting the proper labels back we have:

M1 = |1/(2*A)| * ( (Sign1)*|D|*M2 + (Sign2)*|D|*M8 + (Sign1)*|D|*M4 + (Sign2)*|D|*M6 + |(2*A)|*M9 - sqrt(2) ), M3 = |1/(2*A)| * ( (Sign1)*|D|*M2 + (Sign2)*|D|*M8 + (Sign2)*|D|*M4 + (Sign1)*|D|*M6 + |(2*A)|*M7 )
M1 = |1/(2*A)| * ( (Sign1)*|D|*M2 + (Sign2)*|D|*M8 + (Sign1)*|D|*M4 + (Sign2)*|D|*M6 + |(2*A)|*M9 + sqrt(2) ), M3 = |1/(2*A)| * ( (Sign1)*|D|*M2 + (Sign2)*|D|*M8 + (Sign2)*|D|*M4 + (Sign1)*|D|*M6 + |(2*A)|*M7 )


In practice, the above equations result in a couple of different values:

A - When both signs of A and D are negative, we have:

    M3 = |1/(2*A)| * ( (Sign1)*|D|*M2 + (Sign2)*|D|*M8 + (Sign2)*|D|*M4 + (Sign1)*|D|*M6 + |(2*A)|*M7 )
    M3 = |1/(2*A)| * ( -|D|*M2 + |D|*M8 + |D|*M4 - |D|*M6 + |(2*A)|*M7 )

    let´s disconsider the Abs sign (we only need the values of A and D, not their signs), so it lead us onto:
    M3 = 1/(2*A) * ( D*(M8-M2) + D*(M4-M6) + 2*A*M7 )

    M1 = |1/(2*A)| * ( (Sign1)*|D|*M2 + (Sign2)*|D|*M8 + (Sign1)*|D|*M4 + (Sign2)*|D|*M6 + |(2*A)|*M9 - sqrt(2) )
    M1 = 1/(2*A) * ( -|D|*M2 + |D|*M8 -|D|*M4 + |D|*M6 + 2*A*M9 - sqrt(2) )

    let´s disconsider the Abs sign (we only need the values of A and D, not their signs), so it lead us onto:
    M1 = 1/(2*A) * ( D*(M8-M2) - D*(M4-M6) + 2*A*M9 - sqrt(2) )

Gx = A*(M1+M7-M3-M9) + D*(M4-M6)



Those are what i´ve found so far, but when i testd, i´ve got weird results.;..Then i stopped this part of the calculations yesterday night to rest a little. :mrgreen:

It seems correct, but, the problem is that, when i put them on excel...The result is always Sqrt(2)/2, no matter what are the values of the input. If I accept M3 and M1 to use values calculated from Item A above, even if we find thing like -458 etc.... when applying these values, the resultant G will always be sqrt(2)/2. And we will end up with values of M3 and M1 outside of the limits of 0 to 255 (0 to 1.0)


What i´m missing here ?
Title: Re: Fast median algorithm
Post by: Siekmanski on August 02, 2020, 06:18:30 AM
Hi guga,

You have made it very complicated.
If you want a certain range back, normalize the edge-detection matrices ( sobel, scharr etc. ),
There are only 6 values for the convolution calculations for Gradient X and 6 for Gradient Y.

For the Sobel Matrix the range = -1020 to 1020
If you want the range -1 to 1, divide the Sobel Matrix members with 1020. ( or multiply with 0.0009803921)
It's handy to have the gray values as real4 in memory.

Quote
    Laplacian is simple   
        0   -1   0
        -1   4  -1
        0   -1   0

The Laplacian matrix looks more like a lowpass filter.
Title: Re: Fast median algorithm
Post by: guga on August 02, 2020, 06:26:33 AM
-765 to 765 ?

How did you found that range ?  Does it have a formula to retrieve it ? Is it the same range as in Scharr or other operators ?
Title: Re: Fast median algorithm
Post by: Siekmanski on August 02, 2020, 06:34:34 AM
Sorry, just corrected my previous post.
It should be -1020 to 1020.

max. value pixel = 255

-1,-2,-1 = -1*255 + -2*255 + -1*255 = -1020
1,2,1 = 1*255 + 2*255 + 1*255 = 1020
Title: Re: Fast median algorithm
Post by: guga on August 02, 2020, 07:50:58 AM
Thanks...Now i see what you were talking about normalizing the values.

Indeed, doing this way (rather then trying to find a equation) does the job and forces the values to stay within their own limits. I made a test now, and the resultant image is too dark, but, that´s not a problem, because it seems that the edges are all there properly identified and the limits were all respected :thumbsup: :thumbsup: :thumbsup: :thumbsup:. If i want eventually to better see the result, all i have to do is make the result be a bit brighter calculating the difference between the max and minimum sobel found on each image and normalize again (as JJ did to calculate the median). But, normalizing again probably won´t be needed in the scope of the watermark remover, because the shape of the watermark can be identified as well using this method. So, it don´t matter if the resultant image is too dark, because the shape can be created anyway and fixed on the way you showed :)  :thumbsup: :thumbsup: :thumbsup:

The good thing is that now we can calculate the proper normalization ratio using different operators, such as sobel or Scharr etc. To calculate the normalization ratio it is as simple as this:

Ratio = 1/((|ValA|*2+|ValD|)*255)

For Sobel:
ValA = -1 (in abs, only the number and not the sign)
ValD = -2 (in abs, only the number and not the sign)

For Scharr:
ValA = 47 (in abs, only the number and not the sign)
ValD = 162 (in abs, only the number and not the sign)


And the apply those values to Gx or Gy and multiply by 255 to we get the proper normalization back in respect to -255 to 255, right ?


Gx = (A*(M1+M7-M3-M9) + D*(M4-M6)) * Ratio * 255
Gy = (A*(M1+M3-M7-M9) + D*(M2-M8)) * Ratio * 255

On this way, Gx and Gy will be normalized to -1 to 1 and also stay within -255 to 255 if we multiply the result by 255, right ?


Or, simply doing this, right (When M1, M2...xxx are the integers)?

Ratio = 255/((|ValA|*2+|ValD|)*255) = 1/((|ValA|*2+|ValD|)

Title: Re: Fast median algorithm
Post by: mineiro on August 02, 2020, 08:13:38 AM
Quote
Matrix Values

A B C
D E F
G H I

Pixels Positions

M1 M2 M3
M4 M5 M6
M7 M8 M9

Convolution is generally done like this:
Convolute = A*M1+D*M4+G*M7+ B*M2+E*M5+H*M8 +C*M3+F*M6+I*M9
I don't undestand, I need read all this post.
My thinking of whats happening was divide to conquer(1/2, 1/4 and 3/4, ...):
123456789
1*X^8 + 2*X^7 + 3*X^6 + ... + 9*X^0 . Changing base X can be an option.
So, forgot my comment, I was thinking other things. Sorry.
Title: Re: Fast median algorithm
Post by: guga on August 02, 2020, 09:49:52 AM
Hi Marinus

Tested it and the result is amazing. Even though the resultant image is too dar, after i equalized it, it enhances all the edges nearly to perfection.

I´ll make some adjusts to better see the edges (lighter them up a bit) to see what is the result without equalizing. (I equalized only to see the results, it´s not part ot the watermark remover algo)

So, adjusting the sobel/Scharr etc to it fits to their own limits are the way to go. No matter if we us it for tyhe watermark remover or sobel itself. The edges are thin, but they are all there. Also, there´s no need to smooth the image as it is commonly done with other apps, because it can detect everything regardless the quality of the image.
Title: Re: Fast median algorithm
Post by: guga on August 02, 2020, 04:09:36 PM
Sorry, just corrected my previous post.
It should be -1020 to 1020.

max. value pixel = 255

-1,-2,-1 = -1*255 + -2*255 + -1*255 = -1020
1,2,1 = 1*255 + 2*255 + 1*255 = 1020

Hi Marinus...Another question.

This is to compute the ratio of a 3x3 matrix. How to do it for 5x5 matrix or 9x9 etc ? You also need to sum up the values of the cols ? But, what if we have 2 or more cols after the midle one ? we sum them all too ?

Like this:

   -5  -4  0   4   5
   -8 -10  0  10   8
  -10 -20  0  20  10
   -8 -10  0  10   8
   -5  -4  0   4   5


Is correct to do this ?
2*(5*255 + 8*255) + 10*255 + 2*(4*255+10*255) + 20*255
 which resumes to:
((5+8+4+10)*2  + (10+20) ) * 255 = 21420

Title: Re: Fast median algorithm
Post by: Siekmanski on August 02, 2020, 05:41:43 PM
It's as simple as this, no matter the size of the matrices:

e.g. a maximum edge:

Code: [Select]
Do the Matrix convolution with the maximum pixel value (255).

         [ 255 0 0 ]
Pixels = [ 255 0 0 ]
         [ 255 0 0 ]


     [+1  0 -1 ]            [ 255 0 0 ]
Gx = [+2  0 -2 ] * Pixels = [ 510 0 0 ] = 1*255 + 0*0 + -1*0 + 2*255 + 0*0 + -2*0 + 1*255 + 0*0 + -1*0 = 1020 ( we found a maximum edge on the X-axis )
     [+1  0 -1 ]            [ 255 0 0 ]


     [+1 +2 +1 ]            [ 255 0 0 ]
Gy = [ 0  0  0 ] * Pixels = [   0 0 0 ] = 1*255 + 2*0 + 1*0 + 0*255 + 0*0 + 0*0 + -1*255 + -2*0 + -1*0 = 0 ( we found no edge on the Y-axis )
     [-1 -2 -1 ]            [-255 0 0 ]

As you can see here: 1020 is a maximum edge but also -1020 is a maximum edge.

         [ 0 0 255 ]
Pixels = [ 0 0 255 ] * Gx = -1020
         [ 0 0 255 ]

G = sqrt( Gx * Gx + Gy * Gy ) = 1020

The result of an Edge is now between 0 - 1020

If your result is nearer to 0, the edge will be weaker.

The difference between the minimum and maximum of GradientX and GradientY is the range.
The range for Sobel is -1020 to 1020.

Normalize the Sobel Matrix to the maximum range you need.
If you want the range -255 to 255,
Divide all Sobel Matrix members by 1020 * 255, write it back to a New Matrix ( normalized to your chosen range )

If you want to calculate the gradient's direction, atan( Gy / Gx )
You could also consider to calculate a range between -PI/2 and PI/2 ( -90 to 90 degrees )

Note: as you can see, you only need to convolute 6 values per Sobel matrix.  :cool:

Title: Re: Fast median algorithm
Post by: guga on August 03, 2020, 04:22:16 AM
Hi MArinus...Ok, now i see. You sum up all pixels filling only the 1st col with 255 (putting it on the left side) and multiplying it by the edge operator, no matter what is it´s size. Great ! Easier then i thought :thumbsup: :thumbsup: :thumbsup:


Code: [Select]
Do the Matrix convolution with the maximum pixel value (255).

         [ 255 0 0 ]
Pixels = [ 255 0 0 ]
         [ 255 0 0 ]


     [+1  0 -1 ]            [ 255 0 0 ]
Gx = [+2  0 -2 ] * Pixels = [ 255 0 0 ] = 1*255 + 0*0 + -1*0 + 2*255 + 0*0 + -2*0 + 1*255 + 0*0 + -1*0 = 1020 ( we found a maximum edge on the X-axis )
     [+1  0 -1 ]            [ 255 0 0 ]


     [+1 +2 +1 ]            [ 255 0 0 ]
Gy = [ 0  0  0 ] * Pixels = [   255 0 0 ] = 1*255 + 2*0 + 1*0 + 0*255 + 0*0 + 0*0 + -1*255 + -2*0 + -1*0 = 0 ( we found no edge on the Y-axis )
     [-1 -2 -1 ]            [255 0 0 ]

As you can see here: 1020 is a maximum edge but also -1020 is a maximum edge.

         [ 0 0 255 ]
Pixels = [ 0 0 255 ] * Gx = -1020
         [ 0 0 255 ]

So, the final part to calculate Gx is:

Code: [Select]
         [ 0 0 255 ]
Pixels = [ 0 0 255 ] * Gx = -1020
         [ 0 0 255 ]

Putting the 255 on the right side and do the same thing as above, produces always the same result, right ?

Code: [Select]

     [+1  0 -1 ]            [ 0 0 255 ]
G = [+2  0 -2 ] * Pixels = [ 0 0 255 ] = 1*0 + 0*0 + -1*255 + 2*0 + 0*0 + -2*255 + 1*0 + 0*0 + -1*255 = -1020 ( we found a maximum edge on the Gx axis but in the opposed sign, because its´in the opposite direction)
     [+1  0 -1 ]            [ 0 0 255 ]

About the normalization, i did as you said. I just precalculated the value of the normalization previously to it always stays out of the loop that calculates the Sobel (or whataver other operator), so it could be a bit faster.

What i did was basically:

Code: [Select]
Proc SetSobelData:
    Arguments @pColorData, @ImgWidth, @ImgHeight, @pOutSobelX, @pOutSobelY
    Uses esi, edx, ecx, ebx, edi

    mov esi D@pColorData

    ; initialize min and max values
    mov D$EdgeSobelPlus+EdgeSobelPlus.SobelGMinDis 255
    mov D$EdgeSobelPlus+EdgeSobelPlus.SobelGMaxDis 0

    ; initialize SobelPlus structure
    call PrepareSobelEdges EdgeSobelPlus, 0, 0, EDGE_FILTER_SOBEL
(...)
    mov ebx D@Imgwidth | shl ebx 2 | mov D@NextLine ebx
    mov D@Y 0 | mov eax D@Y
    ...While eax < D@Imgheight
        mov D@X 0
        mov edx D@X
        ..While edx < D@Imgwidth

            ; fill our matrix with the inputed pixels
            call FillMatrix3_3 esi, edx, D@Y, TmpMatrix, D@ImgWidth, D@ImgHeight

            call FindSobelEdgesEx TmpMatrix, EdgeSobelPlus, &FALSE

            ; I´ll Optimize this later removing the labels and using registers, instead
            mov eax D$EdgeSobelPlus+EdgeSobelPlus.SobelGxDis | mov edi D@CurSobelX | mov D$edi eax | add D@CurSobelX 4
            mov eax D$EdgeSobelPlus+EdgeSobelPlus.SobelGyDis | mov edi D@CurSobelY | mov D$edi eax | add D@CurSobelY 4

            mov eax D$EdgeSobelPlus+EdgeSobelPlus.SobelGDis | mov edi D@CurSobel | add D@CurSobel 4
            On eax < D$EdgeSobelPlus+EdgeSobelPlus.SobelGMinDis, mov D$EdgeSobelPlus+EdgeSobelPlus.SobelGMinDis eax
            On eax > D$EdgeSobelPlus+EdgeSobelPlus.SobelGMaxDis, mov D$EdgeSobelPlus+EdgeSobelPlus.SobelGMaxDis eax

            ; Not necessary to set the output here for the scope of the watermar detector. I putted here only because i wanted to see the results 1st
            mov B$edi+BGRA.RedDis al
            mov B$edi+BGRA.GreenDis al
            mov B$edi+BGRA.BlueDis al
            mov B$edi+BGRA.AlphaDis 255

            inc edx
        ..End_While
        inc D@Y
        mov eax D@Y
        add esi D@NextLine
    ...End_While
(...)
EndP


Code: [Select]
;;
           PrepareSobelEdges
             This function precalculates the values of the normalization to be used on a edge detection function according to the operators values you feed in

          Parameters:

               pSobelStruct - Pointer to a EdgeSobelPlus Structure that will hold and deal with the several values retrieved during a sobel, scharr or any other edge operators we want

               SobelValA - Is the 'magic' numbers used in an edge operator. ex: In sobel, this value is 0-1, for scharr, this value is 47   and so on. It is the number related to the M1 position on the matrix index.
                                 You can set whatever value you want to create your own Edge operator indexes. To do that, you must define the TypeFlag member as EDGE_FILTER_USER.
                                 Otherwise, if you want a predefined operator (sobel), you can set this member to 0 and use the EDGE_FILTER_SOBEL equate in the  TypeFlag parameter.

               SobelValB - Is the 'magic' numbers used in an edge operator. ex: In sobel, this value is 0-2, for scharr, this value is 162  and so on. It is the number related to the M4 position on the matrix index.
                                 You can set whatever value you want to create your own Edge operator indexes. To do that, you must define the TypeFlag member as EDGE_FILTER_USER.
                                 Otherwise, if you want a predefined operator (sobel), you can set this member to 0 and use the EDGE_FILTER_SOBEL equate in the  TypeFlag parameter

               TypeFlag - A set of predefined Flags to use for specific operators. Currently, you can use any of these 3 equates:
                                Equate Name                Equate Value     Meaning
                                EDGE_FILTER_USER              0                  Use this flag, if you want to create your own pre-defined operator. You must fill the values on SobelValA and SobelValB
                                EDGE_FILTER_SOBEL            1                  Use this flag to enable only the Sobel Operator. The values on SobelValA and SobelValB will be ignored (you can, as well, simply set them to zero. But for convention, better is use 0 insetad)
                                EDGE_FILTER_SCHARR          2                  Use this flag to enable only the Sobel Operator. The values on SobelValA and SobelValB will be ignored (you can, as well, simply set them to zero. But for convention, better is use 0 insetad)
;;

Proc PrepareSobelEdges:
    Arguments @pSobelStruct, @SobelValA, @SobelValD, @TypeFlag
    Local @MatrixValA, @MatrixValD
    Uses edi

    mov eax D@SobelValA | mov D@MatrixValA eax
    mov eax D@SobelValD | mov D@MatrixValD eax

    .If D@TypeFlag = EDGE_FILTER_SOBEL
        mov D@MatrixValA 0-1
        mov D@MatrixValD 0-2
    .Else_If D@TypeFlag = EDGE_FILTER_SCHARR
        mov D@MatrixValA 47
        mov D@MatrixValD 161; 162 (official value). 161 is the best value i found because the sum of MatrixA+matrixB applying the formula below gives 255
    .Else
         ; if both are zero, avoid division by zero on the normalization ratio
        mov edi D@MatrixValA
        .Test_If_Not_And eax eax, edi edi
            mov edi D@pSobelStruct
            fldz | fst F$edi+EdgeSobelPlus.MatrixValADis | fst F$edi+EdgeSobelPlus.MatrixValBDis | fstp F$edi+EdgeSobelPlus.NormalizeRatioDis
            ExitP
        .Test_End
    .End_If

    mov edi D@pSobelStruct ; All Ok, let´s calculate and put their values
    fild D@MatrixValA | fst F$edi+EdgeSobelPlus.MatrixValADis | fabs | fmul R$Float_Two
    fild D@MatrixValD | fst F$edi+EdgeSobelPlus.MatrixValBDis | fabs | faddp ST1 ST0
    fmul R$Float_255 | fld1 | fdivrp ST0 ST1 | fstp F$edi+EdgeSobelPlus.NormalizeRatioDis

EndP

This is the main Structure used to store the necessary data to calculate thee sobel, Scharr or any other 3x3 operator (including user made). All values are stored in Real4 format.

Code: [Select]
[EdgeSobelPlus:
 EdgeSobelPlus.ValA: F$ 0 ; Is the magic number used in Sobel in Position M1. Ex: 0-1
 EdgeSobelPlus.ValB: F$ 0 ; Is the magic number used in Sobel in Position M4. Ex: 0-2
 EdgeSobelPlus.NormalizeRatio: F$ 0 ; Is the ratio used to normalize the edge detection function.
 EdgeSobelPlus.SobelGx: F$ 0 ; The value of Sobel Gx. Real4 format
 EdgeSobelPlus.SobelGy: F$ 0 ; The value of Sobel Gy. Real4 format
 EdgeSobelPlus.SobelG: F$ 0 ; The value of Sobel G. Real4 format
 EdgeSobelPlus.Angle: F$ 0 ; The Angle (in degrees)of Sobel G. Real4 format
 EdgeSobelPlus.SobelGxMin: F$ 0 ; The Minimum Value of Sobel Gx inside a image. Real4 format
 EdgeSobelPlus.SobelGyMin: F$ 0 ; The Minimum Value of Sobel Gy inside a image. Real4 format
 EdgeSobelPlus.SobelGMin: F$ 0 ; The Minimum Value of Sobel G inside a image. Real4 format
 EdgeSobelPlus.SobelGxMax: F$ 0 ; The Maximum Value of Sobel Gx inside a image. Real4 format
 EdgeSobelPlus.SobelGyMax: F$ 0 ; The Maximum Value of Sobel Gy inside a image. Real4 format
 EdgeSobelPlus.SobelGMax: F$ 0 ; The Maximum Value of Sobel G inside a image. Real4 format
 EdgeSobelPlus.AngleMin: F$ 0 ; The Maximum Value of Sobel G inside a image. Real4 format
 EdgeSobelPlus.AngleMax: F$ 0 ; The Maximum Value of Sobel G inside a image. Real4 format
]

; Equates used


[EdgeSobelPlus.MatrixValADis 0
 EdgeSobelPlus.MatrixValBDis 4
 EdgeSobelPlus.NormalizeRatioDis 8
 EdgeSobelPlus.SobelGxDis 12
 EdgeSobelPlus.SobelGyDis 16
 EdgeSobelPlus.SobelGDis 20
 EdgeSobelPlus.AngleDis 24
 EdgeSobelPlus.SobelGxMinDis 28
 EdgeSobelPlus.SobelGyMinDis 32
 EdgeSobelPlus.SobelGMinDis 36
 EdgeSobelPlus.SobelGxMaxDis 40
 EdgeSobelPlus.SobelGyMaxDis 44
 EdgeSobelPlus.SobelGMaxDis 48
 EdgeSobelPlus.AngleMinDis 52
 EdgeSobelPlus.AngleMaxDis 56]

[Size_of_EdgeSobelPlus 60]


Code: [Select]

;;
             pMatrix (in)- Pointer to the inputed pixels stored on a 3x3 matrix
             pSobelStruct (in/out) - Pointer to a EdgeSobelPlus structure that will grab the values predefined to calcaulate the sobel, and fill thenecessay data accordlying.
                                                 The values that will be filled here are: SobelGx, SobelGy, Sobel G and Angle (in degrees). All value are stored on Real4 format, except Sobel G that is exported as integer
                                                  Sobel Gx and Sobel Gy (or shcarr gx/gy or whatever other operator you used), will be stored on the normalized values between -1 to 1 and always biased on 255.
                                                  So the range is, in fact from -255 to 255. To retrieve this values (the same as the pixel) all you have to do later is multiply the values found to 255.
             UseDegree - If yopu want the function to export the angle of the pixel set this flag to TRUE. Otherwise set it to FALSE.
;;

Proc FindSobelEdgesEx:
    Arguments @pMatrix, @pSobelStruct, @UseDegree
    Local @DataCheck
    Structure @TempAxis 16, @SobelYAxisDis 0, @SobelXAxisDis 8
    Uses esi, edi, edx, ecx, eax

    finit

    mov esi D@pMatrix

    ; calculate normalization is already done in PrepareSobelEdges function

    ;Gx = A*(M1+M7-M3-M9) + D*(M4-M6); where A = 47, D = 162

    mov edi D@pSobelStruct
    fld F$esi+FloatMatricesInt.M1Dis | fadd F$esi+FloatMatricesInt.M7Dis | fsub F$esi+FloatMatricesInt.M3Dis | fsub F$esi+FloatMatricesInt.M9Dis | fmul F$edi+EdgeSobelPlus.MatrixValADis;D@MatrixValA
    fld F$esi+FloatMatricesInt.M4Dis | fsub F$esi+FloatMatricesInt.M6Dis | fmul F$edi+EdgeSobelPlus.MatrixValBDis | faddp ST1 ST0
    ; normalize to 1020, 765 and multiply the result to 255 to we get the proper integer are already predefined in PrepareSobelEdges  function
    fmul F$edi+EdgeSobelPlus.NormalizeRatioDis | fstp F$edi+EdgeSobelPlus.SobelGxDis

    ; Gy = A*(M1+M3-M7-M9) + B*(M2-M8)
    fld F$esi+FloatMatricesInt.M1Dis | fadd F$esi+FloatMatricesInt.M3Dis | fsub F$esi+FloatMatricesInt.M7Dis | fsub F$esi+FloatMatricesInt.M9Dis | fmul F$edi+EdgeSobelPlus.MatrixValADis; | fmul D@MatrixValA
    fld F$esi+FloatMatricesInt.M2Dis | fsub F$esi+FloatMatricesInt.M8Dis | fmul F$edi+EdgeSobelPlus.MatrixValBDis | faddp ST1 ST0
    ; normalize to 1020, 765 etc and multiply the result to 255 to we get the proper integer are already predefined in PrepareSobelEdges function
    fmul F$edi+EdgeSobelPlus.NormalizeRatioDis | fstp F$edi+EdgeSobelPlus.SobelGyDis

    ; Calculate the angle if needed
    If D@UseDegree <> 0
        lea eax D@SobelYAxisDis | fld F$edi+EdgeSobelPlus.SobelGyDis | fstp R$eax
        lea edx D@SobelXAxisDis | fld F$edi+EdgeSobelPlus.SobelGxDis | fstp R$edx
        call atan2 eax, edx, &True
        fstp F$edi+EdgeSobelPlus.AngleDis
    End_If

    ; And finally create the Sobel G after Gx and Gy are already fixed

    ; Soble = sqrt((255*FractionX)^2+(255*FractionY)^2)) = G = sqrt(Gx^2+Gy^2)
    ; since FractionX and FractionY can have a maximum of 1 and -1, therefore sobleMax = (255/sqrt(2)) * sqrt(FractionX^2+FractionY^2)

    fld F$edi+EdgeSobelPlus.SobelGxDis | fmul ST0 ST0 | fld F$edi+EdgeSobelPlus.SobelGyDis | fmul ST0 ST0 | faddp ST1 ST0 | fsqrt | fmul R$Float_SobleVarG
    fistp F$edi+EdgeSobelPlus.SobelGDis
    mov eax D$edi+EdgeSobelPlus.SobelGDis
    If eax > 255
        mov eax 255
    End_If
    mov D$edi+EdgeSobelPlus.SobelGDis eax

EndP

As you see, i made the necessary functions to work outside the maain loop to precalculate everything needed to initiate the sobel routine. This gained soime speed (although the fucntions are nopt yet optimized).


I´ll clean up the code and ppost here an small working example.



Some notes:

Scharr Operator defines the 'magic numbers" as: ValA = 47 and ValB = 162, but this lead to a bit of innacuracy, because the normalization is, therefore biased on 255 and it will produce a result slightly different. The better is used ValB as 161, instead

Also, we can use our own values to create a costumized operator. I found that using ValA = -300 and ValB as 120000 gives a slighly better result, since it calculates all edges and the image becomes prettier. (although, contains some noise as well, but prettier)
Title: Re: Fast median algorithm
Post by: Siekmanski on August 03, 2020, 05:50:46 AM
Quote
Putting the 255 on the right side and do the same thing as above, produces always the same result, right ?

The sign tells you if the edge is left or right, for Gx
Up or down for Gy.

I'm not that far yet with my programming.
Still at the stage of generating different gray color conversions.  :tongue: ( 4 pixels at once, and handle image widths not divisible by 4 )
I have written the code this way, that every stage can access all next stages etc. to see all the possibilities on screen.

stage 1: different gray color conversions
stage 2: different lowpass filters
stage 3: different types of edge detection operators ( Sobel, Scharr etc )
stage 4: creating a Gx,Gy table from one of the previous stages
stage 5: finding the directions per edge pixel and give it a color for that direction. ( stage 5: just for the fun, it should look awesome )

But first step is, getting the math correct ( think I got it covered, else, have to study it more ) then after that, optimize it.
Still thinking what to do with G, maybe clamp it to get a higher intensity color, I will experiment with that when I reach that stage.

Title: Re: Fast median algorithm
Post by: guga on August 03, 2020, 12:39:08 PM
Quote
Putting the 255 on the right side and do the same thing as above, produces always the same result, right ?

The sign tells you if the edge is left or right, for Gx
Up or down for Gy.

I'm not that far yet with my programming.
Still at the stage of generating different gray color conversions.  :tongue: ( 4 pixels at once, and handle image widths not divisible by 4 )
I have written the code this way, that every stage can access all next stages etc. to see all the possibilities on screen.

stage 1: different gray color conversions
stage 2: different lowpass filters
stage 3: different types of edge detection operators ( Sobel, Scharr etc )
stage 4: creating a Gx,Gy table from one of the previous stages
stage 5: finding the directions per edge pixel and give it a color for that direction. ( stage 5: just for the fun, it should look awesome )

But first step is, getting the math correct ( think I got it covered, else, have to study it more ) then after that, optimize it.
Still thinking what to do with G, maybe clamp it to get a higher intensity color, I will experiment with that when I reach that stage.
.

Hi Marinus. About different color conversion of gray, it depends of which workspace you are using. Each workspace has his own set of gamma, offset  and matrix values to be settled.

I can´t put all of the code here, because the code i made to work with these is a kind of a mess and is big. But, if you want the values, i put them in one single structure with the matrices like this (Btw, R$ means Real8):

Code: [Select]
[NativeMatrix:
  WSData_NTSC_RGB_C.Name: D$ Sz_NTSC_RGB
  WsData_WhiteRef_NTSC_RGB: D$ WS_C_OBS2
  WSData_NTSC_RGB_C_Red_M1: R$ 0.6068909        WSData_NTSC_RGB_C_Green_M2: R$ 0.1735011        WSData_NTSC_RGB_C_Blue_M3: R$ 0.200348
  WSData_NTSC_RGB_C_Red_M4: R$ 0.2989164        WSData_NTSC_RGB_C_Green_M5: R$ 0.586599         WSData_NTSC_RGB_C_Blue_M6: R$ 0.1144846
  WSData_NTSC_RGB_C_Red_M7: R$ 0                WSData_NTSC_RGB_C_Green_M8: R$ 0.0660957        WSData_NTSC_RGB_C_Blue_M9: R$ 1.1162243
  WSData_GammaDecEnc_NTSC_RGB_C.Gamma: R$ 2.2
  WSData_GammaDecEnc_NTSC_RGB_C.Offset: R$ 0.055

  WSData_Best_RGB_D50.Name: D$ Sz_Best_RGB
  WsData_WhiteRef_Best_RGB: D$ WS_D50_OBS2
  WSData_Best_RGB_D50_Red_M1: R$ 0.6326696      WSData_Best_RGB_D50_Green_M2: R$ 0.2045558      WSData_Best_RGB_D50_Blue_M3: R$ 0.1269946
  WSData_Best_RGB_D50_Red_M4: R$ 0.2284569      WSData_Best_RGB_D50_Green_M5: R$ 0.7373523      WSData_Best_RGB_D50_Blue_M6: R$ 0.0341908
  WSData_Best_RGB_D50_Red_M7: R$ 0              WSData_Best_RGB_D50_Green_M8: R$ 0.0095142      WSData_Best_RGB_D50_Blue_M9: R$ 0.8156958
  WSData_GammaDecEnc_Best_RGB_D50.Gamma: R$ 2.2
  WSData_GammaDecEnc_Best_RGB_D50.Offset: R$ 0.055

  WSData_Beta_RGB_D50.Name: D$ Sz_Beta_RGB
  WsData_WhiteRef_Beta_RGB: D$ WS_D50_OBS2
  WSData_Beta_RGB_D50_Red_M1: R$ 0.6712537      WSData_Beta_RGB_D50_Green_M2: R$ 0.1745834      WSData_Beta_RGB_D50_Blue_M3: R$ 0.1183829
  WSData_Beta_RGB_D50_Red_M4: R$ 0.3032726      WSData_Beta_RGB_D50_Green_M5: R$ 0.6637861      WSData_Beta_RGB_D50_Blue_M6: R$ 0.0329413
  WSData_Beta_RGB_D50_Red_M7: R$ 0              WSData_Beta_RGB_D50_Green_M8: R$ 0.040701       WSData_Beta_RGB_D50_Blue_M9: R$ 0.784509
  WSData_GammaDecEnc_Beta_RGB_D50.Gamma: R$ 2.2
  WSData_GammaDecEnc_Beta_RGB_D50.Offset: R$ 0.055

  WSData_ColorMatch_RGB_D50.Name: D$ Sz_ColorMatch
  WsData_WhiteRef_ColorMatch_RGB: D$ WS_D50_OBS2
  WSData_ColorMatch_RGB_D50_Red_M1: R$ 0.5093439    WSData_ColorMatch_RGB_D50_Green_M2: R$ 0.3209071    WSData_ColorMatch_RGB_D50_Blue_M3: R$ 0.133969
  WSData_ColorMatch_RGB_D50_Red_M4: R$ 0.274884     WSData_ColorMatch_RGB_D50_Green_M5: R$ 0.6581315    WSData_ColorMatch_RGB_D50_Blue_M6: R$ 0.0669845
  WSData_ColorMatch_RGB_D50_Red_M7: R$ 0.0242545    WSData_ColorMatch_RGB_D50_Green_M8: R$ 0.108782     WSData_ColorMatch_RGB_D50_Blue_M9: R$ 0.6921735
  WSData_GammaDecEnc_ColorMatch_RGB_D50.Gamma: R$ 1.8
  WSData_GammaDecEnc_ColorMatch_RGB_D50.Offset: R$ 0.055

  WSData_Don_RGB_4_D50.Name: D$ Sz_Don_RGB4
  WsData_WhiteRef_Don_RGB_4: D$ WS_D50_OBS2
  WSData_Don_RGB_4_D50_Red_M1: R$ 0.6457711         WSData_Don_RGB_4_D50_Green_M2: R$ 0.1933511     WSData_Don_RGB_4_D50_Blue_M3: R$ 0.1250978
  WSData_Don_RGB_4_D50_Red_M4: R$ 0.2783496         WSData_Don_RGB_4_D50_Green_M5: R$ 0.6879702     WSData_Don_RGB_4_D50_Blue_M6: R$ 0.0336802
  WSData_Don_RGB_4_D50_Red_M7: R$ 0.0037113         WSData_Don_RGB_4_D50_Green_M8: R$ 0.0179861     WSData_Don_RGB_4_D50_Blue_M9: R$ 0.8035126
  WSData_GammaDecEnc_Don_RGB_4_D50.Gamma: R$ 2.2
  WSData_GammaDecEnc_Don_RGB_4_D50.Offset: R$ 0.055

  WSData_ECI_RGB_D50.Name: D$ Sz_ECI_RGB
  WsData_WhiteRef_ECI_RGB: D$ WS_D50_OBS2
  WSData_ECI_RGB_D50_Red_M1: R$ 0.6502043       WSData_ECI_RGB_D50_Green_M2: R$ 0.1780774       WSData_ECI_RGB_D50_Blue_M3: R$ 0.1359383
  WSData_ECI_RGB_D50_Red_M4: R$ 0.3202499       WSData_ECI_RGB_D50_Green_M5: R$ 0.6020711       WSData_ECI_RGB_D50_Blue_M6: R$ 0.077679
  WSData_ECI_RGB_D50_Red_M7: R$ 0               WSData_ECI_RGB_D50_Green_M8: R$ 0.067839        WSData_ECI_RGB_D50_Blue_M9: R$ 0.757371
  WSData_GammaDecEnc_ECI_RGB_D50.Gamma: R$ 1.8
  WSData_GammaDecEnc_ECI_RGB_D50.Offset: R$ 0.055

  WSData_Ekta_Space_PS5_D50.Name: D$ Sz_Ekta_Space_PS5
  WsData_WhiteRef_Ekta_Space_PS5: D$ WS_D50_OBS2
  WSData_Ekta_Space_PS5_D50_Red_M1: R$ 0.5938914    WSData_Ekta_Space_PS5_D50_Green_M2: R$ 0.2729801    WSData_Ekta_Space_PS5_D50_Blue_M3: R$ 0.0973485
  WSData_Ekta_Space_PS5_D50_Red_M4: R$ 0.2606286    WSData_Ekta_Space_PS5_D50_Green_M5: R$ 0.7349465    WSData_Ekta_Space_PS5_D50_Blue_M6: R$ 0.0044249
  WSData_Ekta_Space_PS5_D50_Red_M7: R$ 0            WSData_Ekta_Space_PS5_D50_Green_M8: R$ 0.0419969    WSData_Ekta_Space_PS5_D50_Blue_M9: R$ 0.7832131
  WSData_GammaDecEnc_Ekta_Space_PS5_D50.Gamma: R$ 2.2
  WSData_GammaDecEnc_Ekta_Space_PS5_D50.Offset: R$ 0.055

  WsData_ProPhoto_RGB.Name: D$ Sz_ProPhoto
  WsData_WhiteRef_ProPhoto_RGB: D$ WS_D50_OBS2
  WSData_ProPhoto_RGB_D50_Red_M1: R$ 0.7976749      WSData_ProPhoto_RGB_D50_Green_M2: R$ 0.1351917      WSData_ProPhoto_RGB_D50_Blue_M3: R$ 0.0313534
  WSData_ProPhoto_RGB_D50_Red_M4: R$ 0.2880402      WSData_ProPhoto_RGB_D50_Green_M5: R$ 0.7118741      WSData_ProPhoto_RGB_D50_Blue_M6: R$ 0.0000857
  WSData_ProPhoto_RGB_D50_Red_M7: R$ 0              WSData_ProPhoto_RGB_D50_Green_M8: R$ 0              WSData_ProPhoto_RGB_D50_Blue_M9: R$ 0.82521
  WSData_GammaDecEnc_ProPhoto_RGB_D50.Gamma: R$ 1.8
  WSData_GammaDecEnc_ProPhoto_RGB_D50.Offset: R$ 0.055

  WSData_Wide_Gamut_RGB_D50.Name: D$ Sz_Wide_Gamut
  WsData_WhiteRef_Wide_Gamut_RGB: D$ WS_D50_OBS2
  WSData_Wide_Gamut_RGB_D50_Red_M1: R$ 0.7161046    WSData_Wide_Gamut_RGB_D50_Green_M2: R$ 0.1009296    WSData_Wide_Gamut_RGB_D50_Blue_M3: R$ 0.1471858
  WSData_Wide_Gamut_RGB_D50_Red_M4: R$ 0.2581874    WSData_Wide_Gamut_RGB_D50_Green_M5: R$ 0.7249378    WSData_Wide_Gamut_RGB_D50_Blue_M6: R$ 0.0168748
  WSData_Wide_Gamut_RGB_D50_Red_M7: R$ 0            WSData_Wide_Gamut_RGB_D50_Green_M8: R$ 0.0517813    WSData_Wide_Gamut_RGB_D50_Blue_M9: R$ 0.7734287
  WSData_GammaDecEnc_Wide_Gamut_RGB_D50.Gamma: R$ 2.2
  WSData_GammaDecEnc_Wide_Gamut_RGB_D50.Offset: R$ 0.055

  WSData_Adobe_RGB_1998.Name: D$ Sz_Adobe_RGB_1998
  WsData_WhiteRef_Adobe_RGB_1998: D$ WS_D65_OBS2
  WSData_Adobe_RGB_1998_D65_Red_M1: R$ 0.5767309    WSData_Adobe_RGB_1998_D65_Green_M2: R$ 0.185554     WSData_Adobe_RGB_1998_D65_Blue_M3: R$ 0.1881851
  WSData_Adobe_RGB_1998_D65_Red_M4: R$ 0.2973769    WSData_Adobe_RGB_1998_D65_Green_M5: R$ 0.6273491    WSData_Adobe_RGB_1998_D65_Blue_M6: R$ 0.075274
  WSData_Adobe_RGB_1998_D65_Red_M7: R$ 0.0270343    WSData_Adobe_RGB_1998_D65_Green_M8: R$ 0.0706872    WSData_Adobe_RGB_1998_D65_Blue_M9: R$ 0.9911085
  WSData_GammaDecEnc_Adobe_RGB_1998_D65.Gamma: R$ 2.19921875
  WSData_GammaDecEnc_Adobe_RGB_1998_D65.Offset: R$ 0.055

  WSData_AppleRGB_D65.Name: D$ Sz_AppleRGB
  WsData_WhiteRef_AppleRGB: D$ WS_D65_OBS2
  WSData_AppleRGB_D65_Red_M1: R$ 0.4497288          WSData_AppleRGB_D65_Green_M2: R$ 0.3162486          WSData_AppleRGB_D65_Blue_M3: R$ 0.1844926
  WSData_AppleRGB_D65_Red_M4: R$ 0.2446525          WSData_AppleRGB_D65_Green_M5: R$ 0.6720283          WSData_AppleRGB_D65_Blue_M6: R$ 0.0833192
  WSData_AppleRGB_D65_Red_M7: R$ 0.0251848          WSData_AppleRGB_D65_Green_M8: R$ 0.1411824          WSData_AppleRGB_D65_Blue_M9: R$ 0.9224628
  WSData_GammaDecEnc_AppleRGB_D65.Gamma: R$ 1.8
  WSData_GammaDecEnc_AppleRGB_D65.Offset: R$ 0.055

  WSData_Bruce_RGB_D65.Name: D$ Sz_Bruce_RGB
  WsData_WhiteRef_Bruce_RGB: D$ WS_D65_OBS2
  WSData_Bruce_RGB_D65_Red_M1: R$ 0.4674162         WSData_Bruce_RGB_D65_Green_M2: R$ 0.2944512         WSData_Bruce_RGB_D65_Blue_M3: R$ 0.1886026
  WSData_Bruce_RGB_D65_Red_M4: R$ 0.2410115         WSData_Bruce_RGB_D65_Green_M5: R$ 0.6835475         WSData_Bruce_RGB_D65_Blue_M6: R$ 0.075441
  WSData_Bruce_RGB_D65_Red_M7: R$ 0.0219101         WSData_Bruce_RGB_D65_Green_M8: R$ 0.0736128         WSData_Bruce_RGB_D65_Blue_M9: R$ 0.9933071
  WSData_GammaDecEnc_Bruce_RGB_D65.Gamma: R$ 2.2
  WSData_GammaDecEnc_Bruce_RGB_D65.Offset: R$ 0.055

  WSData_PAL_SECAM_RGB_D65.Name: D$ Sz_PAL_SECAM
  WsData_WhiteRef_PAL_SECAM_RGB: D$ WS_D65_OBS2
  WSData_PAL_SECAM_RGB_D65_Red_M1: R$ 0.430619      WSData_PAL_SECAM_RGB_D65_Green_M2: R$ 0.3415419     WSData_PAL_SECAM_RGB_D65_Blue_M3: R$ 0.1783091
  WSData_PAL_SECAM_RGB_D65_Red_M4: R$ 0.2220379     WSData_PAL_SECAM_RGB_D65_Green_M5: R$ 0.7066384     WSData_PAL_SECAM_RGB_D65_Blue_M6: R$ 0.0713237
  WSData_PAL_SECAM_RGB_D65_Red_M7: R$ 0.0201853     WSData_PAL_SECAM_RGB_D65_Green_M8: R$ 0.1295504     WSData_PAL_SECAM_RGB_D65_Blue_M9: R$ 0.9390943
  WSData_GammaDecEnc_PAL_SECAM_RGB_D65.Gamma: R$ 2.2
  WSData_GammaDecEnc_PAL_SECAM_RGB_D65.Offset: R$ 0.055

  WSData_SMPTE_C_RGB_D65.Name: D$ Sz_SMPTE_C
  WsData_WhiteRef_SMPTE_C_RGB: D$ WS_D65_OBS2
  WSData_SMPTE_C_RGB_D65_Red_M1: R$ 0.3935891       WSData_SMPTE_C_RGB_D65_Green_M2: R$ 0.3652497       WSData_SMPTE_C_RGB_D65_Blue_M3: R$ 0.1916312
  WSData_SMPTE_C_RGB_D65_Red_M4: R$ 0.2124132       WSData_SMPTE_C_RGB_D65_Green_M5: R$ 0.7010437       WSData_SMPTE_C_RGB_D65_Blue_M6: R$ 0.0865431
  WSData_SMPTE_C_RGB_D65_Red_M7: R$ 0.0187423       WSData_SMPTE_C_RGB_D65_Green_M8: R$ 0.1119313       WSData_SMPTE_C_RGB_D65_Blue_M9: R$ 0.9581564
  WSData_GammaDecEnc_SMPTE_C_RGB_D65.Gamma: R$ 2.2
  WSData_GammaDecEnc_SMPTE_C_RGB_D65.Offset: R$ 0.055

  WSData_sRGB_D65.Name: D$ Sz_sRGB
  WsData_WhiteRef_sRGB: D$ WS_D65_OBS2
  WSData_sRGB_D65_Red_M1: R$ 0.4124564              WSData_sRGB_D65_Green_M2: R$ 0.3575761              WSData_sRGB_D65_Blue_M3: R$ 0.1804375
  WSData_sRGB_D65_Red_M4: R$ 0.2126729              WSData_sRGB_D65_Green_M5: R$ 0.7151522              WSData_sRGB_D65_Blue_M6: R$ 0.0721749
  WSData_sRGB_D65_Red_M7: R$ 0.0193339              WSData_sRGB_D65_Green_M8: R$ 0.1191920              WSData_sRGB_D65_Blue_M9: R$ 0.9503041
  WSData_GammaDecEnc_sRGB_D65.Gamma: R$ 2.4
  WSData_GammaDecEnc_sRGB_D65.Offset: R$ 0.055

  WSData_sRGB_D65_Red_HDTV.Name: D$ Sz_sRGB_HDTV
  WsData_WhiteRef_sRGB_HDTV: D$ WS_D65_OBS2
  WSData_sRGB_D65_Red_HDTV_M1: R$ 0.4124564         WSData_sRGB_D65_Green_HDTV_M2: R$ 0.3575761         WSData_sRGB_D65_Blue_HDTV_M3: R$ 0.1804375
  WSData_sRGB_D65_Red_HDTV_M4: R$ 0.2126729         WSData_sRGB_D65_Green_HDTV_M5: R$ 0.7151522         WSData_sRGB_D65_Blue_HDTV_M6: R$ 0.0721749
  WSData_sRGB_D65_Red_HDTV_M7: R$ 0.0193339         WSData_sRGB_D65_Green_HDTV_M8: R$ 0.1191920         WSData_sRGB_D65_Blue_HDTV_M9: R$ 0.9503041
  WSData_GammaDecEnc_sRGB_D65_HDTV.Gamma: R$ 2.2
  WSData_GammaDecEnc_sRGB_D65_HDTV.Offset: R$ 0.099

  WSData_CIE_RGB_E.Name: D$ Sz_CIE_RGB
  WsData_WhiteRef_CIE_RGB: D$ WS_E_OBS2
  WSData_CIE_RGB_E_Red_M1: R$ 0.488718              WSData_CIE_RGB_E_Green_M2: R$ 0.3106803             WSData_CIE_RGB_E_Blue_M3: R$ 0.2006017
  WSData_CIE_RGB_E_Red_M4: R$ 0.1762044             WSData_CIE_RGB_E_Green_M5: R$ 0.8129847             WSData_CIE_RGB_E_Blue_M6: R$ 0.0108109
  WSData_CIE_RGB_E_Red_M7: R$ 0                     WSData_CIE_RGB_E_Green_M8: R$ 0.0102048             WSData_CIE_RGB_E_Blue_M9: R$ 0.9897952
  WSData_GammaDecEnc_CIE_RGB_E.Gamma: R$ 2.2
  WSData_GammaDecEnc_CIE_RGB_E.Offset: R$ 0.055]

Code: [Select]
[Sz_NTSC_RGB: B$ "NTSC RGB", 0
 Sz_Best_RGB: B$ "Best RGB", 0
 Sz_Beta_RGB: B$ "Beta RGB", 0
 Sz_ColorMatch: B$ "ColorMatch", 0
 Sz_Don_RGB4: B$ "Don RGB4", 0
 Sz_ECI_RGB: B$ "ECI RGB", 0
 Sz_Ekta_Space_PS5: B$ "Ekta Space PS5", 0
 Sz_ProPhoto: B$ "ProPhoto", 0
 Sz_Wide_Gamut: B$ "Wide Gamut", 0
 Sz_Adobe_RGB_1998: B$ "Adobe RGB (1998)", 0
 Sz_AppleRGB: B$ "Apple RGB", 0
 Sz_Bruce_RGB: B$ "Bruce RGB", 0
 Sz_PAL_SECAM: B$ "PAL SECAM", 0
 Sz_SMPTE_C: B$ "SMPTE-C", 0
 Sz_sRGB: B$ "sRGB", 0
 Sz_sRGB_HDTV: B$ "sRGB HDTV", 0
 Sz_CIE_RGB: B$ "CIE RGB", 0]

And here is a list of all available White Reference that can be used per workspace or alone.

Code: [Select]

[WHITE_SPACE_A_X_OBS2: R$ 109.850       WHITE_SPACE_A_Y_OBS2: R$ 100        WHITE_SPACE_A_Z_OBS2: R$ 35.585     ; Incandescent/tungsten
 WHITE_SPACE_A_X_OBS10: R$ 111.144      WHITE_SPACE_A_Y_OBS10: R$ 100       WHITE_SPACE_A_Z_OBS10: R$ 35.200    ; Incandescent/tungsten

 WHITE_SPACE_B_X_OBS2: R$ 99.0927       WHITE_SPACE_B_Y_OBS2: R$ 100        WHITE_SPACE_B_Z_OBS2: R$ 85.313     ; Old direct sunlight at noon
 WHITE_SPACE_B_X_OBS10: R$ 99.178       WHITE_SPACE_B_Y_OBS10: R$ 100       WHITE_SPACE_B_Z_OBS10: R$ 84.3493   ; Old direct sunlight at noon

 WHITE_SPACE_C_X_OBS2: R$ 98.074        WHITE_SPACE_C_Y_OBS2: R$ 100        WHITE_SPACE_C_Z_OBS2: R$ 118.232    ; Old daylight
 WHITE_SPACE_C_X_OBS10: R$ 97.285       WHITE_SPACE_C_Y_OBS10: R$ 100       WHITE_SPACE_C_Z_OBS10: R$ 116.145   ; Old daylight

 WHITE_SPACE_D50_X_OBS2: R$ 96.422      WHITE_SPACE_D50_Y_OBS2: R$ 100      WHITE_SPACE_D50_Z_OBS2: R$ 82.521   ; ICC profile PCS
 WHITE_SPACE_D50_X_OBS10: R$ 96.720     WHITE_SPACE_D50_Y_OBS10: R$ 100     WHITE_SPACE_D50_Z_OBS10: R$ 81.427  ; ICC profile PCS

 WHITE_SPACE_D55_X_OBS2: R$ 95.682      WHITE_SPACE_D55_Y_OBS2: R$ 100      WHITE_SPACE_D55_Z_OBS2: R$ 92.149   ; Mid-morning daylight
 WHITE_SPACE_D55_X_OBS10: R$ 95.799     WHITE_SPACE_D55_Y_OBS10: R$ 100     WHITE_SPACE_D55_Z_OBS10: R$ 90.926  ; Mid-morning daylight

 WHITE_SPACE_D65_X_OBS2: R$ 95.047      WHITE_SPACE_D65_Y_OBS2: R$ 100      WHITE_SPACE_D65_Z_OBS2: R$ 108.883  ; Daylight, sRGB, Adobe-RGB
 WHITE_SPACE_D65_X_OBS10: R$ 94.811     WHITE_SPACE_D65_Y_OBS10: R$ 100     WHITE_SPACE_D65_Z_OBS10: R$ 107.304 ; Daylight, sRGB, Adobe-RGB

 WHITE_SPACE_D75_X_OBS2: R$ 94.972      WHITE_SPACE_D75_Y_OBS2: R$ 100      WHITE_SPACE_D75_Z_OBS2: R$ 122.638  ; North sky daylight
 WHITE_SPACE_D75_X_OBS10: R$ 94.416     WHITE_SPACE_D75_Y_OBS10: R$ 100     WHITE_SPACE_D75_Z_OBS10: R$ 120.641 ; North sky daylight

 WHITE_SPACE_E_X_OBS2: R$ 100           WHITE_SPACE_E_Y_OBS2: R$ 100        WHITE_SPACE_E_Z_OBS2: R$ 100        ; Equal energy
 WHITE_SPACE_E_X_OBS10: R$ 100          WHITE_SPACE_E_Y_OBS10: R$ 100       WHITE_SPACE_E_Z_OBS10: R$ 100       ; Equal energy

 WHITE_SPACE_F1_X_OBS2: R$ 92.834       WHITE_SPACE_F1_Y_OBS2: R$ 100       WHITE_SPACE_F1_Z_OBS2: R$ 103.665   ; Daylight Fluorescent
 WHITE_SPACE_F1_X_OBS10: R$ 94.791      WHITE_SPACE_F1_Y_OBS10: R$ 100      WHITE_SPACE_F1_Z_OBS10: R$ 103.191  ; Daylight Fluorescent

 WHITE_SPACE_F2_X_OBS2: R$ 99.187       WHITE_SPACE_F2_Y_OBS2: R$ 100       WHITE_SPACE_F2_Z_OBS2: R$ 67.395    ; Cool fluorescent
 WHITE_SPACE_F2_X_OBS10: R$ 103.280     WHITE_SPACE_F2_Y_OBS10: R$ 100      WHITE_SPACE_F2_Z_OBS10: R$ 69.026   ; Cool fluorescent

 WHITE_SPACE_F3_X_OBS2: R$ 103.754      WHITE_SPACE_F3_Y_OBS2: R$ 100       WHITE_SPACE_F3_Z_OBS2: R$ 49.861    ; White Fluorescent
 WHITE_SPACE_F3_X_OBS10: R$ 108.968     WHITE_SPACE_F3_Y_OBS10: R$ 100      WHITE_SPACE_F3_Z_OBS10: R$ 51.965   ; White Fluorescent

 WHITE_SPACE_F4_X_OBS2: R$ 109.147      WHITE_SPACE_F4_Y_OBS2: R$ 100       WHITE_SPACE_F4_Z_OBS2: R$ 38.813    ; Warm White Fluorescent
 WHITE_SPACE_F4_X_OBS10: R$ 114.961     WHITE_SPACE_F4_Y_OBS10: R$ 100      WHITE_SPACE_F4_Z_OBS10: R$ 40.963   ; Warm White Fluorescent

 WHITE_SPACE_F5_X_OBS2: R$ 90.872       WHITE_SPACE_F5_Y_OBS2: R$ 100       WHITE_SPACE_F5_Z_OBS2: R$ 98.723    ; Daylight Fluorescent
 WHITE_SPACE_F5_X_OBS10: R$ 93.369      WHITE_SPACE_F5_Y_OBS10: R$ 100      WHITE_SPACE_F5_Z_OBS10: R$ 98.636   ; Daylight Fluorescent

 WHITE_SPACE_F6_X_OBS2: R$ 97.309       WHITE_SPACE_F6_Y_OBS2: R$ 100       WHITE_SPACE_F6_Z_OBS2: R$ 60.191    ; Lite White Fluorescent
 WHITE_SPACE_F6_X_OBS10: R$ 102.148     WHITE_SPACE_F6_Y_OBS10: R$ 100      WHITE_SPACE_F6_Z_OBS10: R$ 62.074   ; Lite White Fluorescent

 WHITE_SPACE_F7_X_OBS2: R$ 95.044       WHITE_SPACE_F7_Y_OBS2: R$ 100       WHITE_SPACE_F7_Z_OBS2: R$ 108.755   ; Daylight fluorescent, D65 simulator
 WHITE_SPACE_F7_X_OBS10: R$ 95.792      WHITE_SPACE_F7_Y_OBS10: R$ 100      WHITE_SPACE_F7_Z_OBS10: R$ 107.687  ; Daylight fluorescent, D65 simulator

 WHITE_SPACE_F8_X_OBS2: R$ 96.413       WHITE_SPACE_F8_Y_OBS2: R$ 100       WHITE_SPACE_F8_Z_OBS2: R$ 82.333    ; Sylvania F40, D50 simulator
 WHITE_SPACE_F8_X_OBS10: R$ 97.115      WHITE_SPACE_F8_Y_OBS10: R$ 100      WHITE_SPACE_F8_Z_OBS10: R$ 81.135   ; Sylvania F40, D50 simulator

 WHITE_SPACE_F9_X_OBS2: R$ 100.365      WHITE_SPACE_F9_Y_OBS2: R$ 100       WHITE_SPACE_F9_Z_OBS2: R$ 67.868    ; Cool White Fluorescent
 WHITE_SPACE_F9_X_OBS10: R$ 102.116     WHITE_SPACE_F9_Y_OBS10: R$ 100      WHITE_SPACE_F9_Z_OBS10: R$ 67.826   ; Cool White Fluorescent

 WHITE_SPACE_F10_X_OBS2: R$ 96.174      WHITE_SPACE_F10_Y_OBS2: R$ 100      WHITE_SPACE_F10_Z_OBS2: R$ 81.712   ; Ultralume 50, Philips TL85
 WHITE_SPACE_F10_X_OBS10: R$ 99.001     WHITE_SPACE_F10_Y_OBS10: R$ 100     WHITE_SPACE_F10_Z_OBS10: R$ 83.134  ; Ultralume 50, Philips TL85

 WHITE_SPACE_F11_X_OBS2: R$ 100.966     WHITE_SPACE_F11_Y_OBS2: R$ 100      WHITE_SPACE_F11_Z_OBS2: R$ 64.370   ; Ultralume 40, Philips TL84
 WHITE_SPACE_F11_X_OBS10: R$ 103.866    WHITE_SPACE_F11_Y_OBS10: R$ 100     WHITE_SPACE_F11_Z_OBS10: R$ 65.627  ; Ultralume 40, Philips TL84

 WHITE_SPACE_F12_X_OBS2: R$ 108.046     WHITE_SPACE_F12_Y_OBS2: R$ 100      WHITE_SPACE_F12_Z_OBS2: R$ 39.228   ; Ultralume 30, Philips TL83
 WHITE_SPACE_F12_X_OBS10: R$ 111.428    WHITE_SPACE_F12_Y_OBS10: R$ 100     WHITE_SPACE_F12_Z_OBS10: R$ 40.353] ; Ultralume 30, Philips TL83

[WhiteSpaceXDis 0
 WhiteSpaceYDis 8
 WhiteSpaceZDis 16]

[Size_Of_WhiteSpace 24]

[WS_A_OBS2 0
 WS_A_OBS10 1
 WS_B_OBS2 2
 WS_B_OBS10 3
 WS_C_OBS2 4
 WS_C_OBS10 5
 WS_D50_OBS2 6
 WS_D50_OBS10 7
 WS_D55_OBS2 8
 WS_D55_OBS10 9
 WS_D65_OBS2 10
 WS_D65_OBS10 11
 WS_D75_OBS2 12
 WS_D75_OBS10 13
 WS_E_OBS2 14
 WS_E_OBS10 15
 WS_F1_OBS2 16
 WS_F1_OBS10 17
 WS_F2_OBS2 18
 WS_F2_OBS10 19
 WS_F3_OBS2 20
 WS_F3_OBS10 21
 WS_F4_OBS2 22
 WS_F4_OBS10 23
 WS_F5_OBS2 24
 WS_F5_OBS10 25
 WS_F6_OBS2 26
 WS_F6_OBS10 27
 WS_F7_OBS2 28
 WS_F7_OBS10 29
 WS_F8_OBS2 30
 WS_F8_OBS10 31
 WS_F9_OBS2 32
 WS_F9_OBS10 33
 WS_F10_OBS2 34
 WS_F10_OBS10 35
 WS_F11_OBS2 36
 WS_F11_OBS10 37
 WS_F12_OBS2 38
 WS_F12_OBS10 39]
Title: Re: Fast median algorithm
Post by: guga on August 03, 2020, 12:41:55 PM
; Native Matrix Structure format is like this and all the equates used are right below.

[NativeMatrix.Name: D$ 0 ; Pointer to the name of the model (camera, color space such as: Adobe, Pal/SECAM, Bruce etc)
 NativeMatrix.WhiteRef: D$ 0
 NativeMatrix.RedM1: R$ 0   NativeMatrix.GreenM2: R$ 0  NativeMatrix.BlueM3: R$ 0
 NativeMatrix.RedM4: R$ 0   NativeMatrix.GreenM5: R$ 0  NativeMatrix.BlueM6: R$ 0
 NativeMatrix.RedM7: R$ 0   NativeMatrix.GreenM8: R$ 0  NativeMatrix.BlueM9: R$ 0
 NativeMatrix.GammaDecEnc.Gamma: R$ 0
 NativeMatrix.GammaDecEnc.Offset: R$ 0]

Equates used in NativeMatrix structure
Code: [Select]

; Native Working Spaces and their Matrices

[CS_MATRIX_NTSC_RGB_C 0]
[CS_MATRIX_BEST_RGB_D50 1]
[CS_MATRIX_BETA_RGB_D50 2]
[CS_MATRIX_COLORMATCH_RGB_D50 3]
[CS_MATRIX_DON_RGB_4_D50 4]
[CS_MATRIX_ECI_RGB_D50 5]
[CS_MATRIX_EKTA_SPACE_PS5_D50 6]
[CS_MATRIX_PROPHOTO_RGB_D50 7]
[CS_MATRIX_WIDE_GAMUT_RGB_D50 8]
[CS_MATRIX_ADOBE_RGB_1998_D65 9]
[CS_MATRIX_APPLERGB_D65 10]
[CS_MATRIX_BRUCE_RGB_D65 11]
[CS_MATRIX_PAL_SECAM_RGB_D65 12]
[CS_MATRIX_SMPTE_C_RGB_D65 13]
[CS_MATRIX_SRGB_D65 14]
[CS_MATRIX_SRGB_D65_HDTV 15]
[CS_MATRIX_CIE_RGB_E 16]



[NativeMatrix.NameDis 0
 NativeMatrix.WhiteRefDis 4
 NativeMatrix.RedM1Dis 8
 NativeMatrix.GreenM2Dis 16
 NativeMatrix.BlueM3Dis 24
 NativeMatrix.RedM4Dis 32
 NativeMatrix.GreenM5Dis 40
 NativeMatrix.BlueM6Dis 48
 NativeMatrix.RedM7Dis 56
 NativeMatrix.GreenM8Dis 64
 NativeMatrix.BlueM9Dis 72
 NativeMatrix.GammaDecEnc.GammaDis 80
 NativeMatrix.GammaDecEnc.OffsetDis 88

 Size_Of_NativeMatrix 96]




Btw...about the values of the matrices, their inverted form are also calculated inside a function i created for it. All the data used by workspaces are called in one single function called at once when an app starts, for example.

The function is called: "SetupWorkSpaceMatrixDataEx"

You call it once using whatever workspace needed, and it does the calculations of all necessary data (gamma, tresholds, offsets, luma ranges to be applied to gray etc etc)
Title: Re: Fast median algorithm
Post by: Siekmanski on August 03, 2020, 08:24:44 PM
That's a big collection of color spaces.  :cool:
Title: Re: Fast median algorithm
Post by: guga on August 04, 2020, 08:20:32 AM
Hi Marinus

Any idea how to deal with Sobel G results applied to the luminosity of  image ?

I´m trying to create a equation to apply the sobel gradient G to the overall luminosity on a image to enhance the edges and the other non-edge areas.

The idea is like this:

SobelG properly identifies an edge when the G value is bigger then 127. Then the pixel are most likely belong to and edge.
When G value is below or equal to 127, it means that the pixel most likely is not an edge.

Therefore i create a equation similar to the one we compute the sobel G like this:

Target Pixel = The image we want to enhance the edges using sobel values.

If the Sobel Pixel G is bigger then 127 (meaning we have an edge) we apply this equation:

sqrt(((SobelG-128)*2)^2 + Luma^2)

Where Luma is the value of the target pixel (Luma = Gray, in fact).

It worked, but do have issues in colored images, because the Luma of the target pixel near the edge (not all of them) sometimes is too bright, resulting in a completely change of the original color.


(https://i.ibb.co/Z2v2GdW/gfdsg-Original.png) (https://ibb.co/Z2v2GdW)

The goal is calculate a new magnitude (luma/Gray) biased on the value found in Sobel (when it is an edge, thus, "G" is above 127) and, at the same time, knows what to do when the target pixel is excessively bright (> 127) or excessively dark (<= 127).

For example, if the target pixel also has a value above 127, then it will apply a sobel G upon it and decrease the level a little bit using the proposed equation.

And do the opposite when the target pixel is below or equal to 127, but, this time, increasing the luma/gray of it a little bit, biased also on what was found on the sobel value.

Plus, know what do to when the target pixel is an edge, but it is completely black (pix val = 0) and the sobel G is too high. Ex: We have a Sobel G of 255 and a pixel target of 0. We can´t simply take i´ts average (neither use the equation above) because we will end up transforming a pure black pixel in gray (127 or something)

All of the above is when it finds a Edge (so, when Sobel is above 127.)



And also, once it is fixed, do the same on the areas that are not edge. Using the values of sobel (which will be this time <= then 127 and increasing or decreasing the pixel luminosity a little bit too.


Any ideas on what could be a possible equation to handle all situations ? (edge and non-edge areas)


Note: In order to enhance the errors (so i could see where they are) i previously equalized the image before applying the sobel over it. The steps i did to magnify and find the errors were:
1 - Opened a colored image
2 - Converted to gray
3 - Applied a Histogram equalization on the gray image created.
4 - Applied Sobel over the equalized gray image
5 - Used the resultant values of  Sobel G as a luma guidance applied over the original image. (This step is where i used the equation)
Title: Re: Fast median algorithm
Post by: Siekmanski on August 04, 2020, 10:41:00 AM
The range back from Sobel G = 0 to 1020.
I have thought about it how to turn it into a usable color. ( 0 to 255 )
We have to find out how the distribution is laid out in the G values.

Maybe clamp a value greater than 127 to 127?
Or move the values 0 - 127 to 128 - 255?
Or try other limits between 127 and 255?
Or stretch, shrink the values?
Or divide G by 1020*255?
Or whatever you can come up with....
I don't have a clue yet, but will see when we get there and use the best solution for our purpose.

I'm almost there in my coding progress, I'm not that fast anymore.  :biggrin:
I'll skip the lowpass filter for now, and do it later and start with the Sobel operator.
Then see what is best to do.

Have the gray (luminosity) conversions ready, PAL NTSC SECAM, HDTV and HDR for now.
In my code I load a bitmap image convert it to 32bit ARGB and save it to memory.
Then create a table of floating point luminosity conversions from the the ARGB values in memory and create a 4 pixel empty boundary around the luminosity table. ( to handle pixels near the edges of the bitmap image. )
So there is room for max. 5*5 matrices to play with.

When I have completed the Sobel-operator code, it is time to experiment.

Step by step, and check every step if the math is correct.

Title: Re: Fast median algorithm
Post by: Siekmanski on August 04, 2020, 11:13:29 AM
Example of the HDTV luminosity conversion with the 4 pixel boundary.

(http://members.home.nl/siekmanski/Luma_HDTV.png)
Title: Re: Fast median algorithm
Post by: guga on August 04, 2020, 11:18:13 AM
Hi Marinus,
About how to normalize it, i've done on the way you suggested and it worked like a charm.

The function is the one i posted on the previous comment:

Code: [Select]

;;
           PrepareSobelEdges
             This function precalculates the values of the normalization to be used on a edge detection function according to the operators values you feed in

          Parameters:

               pSobelStruct - Pointer to a EdgeSobelPlus Structure that will hold and deal with the several values retrieved during a sobel, scharr or any other edge operators we want

               SobelValA - Is the 'magic' numbers used in an edge operator. ex: In sobel, this value is 0-1, for scharr, this value is 47   and so on. It is the number related to the M1 position on the matrix index.
                                 You can set whatever value you want to create your own Edge operator indexes. To do that, you must define the TypeFlag member as EDGE_FILTER_USER.
                                 Otherwise, if you want a predefined operator (sobel), you can set this member to 0 and use the EDGE_FILTER_SOBEL equate in the  TypeFlag parameter.

               SobelValB - Is the 'magic' numbers used in an edge operator. ex: In sobel, this value is 0-2, for scharr, this value is 162  and so on. It is the number related to the M4 position on the matrix index.
                                 You can set whatever value you want to create your own Edge operator indexes. To do that, you must define the TypeFlag member as EDGE_FILTER_USER.
                                 Otherwise, if you want a predefined operator (sobel), you can set this member to 0 and use the EDGE_FILTER_SOBEL equate in the  TypeFlag parameter

               TypeFlag - A set of predefined Flags to use for specific operators. Currently, you can use any of these 3 equates:
                                Equate Name                Equate Value     Meaning
                                EDGE_FILTER_USER              0                  Use this flag, if you want to create your own pre-defined operator. You must fill the values on SobelValA and SobelValB
                                EDGE_FILTER_SOBEL            1                  Use this flag to enable only the Sobel Operator. The values on SobelValA and SobelValB will be ignored (you can, as well, simply set them to zero. But for convention, better is use 0 insetad)
                                EDGE_FILTER_SCHARR          2                  Use this flag to enable only the Sobel Operator. The values on SobelValA and SobelValB will be ignored (you can, as well, simply set them to zero. But for convention, better is use 0 insetad)

;;


; Equates used for the EdgeSobelPlus structure

[EdgeSobelPlus.MatrixValADis 0
 EdgeSobelPlus.MatrixValBDis 4
 EdgeSobelPlus.NormalizeRatioDis 8
 EdgeSobelPlus.SobelGxDis 12
 EdgeSobelPlus.SobelGyDis 16
 EdgeSobelPlus.SobelGDis 20
 EdgeSobelPlus.AngleDis 24
 EdgeSobelPlus.SobelGxMinDis 28
 EdgeSobelPlus.SobelGyMinDis 32
 EdgeSobelPlus.SobelGMinDis 36
 EdgeSobelPlus.SobelGxMaxDis 40
 EdgeSobelPlus.SobelGyMaxDis 44
 EdgeSobelPlus.SobelGMaxDis 48
 EdgeSobelPlus.AngleMinDis 52
 EdgeSobelPlus.AngleMaxDis 56]

[Size_of_EdgeSobelPlus 60]


; Equates Flags used to make easier the computation

[EDGE_FILTER_USER 0
 EDGE_FILTER_SOBEL 1
 EDGE_FILTER_SCHARR 2]


Proc PrepareSobelEdges:
    Arguments @pSobelStruct, @SobelValA, @SobelValD, @TypeFlag
    Local @MatrixValA, @MatrixValD
    Uses edi

    mov eax D@SobelValA | mov D@MatrixValA eax
    mov eax D@SobelValD | mov D@MatrixValD eax

    .If D@TypeFlag = EDGE_FILTER_SOBEL
        mov D@MatrixValA 0-1
        mov D@MatrixValD 0-2
    .Else_If D@TypeFlag = EDGE_FILTER_SCHARR
        mov D@MatrixValA 47
        mov D@MatrixValD 161; 162 (official value). 161 is the best value i found because the sum of MatrixA+matrixB applying the formula below gives 255
    .Else
         ; if both are zero, avoid division by zero on the normalization ratio. The ratio is, therefore: zero.
        mov edi D@MatrixValA
        .Test_If_Not_And eax eax, edi edi
            mov edi D@pSobelStruct
            fldz | fst F$edi+EdgeSobelPlus.MatrixValADis | fst F$edi+EdgeSobelPlus.MatrixValBDis | fstp F$edi+EdgeSobelPlus.NormalizeRatioDis
            ExitP
        .Test_End
    .End_If

    mov edi D@pSobelStruct ; All Ok, let´s calculate and put their values
    fild D@MatrixValA | fst F$edi+EdgeSobelPlus.MatrixValADis | fabs | fmul R$Float_Two
    fild D@MatrixValD | fst F$edi+EdgeSobelPlus.MatrixValBDis | fabs | faddp ST1 ST0
    fmul R$Float_255 | fld1 | fdivrp ST0 ST1 | fstp F$edi+EdgeSobelPlus.NormalizeRatioDis

EndP

The math is what you told me:

Normalization Ratio = 1/(255*(|ValA|*2 + |ValD|))

Where:
ValA is the Sobel Matrix Value at Position M1 = 0-1
ValD is the Sobel Matrix Value at Position M4 = 0-2

For Sobel, it results in: 1/(255*(|-1|*2 + |-2|)) = 1/1020

Or...for scharr it is
ValA = 47
ValD = 161 ; (The official value is 162, but i figure it out that using 161 is a bit more accurated)

For Scharr, it results in: 1/(255*(|47|*2 + |161|)) = 1/65025


Or..you can use whatever value you like on a 3x3 matrix on the style of a Sobel Operator.


I used the normalization ratio on other function which is the one that do the sobel computation. (FindSobelEdgesEx)

Basically, it does this:

Gx = (ValA*(M1+M7-M3-M9) + ValD*(M4-M6)) * Normalization Ratio.
Gy = (ValA*(M1+M3-M7-M9) + ValD*(M2-M8)))* Normalization Ratio.

Where M1 to M9 are the values of the pixels (in Real4 from 0 to 255). and ValA and ValD are the Sobel/Scharr values we use (or any other values we want. The sign is kept intact, btw). When we multiply this with the normalization ratio, the values of Gx and GY will already be normalized to -1 to 1 obeying the range of -255 to 255.


For the tests i made so far, there´s no need to clamp


The issue i was talking about was other test i´m currently doing. I´m checking to see how to apply the sobel values we´ve got on a normal picture to we enhance the edges and overall illumination of the image. I made a quick draw. the idea is it obey this rule:

(https://i.ibb.co/bQpWRsV/222-Image1.png) (https://ibb.co/bQpWRsV)

When we are in a edge, we need to choose if we will increase the bright of the pixel or decrease it. Pixels too bright with a high Sobel G (64 in this case, representing 65 to 128 because we are already on a edge), we need to decrease a little bit. On the other hand, edge pixels too dark (from 0 to 127) needs to be brighten a little bit, also obeying the rule to the sobel G value (this time on a range of 0 to 64 meaning we need to increase the image pixel)

It basically compress a bit the pixel brightness. So, when you are on a edge, the goal is do something like this. (We are inside a edge, ok ? So, Sobel G range here is from 128 to 255):

Pixel image inside a edge, also varies from: 0.......255. So we need to find a way to recalculate their values taking onto account the fact we are on a edge (so, in Sobel G brighter values from 128 to 255)

Too dark..(need to increase a bit)                                     Too bright (need to decrease a bit)
----------------------------------------> New Edge Pixel Value <--------------------------------------------

Btw...I´m talking about this routine i´m rying to make to use sobel over a regular image, and not sobel itself.


Title: Re: Fast median algorithm
Post by: guga on August 04, 2020, 11:20:46 AM
Example of the HDTV luminosity conversion with the 4 pixel boundary.



Wow..Great !

But you applied sobel over it or only converted to GrayScale ? I see some black borders around it.
Title: Re: Fast median algorithm
Post by: Siekmanski on August 04, 2020, 11:42:24 AM
No, only the luminosity conversion.
BTW both images are stretched to the same size
The gray version is in reality 8 by 8 pixels larger than the original image, because of the borders.

Quote
I see some black borders around it.

See Reply #141 , I have explained it there.
Title: Re: Fast median algorithm
Post by: guga on August 04, 2020, 01:40:07 PM
Ok :)


One thing...i´ll give a try on this equation


https://www.wolframalpha.com/input/?i=1%2F%28sqrt%2864%5E2%2B128%5E2%29%29+*+128+*+%28sqrt%28%28%28%28x-128%29%2B%28y-128%29%29%2F2%29%5E2%2B%28%28y-128%29%2F2%29%5E2%29%29+%2B128 (https://www.wolframalpha.com/input/?i=1%2F%28sqrt%2864%5E2%2B128%5E2%29%29+*+128+*+%28sqrt%28%28%28%28x-128%29%2B%28y-128%29%29%2F2%29%5E2%2B%28%28y-128%29%2F2%29%5E2%29%29+%2B128)

1/(sqrt(64^2+128^2)) * 128 * (sqrt((((x-128)+(y-128))/2)^2+((y-128)/2)^2)) +128

where:

x = Sobel Value From 128 to 255
y = Pixel Value from 128 to 255

This seems to produce a somewhat compressed values. The graphic shows it produces some oval image, meaning that the edges of too bright pixels (above 127) maybe represented like this.

I´ll make a tests. If it works, i´ll give a try on other equation to work with darker pixels that also belongs to the edges.

If the results are ok...i´ll then try with non edge pixels
Title: Re: Fast median algorithm
Post by: guga on August 04, 2020, 02:39:21 PM
1st test

It found the high frequencies of Sobel applied only to the brighter pixels and balanced the luma pixels on them. It working in 4 different pieces when it is inside the edges. Edges too bright, it increases or decreases the pixel accord to the SObel Values and also will have to do the same when the pixels on the edges are too dark. I´ll test it to see if i can find a equations for that.

It do have a bit of noise (near the hair and borders) becausue the images was previously equalized. I did that too search for errors.

(https://i.ibb.co/fN1TjPg/Fixing-Light-Default-User-Defined4.png) (https://ibb.co/fN1TjPg)

The edges were properly balanced on the higher frequencies on the cases where the values of the pixel is higher then 127.

I´ll now try to see if it also works for edges whose pixels are too dark.

Later i´ll try on the non-edge areas


Note: There´s a small error on mine CieLab function. Colored Pixels too close to gray (R G and B are not equal, but their distances are too close to each other) are turned onto a colored variation, because the chroma (a and b factors on CieLab)_ were readjusted. I´ll review this later. Probably i´ll use HSL just to see what it looks like without the tiny errors in cielab.
Title: Re: Fast median algorithm
Post by: guga on August 04, 2020, 04:05:22 PM
Hypnotoad :biggrin: :biggrin: :biggrin: :biggrin: :greensml: :greensml: :greensml: :greensml:

(https://i.ibb.co/x8pNLMv/Fixing-Light-Default-User-Defined4.jpg) (https://ibb.co/x8pNLMv)
Title: Re: Fast median algorithm
Post by: Siekmanski on August 04, 2020, 06:51:08 PM
 :biggrin:
Nice toad.

I don't calculate the complete colorspace because we only need the luma (Y') component.
They add up to 1.0 for the luma channel. ( I don't use the Chroma channels )

Code: [Select]
                                    ;   Blue    Green    Red ( Alpha,  unused = 0.0 )
Luma_PAL_SECAM_NTSC             real4  0.1140, 0.5870, 0.2990, 0.0  ; rec601 luma (Y') component
Luma_HDTV                       real4  0.0722, 0.7152, 0.2126, 0.0  ; ITU-R BT.709 standard luma (Y') component
Luma_HDR                        real4  0.0593, 0.6780, 0.2627, 0.0  ; ITU-R BT.2100 standard luma (Y') component

Think the Luma_HDTV coefficients are the best choice.....
Title: Re: Fast median algorithm
Post by: mineiro on August 04, 2020, 09:29:44 PM
Hypnotoad :biggrin: :biggrin: :biggrin: :biggrin: :greensml: :greensml: :greensml: :greensml:
(https://i.ibb.co/x8pNLMv/Fixing-Light-Default-User-Defined4.jpg) (https://ibb.co/x8pNLMv)
Retinex!?
https://spinoff.nasa.gov/spinoff2002/ch_6.html
Good work.
Title: Re: Fast median algorithm
Post by: mineiro on August 04, 2020, 11:05:33 PM
Hello sir guga;
I build this image years ago, can you apply your tool in this image and upload result?
Original image is .gif, if you adjust zoom to 50% and rotate/mirror/flip image you should get these others images showed below.
Original image:
(https://i.ibb.co/7YRQjV7/bigtrash.gif) (https://imgbb.com/)
This is what you can get from that image:

(https://i.ibb.co/vk2RQxd/desk1.png) (https://imgbb.com/)
(https://i.ibb.co/PGNrzrg/desk2.png) (https://imgbb.com/)
(https://i.ibb.co/vVdTtc2/desk3.png) (https://imgbb.com/)




Title: Re: Fast median algorithm
Post by: daydreamer on August 05, 2020, 03:12:38 AM
Hypnotoad :biggrin: :biggrin: :biggrin: :biggrin: :greensml: :greensml: :greensml: :greensml:

(https://i.ibb.co/x8pNLMv/Fixing-Light-Default-User-Defined4.jpg) (https://ibb.co/x8pNLMv)
Oops ended up in game forum,preview of the soon to be released "froggy 2020"  :greenclp: :greenclp: :greenclp:
Title: Re: Fast median algorithm
Post by: guga on August 05, 2020, 04:21:48 AM
Hi Mineiro.

You mean, this ?

(https://i.ibb.co/WHsh7m8/Mineiro.png) (https://ibb.co/WHsh7m8)

This is the result i´ve got after applying the code onto your image. But the tool is not finished. This part of the code i was fixing last night and seeing the results just for fun. The function was not supposed to do this :greensml: :greensml: :greensml:

But if you want the steps to do it, basically all i did was:

1 - Convert the image to gray
2 - Equalized the image (with a general histogram equalization algorithm, not Clahe)
3 - Applied the sobel algo over the resultant image
4 - Equalized the result again. So, i equalized over the generated sobel image)
5 - Applied the generated equalized image over the original colored one. But, the routine to do this only applies on high frequencies of sobel, all the rest is left as 0 (black).  The math to do this is: sqrt(((SobelG-128)*2)^2 + Luma^2). or a pseudocode:

If SobelG > 127
  NewPix =     sqrt(((SobelG-128)*2)^2 + Luma^2)
Else
   NewPix = 0
End_If

6 - Applied the value of NewPix (which is a new luma) on the original image with CieLab. So, i stored previously the values of Luma, a and b factor of CieLab of the original image and simply replaced the Luma on the original image with this new value.


But..again..this is not what the algo was supposed to do :mrgreen: :mrgreen: :mrgreen:. The goal is to apply sobel values over the whole image without ruining it to enhance the edges
Title: Re: Fast median algorithm
Post by: guga on August 05, 2020, 04:27:45 AM
Hypnotoad :biggrin: :biggrin: :biggrin: :biggrin: :greensml: :greensml: :greensml: :greensml:

(https://i.ibb.co/x8pNLMv/Fixing-Light-Default-User-Defined4.jpg) (https://ibb.co/x8pNLMv)
Oops ended up in game forum,preview of the soon to be released "froggy 2020"  :greenclp: :greenclp: :greenclp:

 :greensml: :greensml: :greensml: :greensml: :greensml:
Title: Re: Fast median algorithm
Post by: mineiro on August 05, 2020, 08:33:05 AM
Hello sir guga;
Yes, I mean that. Thank you.
I just like to check if main image stand out over others.
Title: Re: Fast median algorithm
Post by: Siekmanski on August 06, 2020, 02:31:15 AM
Played around with the Sobel Operator convolution calculations.
Simplifying the math and keeping it logical.
Reason, make them suitable for fast parallel SSE2.

Faster Sobel Operator Convolution:

Code: [Select]
         [-1 -2 -1 ]
SobelX = [ 0  0  0 ] = 0 degree
         [+1 +2 +1 ]

         [-1  0 +1 ]
SobelY = [-2  0 +2 ] = 90 degree
         [-1  0 +1 ]

         [ P1 P2 P3 ]
Pixels = [ P4 P5 P6 ]
         [ P7 P8 P9 ]
Code: [Select]
Convolution with simplified math:
GradientX = (P7 + 2*P8 + P9) - (P1 + 2*P2 + P3)         ; - if edge is on the left side, + if on the right side
GradientY = (P3 + 2*P6 + P9) - (P1 + 2*P4 + P7)         ; - if edge is on the top side, + if on the bottom side
Code: [Select]
Convolution without multiplications:
GradientX = (P7 + P8 + P8 + P9) - (P1 + P2 + P2 + P3)   ; - if edge is on the left side, + if on the right side
GradientY = (P3 + P6 + P6 + P9) - (P1 + P4 + P4 + P7)   ; - if edge is on the top side, + if on the bottom side

Now we can use fast SIMD, shuffle them in place and add them together and do 2 subtractions.  :cool:
Title: Re: Fast median algorithm
Post by: guga on August 06, 2020, 03:40:25 AM
Great  :thumbsup: :thumbsup: :thumbsup: :thumbsup:

Also we need to normalize the result, right ?
Title: Re: Fast median algorithm
Post by: hutch-- on August 06, 2020, 06:14:32 AM
I have not really kept up with this topic but it sounds like an algorithm that does sharpening and smoothing, depending on the settings. I use an ffmpeg option for doing this type of work on 1080 and 4k video.
Title: Re: Fast median algorithm
Post by: Siekmanski on August 06, 2020, 06:33:51 AM
Great  :thumbsup: :thumbsup: :thumbsup: :thumbsup:

Also we need to normalize the result, right ?

Maybe, maybe not, that's the question.  :biggrin:
Have to implement it first, and see if threshold clamping is an option.

Just finished a routine to save the gray color conversions to memory.
It handles 4 pixels at once and takes care of any possible "rest pixels".
It has a Stride for the width, wich is always a multiple of 16 bytes, no matter if the bitmap width is not divisible by 4.
Works like charm and is very fast.
I wrote it for SSE2 but it can be made faster for SSE4.1 if needed.
And this routine is a good candidate for multithreading if it is needed in realtime video processing.

Hi Hutch,

It's for edge detection in bitmaps.
guga want's to use it for removing watermarks.
I collaborate with him to learn a few new things.
Title: Re: Fast median algorithm
Post by: guga on August 06, 2020, 08:36:43 AM
I have not really kept up with this topic but it sounds like an algorithm that does sharpening and smoothing, depending on the settings. I use an ffmpeg option for doing this type of work on 1080 and 4k video.

Hi Steve

The algorithm is to detect edges on a image (or video). What i´m trying to do is adapt a amazing algorithm developed by google that removes watermarks and makes the image be perfect. And later (if we suceed) this can be adapted to do the same for videos.

FFMpeg, for example, have a built in function (or plugin, i don´t remember) that removes the logo of videos or watermarks etc, but it is far from perfect. With google algorithm we can extend the functionality to remove whatever imperfections we have on a video or image.

It was designed to remove watermarks, but, in fact, it can remove all sort of imperfections, such as lens crack on a camera, fog, scratches, spots, dirt etc. So, whatever is in front of the image that works as a unwanted layer can be completely removed.

For example, if we have a video containing scenes that were ruined by a lens crack such as the image below:
(https://i.ibb.co/86pCVYv/bvcxbtropical-bird-1390996.png) (https://ibb.co/86pCVYv)

This can be completely removed (as long the layer (watermark, crack, dirt etc) is fixed). It´s an amazing algorithm and seems to produce a quite perfect result. But it is a hell to port , since it´s written in python and i don´t know python language, i´m struggling to understand the logic behinds it.

Note: Also, if we would apply the same logic (and with some extra work), then we could also remove any background (or any other object) from a video or image and replace with whatever we want (as long the background is fixed) on a way faster, easier and more accurate then premiere, Mocha etc.
Title: Re: Fast median algorithm
Post by: guga on August 06, 2020, 09:36:43 AM
Great  :thumbsup: :thumbsup: :thumbsup: :thumbsup:

Also we need to normalize the result, right ?

Maybe, maybe not, that's the question.  :biggrin:
Have to implement it first, and see if threshold clamping is an option.

Just finished a routine to save the gray color conversions to memory.
It handles 4 pixels at once and takes care of any possible "rest pixels".
It has a Stride for the width, wich is always a multiple of 16 bytes, no matter if the bitmap width is not divisible by 4.
Works like charm and is very fast.
I wrote it for SSE2 but it can be made faster for SSE4.1 if needed.
And this routine is a good candidate for multithreading if it is needed in realtime video processing.

Hi Hutch,

It's for edge detection in bitmaps.
guga want's to use it for removing watermarks.
I collaborate with him to learn a few new things.

Hi Marinus

In SSE2, please. So it would be good to people uses in older machines. Also, i don´t have the way to port it to SSE4 with RosAsm yet. (Didn´t implemented SSE3 and SSE4 opcodes yet onto it)


Btw... how do i multiply 4 Floats (or dword) in SSE2 register with one single Float ? And also how to add or subtract specific floats from a register ?

Ex:

Say i have in xmm0 this:

15 25 99 100

and i want to multiply each one of these values with let´s say 1.5

How do i do it ?


I´m asking to see how to port to SSE this:

Gx = A*(M1+M7-M3-M9) + D*(M4-M6)
Gy = A*(M1+M3-M7-M9) + B*(M2-M8)
Title: Re: Fast median algorithm
Post by: Siekmanski on August 06, 2020, 09:58:12 AM
Code: [Select]
15 25 99 100 in xmm0

movss  xmm1,FLT4(1.5) ; move a single float in xmm1
shufps xmm1,xmm1,0      ; xmm1 =  1.5,  1.5,   1.5,   1.5
mulps  xmm0,xmm1        ; xmm0 = 22.5, 37.5, 148.5, 150.0

subtract and add single floats:
Code: [Select]
subss xmm0,xmm1
addss xmm0,xmm1

subtract and add 4 floats at once:
Code: [Select]
subps xmm0,xmm1
addps xmm0,xmm1
Title: Re: Fast median algorithm
Post by: guga on August 06, 2020, 11:41:46 AM
Code: [Select]
15 25 99 100 in xmm0

movss  xmm1,FLT4(1.5) ; move a single float in xmm1
shufps xmm1,xmm1,0      ; xmm1 =  1.5,  1.5,   1.5,   1.5
mulps  xmm0,xmm1        ; xmm0 = 22.5, 37.5, 148.5, 150.0

subtract and add single floats:
Code: [Select]
subss xmm0,xmm1
addss xmm0,xmm1

subtract and add 4 floats at once:
Code: [Select]
subps xmm0,xmm1
addps xmm0,xmm1

Great. Thank you a lot. I´ll give a try in optimize one routine a little bit before go further with the crop_watermark routine.


Take a look at the result after using the normalization. It found the watermark from only 3 true different images among 107 similar ones.   :thumbsup: :thumbsup: :thumbsup:

So, to identify the watermark, we mainly need a couple of different images, indeed :) This will be awesome if we succeed to port it to work with videos :thumbsup: :thumbsup: :thumbsup:
(https://i.ibb.co/T8KQwdW/bvcxbtropical-bird-1390996.png) (https://ibb.co/T8KQwdW)

Note: The image is a bit too dark, because i didn´t enhanced the edges pixels. For now, it seems not to be necessary to create the shape :)
Title: Re: Fast median algorithm
Post by: hutch-- on August 06, 2020, 12:29:17 PM
Hi Guga,

I think I understand what you are doing and an edge detection algo is a lot more useful than just removing water marks, it can also be used for both smoothing rough images or sharpening slightly blurry ones. What I suggested originally was a technique that I have been using for years, close proximity cloning to get the field depth correct and match the texture where possible.

(http://masm32.com/private/both.jpg)

The original on the left, the modified on the right. I used some Russian software to remove the buttons and manually cloned the background to get rid of the insignia. My comment is you can do both as both techniques have their advantages and between them the watermarks you want to remove can be filled in with a texture that maintains the correct field depth.
Title: Re: Fast median algorithm
Post by: guga on August 06, 2020, 01:07:06 PM
Hi Guga,

I think I understand what you are doing and an edge detection algo is a lot more useful than just removing water marks, it can also be used for both smoothing rough images or sharpening slightly blurry ones. What I suggested originally was a technique that I have been using for years, close proximity cloning to get the field depth correct and match the texture where possible.

(http://masm32.com/private/both.jpg)

The original on the left, the modified on the right. I used some Russian software to remove the buttons and manually cloned the background to get rid of the insignia. My comment is you can do both as both techniques have their advantages and between them the watermarks you want to remove can be filled in with a texture that maintains the correct field depth.

Very, very good !


The buttons you removed with which software ? You told is a russian, but, what is it ? It seems that what it does is paint over the good areas. If it is this what it´s doing, i´ use a program called inpaint that does a similar thing.


I saw some awesome program that can restore old movies near perfection. It´s called Viva Film Restoration. Unfortunatelly, it uses Cuda, so i can´t be able to give a try.

Take a look on what it does. It´s amazing. Too bad it uses only cuda  :sad:

https://www.youtube.com/watch?v=_LBnlhJTDrE

Sure, that if we ever succeed to make this algorithm works (even though google designed it only to watermark removal), a simple adaptation can make it work wonders specially because what those other apps does (basically) is calculate the area of the edges where they want things to be removed or cleaned. The main problem i see on such apps is that they uses (in general) huge libraries such as opencv (or cuda, in this case) that are absolutely unnecessary if could simply rebuild the same functions from scratch and improve them to produce the very same result (or even better).

There are other apps  (an old one), such as Diamond Film restoration and PFClean  that also do a good job. The main problem of diamond and PFClean is that it is not perfect and it is sloooowww. :greensml: :greensml: :greensml:
Title: Re: Fast median algorithm
Post by: hutch-- on August 06, 2020, 03:08:26 PM
Hi Guga,

The app is called "Movavi Photo Editor 6". It does exactly what I did not need, tweaking pretty girls to look better but its fun to play with. I did the cloning with my very old and now buggy MicroGrafx Picture Publisher as it has exactly the right tools for doing this.

My suggestion was once you get the edge detection algo up and going, try sampling close to the watermark area to get the texture and field depth then selectively clone small parts to get rid of any blurring. I have seen robot watermark removal and some of it is real junk, a blurred mess that stands out very badly due to no compatibility of field depth.
Title: Re: Fast median algorithm
Post by: hutch-- on August 06, 2020, 03:16:58 PM
Something I should add, in reverse of normal video restoration, I have occasionally tried to make modern video look like old very slow frame rate film and while I have software that can add scratches, vignette and convert to either monochrome or sepia at 8 frames a second, its very hard to get it right. Hard to add in the flicker.
Title: Re: Fast median algorithm
Post by: guga on August 06, 2020, 04:24:49 PM
Hi Guga,

The app is called "Movavi Photo Editor 6". It does exactly what I did not need, tweaking pretty girls to look better but its fun to play with. I did the cloning with my very old and now buggy MicroGrafx Picture Publisher as it has exactly the right tools for doing this.

My suggestion was once you get the edge detection algo up and going, try sampling close to the watermark area to get the texture and field depth then selectively clone small parts to get rid of any blurring. I have seen robot watermark removal and some of it is real junk, a blurred mess that stands out very badly due to no compatibility of field depth.

Indeed. Sampling closer to the edge seems to be the way to go. The python version seems to do that using a technique called "Poisson reconstruction" that seems to do this. I tried Movavi once, but i quit. :greensml: :greensml: :greensml: personally i´m used to PaintShop pro for image edition, irfanview when i need to decrease the size without quality loss and Inpaint when i need to reconstruct some areas.

For video, i´m used to Sony vegas, plus VirtualDub and now, i´m trying to learn this huge ffmpeg  :greensml: :greensml:

If you want to make modern video looks like old, there´s a good forum where people teach doing this using avisynth (forum video help)

The guys there do a great job in video processing. Take a look a this example. (It´s for restoring old movies, not the opposite, but seems good, although i didn´t tested yet)
https://forum.videohelp.com/threads/369010-FILM9-Restoration-software-for-old-films-%288mm-Super8-16mm%29-and-video?
Title: Re: Fast median algorithm
Post by: hutch-- on August 06, 2020, 07:15:34 PM
 :biggrin:

> irfanview when i need to decrease the size without quality loss

I could lead you astray here, my JpgTool complete with sauce (whoops I mean source) does that just fine. With input from Marinus and Vortex the GDI+ code works really well.
Title: Re: Fast median algorithm
Post by: guga on August 06, 2020, 10:30:39 PM
:biggrin:

> irfanview when i need to decrease the size without quality loss

I could lead you astray here, my JpgTool complete with sauce (whoops I mean source) does that just fine. With input from Marinus and Vortex the GDI+ code works really well.
Wow. Tks you, steve

If you can, please release the source, i[ll like to see how to work with gditools to open image files and grab the pixels data.
Title: Re: Fast median algorithm
Post by: Siekmanski on August 06, 2020, 11:44:12 PM
Hi guga,

At your service,

Bare Minimum GDIplus code to save different Image Types and PixelFormat conversions:

http://masm32.com/board/index.php?topic=8483.msg92918#msg92918

You can skip "GdipCloneImage" in the sources, it's only used to process the same image over and over again.


Here is a routine to load a bitmap image to memory forced to 32bit ARGB pixels.
Now you have access to the raw ARGB pixels.  :cool:
No matter the format of the original bitmap image.

Code: [Select]
BitmapData struct                         
    dwWidth      dd ?   
    dwHeight     dd ?   
    Stride       dd ?   
    PixelFormat  dd ?   
    Scan0        dd ?   
    Reserved     dd ?
BitmapData ends

.data?
align 4
GDIplusBitmapData BitmapData <?>

.data
align 4
SourceBitmapData    dd NULL

.code
align 4
ReleaseBitmapMemory proc mem_ptr:DWORD

    .if mem_ptr != NULL
        invoke  VirtualFree,mem_ptr,0,MEM_RELEASE
        mov     mem_ptr,NULL
    .else
        mov     eax,1
    .endif
    ret
ReleaseBitmapMemory endp


align 4
LoadBitmapData proc uses ebx esi edi ImageName:DWORD,pImageData:DWORD

    mov         ReturnMessage,E_FAIL

    invoke      MultiByteToWideChar,CP_ACP,0,ImageName,-1,offset FilenameW,MAX_PATH-1
    invoke      GdipCreateBitmapFromFile,offset FilenameW,offset pImage
    test        eax,eax
    jnz         Close_Gdiplus

    invoke      GdipBitmapLockBits,pImage,NULL,ImageLockModeRead,PixelFormat32bppARGB,offset GDIplusBitmapData
    test        eax,eax
    jnz         Close_Image

    mov         esi,pImageData

    invoke      ReleaseBitmapMemory,dword ptr [esi]
    test        eax,eax
    jz          UnlockBitmap
       
    mov         eax,GDIplusBitmapData.dwWidth
    shl         eax,2
    mul         GDIplusBitmapData.dwHeight
    invoke      VirtualAlloc,0,eax,MEM_COMMIT or MEM_RESERVE,PAGE_READWRITE
    test        eax,eax
    jz          UnlockBitmap

    mov         dword ptr [esi],eax             ; save the virtual memory pointer
    mov         edi,eax                         ; pointer to the raw 32 bit bitmap data

CopyBitMap:   
    mov         esi,GDIplusBitmapData.Scan0     ; pointer to the bitmap data
   
    mov         ecx,GDIplusBitmapData.dwHeight
Height_lp:
    mov         edx,GDIplusBitmapData.dwWidth
    xor         ebx,ebx
Width_lp:
    mov         eax,dword ptr [esi+ebx]
    mov         dword ptr [edi],eax
    add         ebx,4
    add         edi,4
    dec         edx
    jnz         Width_lp
    add         esi,GDIplusBitmapData.Stride   
    dec         ecx
    jnz         Height_lp   

    mov         ReturnMessage,D3D_OK
UnlockBitmap:
    invoke      GdipBitmapUnlockBits,pImage,offset GDIplusBitmapData
Close_Image:
    invoke      GdipDisposeImage,pImage
Close_Gdiplus:
    mov         ecx,GDIplusBitmapData.dwWidth
    mov         edx,GDIplusBitmapData.dwHeight
Exit_LoadImage:
    mov         eax,ReturnMessage
    ret

LoadBitmapData endp

; function:
 invoke LoadBitmapData,TEXT_("Lena320.png"),offset SourceBitmapData


EDIT: added the missing BitmapData structure
Title: Re: Fast median algorithm
Post by: Siekmanski on August 07, 2020, 12:14:14 AM
This is how to create a pImage from the raw 32bit ARGB bitmap data in memory

Code: [Select]
I_pBits = the memory pointer to the raw 32bit ARGB bitmap data in memory
I_Stride = I_Width * 4

    invoke  GdipCreateBitmapFromScan0,I_Width,I_Height,I_Stride,PixelFormat32bppRGB,I_pBits,offset pImage

Now you can use "pImage" with one of the routines in GdiPlusColors.zip to save it in the format you like.
Title: Re: Fast median algorithm
Post by: hutch-- on August 07, 2020, 12:24:49 AM
Hi Guga,

Using the code by Marinus is a better choice, I converted his and Vortex's code to 64 bit MASM but it would be no joy to convert as the architecture is different between 32 and 64 bit. I have it ready as example code for 64 bit MASM but the procedures are in the library so they can be reused in 64 bit MASM code.
Title: Re: Fast median algorithm
Post by: hutch-- on August 07, 2020, 10:56:39 AM
Here is my most recent attempt to emulate old film, its an old stone church in the area where I live. Its almost OK but I have not yet worked out how to do the flickering that very old film had. I want it to look like the technology of the very old movie Nosferatu.

https://www.youtube.com/watch?v=4ZLL_P84IoI
Title: Re: Fast median algorithm
Post by: guga on August 07, 2020, 11:27:42 AM
Here is my most recent attempt to emulate old film, its an old stone church in the area where I live. Its almost OK but I have not yet worked out how to do the flickering that very old film had. I want it to look like the technology of the very old movie Nosferatu.
 :greensml: :greensml: :greensml: I love that movie.

Excellent work :)   :greenclp: :greenclp: :greenclp: :greenclp:

What program do you use to work with video ? If you use sony vegas  or premiere, there´s a filter called "Ignite Flicker" that you can play with.
(https://i.ibb.co/MctcSyj/gfdsg-Image1.png) (https://ibb.co/MctcSyj)

Or, you can also try with virtualdub. I remember seeing a flickering filter that works on it. I don´t remember the name because my HD where Vdub was stored, simply fried last month  :dazzled: :dazzled: :dazzled: :sad: :sad:

But, it do exists a filter for vdub that can also do the flickering for you. Other filters have those large black and white spots simulating a damage on the film which also brings more realism when you want to make a new video looks like and old one. Or even adding some effects of increasing the contrast and blurring a little a couple of frames also makes it looks like something filmed back in the 1910´s.

You could also try decrease the framerate in something in between 16 to 20 fps and later rebuild the video to 29.970.  This would delete some frames and simulate those "stops motions" that old movies does when people are moving, for example. That thing that happens when you see a guy with his hand raised and the next frame it immediately have his hands on the floor :mrgreen: :mrgreen: :mrgreen:

Title: Re: Fast median algorithm
Post by: guga on August 07, 2020, 11:34:30 AM
This is how to create a pImage from the raw 32bit ARGB bitmap data in memory

Code: [Select]
I_pBits = the memory pointer to the raw 32bit ARGB bitmap data in memory
I_Stride = I_Width * 4

    invoke  GdipCreateBitmapFromScan0,I_Width,I_Height,I_Stride,PixelFormat32bppRGB,I_pBits,offset pImage

Now you can use "pImage" with one of the routines in GdiPlusColors.zip to save it in the format you like.

Thank you a lot marinus and steve.

Marinus, what´s the format of the structure"GDIplusBitmapData" ? I can´t find it .
Title: Re: Fast median algorithm
Post by: hutch-- on August 07, 2020, 11:47:24 AM
Hi Guga,

I use MovAvi as its more or less useful. It started out a bit "cheap and cheerful" but has got a lot better over time (Software from Novosibirsk in Siberia). For doing the final finishing I use FFMPEG for image sizing, fine control of sharpening/smoothing and bitrate. If you look in the Microsoft 64 bit MASM »Tools & Toys sub forum I have a number of interfaces that use FFMPEG and all you need is to have FFMPEG in your path and they work fine.

RE : The cooked HDD, something I learn long ago was to put all of my HDDs behind a front case fan as disks last a lot longer if you don't cook them.
Title: Re: Fast median algorithm
Post by: hutch-- on August 07, 2020, 01:56:49 PM
Guga,

Here are some scaling algos. These are in 64 bit but easy enough to convert to 32 bit.

IMPORTANT : Use this option as it improves the quality. ====>>>> SetStretchBltMode,dDC,HALFTONE

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

scale_image_by_size proc hWin:QWORD,bitmap:QWORD,nWid:QWORD,nHgt:QWORD

    LOCAL hDC   :QWORD                                          ; original window DC
    LOCAL sDC   :QWORD                                          ; source bitmap DC
    LOCAL dDC   :QWORD                                          ; destination bitmap DC

    LOCAL srcWd :QWORD
    LOCAL srcHt :QWORD

    LOCAL nubmp :QWORD                                          ; new bmp handle
    LOCAL hOld  :QWORD

    rcall GetBmpSize,bitmap                                     ; get source bitmap size
    mov srcWd, rax
    mov srcHt, rcx

    mov hDC, rvcall(GetDC,hWin)                                 ; get the window DC
    mov sDC, rvcall(CreateCompatibleDC,hDC)                     ; create the source DC
    mov dDC, rvcall(CreateCompatibleDC,hDC)                     ; create the destination DC

    rcall SelectObject,sDC,bitmap                               ; select source bitmap
    mov hOld, rax

    mov nubmp, rvcall(CreateCompatibleBitmap,hDC,nWid,nHgt)     ; create the new bitmap
    rcall SelectObject,dDC,nubmp                                ; select destination bitmap

    rcall SetStretchBltMode,dDC,HALFTONE                        ; set mode to HALFTONE
    rcall SetBrushOrgEx,dDC,0,0,0
    invoke StretchBlt,dDC,0,0,nWid,nHgt, \
                      sDC,0,0,srcWd,srcHt,SRCCOPY               ; scale source to destination

    rcall SelectObject,sDC,hOld                                 ; re select old object

    rcall DeleteDC,sDC                                          ; deallocate the device contexts
    rcall DeleteDC,dDC
    rcall ReleaseDC,hWin,hDC

    mov rax, nubmp                                              ; return new bitmap handle

    ret

scale_image_by_size endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

By percent.

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

scale_image_by_percent proc hWin:QWORD,bitmap:QWORD,percnt:QWORD

    LOCAL hDC   :QWORD                                          ; original window DC
    LOCAL sDC   :QWORD                                          ; source bitmap DC
    LOCAL dDC   :QWORD                                          ; destination bitmap DC

    LOCAL srcWd :QWORD
    LOCAL srcHt :QWORD

    LOCAL nWid  :QWORD                                          ; width of new bmp
    LOCAL nHgt  :QWORD                                          ; height of new bmp
    LOCAL nubmp :QWORD                                          ; new bmp handle
    LOCAL hOld  :QWORD

    rcall GetBmpSize,bitmap                                     ; get source bitmap size
    mov srcWd, rax
    mov srcHt, rcx

    mov hDC, rvcall(GetDC,hWin)                                 ; get the window DC
    mov sDC, rvcall(CreateCompatibleDC,hDC)                     ; create the source DC
    mov dDC, rvcall(CreateCompatibleDC,hDC)                     ; create the destination DC

    rcall SelectObject,sDC,bitmap                               ; select source bitmap
    mov hOld, rax

    mov nWid, rvcall(getpercent,srcWd,percnt)                   ; calculate new width
    mov nHgt, rvcall(getpercent,srcHt,percnt)                   ; calculate new height

    add nWid, 1                                                 ; correct for zero based index
    add nHgt, 1

    mov nubmp, rvcall(CreateCompatibleBitmap,hDC,nWid,nHgt)     ; create the new bitmap
    rcall SelectObject,dDC,nubmp                                ; select destination bitmap

    rcall SetStretchBltMode,dDC,HALFTONE                        ; set mode to HALFTONE
    rcall SetBrushOrgEx,dDC,0,0,0
    invoke StretchBlt,dDC,0,0,nWid,nHgt, \
                      sDC,0,0,srcWd,srcHt,SRCCOPY               ; scale source to destination

    rcall SelectObject,sDC,hOld                                 ; re select old object

    rcall DeleteDC,sDC                                          ; deallocate the device contexts
    rcall DeleteDC,dDC
    rcall ReleaseDC,hWin,hDC

    mov rax, nubmp                                              ; return new bitmap handle

    ret

scale_image_by_percent endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

getpercent

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

getpercent proc src:QWORD,pcnt:QWORD

    mov r10, pcnt
    shl src, 10
    shl pcnt, 10

    fn intdiv,src,100
    fn intmul,rax,r10

    shr rax, 10

    ret

getpercent endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

Title: Re: Fast median algorithm
Post by: guga on August 07, 2020, 02:59:18 PM
Great. Tks Steve


Btw, marinus and Steve


Marinus, if you would like to add more equates to GdiPlus header. here are some more:

Hvec and Webp guids and also EMF, ico

IMAGE_BMP     0
IMAGE_JPG      1
IMAGE_GIF      2
IMAGE_EMF      3
IMAGE_WMF      4
IMAGE_TIF      5
IMAGE_PNG      6
IMAGE_ICO      7
IMAGE_HEIF     8
IMAGE_WEBP     9


[CLSID_BMP: D$ 0557CF400, W$ 01A04, 011D3,  B$ 09A, 073, 0, 0, 0F8, 01E, 0F3, 02E
 CLSID_JPG: D$ 0557CF401, W$ 01A04, 011D3,  B$ 09A, 073, 0, 0, 0F8, 01E, 0F3, 02E
 CLSID_GIF: D$ 0557CF402, W$ 01A04, 011D3,  B$ 09A, 073, 0, 0, 0F8, 01E, 0F3, 02E
 CLSID_TIF: D$ 0557CF405, W$ 01A04, 011D3,  B$ 09A, 073, 0, 0, 0F8, 01E, 0F3, 02E
 CLSID_PNG: D$ 0557CF406, W$ 01A04, 011D3,  B$ 09A, 073, 0, 0, 0F8, 01E, 0F3, 02E
 CLSID_ICO: D$ 0557CF407, W$ 01A04, 011D3,  B$ 09A, 073, 0, 0, 0F8, 01E, 0F3, 02E]

[CLSID_EMF: D$ 0557CF403, W$ 01A04, 011D3,  B$ 09A, 073, 0, 0, 0F8, 01E, 0F3, 02E]
[CLSID_WMF: D$ 0557CF404, W$ 01A04, 011D3,  B$ 09A, 073, 0, 0, 0F8, 01E, 0F3, 02E]
[CLSID_HEIF: D$ 0557CF408, W$ 01A04, 011D3,  B$ 09A, 073, 0, 0, 0F8, 01E, 0F3, 02E]
[CLSID_WEBP: D$ 0557CF409, W$ 01A04, 011D3,  B$ 09A, 073, 0, 0, 0F8, 01E, 0F3, 02E]

For the PixelFormat, i found these

value in hexa             Corresponding Guid
030101                      GUID_WICPixelFormat1bppIndexed
010212                      GUID_WICPixelFormat2bppIndexed
030402                      GUID_WICPixelFormat4bppIndexed
030803                      GUID_WICPixelFormat8bppIndexed
0811                          GUID_WICPixelFormat8bppGray
0101004                    GUID_WICPixelFormat16bppGray
021005                      GUID_WICPixelFormat16bppBGR555
021006                      GUID_WICPixelFormat16bppBGR565
061007                      GUID_WICPixelFormat16bppBGRA5551
021808                      GUID_WICPixelFormat24bppBGR
021810                      GUID_WICPixelFormat24bppRGB
022009                      GUID_WICPixelFormat32bppBGR
026200A                    GUID_WICPixelFormat32bppBGRA
0E200B                      GUID_WICPixelFormat32bppPBGRA
0200F                        GUID_WICPixelFormat32bppCMYK
010300C                    GUID_WICPixelFormat48bppBGRFixedPoint
034400D                    GUID_WICPixelFormat64bppBGRAFixedPoint
01C400E                    GUID_WICPixelFormat64bppBGRAFixedPoint

Used guids values from above
Code: [Select]
DEFINE_GUID IMGFMT_WEBP, 0x0B96B3CB7,0x0728,0x11d3,0x9d,0x7b,0x00,0x00,0xf8,0x1e,0xf3,0x2e);
DEFINE_GUID IMGFMT_HEIF, 0x0B96B3CB6,0x0728,0x11d3,0x9d,0x7b,0x00,0x00,0xf8,0x1e,0xf3,0x2e);

GUID_WICPixelFormatDontCare dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 0
GUID_WICPixelFormat1bppIndexed dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 1
GUID_WICPixelFormat2bppIndexed dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 2
GUID_WICPixelFormat4bppIndexed dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 3
GUID_WICPixelFormat8bppIndexed dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 4
GUID_WICPixelFormatBlackWhite dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 5
GUID_WICPixelFormat2bppGray dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 6
GUID_WICPixelFormat4bppGray dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 7
GUID_WICPixelFormat8bppGray dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 8
GUID_WICPixelFormat16bppBGR555 dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 9
GUID_WICPixelFormat16bppBGR565 dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 0Ah
GUID_WICPixelFormat16bppBGRA5551 dd 5EC7C2Bh
dw 0F1E6h
dw 4961h
db 0ADh, 46h, 0E1h, 0CCh, 81h, 0Ah, 87h, 0D2h
GUID_WICPixelFormat16bppGray dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 0Bh
GUID_WICPixelFormat24bppBGR dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 0Ch
GUID_WICPixelFormat24bppRGB dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 0Dh
GUID_WICPixelFormat32bppBGR dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 0Eh
GUID_WICPixelFormat32bppBGRA dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 0Fh
GUID_WICPixelFormat32bppPBGRA dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 10h
GUID_WICPixelFormat32bppGrayFloat dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 11h
GUID_WICPixelFormat32bppRGBA dd 0F5C7AD2Dh
dw 6A8Dh
dw 43DDh
db 0A7h, 0A8h, 0A2h, 99h, 35h, 26h, 1Ah, 0E9h
GUID_WICPixelFormat32bppPRGBA dd 3CC4A650h
dw 0A527h
dw 4D37h
db 0A9h, 16h, 31h, 42h, 0C7h, 0EBh, 0EDh, 0BAh
GUID_WICPixelFormat48bppRGB dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 15h
GUID_WICPixelFormat48bppBGR dd 0E605A384h
dw 0B468h
dw 46CEh
db 0BBh, 2Eh, 36h, 0F1h, 80h, 0E6h, 43h, 13h
GUID_WICPixelFormat64bppRGBA dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 16h
GUID_WICPixelFormat64bppBGRA dd 1562FF7Ch
dw 0D352h
dw 46F9h
db 97h, 9Eh, 42h, 97h, 6Bh, 79h, 22h, 46h
GUID_WICPixelFormat64bppPRGBA dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 17h
GUID_WICPixelFormat64bppPBGRA dd 8C518E8Eh
dw 0A4ECh
dw 468Bh
db 0AEh, 70h, 0C9h, 0A3h, 5Ah, 9Ch, 55h, 30h
GUID_WICPixelFormat16bppGrayFixedPoint dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 13h
GUID_WICPixelFormat48bppBGRFixedPoint dd 49CA140Eh
dw 0CAB6h
dw 493Bh
db 9Dh, 0DFh, 60h, 18h, 7Ch, 37h, 53h, 2Ah
GUID_WICPixelFormat32bppCMYK dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 1Ch
GUID_WICPixelFormat64bppRGBAFixedPoint dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 1Dh
GUID_WICPixelFormat64bppBGRAFixedPoint dd 356DE33Ch
dw 54D2h
dw 4A23h
db 0BBh, 4, 9Bh, 7Bh, 0F9h, 0B1h, 0D4h, 2Dh
GUID_WICPixelFormat64bppRGBFixedPoint dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 40h
GUID_WICPixelFormat64bppRGBAHalf dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 3Ah
GUID_WICPixelFormat64bppRGBHalf dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 42h
GUID_WICPixelFormat16bppGrayHalf dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 3Eh
GUID_WICPixelFormat32bppGrayFixedPoint dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 3Fh
GUID_WICPixelFormat64bppCMYK dd 6FDDC324h
dw 4E03h
dw 4BFEh
db 0B1h, 85h, 3Dh, 77h, 76h, 8Dh, 0C9h, 1Fh


Btw...The above  guids and values (including yours)  are accessed in GdiPlus at: GdipGetImageEncodersSize and GdipGetImageDecodersSize apis that leads to a internal function called "InitializeBuiltinCodecs".


Those data are stored as a array of structures pointed from inside that InitializeBuiltinCodecs function. I presume it is something like: ImageCodecInfo. Not sure it if is documented, but, anyway, i named the structure to what it seems to be doing as:

Code: [Select]
GugaImageCodecInfo struc
        CLSID dd ?
        FormatID dd ?
        CodecName dd ?
        FormatDescription dd ?
        FilenameExtension dd ?
        MimeType dd ?
        Version dd 4 dup(?)
        Signature dd ?
        SignatureReserved dd ?
        pInstanceFunction dd ?
GugaImageCodecInfo ends


The above structure, btw, is almost equal to this:
https://www.sagara.net/2010/01/25/imagecodecinfo-encoder-and-decoder-information/
Title: Re: Fast median algorithm
Post by: Siekmanski on August 07, 2020, 05:40:48 PM
Marinus, what´s the format of the structure"GDIplusBitmapData" ? I can´t find it .

Here it is.  :biggrin:

Code: [Select]
BitmapData struct                         
    dwWidth      dd ?   
    dwHeight     dd ?   
    Stride       dd ?   
    PixelFormat  dd ?   
    Scan0        dd ?   
    Reserved     dd ?
BitmapData ends

GDIplusBitmapData BitmapData <?>
Title: Re: Fast median algorithm
Post by: guga on August 07, 2020, 07:12:28 PM
Hi marinus. Tks :)


Btw, i succeed to load and save webp files (unfortunatelly, not with gdiplus yet. Don´t know how to access those interfaces for webp or hevc using gdiplus) I've done it using libwebp library from google.

The routine i did was (I´ll change the name later, this was just a test to load and save this webp stuff):

Code: [Select]
; https://www.autohotkey.com/boards/viewtopic.php?style=17&t=66335
; https://git.cs.usask.ca/SumatraPDF/SumatraPDF/blob/master/src/utils/WebpReader.cpp
; https://chromium.googlesource.com/webm/libwebp/+/master/src/webp/decode.h
; https://developers.google.com/speed/webp/docs/api

Proc DecodeWebpFile:
    Arguments @pFileName
    Local @hFile, @FileSize, @pImgWidth, @pImgHeight, @pBits, @TmpOpenedFile

    lea ebx D@TmpOpenedFile ; Buffer to hold the loaded webp in memory
    lea ecx D@FileSize ; and store it´s size
    call ReadOpenedFile 0, D@pFileName, ebx, ecx ; A function to open the webp file and read it´s contents.

    ; Just to get some info of the web p image. It´s not needed since the width and height will already be retrieved with the function below
;    lea ebx D@pImgWidth
;    lea ecx D@pImgHeight
;    C_call 'libwebp.WebPGetInfo' eax, D@FileSize, ebx, ecx

    ; This function gets the Pixel data from the webp image and also it´s width and height
    lea ebx D@pImgWidth
    lea ecx D@pImgHeight
    C_call 'libwebp.WebPDecodeBGRA' D@TmpOpenedFile, D@FileSize, ebx, ecx ; libwebp library is in cdecl calling convention. The stack needs to be adjusted, that´s why i´m using RosAsm "C_call" macro that simply add XXX bytes to the stack after the call.
    mov D@pBits eax ; Copy the image pixels to export to other formats

    call CreateImageFromMemory D@pBits, D@pImgWidth, D@pImgHeight, pImage ; Marinus routine to create a pImage from the raw 32bit ARGB bitmap data in memory
    call SaveNonIndexedImage D$pImage, {B$ "GugaWebP.png", 0}, Image_PNG, PixelFormat32bppRGB
    call SaveJpgQualityImage D$pImage, {B$ "GugaWebP.jpg", 0}, 40

    C_call 'libwebp.WebPFree' D@pBits ; Must release the data loaded from WebPDecodeBGRA
    call 'RosMem.VMemFree' D@TmpOpenedFile ; and released our allocated memory

EndP

______________

;;
I_pBits = the memory pointer to the raw 32bit ARGB bitmap data in memory
I_Stride = I_Width * 4
;;

Proc CreateImageFromMemory:
    Arguments @ImgBits, @ImageWidth, @ImageHeight, @pImage
    Local @Stride

    mov eax D@ImageWidth | shl eax 2
    call 'gdiplus.GdipCreateBitmapFromScan0' D@ImageWidth, D@ImageHeight, eax, PIXELFORMAT_32BPPARGB, D@ImgBits, D@pImage

EndP
____________________

; Just a function to open the file onto a memory buffer
[NumberOfReadBytes: 0]
Proc ReadOpenedFile:
    Arguments @Adresseem, @Filename, @pFileData, @pFileLenght
    Local @hFile, @hFileSize, @MemFile
    Uses edi, ecx, edx

    call 'KERNEL32.CreateFileA' D@Filename, &GENERIC_READ,
                                &FILE_SHARE_READ+&FILE_SHARE_WRITE, 0, &OPEN_EXISTING,
                                &FILE_ATTRIBUTE_NORMAL, 0
    If eax = &INVALID_HANDLE_VALUE
        call 'USER32.MessageBoxA' 0, {B$ "File cannot be opened!", 0}, {'Error', 0}, &MB_OK__&MB_ICONWARNING__&MB_SYSTEMMODAL
        xor eax eax | ExitP
    End_If
    mov D@hFile eax

    call 'KERNEL32.GetFileSize' eax, 0
    mov D@hFileSize eax
    On eax = 0, ExitP
    mov edi D@pFileLenght

    add eax 10 | mov edx eax
    mov D$edi edx

     ; Allocate enough memory
    mov D@MemFile 0 | lea eax D@MemFile
    call 'RosMem.VMemAlloc' eax, edx
    mov edi D@pFileData
    mov D$edi eax


    call 'KERNEL32.ReadFile' D@hFile, eax,
                             D@hFileSize, NumberOfReadBytes, 0

    call 'KERNEL32.CloseHandle' D@hFile

EndP



I found how to do it reading these:

Code: [Select]
webpUrl := "https://upload.wikimedia.org/wikipedia/commons/b/b2/Vulphere_WebP_OTAGROOVE_demonstration_2.webp"
libwebp32Url := "https://s3.amazonaws.com/resizer-dynamic-downloads/webp/0.5.2/x86/libwebp.dll"
libwebp64Url := "https://s3.amazonaws.com/resizer-dynamic-downloads/webp/0.5.2/x86_64/libwebp.dll"
filePath := A_ScriptDir . "\test.webp"
bitness := A_PtrSize*8
lib := A_ScriptDir . "\libwebp" . bitness . ".dll"

if !FileExist(filePath)
   URLDownloadToFile, % webpUrl, % filePath

if !FileExist(lib)
   URLDownloadToFile, % libwebp%bitness%Url, % lib

hBitmap := HBitmapFromWebP(lib, filePath, width, height)
Gui, Margin, 0, 0
Gui, Add, Pic, w600 h-1, HBITMAP:%hBitmap%
Gui, Show
Return

GuiClose() {
   ExitApp
}

HBitmapFromWebP(libwebp, WebpFilePath, ByRef width, ByRef height) {
   file := FileOpen(WebpFilePath, "r")
   len := file.RawRead(buff, file.Length)
   file.Close()
   if !len
      throw Exception("Failed to read the image file")
   
   if !hLib := DllCall("LoadLibrary", Str, libwebp, Ptr)
      throw Exception("Failed to load library. Error:" . A_LastError)
   
   if !pBits := DllCall(libwebp . "\WebPDecodeBGRA", Ptr, &buff, Ptr, len, IntP, width, IntP, height) {
      DllCall("FreeLibrary", Ptr, hLib)
      throw Exception("Failed to decode the image file")
   }
   
   oGDIp := new GDIp
   pBitmap := oGDIp.CreateBitmapFromScan0(width, height, pBits)
   hBitmap := oGDIp.CreateHBITMAPFromBitmap(pBitmap)
   DllCall(libwebp . "\WebPFree", Ptr, pBits)
   oGDIp.DisposeImage(pBitmap)
   DllCall("FreeLibrary", Ptr, hLib)
   Return hBitmap
}

class GDIp   {
   __New() {
      if !DllCall("GetModuleHandle", Str, "gdiplus", Ptr)
         DllCall("LoadLibrary", Str, "gdiplus")
      VarSetCapacity(si, A_PtrSize = 8 ? 24 : 16, 0), si := Chr(1)
      DllCall("gdiplus\GdiplusStartup", UPtrP, pToken, Ptr, &si, Ptr, 0)
      this.token := pToken
   }
   
   __Delete()  {
      DllCall("gdiplus\GdiplusShutdown", Ptr, this.token)
      if hModule := DllCall("GetModuleHandle", Str, "gdiplus", Ptr)
         DllCall("FreeLibrary", Ptr, hModule)
   }
   
   CreateBitmapFromScan0(Width, Height, pBits, PixelFormat := 0x26200A, Stride := "") {
      if !Stride {
         bpp := (PixelFormat & 0xFF00) >> 8
         Stride := ((Width * bpp + 31) & ~31) >> 3
      }
      DllCall("gdiplus\GdipCreateBitmapFromScan0", Int, Width, Int, Height
                                                 , Int, Stride, Int, PixelFormat
                                                 , Ptr, pBits, PtrP, pBitmap)
      Return pBitmap
   }
   
   CreateHBITMAPFromBitmap(pBitmap, Background=0xffffffff) {
      DllCall("gdiplus\GdipCreateHBITMAPFromBitmap", Ptr, pBitmap, PtrP, hbm, Int, Background)
      return hbm
   }
   
   DisposeImage(pBitmap) {
      return DllCall("gdiplus\GdipDisposeImage", Ptr, pBitmap)
   }
}


And this...

Code: [Select]
/* Copyright 2015 the SumatraPDF project authors (see AUTHORS file).
   License: Simplified BSD (see COPYING.BSD) */

#include "BaseUtil.h"
#include "WebpReader.h"

#ifndef NO_LIBWEBP

#include <webp/decode.h>
using namespace Gdiplus;

namespace webp {

// checks whether this could be data for a WebP image
bool HasSignature(const char *data, size_t len)
{
    return len > 12 && str::StartsWith(data, "RIFF") && str::StartsWith(data + 8, "WEBP");
}

Size SizeFromData(const char *data, size_t len)
{
    Size size;
    WebPGetInfo((const uint8_t *)data, len, &size.Width, &size.Height);
    return size;
}

Bitmap *ImageFromData(const char *data, size_t len)
{
    int w, h;
    if (!WebPGetInfo((const uint8_t *)data, len, &w, &h))
        return nullptr;

    Bitmap bmp(w, h, PixelFormat32bppARGB);
    Rect bmpRect(0, 0, w, h);
    BitmapData bmpData;
    Status ok = bmp.LockBits(&bmpRect, ImageLockModeWrite, PixelFormat32bppARGB, &bmpData);
    if (ok != Ok)
        return nullptr;
    if (!WebPDecodeBGRAInto((const uint8_t *)data, len, (uint8_t *)bmpData.Scan0, bmpData.Stride * h, bmpData.Stride))
        return nullptr;
    bmp.UnlockBits(&bmpData);

    // hack to avoid the use of ::new (because there won't be a corresponding ::delete)
    return bmp.Clone(0, 0, w, h, PixelFormat32bppARGB);
}

}

#else

namespace webp {
    bool HasSignature(const char *data, size_t len) { UNUSED(data); UNUSED(len); return false; }
    Gdiplus::Size SizeFromData(const char *data, size_t len) { UNUSED(data); UNUSED(len); return Gdiplus::Size(); }
    Gdiplus::Bitmap *ImageFromData(const char *data, size_t len) { UNUSED(data); UNUSED(len); return nullptr; }
}

#endif


References:
https://www.autohotkey.com/boards/viewtopic.php?style=17&t=66335
https://git.cs.usask.ca/SumatraPDF/SumatraPDF/blob/master/src/utils/WebpReader.cpp
https://chromium.googlesource.com/webm/libwebp/+/master/src/webp/decode.h
https://developers.google.com/speed/webp/docs/api
https://cpp.hotexamples.com/examples/-/Bitmap/-/cpp-bitmap-class-examples.html

Note: Attached are the webp file and the one i converted to jpg. Also it contains the libwebp.dll library.
Title: Re: Fast median algorithm
Post by: Siekmanski on August 07, 2020, 07:19:43 PM
Marinus, if you would like to add more equates to GdiPlus header. here are some more:

Hi guga,

The code was designed to keep it as small as possible.

It doesn't need all the Format and Type GUIDS in memory.
Just one GUID for all the Image Types, and one GUID for the JPG EncoderQuality.

Here comes the trick:
The first byte of the first dword of the Image Types GUID, represents the Image Type.
Image_BMP equ 0
Image_JPG equ 1
Image_GIF equ 2
Image_TIF equ 5
Image_PNG equ 6
So fill the byte with the desired Image Type and you're good to go.

These Image types are the only ones of my interest and always available.
But you can also add the other Types and Formats if desired. ( Be sure they are available.... )

In my code you don't need all the enumerate functions to get all the GUIDS or use the other wrapper functions to fill in the values for you.
And we are not wasting memory for all the Info structures that those wrapper functions need.

That's the reason I called it "Bare Minimum GDIplus code to save different Image Types and PixelFormat conversions"  :cool:
Title: Re: Fast median algorithm
Post by: guga on August 07, 2020, 07:34:09 PM
yeah, i saw it. Tinny functions that do the job correctly :thumbsup: :thumbsup:

I was curious about this gdiplus and gave a test (Never used before  :greensml: :mrgreen:). Indeed it seems better to work with then FreeImage, SDL, etc. I´ll later try to convert this routines for the watermark remover function and replace the Api i was using to load the images to gdiplus.

I tested also this webp format a couple of hours agos while i was trying to learsn and understand your code for gdiplus. It seems that webp is also very handy. The size of the images are really small and i couldn't´t see that much loss of quality.
Title: Re: Fast median algorithm
Post by: Siekmanski on August 07, 2020, 07:46:19 PM
WebP images will not load with GdiPlus
Maybe you need to install a decoder/encoder for GdiPlus???
I say this because you provided the "IMAGE_WEBP     9" Image Type.
Title: Re: Fast median algorithm
Post by: guga on August 07, 2020, 07:51:57 PM
WebP images will not load with GdiPlus
Maybe you need to install a decoder/encoder for GdiPlus???
I say this because you provided the "IMAGE_WEBP     9" Image Type.

That´s weird because gdiplus.dll contain the routines to load and save webp and heic. That´s where i got´those data for the guid and the other variables. Take a look on the images below.It´s from my gdiplus in windows10

(https://i.ibb.co/4V7VxP3/Image3.png) (https://ibb.co/4V7VxP3)

(https://i.ibb.co/QvVQ4WM/Image3.png) (https://ibb.co/QvVQ4WM)

The routines to load and save heic and webp are inside gdiplus. I don´t know how to load them, but they are built internally

Title: Re: Fast median algorithm
Post by: Siekmanski on August 07, 2020, 07:57:04 PM
It doesn't on my computer, Win8.1
Then we have to update the Masm gdiplus.lib and gdiplus.inc again...  :biggrin:

Title: Re: Fast median algorithm
Post by: guga on August 09, 2020, 08:32:04 AM
Hi Marinus

One question. How do we calculate the laplacian operator for this image processing ?

I saw at http://masm32.com/board/index.php?topic=7420.msg81182#msg81182 that Rui did a Laplacian computation for a determinant of a matrix 3x3. Is it the same thing as what we are doing ? Can we use his function to compute the laplacian to detect edges too ?

If not... how can we estimate the correct laplacian using  2-D Log function with Gaussian standard deviation as described here ?
https://www.javatpoint.com/dip-concept-of-edge-detection

And most important..what a hell is this laplacian ? :mrgreen: :mrgreen: :mrgreen: :mrgreen:

I gave a test on this logarithm function using wolframalpha and came onto this:

If Log(x) = 1 . So our laplacian of x, y pixels, right ? So, at the end, the laplace (not normalized) can only results in values varying from 0 to 255 to make it works similar as what Sobel does)

Laplace = Log(x,y) = (1-(z)/(2*k)) * (exp(-(z)/(2*k))) * (-1/(pi*k))

If the final laplace/Log(x,y) = 1, then we have:

1 = (1-(z)/(2*k)) * (exp(-(z)/(2*k))) * (-1/(pi*k))

where z = x^2+y^2
k = standard deviation of the whole image

So...when the result of log(x,y) = 1  Wolfram Alpha gave as a solution this:

z = -2 (k W(-e k pi) - k)

1 = (1-(z)/(2*k)) * (exp(-(z)/(2*k))) * (-1/(pi*k))

https://www.wolframalpha.com/input/?i=1+%3D+%281-%28z%29%2F%282*k%29%29+*+%28exp%28-%28z%29%2F%282*k%29%29%29+*+%28-1%2F%28pi*k%29%29 (https://www.wolframalpha.com/input/?i=1+%3D+%281-%28z%29%2F%282*k%29%29+*+%28exp%28-%28z%29%2F%282*k%29%29%29+*+%28-1%2F%28pi*k%29%29)

Where W = product Log function (whatever that is  :greensml: :greensml: :greensml:)

Also, if we consider log(x,y) = 2 we then have this formula:

z = -2 (k W(-2 exp(k pi)) - k)
2 = (1-(z)/(2*k)) * (exp(-(z)/(2*k))) * (-1/(pi*k))


If log(x,y) = 3, we have this:
z = -2 (k W(-3 e k pi) - k)
3 = (1-(z)/(2*k)) * (exp(-(z)/(2*k))) * (-1/(pi*k))


So, it seems to me that for log(x,y) = N we have

z = -2 (k W(-N e k pi) - k)


z = -2*k (W(-N e k pi) - 1)

where z  = x^2+y^2


So, in theory, all we need to know is how to calculate this W stuff, right ?

Knowing that "k" is a constant for each image (it´s simply their standard deviation), then the values of this W function can all be pre calculated and put onto a table from 0 to 255, right ?

Since, "k" and pi are constants, and we also have the resultant N varying from 0 to 255, we can build a 256 dword table containing the results of W(-N e k pi), right ?

If is that so, to calculate the laplace (the exact values and not the approximations), we could 1st solve this W function and later, solve z for each values from 0 to 255, like this:


Val0 = -2*k (W(-0 e k pi) - 1) => related to Laplace Val = 0
Val1 = -2*k (W(-1 e k pi) - 1) => related to Laplace Val = 1
Val2 = -2*k (W(-2 e k pi) - 1) => related to Laplace Val = 2
Val3 = -2*k (W(-3 e k pi) - 1) => related to Laplace Val = 3
(...)


Since, z varies according to the value of each pixel at x, y pos, we could simply point the result of x^2+y^2 to the value that is more closer to the Val1, Val2..... right ?

On this way we would create only 256 values for this "-2*k (W(-N e k pi) - 1)" stuff and compare the values of x^2+y^2 to them to get the proper laplace value.

The question is how do we calculate this W stuff ????


Title: Re: Fast median algorithm
Post by: Siekmanski on August 09, 2020, 09:54:32 AM
 :thumbsup:
You are going full steam ahead.  :cool:
I'm not that far yet, to dive into other edge dectection operators and techniques.
Sounds interesting, and a Laplacian filter as a preprocessing step maybe possible.
First I want the Sobel routine to be ready, than we will see what else is cool to implement.

I have almost finished the Sobel operator.
And I think it will be as fast as lightning....
It reads 16 pixels with 4 read instructions.
And it saves 4 Sobel G values as pixels with 2 write instruction. ( a block of 2 by 2 pixels )
It's all 4 lane parallel SSE2 code.
4 lane convolution at once without multiplications, and other tricks.
For example it uses a 4 by 4 Pixel Matrix, so we can abuse SIMD to the max.
The advantage is, a 4 by 4 Matrix can hold 4 single Sobel operators without the need of loading the pixels separately.
Yes, this one was a brain teaser, but I knew it should be possible.
Still have to test it on real data, hope it works out ( on paper it does.... )  :eusa_pray:
Title: Re: Fast median algorithm
Post by: Siekmanski on August 09, 2020, 09:29:49 PM
Hi guga,

We have to calculate with square roots for the magnitudes.
How much precision do we need?

We can calculate with high precision at the cost of +/- 39 clock cycles.
Or we can calculate the reciprocal square roots with 12 bit precision at +/- 8 clock cycles.
Or do it the rude way ( not very accurate ) and make the X Y gradients absolute values and add them together at +/- 3 clock cycles.

I myself would go for the reciprocal square roots, it's more than enough precision we need for the pixel bytes, which are actually rounded to natural numbers.
Title: Re: Fast median algorithm
Post by: guga on August 10, 2020, 12:23:06 AM
Hi guga,

We have to calculate with square roots for the magnitudes.
How much precision do we need?

We can calculate with high precision at the cost of +/- 39 clock cycles.
Or we can calculate the reciprocal square roots with 12 bit precision at +/- 8 clock cycles.
Or do it the rude way ( not very accurate ) and make the X Y gradients absolute values and add them together at +/- 3 clock cycles.

I myself would go for the reciprocal square roots, it's more than enough precision we need for the pixel bytes, which are actually rounded to natural numbers.

Although 39 clock cycles is very very fast, a precision of 12 bits seems to me good enough to make the result be accurate. I saw a small issue when it concerns the strategy on what operator to choose.  Since we normalized the operator value to stays withing the limits, i started to test the different operators and check the results.

With sobel we have the edges correctly detected, but it has some small gaps in  some pixels, which is hard to trace. I then tested Scharr and the result was the opposite as sobel. I mean, it detected the edges a bit more accurated, but on the other hand, produced a bit more noise outside the edges.

I´m trying to figure it out what exact values we can use as a good edge detector. You are using fixed values for sobel, right ? I mean those 0-1 and 0-2.  I extended your algo a little bit and allowed to insert whatever values we want. Doing this way we can try to track what values (those ValA and ValB i posted previously) generates a better result. Or...we can try to calculate the reverse operation that sobel/Scharr/Prewitt are using in order to we find the correct values of ValA and ValB.

We only need 2 values in fact, to write a edge detector routine. 

I made a workaround on these problems writing a routine to Estimate the Threshold automatically. It works in 90% of the cases. The main problem relies on watermarks whose pixels that forms the edges are too dark. For example, i saw pixels whose values are in between 14 and 18 that still belongs to edges. The new routine calculates the median of the darkers values of the image (after the sobel routines do their job) and after then once it is found, it calculates the average of the remaining pixels (the ones too dark closer to black) and tries to exclude which pixels  are heavily distributed (pixels from 0 to 3 represents more then 90% of the whole image, in general).

I´m just giving a rest, because last night i started the routine to test the laplacian operator. Currently, i´m trying to create a routine to compute that damn Lambert W-Function existant in wolfram alpha. I´m close to a result, but i got stuck trying to create a routine in SSE to calculate log and exp values. This laplacian stuff will be a pain to implement, but i think we can make it work :)
Title: Re: Fast median algorithm
Post by: Siekmanski on August 10, 2020, 12:51:54 AM
Just take a moment rest, then I have a chance to keep up with you.  :biggrin:

Log and Exp routines are not available in SSE, we have to create them ourselves.

Did a test with the machine image from wikipedia's Sobel example.
The principal of the 16 pixel sse2 parallel sobel routine works, but there is some fine tuning needed.
There are misalignments on the Y axis, as you can see in the attachment.
Title: Re: Fast median algorithm
Post by: guga on August 10, 2020, 01:06:20 AM
Just take a moment rest, then I have a chance to keep up with you.  :biggrin:

Log and Exp routines are not available in SSE, we have to create them ourselves.

Did a test with the machine image from wikipedia's Sobel example.
The principal of the 16 pixel sse2 parallel sobel routine works, but there is some fine tuning needed.
There are misalignments on the Y axis, as you can see in the attachment.

Hi Marinus. The app doesn´t open here.
Title: Re: Fast median algorithm
Post by: Siekmanski on August 10, 2020, 01:09:37 AM
did you depack it to your HD?
Title: Re: Fast median algorithm
Post by: guga on August 10, 2020, 01:13:47 AM
Yep. But i found it. It was looking for the image in a "Info' directory. I simply create this dir and set the machine.png to there

I see the cross lines in sobel now. The image in not aligned
Title: Re: Fast median algorithm
Post by: Siekmanski on August 10, 2020, 01:15:59 AM
(http://members.home.nl/siekmanski/SobelTest.png)
Title: Re: Fast median algorithm
Post by: guga on August 10, 2020, 01:19:42 AM
This is what shows to me

(https://i.ibb.co/ZTZxPzQ/Image22aa.png) (https://ibb.co/ZTZxPzQ)

Title: Re: Fast median algorithm
Post by: Siekmanski on August 10, 2020, 01:23:50 AM
That's not okay.
Work to do....  :biggrin:
Title: Re: Fast median algorithm
Post by: guga on August 10, 2020, 01:26:36 AM
 :rofl: :rofl: :rofl:


This is with the version i did with the normalization. (This one needs a small fix. It is the  outdated version i used to compute the Sobel alone and not insid ethe main routine of the watermark remover. I´ll later join them both)

(https://i.ibb.co/djV3gF6/machine-Guga.png) (https://ibb.co/djV3gF6)
Title: Re: Fast median algorithm
Post by: Siekmanski on August 10, 2020, 01:41:38 AM
That's how it supposed to be.  :thumbsup:
Your post of my distorted image gave me a hint, it probably is the routine that draws it to screen.
I'll have a look tomorrow.
Title: Re: Fast median algorithm
Post by: guga on August 10, 2020, 03:33:28 PM
Hi Marinus

Finished analyzing the Laplace algorithm and this Lambert W Function. I succeeded to create this W function (with SSE2  :tongue: :tongue: :tongue:)  and i´ll give a try to see if it is worthfull to use laplace as an edge detector. I also implemented a log and exp function using SSE2, but the code is a kinda mess. So i´ll pout them later (here on on another thread to we test the speed :) )

Attached is a tutorial i wrote about this Laplace stuff to don´t forget what i´m doing :mrgreen: :mrgreen: :mrgreen:
Title: Re: Fast median algorithm
Post by: guga on August 10, 2020, 07:58:59 PM
Hi Marinus


I-´m not sure if i implemented this correctly, but...if i did, then....this is what Laplace results


(https://i.ibb.co/6R6bJBf/Laplace.png) (https://ibb.co/6R6bJBf)
Title: Re: Fast median algorithm
Post by: Siekmanski on August 10, 2020, 08:17:21 PM
Hi Gustavo,

Great.  :cool:
I will digest the information you wrote, later this week.
Have a busy week ahead, so less time for coding.
Title: Re: Fast median algorithm
Post by: guga on August 10, 2020, 08:35:21 PM
No problem  :thumbsup: :thumbsup: :thumbsup:

I´m still trying to understand this whole laplace concept. The result using is 2-D Log function with Gaussian (without those matrices) is a sort of a mask. So i don´t know if i should use these 256 new values as a 16x16 matrix (that will be biased on each image) or if i should try to search the pixel values and compare with the results of the table (This is what i did, in fact).

I´m watching these videos
https://www.youtube.com/watch?v=kJKgCwUzkmc
https://www.youtube.com/watch?v=zQKNVept4bU

to try to understand.

Well.. at leats i got that damn W lambert function working with SSE. Despite the results i´ve got, the algo is pretty fast :bgrin: :bgrin: :bgrin:
Title: Re: Fast median algorithm
Post by: Siekmanski on August 10, 2020, 09:02:06 PM
Cool.

So, you're in love now with SSE...  :bgrin:
Maybe it is possible to transfer it into a matrix with pre-calculated coefficients???
Title: Re: Fast median algorithm
Post by: Siekmanski on August 10, 2020, 09:33:31 PM
Just watched the first video, it shows the coefficients for the 3*3 matrix.
As I understand it, it's used as a filter.
If so, it has to be used in a step before you apply the Sobel operations.

(http://members.home.nl/siekmanski/Laplacian.jpg)
Title: Re: Fast median algorithm
Post by: guga on August 10, 2020, 10:05:59 PM
Yes, i´m liking to learn SSE :bgrin: :bgrin: :bgrin:

A bit painful at 1st, but the results are great.

What i did not understood is what a hell is this thing doing ?

(https://static.javatpoint.com/tutorial/dip/images/dip-concept-of-edge-detection10.png)

I succeeded to create a formula and also made a function to work with this W Lambert stuff. But, what i did was assume that Log(x,y) = 0 to 255 and i´m not sure if it´s correct to do this way.  The resultant image is a sort of a mask but i´m not sure if all i did was correct. I tried to not use the matrix values and go straight to this formula.

The result was 256 values i used as a table, but i have no idea if it´s correct to do that way.

What´s log(x,y) all about ? I know what is log(x*y), log(x/y) etc, but with a ",", i have no idea. :greensml: :greensml: :greensml:


And also, the above formula is different from the one below:
https://academic.mu.edu/phys/matthysd/web226/Lab02.htm

(https://academic.mu.edu/phys/matthysd/web226/images/Image174.gif)

Here it uses a power of 4 to calculate the Standard deviation, but on the image it displays a power of 2  :sad: :sad: :sad:
Title: Re: Fast median algorithm
Post by: guga on August 10, 2020, 11:14:50 PM
Ok, i guess i found how this stuff really works.

It´s used a a kernel generator, in fact

https://www.geeksforgeeks.org/gaussian-filter-generation-c
Title: Re: Fast median algorithm
Post by: daydreamer on August 10, 2020, 11:50:37 PM
Cool.

So, you're in love now with SSE...  :bgrin:
Maybe it is possible to transfer it into a matrix with pre-calculated coefficients???
learn from RCP** instructions the backward math:way of faster coding together with replace x DIV** coefficients to x MUL** 1/coeffcients with upto REAL8 precisicion
Title: Re: Fast median algorithm
Post by: Siekmanski on August 10, 2020, 11:59:23 PM
Thinking it over, I think we can skip the overcomplicated approach.
We are not trying to get the highest accuracy.
Not an issue with only 3 coefficients.
Let me try to explain what I mean.

Let's think backwards:

The whole idea is to find coefficients for a lowpass filter.
The choices are, how do we want the curve to behave.
In other words, what type of filter.

filter example in 1 dimension:  [ -0.5 1.0 -0.5 ]

For the computation of a 3*3 matrix we only need 3 main coefficients, which represents a 2D first second order filter.

The Laplace filter uses the Standard Deviation to calculate the curve properties.

For 2D Laplace filter, it turns out to be these coefficients:
Very effective for the edge detecting routines. ( you see the cross in it? )

in 2 dimensions:

[ 0  1  0 ]
[ 1 -4  1 ]
[ 0  1  0 ]

1+1+1+1-4= 0

or try something like this:

[ 1  1  1 ]
[ 1 -8  1 ]
[ 1  1  1 ]

1+1+1+1+1+1+1+1-8= 0

With so few coefficients you could create your own filter type, by experimenting with the values.
As long as the surrounding values add up to zero including the middle value, you have created a filter with your own roll-off factor.

This could be a nice one too:

[ 0.5  1  0.5 ]
[ 1   -6  1    ]
[ 0.5  1  0.5 ]
Title: Re: Fast median algorithm
Post by: Siekmanski on August 11, 2020, 12:22:01 AM
Maby we could use this as the Laplace edge detection operator????

[ 0  -1  0 ]
[-1   4 -1 ]
[ 0  -1  0 ]

Not sure though.......