Test results for AVX and AVX-512 needed

[Gunther]

Steve,

now I'm understanding. My previous answer is based on a misunderstanding. You talk about the auto-correlation function and the cross-correlation function, right? This is a method of the correlation electronics that lets you distinguish between Nyquist noise and hidden patterns, in the frequency domain of course. Very refined, indeed. It is used in the construction of radar stations that can look far beyond the horizon.

That was one of our first approaches (using such statistical parameters) some years ago, but we rejected that because the unpacked images did not meet the quality standards. We are now looking for the patterns directly in the original area and not via the detour of the frequency domain.

But your suggestion is very good and has made me think. For very accurate images, where time is of minor importance, this path can be a backup for the main process. This will be the subject of another Master's thesis. Please ask me later about the results. The graduates are young and ambitious and want to achieve good results. But you also have to give them time to get the chance. At least that's my experience.

Gunther

[Gunther]

That sounds like a detective  or maybe even a thriller. It's a pity that we can't hear all your story.

Oh yes, life writes the best stories, but that was also a very sad story. Those who are further interested in such questions and the very dark pages of the Wikipedia itself, should read the following book, David Talbot: The Devil's chessboard: Allen Dulles, the CIA, and the Rise of America's Secret Government. It's published by Harper Collins, in 2015. The content has opened my eyes and also has to do with Tron's case. Who gets between such millstones, has very bad cards. But I should open a separate thread in the Colosseum about this, because that is far away from the programming, but humanly very close.

Gunther

[GoneFishing]

Thanks, Gunther
I'll try to find that book

[Gunther]

Thanks, Gunther
I'll try to find that book

It's worth reading in any case.

Gunther

[hutch--]

It is unfortunate that information is subject to either commercial or government interests when both have an unhappy history in what they will do to obtain it, either for control or financial reasons so your choice to protect the identities of authors and the contents of their work is in fact a good idea. Both government agencies and private companies protect the identities and content of information they control so it makes sense that you should do the same.

From a quick read of the Wikipedia page you linked to, it sounds like Tron was "assisted" in his suicide because he knew something that someone else did not want him to know. You do have examples of some countries in the middle east killing scientists in other countries in the middle east so conduct like this is not unknown.

[Gunther]

Steve,

From a quick read of the Wikipedia page you linked to, it sounds like Tron was "assisted" in his suicide because he knew something that someone else did not want him to know.

That's exactly the point. When I finally realized that, it was already too late. That will attach to me forever. I should have taken him out of the line of fire much earlier.

Gunther

[FORTRANS]

Hi,

Steve,

now I'm understanding. My previous answer is based on a misunderstanding. You talk about the auto-correlation function and the cross-correlation function, right?

   Yes.  Correlation between your image and a specified feature.

This is a method of the correlation electronics that lets you distinguish between Nyquist noise and hidden patterns, in the frequency domain of course. Very refined, indeed. It is used in the construction of radar stations that can look far beyond the horizon.

That was one of our first approaches (using such statistical parameters) some years ago, but we rejected that because the unpacked images did not meet the quality standards. We are now looking for the patterns directly in the original area and not via the detour of the frequency domain.

   That answers my question.  You have already evaluated my
suggested action and are not currently using it.  Not due to
calculation cost as I thought, but due to poor performance.  Thank
you, very informative.

But your suggestion is very good and has made me think. For very accurate images, where time is of minor importance, this path can be a backup for the main process. This will be the subject of another Master's thesis. Please ask me later about the results. The graduates are young and ambitious and want to achieve good results. But you also have to give them time to get the chance. At least that's my experience.

   Best of luck to you and them.  Keep us informed if something
shows up.  Thank you for your response.

Regards,

Steve

[Gunther]

Steve,

   Yes.  Correlation between your image and a specified feature.

Of course, the auto- and cross-correlation functions have to do with the Fourier Transformation; the convolution also plays a role. This can be very interesting things. We found that, for example, the convolution of two Bessel functions gives the harmonic sinusoidal oscillation. You can even recalculate that. This is important for the fractal encoding.

Gunther

[HSE]

Hi Gunther!

    Just reading from the more complete ignorance: You are searching only for continuous patterns. Because if the pattern is discontinuous, you can't split the search in different threads. ¿?

[Gunther]

    Just reading from the more complete ignorance: You are searching only for continuous patterns. Because if the pattern is discontinuous, you can't split the search in different threads. ¿?

That's a really good question. The basis of the described technique is the fractal geometry. The emphasis is on the geometry. In vivid terms, we search for self-similarities within a given image. These can be big - the bigger the better, because then the compression gets better - or small. Last but not least, these are geometric aspects that we assume. In this sense, the patterns must already be coherent. But to come back to your suggestion: At present a graduate is working on various new approaches to parallelization. His interim results look very good. The deadline for submitting his master's thesis is mid-February. If he defended his theses in April next year, we know more. We are already looking for different processes and sometimes several threads within a process. But with the new results we should be able to take decisive steps forward, because the data volume involved in film sequences or at CERN is gigantic.

Another problem that occurs when decoding movie clips or entire movies is the synchronization of the video and audio tracks. However, we have that under control since a few days, which makes me very happy.

Gunther

[HSE]

 :t Thanks.

[Gunther]

:t Thanks.

You're welcome. I just hope my explanation was reasonably clear and understandable.

Gunther

[hutch--]

Gunther,

I would be interested to know what video resolution you are working with on a project like CERN. I remember years ago that Silicon Graphics had the hardware to animate a 20 megapixel image at 100 frames a second but JPL were using very big computer grunt for rotating images of the known universe. I wondered if you guys are doing similar stuff.

I have a couple of cameras that will shoot decent 4k and I know there are a few that will shoot 8k but I got the impression that you may be working in much higher resolution than the normal standards offer.

[Gunther]

Steve,

the CERN images are not the question, because we've no audio track. There, only the sheer mass is the problem.

The real challenge are the 4K movies. This is a parallel project and located at the University. We aim for a resolution of 3840x2160, 24 images per second (cinema quality) with Dolby Surround soundtrack. But that's all easier said than done. At the moment it looks a bit bumpy on my screen, but it gets better every day. The graduates are working with full steam. Well, they are young and ambitious, they want to show what they have learned and what they have on it.

Although you can do a lot more tricky with movies, we do not get around to do decoding with the good old assembler. We do that with the inline assembler, leave the old C++ code as comment, so that we can later do migration easier (UNIX, Mac, PowerPC, what do I know). In particular, the iterations and the conversion from YUV to RGB are critical. There is no free register and all cores are running full load; the cache could well be three times as big.

On the other hand, C++ comes with a lot of unnecessary overhead. In the last few days I've often thought about not going back to the good old C. You know what you have, you do not need constructors, destructors, and you do not have to be careful about bending the This pointer by mistake. But it's difficult in the middle of the work. Some design decisions avenge later, but they take their revenge.

But fine, I let myself be persuaded by the young people to this madness project and must now cope with it. What else is left for me? It is my duty to show those young people the right way, to eliminate difficulties and sometimes to give comfort. Whining does not help, just bite your teeth together and carry on. The crazy thing is: sometimes I have bad moments, like every human being. At such moments, I always hear the old Vince Guaraldi tune. There's also a guitar version by the great Earl Klugh with a bit help by Vince Gill; the music starts at 1:29. Believe it or not, my graduates, who usually only listen to rap and hip-hop, find it really good and have come to the taste. That's a crazy world.

Gunther