Playing with Neural Network

[HSE]

Hi All!

I was using  the Artificial Neural Network included in the Package to predict some variable in differents treatments. Original from Thomas Bleeker was converted to OOP by Homer.

It's amazing 

[Biterider]

Hi
Homer and Thomas Bleek are very talented programmers. It is a pity that they are no longer here.

Biterider

[daydreamer]

Hi
Homer and Thomas Bleek are very talented programmers. It is a pity that they are no longer here.

Biterider

I totally agree

[Biterider]

Hi HSE
I'm not sure what you are optimizing, but I think you are using the NN for curve fitting.
I've read a few articles on the subject, but I've never really gone in that direction.

Some time ago, I coded an algorithm in asm to generate the profile of double cams using a custom adaptive strategy. I always had to fight against conversion speed and stability.
Perhaps an approach like yours is more efficient. Can I have a look?

Biterider

[HSE]

Hi Biterider!!

I'm not sure what you are optimizing, but I think you are using the NN for curve fitting.

Correct. I'm not interested in the fitting itself, but in selection of variables that allow some fitting.

Can I have a look?

No problem. I will try to fake the true thing.

Regards. HSE

[HSE]

I think is working well.

Also there is a little ANN extension for "Knowledge extraction from artificial neural network models" (from Boger & Guterman, 1997).  Everybody is invited to complete that 


Now assemble in 32 bits but using ObjAsm, instead of ObjAsm32

[Biterider]

Hi HSE
Great work 

I looked into the code and found what seems to be the core (Boger, Z. & Guterman stuff).
I will dig deeper in the next few days and maybe I'll come back with a few questions ...


Biterider

[HSE]

Hi Biterider.

I looked into the code and found what seems to be the core (Boger, Z. & Guterman stuff).

The core is Bleek stuff, indeed (what it's in the package, by Homer). That requiere data normalization. And most macros are about normalization, storage, retrival and denormalization. Boger and Guterman stuff enter in scene when you already have the solution.

I will dig deeper in the next few days and maybe I'll come back with a few questions ...

HSE

[LiaoMi]

Hi HSE,

if you press the button twice, the program crashes. What should happen after clicking ?!

Code Select Expand

──────────────────────────────────────────────────────────────────────
eax = 3t, Number of input neurons [NNet.inc, 101]
eax = 1t, Number of output neurons [NNet.inc, 110]
──────────────────────────────────────────────────────────────────────
edx = 4t, Number of hidden neurons per layer [NNet.inc, 120]
eax = 3t, Number of hidden layers [NNet.inc, 126]
eax = 5t, Total number of layers [NNet.inc, 134]
──────────────────────────────────────────────────────────────────────
eax = 25t, Total number of output values = (neuronsperlayer+1 * totallayers) [NNet.inc, 139]
eax = 20t, Total number of weights per layer [NNet.inc, 152]
eax = 80t, Total number of weights for all layers [NNet.inc, 156]
──────────────────────────────────────────────────────────────────────
Faked.OnCreate  [Faked_Main.inc, 104]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
xname -> pmxR [ShowMatrix.inc, 15]
[esi].dRows = 3t [ShowMatrix.inc, 17]
[esi].dColumns = 3t [ShowMatrix.inc, 18]
Mtx Type: MXI_SQUARE [ShowMatrix.inc, 28]
                             [ShowMatrix.inc, 36]
Row =   -6.98e-017   3.03e-018   4.04e-017 [ShowMatrix.inc, 50]
Row =    3.03e-018   0.00e+000  -6.15e-018 [ShowMatrix.inc, 50]
Row =    4.04e-017  -6.15e-018   3.47e-016 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
xname -> pmxWh [ShowMatrix.inc, 15]
[esi].dRows = 3t [ShowMatrix.inc, 17]
[esi].dColumns = 4t [ShowMatrix.inc, 18]
Mtx Type: MXI_NORMAL [ShowMatrix.inc, 22]
                             [ShowMatrix.inc, 36]
Row =    6.06e-001   1.37e+000   3.76e-002   1.50e+000 [ShowMatrix.inc, 50]
Row =    1.50e+000   3.52e+000  -3.92e+000  -2.98e+000 [ShowMatrix.inc, 50]
Row =   -2.98e+000  -2.60e+000   7.72e+000   4.25e+000 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
xname -> pmxWhT [ShowMatrix.inc, 15]
[esi].dRows = 4t [ShowMatrix.inc, 17]
[esi].dColumns = 3t [ShowMatrix.inc, 18]
Mtx Type: MXI_NORMAL [ShowMatrix.inc, 22]
                             [ShowMatrix.inc, 36]
Row =    6.06e-001   1.50e+000  -2.98e+000 [ShowMatrix.inc, 50]
Row =    1.37e+000   3.52e+000  -2.60e+000 [ShowMatrix.inc, 50]
Row =    3.76e-002  -3.92e+000   7.72e+000 [ShowMatrix.inc, 50]
Row =    1.50e+000  -2.98e+000   4.25e+000 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
Row =   -2.59e-016 [ShowMatrix.inc, 50]
Row =    3.52e-016 [ShowMatrix.inc, 50]
Row =    3.28e-014 [ShowMatrix.inc, 50]
Vh percentage -------------- [ShowMatrix.inc, 281]
porH[eax*8] =  7.752153662707E-0003 [ShowMatrix.inc, 286]
porH[eax*8] =  1.052502820510E-0002 [ShowMatrix.inc, 286]
porH[eax*8] =  9.817228181322E-0001 [ShowMatrix.inc, 286]


Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)

Code Select Expand

Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)

[HSE]

Thanks LiaoMi !

What crash is DebugCenter, this program use OA32 DebugCenter.

I have a different setting now and something of ObjAsm32 it's not working very well   

Between things differents I don't found my old DebugCenter with black background

[HSE]

Hi!!

The project was modified to work with ObjAsm (only in 32 bits now  ), mainly Matrix object.

Some object error names in ObjErrs.inc must be actualized.

Solved the problem of object error (that don't affected the results).

After crashes because DebugCenter type, there was a "final crash" because a register is not preserved 

Still is in development.

Updated in previous post.

Regards, HSE

[LiaoMi]

Hi!!

The project was modified to work with ObjAsm (only in 32 bits now  ), mainly Matrix object.

Some object error names in ObjErrs.inc must be actualized.

Solved the problem of object error (that don't affected the results).

After crashes because DebugCenter type, there was a "final crash" because a register is not preserved 

Still is in development.

Updated in previous post.

Regards, HSE

Hi HSE,

now the program does not start 

Code Select Expand

Row =    6.83e-001   5.06e-001  -3.00e+000 [ShowMatrix.inc, 54]
Row =    7.79e-001   3.03e+000  -2.75e+000 [ShowMatrix.inc, 54]
Row =    1.51e-001  -3.39e+000   6.74e+000 [ShowMatrix.inc, 54]
Row =    5.06e-001  -3.00e+000   3.48e+000 [ShowMatrix.inc, 54]
──────────────────────────────────────────────────────────────────────
Row =   -1.49e-016 [ShowMatrix.inc, 54]
Row =    2.67e-016 [ShowMatrix.inc, 54]
Row =    2.59e-014 [ShowMatrix.inc, 54]
Vh percentage -------------- [NNet_BG97.inc, 302]
porH[eax*8] =  5.666900849863E-0003 [NNet_BG97.inc, 307]
porH[eax*8] =  1.015015881378E-0002 [NNet_BG97.inc, 307]
porH[eax*8] =  9.841829403364E-0001 [NNet_BG97.inc, 307]
──────────────────────────────────────────────────────────────────────
----------------------------- [Faked.asm, 146]
          Terminу             [Faked.asm, 147]
----------------------------- [Faked.asm, 148]


[HSE]

Hi LiaoMi!!

Solution it's not showed. That is part of Boger and Guterman analysis of solution.

The problem with Neural Network is exactly that solution is beyond understanding. Novelty of Bogdan and others was that some understanding is posible. But today, the idea is that understanding still is minimum.


You can see results of the evolution of solution sending predictions to whatever graphic. In method Muestra2:

Code Select Expand

    OCall lpXorNet::NNet.run, addr inputs_train
    NeuronStage = 2
    NeuronsOut
    CargaPunto gral_1 , outputs_train  ; <------ Graphic (x,y)


Regards, HSE.

[mineiro]

The problem with Neural Network is exactly that solution is beyond understanding.

Hello sir HSE, I hope you're fine. Maybe I don't understood your phrase.

Neural networks are in theory simple to understand.
Neurons have dendrites (input), axons (output), like any truth Boolean table. However, in the middle of the input and output there is some mathematical function, called synthesis, as a math point of view.
As in electronics, there can be feedback, the output becoming a variable for the input, in order to readjust the criteria when an error occurs.
Most of the mathematical functions that I perceive are exponential, factorial, polynomial ... . We know that today when dealing with calculations of these types, computers become abacious in terms of processing speed.

Basically we provide values ​​for the input, synthesize the data and compare the output with the desired result. If an error occurs, I mean, if the desired output was not obtained, we change and retrain the values. In easy words, it's a math function that auto adjust values. The learning process, the knowledge is not retain in only one neuron, that's shared between all neurons, variables.

[LiaoMi]

@HSE,
thanks, I will study this example 

Another assembly project 
A multy-layer feed-forward neural network implementation in assembly x86 32 bits
https://github.com/idoheinemann/Assembly-Neural-Network
Source - https://github.com/idoheinemann/Assembly-Neural-Network/archive/refs/heads/master.zip