The MASM Forum

Projects => ObjAsm => Topic started by: HSE on June 05, 2020, 10:20:38 AM

Title: Playing with Neural Network
Post by: HSE on June 05, 2020, 10:20:38 AM
Hi All!

I was using  the Artificial Neural Network included in the Package to predict some variable in differents treatments. Original from Thomas Bleeker was converted to OOP by Homer.

It's amazing  :biggrin:


(http://www.prodanimal.com.ar/gif/Ajuste.gif)
Title: Re: Playing with Neural Network
Post by: Biterider on June 05, 2020, 03:30:27 PM
Hi
Homer and Thomas Bleek are very talented programmers. It is a pity that they are no longer here.

Biterider
Title: Re: Playing with Neural Network
Post by: daydreamer on June 06, 2020, 05:47:56 AM
Quote from: Biterider on June 05, 2020, 03:30:27 PM
Hi
Homer and Thomas Bleek are very talented programmers. It is a pity that they are no longer here.

Biterider
I totally agree :thumbsup:
Title: Re: Playing with Neural Network
Post by: Biterider on June 13, 2020, 05:59:50 PM
Hi HSE
I'm not sure what you are optimizing, but I think you are using the NN for curve fitting.
I've read a few articles on the subject, but I've never really gone in that direction.

Some time ago, I coded an algorithm in asm to generate the profile of double cams (https://www.youtube.com/watch?v=vWIyxkMVBwc) using a custom adaptive strategy. I always had to fight against conversion speed and stability. :rolleyes:
Perhaps an approach like yours is more efficient. Can I have a look?

Biterider
Title: Re: Playing with Neural Network
Post by: HSE on June 14, 2020, 02:13:03 AM
Hi Biterider!!

Quote from: Biterider on June 13, 2020, 05:59:50 PM
I'm not sure what you are optimizing, but I think you are using the NN for curve fitting.
Correct. I'm not interested in the fitting itself, but in selection of variables that allow some fitting.

Quote from: Biterider on June 13, 2020, 05:59:50 PM
Can I have a look?
No problem. I will try to fake the true thing.

Regards. HSE
Title: Re: Playing with Neural Network
Post by: HSE on June 14, 2020, 08:29:39 AM
I think is working well.

Also there is a little ANN extension for "Knowledge extraction from artificial neural network models" (from Boger & Guterman, 1997).  Everybody is invited to complete that  :biggrin: :biggrin: :biggrin:


Now assemble in 32 bits but using ObjAsm, instead of ObjAsm32
Title: Re: Playing with Neural Network
Post by: Biterider on June 14, 2020, 04:48:08 PM
Hi HSE
Great work  :thumbsup:

I looked into the code and found what seems to be the core (Boger, Z. & Guterman stuff).
I will dig deeper in the next few days and maybe I'll come back with a few questions ...


Biterider
Title: Re: Playing with Neural Network
Post by: HSE on June 15, 2020, 05:26:32 AM
Hi Biterider.

Quote from: Biterider on June 14, 2020, 04:48:08 PM
I looked into the code and found what seems to be the core (Boger, Z. & Guterman stuff).
The core is Bleek stuff, indeed (what it's in the package, by Homer). That requiere data normalization. And most macros are about normalization, storage, retrival and denormalization. Boger and Guterman stuff enter in scene when you already have the solution.

Quote from: Biterider on June 14, 2020, 04:48:08 PM
I will dig deeper in the next few days and maybe I'll come back with a few questions ...
:thumbsup:

HSE
Title: Re: Playing with Neural Network
Post by: LiaoMi on May 16, 2021, 03:08:40 AM
Hi HSE,

if you press the button twice, the program crashes. What should happen after clicking ?!

──────────────────────────────────────────────────────────────────────
eax = 3t, Number of input neurons [NNet.inc, 101]
eax = 1t, Number of output neurons [NNet.inc, 110]
──────────────────────────────────────────────────────────────────────
edx = 4t, Number of hidden neurons per layer [NNet.inc, 120]
eax = 3t, Number of hidden layers [NNet.inc, 126]
eax = 5t, Total number of layers [NNet.inc, 134]
──────────────────────────────────────────────────────────────────────
eax = 25t, Total number of output values = (neuronsperlayer+1 * totallayers) [NNet.inc, 139]
eax = 20t, Total number of weights per layer [NNet.inc, 152]
eax = 80t, Total number of weights for all layers [NNet.inc, 156]
──────────────────────────────────────────────────────────────────────
Faked.OnCreate  [Faked_Main.inc, 104]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
xname -> pmxR [ShowMatrix.inc, 15]
[esi].dRows = 3t [ShowMatrix.inc, 17]
[esi].dColumns = 3t [ShowMatrix.inc, 18]
Mtx Type: MXI_SQUARE [ShowMatrix.inc, 28]
                             [ShowMatrix.inc, 36]
Row =   -6.98e-017   3.03e-018   4.04e-017 [ShowMatrix.inc, 50]
Row =    3.03e-018   0.00e+000  -6.15e-018 [ShowMatrix.inc, 50]
Row =    4.04e-017  -6.15e-018   3.47e-016 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
xname -> pmxWh [ShowMatrix.inc, 15]
[esi].dRows = 3t [ShowMatrix.inc, 17]
[esi].dColumns = 4t [ShowMatrix.inc, 18]
Mtx Type: MXI_NORMAL [ShowMatrix.inc, 22]
                             [ShowMatrix.inc, 36]
Row =    6.06e-001   1.37e+000   3.76e-002   1.50e+000 [ShowMatrix.inc, 50]
Row =    1.50e+000   3.52e+000  -3.92e+000  -2.98e+000 [ShowMatrix.inc, 50]
Row =   -2.98e+000  -2.60e+000   7.72e+000   4.25e+000 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
xname -> pmxWhT [ShowMatrix.inc, 15]
[esi].dRows = 4t [ShowMatrix.inc, 17]
[esi].dColumns = 3t [ShowMatrix.inc, 18]
Mtx Type: MXI_NORMAL [ShowMatrix.inc, 22]
                             [ShowMatrix.inc, 36]
Row =    6.06e-001   1.50e+000  -2.98e+000 [ShowMatrix.inc, 50]
Row =    1.37e+000   3.52e+000  -2.60e+000 [ShowMatrix.inc, 50]
Row =    3.76e-002  -3.92e+000   7.72e+000 [ShowMatrix.inc, 50]
Row =    1.50e+000  -2.98e+000   4.25e+000 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
Row =   -2.59e-016 [ShowMatrix.inc, 50]
Row =    3.52e-016 [ShowMatrix.inc, 50]
Row =    3.28e-014 [ShowMatrix.inc, 50]
Vh percentage -------------- [ShowMatrix.inc, 281]
porH[eax*8] =  7.752153662707E-0003 [ShowMatrix.inc, 286]
porH[eax*8] =  1.052502820510E-0002 [ShowMatrix.inc, 286]
porH[eax*8] =  9.817228181322E-0001 [ShowMatrix.inc, 286]


Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Title: Re: Playing with Neural Network
Post by: HSE on May 16, 2021, 04:35:48 AM
Thanks LiaoMi !

What crash is DebugCenter, this program use OA32 DebugCenter.

I have a different setting now and something of ObjAsm32 it's not working very well  :biggrin: 

Between things differents I don't found my old DebugCenter with black background :sad:

Title: Re: Playing with Neural Network
Post by: HSE on May 17, 2021, 05:52:43 AM
Hi!!

The project was modified to work with ObjAsm (only in 32 bits now  :biggrin:), mainly Matrix object.

Some object error names in ObjErrs.inc must be actualized.

Solved the problem of object error (that don't affected the results).

After crashes because DebugCenter type, there was a "final crash" because a register is not preserved  :eusa_boohoo:

Still is in development.

Updated in previous post.

Regards, HSE

Title: Re: Playing with Neural Network
Post by: LiaoMi on May 18, 2021, 08:30:00 PM
Quote from: HSE on May 17, 2021, 05:52:43 AM
Hi!!

The project was modified to work with ObjAsm (only in 32 bits now  :biggrin:), mainly Matrix object.

Some object error names in ObjErrs.inc must be actualized.

Solved the problem of object error (that don't affected the results).

After crashes because DebugCenter type, there was a "final crash" because a register is not preserved  :eusa_boohoo:

Still is in development.

Updated in previous post.

Regards, HSE

Hi HSE,

now the program does not start  :sad:

Row =    6.83e-001   5.06e-001  -3.00e+000 [ShowMatrix.inc, 54]
Row =    7.79e-001   3.03e+000  -2.75e+000 [ShowMatrix.inc, 54]
Row =    1.51e-001  -3.39e+000   6.74e+000 [ShowMatrix.inc, 54]
Row =    5.06e-001  -3.00e+000   3.48e+000 [ShowMatrix.inc, 54]
──────────────────────────────────────────────────────────────────────
Row =   -1.49e-016 [ShowMatrix.inc, 54]
Row =    2.67e-016 [ShowMatrix.inc, 54]
Row =    2.59e-014 [ShowMatrix.inc, 54]
Vh percentage -------------- [NNet_BG97.inc, 302]
porH[eax*8] =  5.666900849863E-0003 [NNet_BG97.inc, 307]
porH[eax*8] =  1.015015881378E-0002 [NNet_BG97.inc, 307]
porH[eax*8] =  9.841829403364E-0001 [NNet_BG97.inc, 307]
──────────────────────────────────────────────────────────────────────
----------------------------- [Faked.asm, 146]
          Terminу             [Faked.asm, 147]
----------------------------- [Faked.asm, 148]

Title: Re: Playing with Neural Network
Post by: HSE on May 18, 2021, 11:04:28 PM
Hi LiaoMi!!

:biggrin: Solution it's not showed. That is part of Boger and Guterman analysis of solution.

The problem with Neural Network is exactly that solution is beyond understanding. Novelty of Bogdan and others was that some understanding is posible. But today, the idea is that understanding still is minimum.


You can see results of the evolution of solution sending predictions to whatever graphic. In method Muestra2:
    OCall lpXorNet::NNet.run, addr inputs_train
    NeuronStage = 2
    NeuronsOut
    CargaPunto gral_1 , outputs_train  ; <------ Graphic (x,y)


Regards, HSE.
Title: Re: Playing with Neural Network
Post by: mineiro on May 19, 2021, 12:28:50 AM
Quote from: HSE on May 18, 2021, 11:04:28 PM
The problem with Neural Network is exactly that solution is beyond understanding.
Hello sir HSE, I hope you're fine. Maybe I don't understood your phrase.

Neural networks are in theory simple to understand.
Neurons have dendrites (input), axons (output), like any truth Boolean table. However, in the middle of the input and output there is some mathematical function, called synthesis, as a math point of view.
As in electronics, there can be feedback, the output becoming a variable for the input, in order to readjust the criteria when an error occurs.
Most of the mathematical functions that I perceive are exponential, factorial, polynomial ... . We know that today when dealing with calculations of these types, computers become abacious in terms of processing speed.

Basically we provide values ​​for the input, synthesize the data and compare the output with the desired result. If an error occurs, I mean, if the desired output was not obtained, we change and retrain the values. In easy words, it's a math function that auto adjust values. The learning process, the knowledge is not retain in only one neuron, that's shared between all neurons, variables.
Title: Re: Playing with Neural Network
Post by: LiaoMi on May 19, 2021, 02:15:08 AM
@HSE,
thanks, I will study this example  :rolleyes:

Another assembly project  :thup:
A multy-layer feed-forward neural network implementation in assembly x86 32 bits
https://github.com/idoheinemann/Assembly-Neural-Network
Source - https://github.com/idoheinemann/Assembly-Neural-Network/archive/refs/heads/master.zip
Title: Re: Playing with Neural Network
Post by: HSE on May 19, 2021, 02:35:35 AM
Hi mineiro!

Quote from: mineiro on May 19, 2021, 12:28:50 AM
Maybe I don't understood your phrase.
That is. You don't understood phrase.

Quote from: HSE on May 18, 2021, 11:04:28 PM
The problem with Neural Network is exactly that solution is beyond understanding.

The problem is not the process, is the solution.

In mathematical functions (exponential, factorial, polynomial ...)  you solve the equations and you can isolate how each variable affect the result, not always easy but in principle always posible. That can help to find real mechanism behind input-ouput asociations.

In Neural Networks you solve input-outputs asociations and you don't know why that work!
Just some months ago Max Tegmark with Brian Keating: https://www.youtube.com/watch?v=pFDqI3pKmec
Title: Re: Playing with Neural Network
Post by: HSE on May 19, 2021, 04:51:15 AM
Quote from: LiaoMi on May 19, 2021, 02:15:08 AM
A multy-layer feed-forward neural network implementation in assembly x86 32 bits

It's same kind of problem addressed in DigiBrain by Thomas Bleeker . Work well  :thumbsup:
Title: Re: Playing with Neural Network
Post by: mineiro on May 19, 2021, 07:19:23 AM
Quote from: HSE on May 19, 2021, 02:35:35 AM
In Neural Networks you solve input-outputs asociations and you don't know why that work!
Plasticity, neuromorphic, organismic. Like a drunken algorithm.
Organismic computing reveals that trained neural network can forgot things and activate that knowledge later.

https://en.wikipedia.org/wiki/Neuromorphic_engineering
https://en.wikipedia.org/wiki/Organismic_computing

John von Neumann:
Anyone who attempts to generate random numbers by deterministic means is, of course, living in a state of sin.
Title: Re: Playing with Neural Network
Post by: HSE on May 19, 2021, 08:07:00 AM
 :biggrin: