Author Topic: Playing with Neural Network  (Read 2600 times)

HSE

  • Member
  • *****
  • Posts: 1741
  • <AMD>< 7-32>
Playing with Neural Network
« on: June 05, 2020, 10:20:38 AM »
Hi All!

I was using  the Artificial Neural Network included in the Package to predict some variable in differents treatments. Original from Thomas Bleeker was converted to OOP by Homer.

It's amazing  :biggrin:


« Last Edit: May 19, 2021, 04:46:53 AM by HSE »

Biterider

  • Moderator
  • Member
  • *****
  • Posts: 723
  • ObjAsm Developer
    • ObjAsm
Re: Playing with Neural Network
« Reply #1 on: June 05, 2020, 03:30:27 PM »
Hi
Homer and Thomas Bleek are very talented programmers. It is a pity that they are no longer here.

Biterider

daydreamer

  • Member
  • *****
  • Posts: 1721
  • building nextdoor
Re: Playing with Neural Network
« Reply #2 on: June 06, 2020, 05:47:56 AM »
Hi
Homer and Thomas Bleek are very talented programmers. It is a pity that they are no longer here.

Biterider
I totally agree :thumbsup:
SIMD fan and macro fan
why assembly is fastest is because its switch has no (brakes) breaks
:P
only in 16bit assembly you can get away with "Only words" :P

Biterider

  • Moderator
  • Member
  • *****
  • Posts: 723
  • ObjAsm Developer
    • ObjAsm
Re: Playing with Neural Network
« Reply #3 on: June 13, 2020, 05:59:50 PM »
Hi HSE
I'm not sure what you are optimizing, but I think you are using the NN for curve fitting.
I've read a few articles on the subject, but I've never really gone in that direction.

Some time ago, I coded an algorithm in asm to generate the profile of double cams using a custom adaptive strategy. I always had to fight against conversion speed and stability. :rolleyes:
Perhaps an approach like yours is more efficient. Can I have a look?

Biterider

HSE

  • Member
  • *****
  • Posts: 1741
  • <AMD>< 7-32>
Re: Playing with Neural Network
« Reply #4 on: June 14, 2020, 02:13:03 AM »
Hi Biterider!!

I'm not sure what you are optimizing, but I think you are using the NN for curve fitting.
Correct. I'm not interested in the fitting itself, but in selection of variables that allow some fitting.

Can I have a look?
No problem. I will try to fake the true thing.

Regards. HSE

HSE

  • Member
  • *****
  • Posts: 1741
  • <AMD>< 7-32>
Re: Playing with Neural Network
« Reply #5 on: June 14, 2020, 08:29:39 AM »
I think is working well.

Also there is a little ANN extension for "Knowledge extraction from artificial neural network models" (from Boger & Guterman, 1997).  Everybody is invited to complete that  :biggrin: :biggrin: :biggrin:


Now assemble in 32 bits but using ObjAsm, instead of ObjAsm32
« Last Edit: May 17, 2021, 05:39:33 AM by HSE »

Biterider

  • Moderator
  • Member
  • *****
  • Posts: 723
  • ObjAsm Developer
    • ObjAsm
Re: Playing with Neural Network
« Reply #6 on: June 14, 2020, 04:48:08 PM »
Hi HSE
Great work  :thumbsup:

I looked into the code and found what seems to be the core (Boger, Z. & Guterman stuff).
I will dig deeper in the next few days and maybe I'll come back with a few questions ...


Biterider

HSE

  • Member
  • *****
  • Posts: 1741
  • <AMD>< 7-32>
Re: Playing with Neural Network
« Reply #7 on: June 15, 2020, 05:26:32 AM »
Hi Biterider.

I looked into the code and found what seems to be the core (Boger, Z. & Guterman stuff).
The core is Bleek stuff, indeed (what it's in the package, by Homer). That requiere data normalization. And most macros are about normalization, storage, retrival and denormalization. Boger and Guterman stuff enter in scene when you already have the solution.

I will dig deeper in the next few days and maybe I'll come back with a few questions ...
:thumbsup:

HSE

LiaoMi

  • Member
  • ****
  • Posts: 922
Re: Playing with Neural Network
« Reply #8 on: May 16, 2021, 03:08:40 AM »
Hi HSE,

if you press the button twice, the program crashes. What should happen after clicking ?!

Code: [Select]
──────────────────────────────────────────────────────────────────────
eax = 3t, Number of input neurons [NNet.inc, 101]
eax = 1t, Number of output neurons [NNet.inc, 110]
──────────────────────────────────────────────────────────────────────
edx = 4t, Number of hidden neurons per layer [NNet.inc, 120]
eax = 3t, Number of hidden layers [NNet.inc, 126]
eax = 5t, Total number of layers [NNet.inc, 134]
──────────────────────────────────────────────────────────────────────
eax = 25t, Total number of output values = (neuronsperlayer+1 * totallayers) [NNet.inc, 139]
eax = 20t, Total number of weights per layer [NNet.inc, 152]
eax = 80t, Total number of weights for all layers [NNet.inc, 156]
──────────────────────────────────────────────────────────────────────
Faked.OnCreate  [Faked_Main.inc, 104]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
xname -> pmxR [ShowMatrix.inc, 15]
[esi].dRows = 3t [ShowMatrix.inc, 17]
[esi].dColumns = 3t [ShowMatrix.inc, 18]
Mtx Type: MXI_SQUARE [ShowMatrix.inc, 28]
                             [ShowMatrix.inc, 36]
Row =   -6.98e-017   3.03e-018   4.04e-017 [ShowMatrix.inc, 50]
Row =    3.03e-018   0.00e+000  -6.15e-018 [ShowMatrix.inc, 50]
Row =    4.04e-017  -6.15e-018   3.47e-016 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
xname -> pmxWh [ShowMatrix.inc, 15]
[esi].dRows = 3t [ShowMatrix.inc, 17]
[esi].dColumns = 4t [ShowMatrix.inc, 18]
Mtx Type: MXI_NORMAL [ShowMatrix.inc, 22]
                             [ShowMatrix.inc, 36]
Row =    6.06e-001   1.37e+000   3.76e-002   1.50e+000 [ShowMatrix.inc, 50]
Row =    1.50e+000   3.52e+000  -3.92e+000  -2.98e+000 [ShowMatrix.inc, 50]
Row =   -2.98e+000  -2.60e+000   7.72e+000   4.25e+000 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
xname -> pmxWhT [ShowMatrix.inc, 15]
[esi].dRows = 4t [ShowMatrix.inc, 17]
[esi].dColumns = 3t [ShowMatrix.inc, 18]
Mtx Type: MXI_NORMAL [ShowMatrix.inc, 22]
                             [ShowMatrix.inc, 36]
Row =    6.06e-001   1.50e+000  -2.98e+000 [ShowMatrix.inc, 50]
Row =    1.37e+000   3.52e+000  -2.60e+000 [ShowMatrix.inc, 50]
Row =    3.76e-002  -3.92e+000   7.72e+000 [ShowMatrix.inc, 50]
Row =    1.50e+000  -2.98e+000   4.25e+000 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
Row =   -2.59e-016 [ShowMatrix.inc, 50]
Row =    3.52e-016 [ShowMatrix.inc, 50]
Row =    3.28e-014 [ShowMatrix.inc, 50]
Vh percentage -------------- [ShowMatrix.inc, 281]
porH[eax*8] =  7.752153662707E-0003 [ShowMatrix.inc, 286]
porH[eax*8] =  1.052502820510E-0002 [ShowMatrix.inc, 286]
porH[eax*8] =  9.817228181322E-0001 [ShowMatrix.inc, 286]

Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Code: [Select]
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)

HSE

  • Member
  • *****
  • Posts: 1741
  • <AMD>< 7-32>
Re: Playing with Neural Network
« Reply #9 on: May 16, 2021, 04:35:48 AM »
Thanks LiaoMi !

What crash is DebugCenter, this program use OA32 DebugCenter.

I have a different setting now and something of ObjAsm32 it's not working very well  :biggrin: 

Between things differents I don't found my old DebugCenter with black background :sad:


HSE

  • Member
  • *****
  • Posts: 1741
  • <AMD>< 7-32>
Re: Playing with Neural Network
« Reply #10 on: May 17, 2021, 05:52:43 AM »
Hi!!

The project was modified to work with ObjAsm (only in 32 bits now  :biggrin:), mainly Matrix object.

Some object error names in ObjErrs.inc must be actualized.

Solved the problem of object error (that don't affected the results).

After crashes because DebugCenter type, there was a "final crash" because a register is not preserved  :eusa_boohoo:

Still is in development.

Updated in previous post.

Regards, HSE


LiaoMi

  • Member
  • ****
  • Posts: 922
Re: Playing with Neural Network
« Reply #11 on: May 18, 2021, 08:30:00 PM »
Hi!!

The project was modified to work with ObjAsm (only in 32 bits now  :biggrin:), mainly Matrix object.

Some object error names in ObjErrs.inc must be actualized.

Solved the problem of object error (that don't affected the results).

After crashes because DebugCenter type, there was a "final crash" because a register is not preserved  :eusa_boohoo:

Still is in development.

Updated in previous post.

Regards, HSE

Hi HSE,

now the program does not start  :sad:

Code: [Select]
Row =    6.83e-001   5.06e-001  -3.00e+000 [ShowMatrix.inc, 54]
Row =    7.79e-001   3.03e+000  -2.75e+000 [ShowMatrix.inc, 54]
Row =    1.51e-001  -3.39e+000   6.74e+000 [ShowMatrix.inc, 54]
Row =    5.06e-001  -3.00e+000   3.48e+000 [ShowMatrix.inc, 54]
──────────────────────────────────────────────────────────────────────
Row =   -1.49e-016 [ShowMatrix.inc, 54]
Row =    2.67e-016 [ShowMatrix.inc, 54]
Row =    2.59e-014 [ShowMatrix.inc, 54]
Vh percentage -------------- [NNet_BG97.inc, 302]
porH[eax*8] =  5.666900849863E-0003 [NNet_BG97.inc, 307]
porH[eax*8] =  1.015015881378E-0002 [NNet_BG97.inc, 307]
porH[eax*8] =  9.841829403364E-0001 [NNet_BG97.inc, 307]
──────────────────────────────────────────────────────────────────────
----------------------------- [Faked.asm, 146]
          Terminу             [Faked.asm, 147]
----------------------------- [Faked.asm, 148]

HSE

  • Member
  • *****
  • Posts: 1741
  • <AMD>< 7-32>
Re: Playing with Neural Network
« Reply #12 on: May 18, 2021, 11:04:28 PM »
Hi LiaoMi!!

 :biggrin: Solution it's not showed. That is part of Boger and Guterman analysis of solution.

The problem with Neural Network is exactly that solution is beyond understanding. Novelty of Bogdan and others was that some understanding is posible. But today, the idea is that understanding still is minimum.


You can see results of the evolution of solution sending predictions to whatever graphic. In method Muestra2:
Code: [Select]
    OCall lpXorNet::NNet.run, addr inputs_train
    NeuronStage = 2
    NeuronsOut
    CargaPunto gral_1 , outputs_train  ; <------ Graphic (x,y)

Regards, HSE.

mineiro

  • Member
  • ****
  • Posts: 723
Re: Playing with Neural Network
« Reply #13 on: May 19, 2021, 12:28:50 AM »
The problem with Neural Network is exactly that solution is beyond understanding.
Hello sir HSE, I hope you're fine. Maybe I don't understood your phrase.

Neural networks are in theory simple to understand.
Neurons have dendrites (input), axons (output), like any truth Boolean table. However, in the middle of the input and output there is some mathematical function, called synthesis, as a math point of view.
As in electronics, there can be feedback, the output becoming a variable for the input, in order to readjust the criteria when an error occurs.
Most of the mathematical functions that I perceive are exponential, factorial, polynomial ... . We know that today when dealing with calculations of these types, computers become abacious in terms of processing speed.

Basically we provide values ​​for the input, synthesize the data and compare the output with the desired result. If an error occurs, I mean, if the desired output was not obtained, we change and retrain the values. In easy words, it's a math function that auto adjust values. The learning process, the knowledge is not retain in only one neuron, that's shared between all neurons, variables.
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

LiaoMi

  • Member
  • ****
  • Posts: 922
Re: Playing with Neural Network
« Reply #14 on: May 19, 2021, 02:15:08 AM »
@HSE,
 thanks, I will study this example  :rolleyes:

Another assembly project  :thup:
A multy-layer feed-forward neural network implementation in assembly x86 32 bits
https://github.com/idoheinemann/Assembly-Neural-Network
Source - https://github.com/idoheinemann/Assembly-Neural-Network/archive/refs/heads/master.zip