News:

Masm32 SDK description, downloads and other helpful links
Message to All Guests
NB: Posting URL's See here: Posted URL Change

Main Menu

Playing with Neural Network

Started by HSE, June 05, 2020, 10:20:38 AM

Previous topic - Next topic

HSE

Hi All!

I was using  the Artificial Neural Network included in the Package to predict some variable in differents treatments. Original from Thomas Bleeker was converted to OOP by Homer.

It's amazing  :biggrin:



Equations in Assembly: SmplMath

Biterider

Hi
Homer and Thomas Bleek are very talented programmers. It is a pity that they are no longer here.

Biterider

daydreamer

Quote from: Biterider on June 05, 2020, 03:30:27 PM
Hi
Homer and Thomas Bleek are very talented programmers. It is a pity that they are no longer here.

Biterider
I totally agree :thumbsup:
my none asm creations
https://masm32.com/board/index.php?topic=6937.msg74303#msg74303
I am an Invoker
"An Invoker is a mage who specializes in the manipulation of raw and elemental energies."
Like SIMD coding

Biterider

Hi HSE
I'm not sure what you are optimizing, but I think you are using the NN for curve fitting.
I've read a few articles on the subject, but I've never really gone in that direction.

Some time ago, I coded an algorithm in asm to generate the profile of double cams using a custom adaptive strategy. I always had to fight against conversion speed and stability. :rolleyes:
Perhaps an approach like yours is more efficient. Can I have a look?

Biterider

HSE

Hi Biterider!!

Quote from: Biterider on June 13, 2020, 05:59:50 PM
I'm not sure what you are optimizing, but I think you are using the NN for curve fitting.
Correct. I'm not interested in the fitting itself, but in selection of variables that allow some fitting.

Quote from: Biterider on June 13, 2020, 05:59:50 PM
Can I have a look?
No problem. I will try to fake the true thing.

Regards. HSE
Equations in Assembly: SmplMath

HSE

#5
I think is working well.

Also there is a little ANN extension for "Knowledge extraction from artificial neural network models" (from Boger & Guterman, 1997).  Everybody is invited to complete that  :biggrin: :biggrin: :biggrin:


Now assemble in 32 bits but using ObjAsm, instead of ObjAsm32
Equations in Assembly: SmplMath

Biterider

Hi HSE
Great work  :thumbsup:

I looked into the code and found what seems to be the core (Boger, Z. & Guterman stuff).
I will dig deeper in the next few days and maybe I'll come back with a few questions ...


Biterider

HSE

Hi Biterider.

Quote from: Biterider on June 14, 2020, 04:48:08 PM
I looked into the code and found what seems to be the core (Boger, Z. & Guterman stuff).
The core is Bleek stuff, indeed (what it's in the package, by Homer). That requiere data normalization. And most macros are about normalization, storage, retrival and denormalization. Boger and Guterman stuff enter in scene when you already have the solution.

Quote from: Biterider on June 14, 2020, 04:48:08 PM
I will dig deeper in the next few days and maybe I'll come back with a few questions ...
:thumbsup:

HSE
Equations in Assembly: SmplMath

LiaoMi

Hi HSE,

if you press the button twice, the program crashes. What should happen after clicking ?!

──────────────────────────────────────────────────────────────────────
eax = 3t, Number of input neurons [NNet.inc, 101]
eax = 1t, Number of output neurons [NNet.inc, 110]
──────────────────────────────────────────────────────────────────────
edx = 4t, Number of hidden neurons per layer [NNet.inc, 120]
eax = 3t, Number of hidden layers [NNet.inc, 126]
eax = 5t, Total number of layers [NNet.inc, 134]
──────────────────────────────────────────────────────────────────────
eax = 25t, Total number of output values = (neuronsperlayer+1 * totallayers) [NNet.inc, 139]
eax = 20t, Total number of weights per layer [NNet.inc, 152]
eax = 80t, Total number of weights for all layers [NNet.inc, 156]
──────────────────────────────────────────────────────────────────────
Faked.OnCreate  [Faked_Main.inc, 104]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
Inner loop complete [rounded.inc, 383]
xname -> pmxR [ShowMatrix.inc, 15]
[esi].dRows = 3t [ShowMatrix.inc, 17]
[esi].dColumns = 3t [ShowMatrix.inc, 18]
Mtx Type: MXI_SQUARE [ShowMatrix.inc, 28]
                             [ShowMatrix.inc, 36]
Row =   -6.98e-017   3.03e-018   4.04e-017 [ShowMatrix.inc, 50]
Row =    3.03e-018   0.00e+000  -6.15e-018 [ShowMatrix.inc, 50]
Row =    4.04e-017  -6.15e-018   3.47e-016 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
xname -> pmxWh [ShowMatrix.inc, 15]
[esi].dRows = 3t [ShowMatrix.inc, 17]
[esi].dColumns = 4t [ShowMatrix.inc, 18]
Mtx Type: MXI_NORMAL [ShowMatrix.inc, 22]
                             [ShowMatrix.inc, 36]
Row =    6.06e-001   1.37e+000   3.76e-002   1.50e+000 [ShowMatrix.inc, 50]
Row =    1.50e+000   3.52e+000  -3.92e+000  -2.98e+000 [ShowMatrix.inc, 50]
Row =   -2.98e+000  -2.60e+000   7.72e+000   4.25e+000 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
xname -> pmxWhT [ShowMatrix.inc, 15]
[esi].dRows = 4t [ShowMatrix.inc, 17]
[esi].dColumns = 3t [ShowMatrix.inc, 18]
Mtx Type: MXI_NORMAL [ShowMatrix.inc, 22]
                             [ShowMatrix.inc, 36]
Row =    6.06e-001   1.50e+000  -2.98e+000 [ShowMatrix.inc, 50]
Row =    1.37e+000   3.52e+000  -2.60e+000 [ShowMatrix.inc, 50]
Row =    3.76e-002  -3.92e+000   7.72e+000 [ShowMatrix.inc, 50]
Row =    1.50e+000  -2.98e+000   4.25e+000 [ShowMatrix.inc, 50]
──────────────────────────────────────────────────────────────────────
Row =   -2.59e-016 [ShowMatrix.inc, 50]
Row =    3.52e-016 [ShowMatrix.inc, 50]
Row =    3.28e-014 [ShowMatrix.inc, 50]
Vh percentage -------------- [ShowMatrix.inc, 281]
porH[eax*8] =  7.752153662707E-0003 [ShowMatrix.inc, 286]
porH[eax*8] =  1.052502820510E-0002 [ShowMatrix.inc, 286]
porH[eax*8] =  9.817228181322E-0001 [ShowMatrix.inc, 286]


Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 6 (Matrix), Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)
Object ID = 0, Error Code = 536871012 (MTX_SRCCOLDIF: Source matrix has an uncompatible column number)

HSE

Thanks LiaoMi !

What crash is DebugCenter, this program use OA32 DebugCenter.

I have a different setting now and something of ObjAsm32 it's not working very well  :biggrin: 

Between things differents I don't found my old DebugCenter with black background :sad:

Equations in Assembly: SmplMath

HSE

Hi!!

The project was modified to work with ObjAsm (only in 32 bits now  :biggrin:), mainly Matrix object.

Some object error names in ObjErrs.inc must be actualized.

Solved the problem of object error (that don't affected the results).

After crashes because DebugCenter type, there was a "final crash" because a register is not preserved  :eusa_boohoo:

Still is in development.

Updated in previous post.

Regards, HSE

Equations in Assembly: SmplMath

LiaoMi

Quote from: HSE on May 17, 2021, 05:52:43 AM
Hi!!

The project was modified to work with ObjAsm (only in 32 bits now  :biggrin:), mainly Matrix object.

Some object error names in ObjErrs.inc must be actualized.

Solved the problem of object error (that don't affected the results).

After crashes because DebugCenter type, there was a "final crash" because a register is not preserved  :eusa_boohoo:

Still is in development.

Updated in previous post.

Regards, HSE

Hi HSE,

now the program does not start  :sad:

Row =    6.83e-001   5.06e-001  -3.00e+000 [ShowMatrix.inc, 54]
Row =    7.79e-001   3.03e+000  -2.75e+000 [ShowMatrix.inc, 54]
Row =    1.51e-001  -3.39e+000   6.74e+000 [ShowMatrix.inc, 54]
Row =    5.06e-001  -3.00e+000   3.48e+000 [ShowMatrix.inc, 54]
──────────────────────────────────────────────────────────────────────
Row =   -1.49e-016 [ShowMatrix.inc, 54]
Row =    2.67e-016 [ShowMatrix.inc, 54]
Row =    2.59e-014 [ShowMatrix.inc, 54]
Vh percentage -------------- [NNet_BG97.inc, 302]
porH[eax*8] =  5.666900849863E-0003 [NNet_BG97.inc, 307]
porH[eax*8] =  1.015015881378E-0002 [NNet_BG97.inc, 307]
porH[eax*8] =  9.841829403364E-0001 [NNet_BG97.inc, 307]
──────────────────────────────────────────────────────────────────────
----------------------------- [Faked.asm, 146]
          Terminу             [Faked.asm, 147]
----------------------------- [Faked.asm, 148]


HSE

Hi LiaoMi!!

:biggrin: Solution it's not showed. That is part of Boger and Guterman analysis of solution.

The problem with Neural Network is exactly that solution is beyond understanding. Novelty of Bogdan and others was that some understanding is posible. But today, the idea is that understanding still is minimum.


You can see results of the evolution of solution sending predictions to whatever graphic. In method Muestra2:
    OCall lpXorNet::NNet.run, addr inputs_train
    NeuronStage = 2
    NeuronsOut
    CargaPunto gral_1 , outputs_train  ; <------ Graphic (x,y)


Regards, HSE.
Equations in Assembly: SmplMath

mineiro

Quote from: HSE on May 18, 2021, 11:04:28 PM
The problem with Neural Network is exactly that solution is beyond understanding.
Hello sir HSE, I hope you're fine. Maybe I don't understood your phrase.

Neural networks are in theory simple to understand.
Neurons have dendrites (input), axons (output), like any truth Boolean table. However, in the middle of the input and output there is some mathematical function, called synthesis, as a math point of view.
As in electronics, there can be feedback, the output becoming a variable for the input, in order to readjust the criteria when an error occurs.
Most of the mathematical functions that I perceive are exponential, factorial, polynomial ... . We know that today when dealing with calculations of these types, computers become abacious in terms of processing speed.

Basically we provide values ​​for the input, synthesize the data and compare the output with the desired result. If an error occurs, I mean, if the desired output was not obtained, we change and retrain the values. In easy words, it's a math function that auto adjust values. The learning process, the knowledge is not retain in only one neuron, that's shared between all neurons, variables.
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

LiaoMi

@HSE,
thanks, I will study this example  :rolleyes:

Another assembly project  :thup:
A multy-layer feed-forward neural network implementation in assembly x86 32 bits
https://github.com/idoheinemann/Assembly-Neural-Network
Source - https://github.com/idoheinemann/Assembly-Neural-Network/archive/refs/heads/master.zip