Baldo's web log

Prerequisites

Hopfield Neural Network able to recall patterns

The g++ version I have used to code this Network is:

 
$ g++ --version 
g++ (Debian 4.3.2-1.1) 4.3.2 
Copyright (C) 2008 Free Software Foundation, Inc. 
This is free software; see the source for copying conditions.  There is NO 
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 
    

Compiling the Network

 
$ g++ main.cpp -o hopfield 
    

Running the Network

 
$ ./hopfield
*******************************
Hopfield Neural Network
able to recall binary patterns
*******************************
Presenting pattern A
 component = 1 output = 1 component matched
 component = 0 output = 0 component matched
 component = 1 output = 1 component matched
 component = 0 output = 0 component matched
* Pattern A recalled correctly
Presenting pattern B
 component = 0 output = 0 component matched
 component = 1 output = 1 component matched
 component = 0 output = 0 component matched
 component = 1 output = 1 component matched
* Pattern B recalled correctly
    

Try to modify the input patterns, for example by presenting a pattern like B = {0, 1, 0, 0}, the output would be like this:

 
.
.
Presenting pattern B
 component = 0 output = 0 component matched
 component = 1 output = 1 component matched
 component = 0 output = 0 component matched
 component = 0 output = 1 component not matched
* Unable to recall pattern B
    

meaning that the network was not trained to recall this pattern.

Source code

hopfield.h

 
    1 class Neuron{
    2 	private:
    3 		int *weightsVec;
    4 	public:
    5 		Neuron(int *, int);
    6         friend class Hopfield;
    7 };
    8 
    9 class Hopfield{
   10 	private:
   11 		Neuron **neurons;
   12 		int *output;
   13 		int n;
   14 		int threshold(int);
   15 		int dotProduct(int *, int *, int);
   16 	public:
   17 		Hopfield(int **, int);
   18 		void run(int  *);
   19 		int getOutput(int);
   20 };
   21 
   22 #include "hopfield.cpp"
    

hopfield.cpp

 
    1 Neuron::Neuron(int *w, int n){
    2 	weightsVec = new int[n];
    3 	for(int i = 0; i < n; i++)
    4 		weightsVec[i] = w[i];
    5 }
    6 
    7 //n = wLen
    8 //every neuron has its weight contribution to other neurons
    9 Hopfield::Hopfield(int **w, int n){
   10 	this -> n = n;
   11 	output = new int[n];
   12 	neurons = new Neuron*[n];
   13 
   14 	for(int i=0;i<n;i++)
   15 		neurons[i] = new Neuron(w[i],n);
   16 }
   17 
   18 //dot product pattern.weights (ej.  A.w1)
   19 int Hopfield::dotProduct(int *pattern, int *weights, int wLen){
   20 	int k = 0;
   21 	for(int i = 0; i < wLen; i++)
   22 		k += pattern[i] * weights[i];
   23 	return k;
   24 }
   25 
   26 /*
   27  f(t) = {1, if t >= theta; 0, if t < theta}
   28         where theta = 0
   29 */
   30 int Hopfield::threshold(int act){
   31 	if(act >= 0) return 1; else return 0;
   32 }
   33 
   34 void Hopfield::run(int *pattern){
   35  int act;
   36  for(int i = 0; i < n; i++){
   37 	 act = dotProduct(pattern, neurons[i]->weightsVec, n);
   38 	 output[i] = threshold(act);
   39    }
   40 }
   41 
   42 //return output of neuron j
   43 int Hopfield::getOutput(int j){
   44 	return output[j];
   45 }
    

main.cpp

 
    1 #include <stdio.h>
    2 #include <iostream>
    3 #include <math.h>
    4 
    5 #include "hopfield.h"
    6 
    7 using namespace std;
    8 
    9 int main(){
   10         // n is the number of neurons in the network
   11 	const int N = 4;
   12 	int recalled = 1;
   13 
   14 	//patterns to recall
   15 	int A[] = {1, 0, 1, 0}, B[] = {0, 1, 0, 1};
   16 
   17 	//weight matrix
   18 	int wm[N][N] = {
   19 			{ 0, -3,  3, -3},
   20 			{-3,  0, -3,  3},
   21 			{ 3, -3,  0, -3},
   22 			{-3,  3, -3,  0}
   23 		       };
   24 
   25 	int **w;
   26 	w = new int*[N];
   27 	for(int i = 0;i < N;i++) w[i] = wm[i];
   28 
   29 	cout<<"*******************************"<<endl;
   30 	cout<<"Hopfield Neural Network"<<endl;
   31 	cout<<"able to recall binary patterns"<<endl;
   32 	cout<<"*******************************"<<endl;
   33 
   34 	Hopfield net(w, N);
   35 
   36 	//run the network by presenting  pattern A.
   37 	cout<<"Presenting pattern A"<<endl;
   38 	net.run(A);
   39 
   40 	for(int i = 0; i < N; i++){
   41 		if(net.getOutput(i) == A[i]){
   42 			cout<<" component = "<<A[i];
   43 			cout<<" output = "<<net.getOutput(i);
   44 			cout<<" component matched"<<endl;
   45 		}
   46 		else{
   47 			cout<<" component = "<<A[i];
   48 			cout<<" output = "<<net.getOutput(i);
   49 			cout<<" component not matched"<<endl;
   50 			recalled = 0;
   51 		}
   52 	}
   53 
   54 	if(!recalled)
   55 		cout<<"* Unable to recall pattern A "<<endl;
   56 	else
   57 		cout<<"* Pattern A recalled correctly"<<endl;
   58 
   59 	recalled = 1;
   60 	//run the network by presenting pattern B.
   61 	cout<<"Presenting pattern B"<<endl;
   62 	net.run(B);
   63 
   64 	for(int i = 0; i < N; i++){
   65 		if(net.getOutput(i) == B[i]){
   66 			cout<<" component = "<<B[i];
   67 			cout<<" output = "<<net.getOutput(i);
   68 			cout<<" component matched"<<endl;
   69 		}
   70 		else{
   71 			cout<<" component = "<<B[i];
   72 			cout<<" output = "<<net.getOutput(i);
   73 			cout<<" component not matched"<<endl;
   74 			recalled = 0;
   75 		}
   76 
   77 	}
   78 
   79 	if(!recalled)
   80 		cout<<"* Unable to recall pattern B"<<endl;
   81 	else
   82 		cout<<"* Pattern B recalled correctly"<<endl;
   83 
   84 	return 1;
   85 }
    

Download this code

hopfield.tgz

---

This is an example of the Hopfield Neural Network with a single layer and four neurons. Two patterns (A,B) will be presented one at a time and the Network will be able to recall them.

This is the picture for this Net.

---

Neural Networks has been an interesting topic for me since I was studying my career. The basic idea about Neural Networks is inspired by the study of human neural processing. In terms of computer programming it's just a simple data structure in which nodes are interconnected.

These are some concepts:

Weights

The weights are used on the connections between different layers.

Training

Steps:

1. Start the neural network with one set of weights.
2. Run the network.
3. Modify some of all the weights.
4. Run the network again.

Repeat these steps until some predetermined goal is met. This process is called training.

No Training

Start the network with one set of weights and do not modify any of them.

Memory

There are two approaches to this concept:

1. Long-term memory (LTM) is memory associated with learning that persists for the long term
2. Short-term memory (STM) is memory associated with a neural network that decays in some time interval.

Supervised Learning

The learning would be supervised if external criteria are used and matched by the network output. You apply the inputs to the supervised network along with an expected response.

Unsupervised Learning / self-organizing

There are only inputs. There is more interaction between neurons, typically with feedback and intra-layer connections between neurons.

Binary and Bipolar Inputs

There are two types of inputs used in neural networks, Binary and Bipolar. Binary inputs have one of two values, 0 and 1; and Bipolar inputs have, 1 and –1; -1 of bipolar correspond to a 0 of binary.

Noise

Is perturbation in the data used to train the neural network. For example, an image may have random speckles in it.

---