The diagram above shows a simple associator net. It is possibly not much smaller or simpler than a real basic neural net might be. Nice and simple; nice and economic; nice and easy. This is a 4 x 4 cell net, providing 16 synapses. In other words 4 cells are interacting with 4 other cells and they make a total of 16 synaptic connections between each other. Because cell processes can be very short or very long, the cells might actually be close or distant. This would make no difference to the function of the net. We draw the net’s components close and in a stylised format for simplicity’s sake. The diagram is highly simplified.

In our diagram let us imagine that this net, at this time, is set up to associate information from the visual cortex and from the olfactory cortex. We will imagine the net associates the sight and smell of roses. Four cells from the visual cortex pass the information that a rose has been seen. Four cells from the olfactory cortex pass the information that a rose has been smelled. They act upon the two sets of four relay cells which connect across our little net as shown. The values and directions (+ or -) at relay cell and synapse level were ‘set’ (according to the ‘Hebb rule’) when our first roses were seen and smelled simultaneously; so long as these values and directions remain as they were originally set the net will always, and automatically, associate the one with the other thereafter.

What is the ‘Hebb rule’? How might a net learn, right down at this level? There must be an innate procedure which enables nets to establish themselves (if that’s what they do). A likely candidate for how a net might be pre-programmed to programme itself, on demand, (for example when a rose was first seen and smelled simultaneously) is the Hebb Rule. This rule tells circuitry how to set itself such as to associate one input with another. As expressed here it may be just a little too simple, but, in modified form, the Hebb rule may be the only rule your head needs to contain.

When units A and B are simultaneously activated, increase the strength of the connection between them. Adjust the strength of the connection between units A and B in proportion to the product of their simultaneous activation. (A stab at a ‘Hebb rule’ from Rumelhart & McClelland 1986 p. 36).

I commend the voluptuous simplicity of this paradigm. Our tiny net learned to associate the sight of a rose with the smell of a rose. It did this using nothing more than a simple switch-setting formula, a few synapses and two sets of input applied across these synapses. The association was learned locally, right down at switch level. The wiring organised itself. The neurons themselves, operating to the Hebb rule, or something like it, managed learning without reference to any imposed procedures. (What is learned, of course, depends on what is presented to the two sides of our nets.)

Let us go back to our diagram to examine its function in a little detail. Let us assume that the pattern stimulated upon the 4 ‘A’ relay cells, which maximally indicates ‘rose seen’, ‘reads out’ as +1, -1, +1, -1. (Full strength stimulation, two excitatory but two inhibitory.) If the ‘readout’ at the 4 ‘B’ relay cells which optimally indicates ‘rose smelled’ is –1, +1, +1, +1, then to achieve this the synapses must be ‘set’ at the values indicated on the diagram. The relay cells are set to deliver, when they fire, the synaptic values shown. The first ‘A’ relay cell delivers an inhibitory stimulation at – 0.25 on its first and second synapses but an excitatory stimulation at + 0.25 on the third and fourth. The second relay cell delivers stimulation to its synapses set exactly opposite, and so on across the 16 synapse net. The synapses are set with a value of 0.25 each, some being excitatory to that value and some inhibitory.

The effect an impulse on an ‘A’ relay cell will have on its ‘B’ relay cell (or vice versa) is the sum of the products of its synaptic impulse and its own impulse. The stimulus delivered by each synapse is the product of the relay cell’s value and the preset synaptic value. The stimulus delivered by synapse one on ‘A’ relay cell one is, for example 1 x - 0.25 = - 0.25. The stimulus delivered by the same cell’s fourth synapse is 1 x 0.25 = 0.25. The stimulus delivered by synapse one on ‘A’ relay cell two is -1 x 0.25 = - 0.25 and that delivered at that cell’s fourth synapse is -1 x – 0.25 = + 0.25. (In order to understand this last product you have to remember that a negative multiplied by a negative makes a positive - inhibiting an inhibition makes an excitation. Hence the cocktail?) Thus the top row of synapses impinging on the uppermost ‘B’ relay cell have values of - 0.25, - 0.25, - 0.25 & - 0.25 which totals – 1. The next ‘B’ relay cell receives synaptic stimuli to the same tune but the third and fourth receive 0.25, 0.25, 0.25 & 0.25, making a total each of + 1.