Difference between revisions of "Contrastive Hebbian learning"

From Eyewire
Jump to: navigation, search
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
<translate>
 +
 
'''Contrastive Hebbian learning''' is an error-driven learning technique. It is a ''supervised'' learning technique, meaning that the desired outputs are known beforehand, and the task of the network is to learn to generate the desired outputs from the inputs.
 
'''Contrastive Hebbian learning''' is an error-driven learning technique. It is a ''supervised'' learning technique, meaning that the desired outputs are known beforehand, and the task of the network is to learn to generate the desired outputs from the inputs.
  
Line 13: Line 15:
 
Given a set of k-dimensional inputs with values between 0 and 1 represented as a column vector:
 
Given a set of k-dimensional inputs with values between 0 and 1 represented as a column vector:
  
<center><math>\vec{x} = [x_1, x_2, \cdots, x_k]^T</math></center>
+
[[File:Hebb1.png|center]]
  
 
and a nonlinear neuron with (initially random, uniformly distributed between -1 and 1) synaptic weights from the inputs:
 
and a nonlinear neuron with (initially random, uniformly distributed between -1 and 1) synaptic weights from the inputs:
  
<center><math>\vec{w} = [w_1, w_2, \cdots, w_k]^T</math></center>
+
[[File:Hebb2.png|center]]
  
then the output <math>y</math> of the neuron is defined as follows:
+
then the output <em>y</em> of the neuron is defined as follows:
  
<center><math>\begin{align}
+
[[File:APRBp1.png|center]]
y &= \varphi \left ( n \right )
+
\end{align}</math></center>
+
  
where <math>\varphi \left ( \cdot \right )</math> is a sigmoidal function such as that used in ordinary [[feedforward backpropagation]] (we will use the logistic function from that page), and <math>n</math> is the net input of the neuron. During the positive phase, the net is written as <math>n^+</math> with the output being <math>y^+</math>, and during the negative phase, the net is written as <math>n^-</math> with the output being <math>y^-</math>.
+
where [[File:RBM3.png]] is a sigmoidal function such as that used in ordinary [[feedforward backpropagation]] (we will use the logistic function from that page), and <em>n</em> is the net input of the neuron. During the positive phase, the net is written as [[File:CHL1.png]] with the output being [[File:CHL2.png]], and during the negative phase, the net is written as [[File:CHL3.png]] with the output being [[File:CHL4.png]].
  
Let there be <math>N</math> neurons where some of the neurons are simple inputs to the network, some are outputs, with the weight of the connection from neuron <math>i</math> to neuron <math>j</math> being <math>w_{ij}</math>. Also make sure that <math>w_{ij} = w_{ji}</math>, that is, the connections and their weights are symmetric. Obviously if one of the neurons is an input, the connection going into it is useless, but nevertheless this is convenient.
+
Let there be <em>N</em> neurons where some of the neurons are simple inputs to the network, some are outputs, with the weight of the connection from neuron <em>i</em> to neuron <em>j</em> being [[File:Sanger2.png]]. Also make sure that [[File:APRBp4.png]], that is, the connections and their weights are symmetric. Obviously if one of the neurons is an input, the connection going into it is useless, but nevertheless this is convenient.
  
The positive phase net <math>n^+_j</math> of neuron <math>j</math> (where <math>j</math> is neither an input neuron nor an output neuron) is computed using a discrete time approximation to the following equation, iteratively applied to all neurons until the nets settle to some equilibrium state. Initially set <math>n^+_j</math> to 0 for all non-input and non-output neurons.
+
The positive phase net [[File:CHL5.png]] of neuron <em>j</em> (where <em>j</em> is neither an input neuron nor an output neuron) is computed using a discrete time approximation to the following equation, iteratively applied to all neurons until the nets settle to some equilibrium state. Initially set [[File:CHL5.png]] to 0 for all non-input and non-output neurons.
  
<center><math>\frac{\mathrm{d} n^+_j}{\mathrm{d} t} = -n^+_j + \sum_{i=1}^N w_{ij} y^+_i</math></center>
+
[[File:CHL6.png|center]]
  
 
where:
 
where:
  
<center><math>y^+_i =
+
[[File:CHL7.png|center]]
\begin{cases}
+
\varphi \left ( n^+_i \right ) & \text{ if } i \text{ is neither an input nor an output} \\
+
x_i & \text{ if } i \text{ is an input } \\
+
t_i & \text{ if } i \text{ is an output }
+
\end{cases}</math></center>
+
  
where <math>t_i</math> is a target output. Effectively, the output neurons are clamped to their target values, the inputs are applied, and the rest of the neurons are allowed to settle to equilibrium. Note that when equilibrium is reached, <math>\mathrm{d} n^+_j / \mathrm{d} t = 0</math>, and the nets are precisely equal to the weighted sum of their inputs, as expected.  
+
where [[File:CHL8.png]] is a target output. Effectively, the output neurons are clamped to their target values, the inputs are applied, and the rest of the neurons are allowed to settle to equilibrium. Note that when equilibrium is reached, [[File:CHL9.png]], and the nets are precisely equal to the weighted sum of their inputs, as expected.  
  
Next, the negative phase nets <math>n^-_j</math> are computed in nearly the same way, except this time start with the nets from the positive phase,<ref>{{cite book|url=http://mplab.ucsd.edu/wp-content/uploads/CHL90.pdf|title=Connectionist Models: Proceedings of the 1990 Summer School|first=Javier R.|last=Movellan|chapter=Contrastive Hebbian learning in the continuous Hopfield model|date=April 1991|publisher=Morgan Kaufmann Publishers|isbn= 978-1558601567}}</ref> and the output neurons are not clamped:
+
Next, the negative phase nets [[File:CHL10.png]] are computed in nearly the same way, except this time start with the nets from the positive phase,<ref>Movellan, Javier R. (April 1991). [http://mplab.ucsd.edu/wp-content/uploads/CHL90.pdf "Contrastive Hebbian learning in the continuous Hopfield model"]. In <em>Connectionist Models: Proceedings of the 1990 Summer School</em>. Morgan Kaufmann Publishers. ISBN 978-1558601567</ref> and the output neurons are not clamped:
  
<center><math>\frac{\mathrm{d} n^-_j}{\mathrm{d} t} = -n^-_j + \sum_{i=1}^N w_{ij} y^-_i</math></center>
+
[[File:CHL11.png|center]]
  
 
where
 
where
  
<center><math>y^-_i =
+
[[File:CHL12.png|center]]
\begin{cases}
+
\varphi \left ( n^-_i \right ) & \text{ if } i \text{ is not an input } \\
+
x_i & \text{ if } i \text{ is an input } \\
+
\end{cases}</math></center>
+
  
Note that if the weights between pairs of neurons are symmetric, that is, <math>w_{ij} = w_{ji}</math>, then the network is guaranteed to settle to an equilibrium state.<ref>{{cite journal|last=Hopfield|first=J. J.|title=Neurons with graded response have collective computational properties like those of two-state neurons|journal=Proceedings of the National Academy of Sciences of the United States of America|volume=81|pages=3088-3092|date=May 1984|url=http://www.pnas.org/content/81/10/3088.full.pdf}}</ref> If symmetry is not held, the network will often settle.<ref=Galland1991>{{cite book|chapter=Deterministic Boltzmann learning in networks with asymmetric connectivity|title=Connectionist Models: Proceedings of the 1990 Summer School|last1=Galland|first1=C. G.|last2=Hinton|first2=G. E.|pages=3-9|date=April 1991|publisher=Morgan Kaufmann Publishers|isbn=978-1558601567}}</ref> Of course, if <math>i</math> is an input, then <math>w_{ji}</math> does not exist.
+
Note that if the weights between pairs of neurons are symmetric, that is, [[File:APRBp4.png]], then the network is guaranteed to settle to an equilibrium state.<ref>Hopfield, J. J. (May 1984). [http://www.pnas.org/content/81/10/3088.full.pdf "Neurons with graded response have collective computational properties like those of two-state neurons"]. <em>Proceedings of the National Academy of Sciences of the United States of America</em>. <strong>81</strong>: 3088-3092</ref> If symmetry is not held, the network will often settle.<ref name=Galland1991>Galland, C. G.; Hinton, G. E. (April 1991). "Deterministic Boltzmann learning in networks with asymmetric connectivity" In <em>Connectionist Models: Proceedings of the 1990 Summer School</em> Morgan Kaufmann Publishers. pp. 39-9 ISBN 978-1558601567</ref> Of course, if <em>i</em> is an input, then [[File:APRBp5.png]] does not exist.
  
 
Once the positive and negative nets of the neurons are determined, the weights are updated according to the following equation:
 
Once the positive and negative nets of the neurons are determined, the weights are updated according to the following equation:
  
<center><math>\Delta w_{ij} = \eta \left [ \left ( y^+_i y^+_j \right ) - \left ( y^-_i y^-_j \right ) \right ] </math></center>
+
[[File:CHL13.png|center]]
  
where <math>\eta</math> is some small learning rate. When the network learns the target, the negative phase exactly cancels out the positive phase, and there is no net change in weight. Also note that <math>\Delta w_{ij} = \Delta w_{ji}</math>, meaning that complementary weights go up and down in lockstep.
+
where η is some small learning rate. When the network learns the target, the negative phase exactly cancels out the positive phase, and there is no net change in weight. Also note that [[File:CHL14.png]], meaning that complementary weights go up and down in lockstep.
  
 
== Relation to cross-entropy ==
 
== Relation to cross-entropy ==
  
If, as discussed in the [[feedforward backpropagation]] derivation, the update to the weight is a gradient descent on the cross-entropy of the network, that is, <math>\Delta w_{ij} \propto - \partial E / \partial w_{ij}</math>, then the change in weight due to the positive phase, where the outputs are clamped to the target, results in a lowering of cross-entropy  where the output is the target. The change in weight due to the negative phase, however, where the outputs are not the target, results in a raising of cross entropy where the output is not the target.
+
If, as discussed in the [[feedforward backpropagation]] derivation, the update to the weight is a gradient descent on the cross-entropy of the network, that is, [[File:CHL15.png]], then the change in weight due to the positive phase, where the outputs are clamped to the target, results in a lowering of cross-entropy  where the output is the target. The change in weight due to the negative phase, however, where the outputs are not the target, results in a raising of cross entropy where the output is not the target.
  
This has the effect of "sculpting" the cross-entropy of the network so that it ends up lower where the output is closer to the target, and higher where the output is farther away from the target.<ref>{{cite web|url=http://hebb.mit.edu/courses/9.641/2006/lectures/chl.ppt.pdf|title=Contrastive Hebbian learning|last=Seung|first=Sebastian|accessdate=Apr 15, 2012}}</ref>
+
This has the effect of "sculpting" the cross-entropy of the network so that it ends up lower where the output is closer to the target, and higher where the output is farther away from the target.<ref>Seung, Sebastian. [http://hebb.mit.edu/courses/9.641/2006/lectures/chl.ppt.pdf "Contrastive Hebbian learning"]. Retrieved Apr 15, 2012.</ref>
  
  
 
==Biological plausibility==
 
==Biological plausibility==
  
Unlike in backpropagation modes such as [[feedforward backpropagation]] or [[Almeida-Pineda recurrent backpropagation]], Contrastive Hebbian learning does not depend on the sending of error information backwards along connections. All the information needed to alter the weight is available locally. However, there are two phases to the model. There is some speculation that this has an analog in biological processing, where the negative phase comes first, followed by a positive phase some 300 milliseconds later.<ref>{{cite journal|url=http://psych.colorado.edu/~oreilly/papers/OReilly96_generec_nc.pdf|last=O'Reilly|first=Randall C.|title=Biologically plausible error-driven learning using local activation differences: the general recirculation algorithm|journal=Neural Computation|volume=8|issue=5|pages=895-938|date=1996}}</ref>
+
Unlike in backpropagation modes such as [[feedforward backpropagation]] or [[Almeida-Pineda recurrent backpropagation]], Contrastive Hebbian learning does not depend on the sending of error information backwards along connections. All the information needed to alter the weight is available locally. However, there are two phases to the model. There is some speculation that this has an analog in biological processing, where the negative phase comes first, followed by a positive phase some 300 milliseconds later.<ref>O'Reilly, Randall C. (1996). [http://psych.colorado.edu/~oreilly/papers/OReilly96_generec_nc.pdf "Biologically plausible error-driven learning using local activation differences: the general recirculation algorithm"] <em>Neural Computation</em> <strong>8</strong> (5): 895-938</ref>
  
Contrastive Hebbian learning requires that weights be symmetric. There is some evidence that there are symmetric connections between cortical areas in the brain.<ref>{{cite journal|title=Distributed hierarchical processing in the primate cerebral cortex|last1=Felleman|first1=D. J.|last2=Van Essen|first2=D. C.|journal=Cerebral Cortex|date=Jan/Feb 1991|issue=1|pages=1-47|url=http://brainvis.wustl.edu/resources/FVE_CC91.pdf}}</ref> In addition, symmetric connectivity between individual neurons does not appear to be critical; as long as there is some bidirectional connectivity between some neurons, error signals can be obtained indirectly by the network.<ref=Galland1991/>
+
Contrastive Hebbian learning requires that weights be symmetric. There is some evidence that there are symmetric connections between cortical areas in the brain.<ref>Felleman, D. J.; Van Essen, D. C. (1991). [http://brainvis.wustl.edu/resources/FVE_CC91.pdf Distributed hierarchical processing in the primate cerebral cortex] <em>Cerebral Cortex</em> (1): 1-47</ref> In addition, symmetric connectivity between individual neurons does not appear to be critical; as long as there is some bidirectional connectivity between some neurons, error signals can be obtained indirectly by the network.<ref name=Galland1991/>
  
 
== References ==
 
== References ==
 
<references/>
 
<references/>
 +
 +
[[Category: Neural computational models]]
 +
 +
</translate>

Latest revision as of 03:10, 24 June 2016

Contrastive Hebbian learning is an error-driven learning technique. It is a supervised learning technique, meaning that the desired outputs are known beforehand, and the task of the network is to learn to generate the desired outputs from the inputs.

As opposed to a feedforward network, a recurrent network is allowed to have connections from any neuron to any neuron in any direction. However, unlike Almeida-Pineda recurrent backpropagation, there is no backpropagation of errors; weights are updated purely via local information.

There are two phases to the model, a positive phase, also called the Hebbian or learning phase, and a negative phase, also called the anti-Hebbian or unlearning phase.

Model

Model of a neuron. j is the index of the neuron when there is more than one neuron. The activation function for backpropagation is sigmoidal.
File:Artificial neural network.svg
A feedforward network. In the Contrastive Hebbian learning model, connections may go from any neuron to any neuron, backwards or forwards.


Given a set of k-dimensional inputs with values between 0 and 1 represented as a column vector:

Error creating thumbnail: Unable to save thumbnail to destination

and a nonlinear neuron with (initially random, uniformly distributed between -1 and 1) synaptic weights from the inputs:

Error creating thumbnail: Unable to save thumbnail to destination

then the output y of the neuron is defined as follows:

Error creating thumbnail: Unable to save thumbnail to destination
where
Error creating thumbnail: Unable to save thumbnail to destination
is a sigmoidal function such as that used in ordinary feedforward backpropagation (we will use the logistic function from that page), and n is the net input of the neuron. During the positive phase, the net is written as
Error creating thumbnail: Unable to save thumbnail to destination
with the output being
Error creating thumbnail: Unable to save thumbnail to destination
, and during the negative phase, the net is written as
Error creating thumbnail: Unable to save thumbnail to destination
with the output being
Error creating thumbnail: Unable to save thumbnail to destination
. Let there be N neurons where some of the neurons are simple inputs to the network, some are outputs, with the weight of the connection from neuron i to neuron j being
Error creating thumbnail: Unable to save thumbnail to destination
. Also make sure that
Error creating thumbnail: Unable to save thumbnail to destination
, that is, the connections and their weights are symmetric. Obviously if one of the neurons is an input, the connection going into it is useless, but nevertheless this is convenient. The positive phase net
Error creating thumbnail: Unable to save thumbnail to destination
of neuron j (where j is neither an input neuron nor an output neuron) is computed using a discrete time approximation to the following equation, iteratively applied to all neurons until the nets settle to some equilibrium state. Initially set
Error creating thumbnail: Unable to save thumbnail to destination
to 0 for all non-input and non-output neurons.
Error creating thumbnail: Unable to save thumbnail to destination

where:

Error creating thumbnail: Unable to save thumbnail to destination
where
Error creating thumbnail: Unable to save thumbnail to destination
is a target output. Effectively, the output neurons are clamped to their target values, the inputs are applied, and the rest of the neurons are allowed to settle to equilibrium. Note that when equilibrium is reached,
Error creating thumbnail: Unable to save thumbnail to destination
, and the nets are precisely equal to the weighted sum of their inputs, as expected. Next, the negative phase nets
Error creating thumbnail: Unable to save thumbnail to destination
are computed in nearly the same way, except this time start with the nets from the positive phase,[1] and the output neurons are not clamped:
Error creating thumbnail: Unable to save thumbnail to destination

where

Error creating thumbnail: Unable to save thumbnail to destination
Note that if the weights between pairs of neurons are symmetric, that is,
Error creating thumbnail: Unable to save thumbnail to destination
, then the network is guaranteed to settle to an equilibrium state.[2] If symmetry is not held, the network will often settle.[3] Of course, if i is an input, then
Error creating thumbnail: Unable to save thumbnail to destination
does not exist.

Once the positive and negative nets of the neurons are determined, the weights are updated according to the following equation:

Error creating thumbnail: Unable to save thumbnail to destination
where η is some small learning rate. When the network learns the target, the negative phase exactly cancels out the positive phase, and there is no net change in weight. Also note that
Error creating thumbnail: Unable to save thumbnail to destination
, meaning that complementary weights go up and down in lockstep.

Relation to cross-entropy

If, as discussed in the feedforward backpropagation derivation, the update to the weight is a gradient descent on the cross-entropy of the network, that is,
Error creating thumbnail: Unable to save thumbnail to destination
, then the change in weight due to the positive phase, where the outputs are clamped to the target, results in a lowering of cross-entropy where the output is the target. The change in weight due to the negative phase, however, where the outputs are not the target, results in a raising of cross entropy where the output is not the target.

This has the effect of "sculpting" the cross-entropy of the network so that it ends up lower where the output is closer to the target, and higher where the output is farther away from the target.[4]


Biological plausibility

Unlike in backpropagation modes such as feedforward backpropagation or Almeida-Pineda recurrent backpropagation, Contrastive Hebbian learning does not depend on the sending of error information backwards along connections. All the information needed to alter the weight is available locally. However, there are two phases to the model. There is some speculation that this has an analog in biological processing, where the negative phase comes first, followed by a positive phase some 300 milliseconds later.[5]

Contrastive Hebbian learning requires that weights be symmetric. There is some evidence that there are symmetric connections between cortical areas in the brain.[6] In addition, symmetric connectivity between individual neurons does not appear to be critical; as long as there is some bidirectional connectivity between some neurons, error signals can be obtained indirectly by the network.[3]

References

  1. Movellan, Javier R. (April 1991). "Contrastive Hebbian learning in the continuous Hopfield model". In Connectionist Models: Proceedings of the 1990 Summer School. Morgan Kaufmann Publishers. ISBN 978-1558601567
  2. Hopfield, J. J. (May 1984). "Neurons with graded response have collective computational properties like those of two-state neurons". Proceedings of the National Academy of Sciences of the United States of America. 81: 3088-3092
  3. 3.0 3.1 Galland, C. G.; Hinton, G. E. (April 1991). "Deterministic Boltzmann learning in networks with asymmetric connectivity" In Connectionist Models: Proceedings of the 1990 Summer School Morgan Kaufmann Publishers. pp. 39-9 ISBN 978-1558601567
  4. Seung, Sebastian. "Contrastive Hebbian learning". Retrieved Apr 15, 2012.
  5. O'Reilly, Randall C. (1996). "Biologically plausible error-driven learning using local activation differences: the general recirculation algorithm" Neural Computation 8 (5): 895-938
  6. Felleman, D. J.; Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex Cerebral Cortex (1): 1-47