Difference between revisions of "Oja's rule"

From Eyewire
Jump to: navigation, search
(Model)
 
(4 intermediate revisions by 3 users not shown)
Line 1: Line 1:
'''Oja's rule''', developed by Finnish computer scientist Erkki Oja in 1982, is a stable version of [[Hebb's rule]].<ref name="Oja82">{{cite journal |last=Oja |first=Erkki |year=1982 |month=November |title=Simplified neuron model as a principal component analyzer |journal=Journal of Mathematical Biology |volume=15 |issue=3 |pages=267–273 |id=BF00275687 |url=http://www.springerlink.com/content/u9u6120r003825u1/ |accessdate= 2007-11-22 |quote= |doi=10.1007/BF00275687 |pmid=7153672 }}</ref>
+
<translate>
 +
 
 +
'''Oja's rule''', developed by Finnish computer scientist Erkki Oja in 1982, is a stable version of [[Hebb's rule]].<ref name="Oja82">Oja, Erkki (November 1982). [http://www.springerlink.com/content/u9u6120r003825u1 "Simplified neuron model as a principal component analyzer"]. <em>Journal of Mathematical Biology</em> <strong>15</strong> (3): 267–273</ref>
  
 
==Model==
 
==Model==
  
[[File:ArtificialNeuronModel english.png|thumb|right|400px|Model of a neuron. For a linear neuron, the activation function is not present (or simply the identity function).]]
+
[[File:ArtificialNeuronModel english.png|thumb|right|400px|Model of a neuron. <em>j</em>&nbsp;is the index of the neuron when there is more than one neuron. For a linear neuron, the activation function is not present (or simply the identity function).]]
  
As with Hebb's rule, we use a linear neuron. Given a set of k-dimensional inputs represented as a column vector <math>\vec{x} = [x_1, x_2, \cdots, x_k]^T</math>, and a linear neuron with (initially random) synaptic weights from the inputs <math>\vec{w} = [w_1, w_2, \cdots, w_k]^T</math> the output the neuron is defined as follows:
+
As with Hebb's rule, we use a linear neuron. Given a set of k-dimensional inputs represented as a column vector [[File:Hebb1.png]], and a linear neuron with (initially random) [[Synapse|synaptic]] weights from the inputs [[File:Hebb2.png]] the output the neuron is defined as follows:
  
<center><math>y = \vec{w}^T \vec{x} = \sum_{i=1}^k w_i x_i</math></center>
+
[[File:Hebb3.png|center]]
  
 
Oja's rule gives the update rule which is applied after an input pattern is presented:
 
Oja's rule gives the update rule which is applied after an input pattern is presented:
  
<center><math>\Delta \vec{w} = \eta  y (\vec{x} - y \vec{w})</math></center>
+
[[File:Oja1.png|center]]
  
Oja's rule is simply Hebb's rule with weight normalization, approximated by a Taylor series with terms of <math>\eta^n</math> ignored for n>1 since <math>\eta</math> is small.
+
Oja's rule is simply Hebb's rule with weight normalization, approximated by a Taylor series with terms of [[File:Oja2.png]] ignored for n>1 since η is small.
  
 
It can be shown that Oja's rule extracts the first principal component of the data set. If there are many Oja's rule neurons, then all will converge to the same principal component, which is not useful. [[Sanger's rule]] was formulated to get around this issue.
 
It can be shown that Oja's rule extracts the first principal component of the data set. If there are many Oja's rule neurons, then all will converge to the same principal component, which is not useful. [[Sanger's rule]] was formulated to get around this issue.
Line 19: Line 21:
 
==References==
 
==References==
 
<references/>
 
<references/>
 +
 +
[[Category: Neural computational models]]
 +
 +
</translate>

Latest revision as of 03:23, 24 June 2016

Oja's rule, developed by Finnish computer scientist Erkki Oja in 1982, is a stable version of Hebb's rule.[1]

Model

Model of a neuron. j is the index of the neuron when there is more than one neuron. For a linear neuron, the activation function is not present (or simply the identity function).
As with Hebb's rule, we use a linear neuron. Given a set of k-dimensional inputs represented as a column vector
生成缩略图出错:无法将缩略图保存到目标地点
, and a linear neuron with (initially random) synaptic weights from the inputs
生成缩略图出错:无法将缩略图保存到目标地点
the output the neuron is defined as follows:
生成缩略图出错:无法将缩略图保存到目标地点

Oja's rule gives the update rule which is applied after an input pattern is presented:

生成缩略图出错:无法将缩略图保存到目标地点
Oja's rule is simply Hebb's rule with weight normalization, approximated by a Taylor series with terms of
生成缩略图出错:无法将缩略图保存到目标地点
ignored for n>1 since η is small.

It can be shown that Oja's rule extracts the first principal component of the data set. If there are many Oja's rule neurons, then all will converge to the same principal component, which is not useful. Sanger's rule was formulated to get around this issue.

References

  1. Oja, Erkki (November 1982). "Simplified neuron model as a principal component analyzer". Journal of Mathematical Biology 15 (3): 267–273