In this thesis we propose various extensions of Principal Component
Analysis in relation to neural network learning theory.
We first propose a new network model (APEX),
for multiple principal component extraction
which utilizes the normalized Hebbian learning rule
for training all the synaptic weights.
The network features a novel structure of
lateral connections among the output units
for the orthogonalization of the weights.
The exponential convergence of the model is formally proved
while there is significant performance improvement
over previous methods. Also the network
is able to grow or shrink
without need for retraining the old units.
Furthermore, APEX is proved to be applicable to
the Constrained PCA problem where the signal variance
is maximized under external orthogonality constraints.
Another PCA extension introduced in this thesis is
Oriented PCA where the variance ratio between two signals is maximized.
We propose two learning models
for OPC extraction,
(justified both mathematically and by simulation)
based again on the Hebbian rule.
The least squares criterion for a supervised
two-layer linear network with reduced hidden units
is related to a Generalized SVD problem which we call Asymmetric
PCA (since PCA is a special case when input and teacher are the same).
We show that back-propagation extracts
the APCA component subspace but we also propose a new model
using the Lateral Orthogonalization Net for extracting the
exact components. Another PCA extension is Cross-correlation
APCA where the cross-correlation between
two signals is maximized.
A new model featuring a LON is proposed and proved
to work for this task.
We then study some of the applications of the new models
which require adaptive solution methods, in particular
signal detection, system identification and image compression
applications.

Finally, we address the problem of motion compensation
for image coding. Standard block-matching techniques
introduce block artifacts in the predicted picture
which are not easily removed at very low bit rates.
We propose a new motion compensation technique using a
histogram-based segmentation scheme that reduces
the artifacts at no cost to the decoder. The technique achieves
reduced bit-rate for the same error compared to the standard algorithm.
