|
NeuroCOLT
Technical Report NC-TR-95-028
Improvement
of Gradient Descent based Algorithms Training Multilayer Perceptrons
with an Evolutionnary Initialization
Cedric
Gegout
Ecole Normale Superieure de Lyon
Lyon
Abstract
Gradient descent algorithms reducing the mean square
error computed on a training set are widely used for training real
valued feedforward networks, because of their easy implementation
and their efficacy. But in some cases they are trapped in a local
optimum and are not able to find a good network. In order to eliminate
theses limitated cases, usually we could only restart the gradient
descent or found an initialization point constructed with unreliable
and training set dependant heuristics. This paper presents a new method
to find a good initialization point. An evolutionary algorithm provides
an individual whose phenotype is a neural network. This individual
is the best one that makes a quick, efficient and robust gradient
descent. The genotypes are real valued vectors containing parameters
of networks. Therefore we use special genetic operators. Simulation
results show that this initialization reduces the neural network training
time, the training complexity and improves the robustness of gradient
descent based algorithms.
Download Compressed
Postscript
|