|
NeuroCOLT
Technical Report NC-TR-99-049
Neural
Systems as Nonlinear Filters
Wolfgang Maass
Technische Universität Graz,
Eduardo D.Sontag
Rutgers University
Abstract
Experimental data show that biological
synapses behave quite differently from the symbolic synapses in all
common artificial neural network models. Biological synapses are dynamic,
i.e., their "weight" changes on a short time scale by several
hundred percent in dependence of the past input to the synapse.
In this article we address the question how this inherent synaptic
dynamics—which should not be confused with long term "learning"—
affects the computational power of a neural network. In particular
we analyze computations on temporal and spatio-temporal patterns,
and we give a complete mathematical characterization of all filters
that can be approximated by feedforward neural networks with dynamic
synapses. It turns out that even with just a single hidden layer such
networks can approximate a very rich class of nonlinear filters: all
filters that can be characterized by Volterra series. This result
is robust with regard to various changes in the model for synaptic
dynamics. Our characterization result provides for all nonlinear filters
that are approximable by Volterra series a new complexity hierarchy
which is related to the cost of implementing such filters in neural
systems.
Download Compressed Postscript
|