Home / yet another political map simulator / Yet Another Neural Network Simulator

Yet Another Neural Network Simulator - yet another political map simulator


Yet Another Neural Network Simulator-yet another political map simulator

Yet Another Neural Network Simulator
R. Bon?, M. Crucianu, J.P. Asselin de Beauville
Equipe Reconnaissance des Formes et Analyse d?Images
Laboratoire d'Informatique
Ecole d?Ing?nieurs en Informatique pour l?Industrie, Universit? de Tours
64, avenue Jean Portalis, 37200, Tours, FRANCE
{bone, crucianu, asselin}@univ-tours.fr
Abstract: YANNS (Yet Another Neural Network Simulator) is a new object-oriented neural network
simulator for feedforward networks as well as general recurrent networks. The goal of this project is to
develop and implement a simulation tool that satisfies the following constraints: flexibility, ease of use,
portability and efficiency. The result is a simulator with the kernel implemented as a collection of C++
classes, and with two interfaces: a high-level network specification language and a Web-based graphical user
interface. These interfaces provide the means for a painless presentation of the features of neural networks to
students or engineers. The object oriented design provides a valuable software environment for the
researchers who wish to develop and study new architectures and algorithms.
1. Introduction 2. Flexibility
Why build another general purpose The YANNS simulator consists of 3 main
simulation tool for neural networks? The last few components: a simulator kernel, a description
years have witnessed the development and language and a graphical user interface. With
distribution of a large number of neural network flexibility in mind, we decided to develop the neural
simulators. We could mention Aspirin/MIGRAINE network simulation kernel in C++. This allowed us
[1], GENESIS [2], PlaNet [3], RCS [4], SNNS [5], to obtain a high level of encapsulation and
Xerion [6], etc. To this non-exhaustive list of modularity. Since there is such a large overlap
freeware tools, one must add several commercial between the different possible neural network
packages. It is not our purpose here to provide a architectures, as well as between the various
complete review of the existing neural networks elementary components of each of these
simulation tools, but we can highlight some common architectures, we believe we were not too ambitious
characteristics for most of these systems: they are when trying to describe most of the neurons and
written in the C language for only a few platforms architectures we know about. The task was made
(Unix or Windows) and provide a system-dependent easier by the very nature of neural networks: neurons
interface. Because of the very specific data structures and their interconnection are common features to
employed, very few of them can deal with general almost all of the neural networks, and multilayer
recurrent neural network models. networks are only a particular case of general
For the purpose of this introduction, we can recurrent neural networks. A large part of the
only mention that no currently available simulation inheritance graph was very natural to conceive.
tool (to the best of our knowledge) is flexible and Routines for building the neurons and the networks,
general enough to provide support for complex and for performing learning or test and for interacting
detailed models of neural networks, in a full-blown with the simulator during the various steps of
object-oriented and multi-platforms approach. network development should be common, whatever
The goal of this project was to create an the architecture or the training algorithm.
efficient and flexible object-oriented neural network The resulting kernel software consists of
simulation environment for teaching, research and approximately 10,000 lines of C++ code in which
application development in the field of neural various C++ classes (Fig. 1) mirror the structure and
networks. As a consequence, we followed four major function of neurons and networks.
guidelines in the design of YANNS: flexibility, ease The first top-level objects are the Networks.
of use, portability, and efficiency. Networks are described by the neurons they contain,
the lists of input and output neurons and possibly a
specific structure as for the layered networks. All the
learning algorithms are methods of these network 3. Ease of use
classes.
A neuron is an object that has a net input, The building of a neural network is
an external input, an output and a transfer function typically a very difficult phase of the simulation,
for mapping the net input to the output. Neurons are especially for beginners. To make the simulator
connected to one another by connections. Each accessible to people with limited experience in
neuron contains the lists of its predecessors and programming or in the field of neural networks, we
successors. Each connection may be weighted for a defined a specification language which is as simple
simple neuron or have a list of weights for FIR or and intuitive as possible.
IIR neurons ([7, 8]). The FIR and IIR neurons In addition, the simulator uses a Web-based
contain connections with delays (Fig. 2 and Fig. 3). interface. An HTML interface with a few Java
To be general enough, we allow for non-consecutive applets (e.g. for error display) is already in use. See
delays ( ). Fig. 5 for an experiment definition form. We actually
develop a fully graphical interface in Java, including
graphical network specification tools and kernel
General Neuron control during the simulation run. We focus here on
Network
the high level network specification language.
This language provides all the network
Layered Bias Simple FIR specification and control functions required. It
Network Neuron Neuron Neuron
allows one to specify the details of the architecture
of the neural network, as well as the different
IIR algorithms and the parameters to employ. Various
Neuron
consistency controls are being performed by the
parser and explicit error messages are available.
Weights Init. TranfertFn Complex networks can be created quickly and easily,
Law
in a way well suited for inexperienced users who
want to learn about neural network models with the
Uniform Gaussian Linear Sigmoid ... help of the simulator.
Law Law Function Function
inheritance use In the next example, we describe a recurrent
network with delays employing the Back-
Fig. 1. Graph of the main classes. Propagation Through Time algorithm [9] for
learning. The network is represented in Fig. 4.
The description language is powerful
enough to allow us to specify a complex simulation
in only a few lines of text.
neuron s j
p) A YANNS description file (see below)
j bi(j
(3)
(2) bij consists of a set of blocks, with one or more blocks
(1) bij
(0) bij for the definition of the neurons, followed by the
bij
neuron
blocks dedicated to the connections. A particular
i block describes the connections with the bias neuron,
if it exists. The script block describes the common
Fig. 2. FIR neuron
data set for the different learning algorithms. A
subset from each data set can be selected.
neuron s j
p) NoNeurons=5;
j bi(j
(3)
(2) bij
(1) bij Neurons [1..5] {
(0) bij
bij TransfertFN=Linear;
neuron }
i
q) Input=[1];
ai(j
(3)
aij
(2) Output=[5];
aij
(1)
aij

Fig. 3. IIR neuron
Bias { ? RTRL and optimized RTRL
target=[2, 3, 4]; ? Momentum term
} ? Weight decay
Connections { ? Stochastic components
source=[1]; ? Update mode: off-line or on-line
target=[2..4];
type=Simple; ? Error function: quadratic or cross-entropy
} ? Stopping criterion: maximum number of
iterations, minimum error on the stop set or on
Connections { the learning set
source=[2..4];
target=[2..4]; ? Transfer functions: linear, sigmoid, tanh, ...
type=Simple; ? The user can specify the values for some of the
} weights and initialize the others randomly, from a
Connections {
uniform or Gaussian distribution
source=[2..4]; ? Default values are available for all the parameters
target=[5];
type=FIR(0 2);
} 4. Portability and efficiency
Script {
Path="./"; Several of the simulators mentioned earlier
Learning="corpus.set"; are limited to a single machine type because of the
Stop="stop.set"; reliance on proprietary graphical systems. An easy
} way to develop a program for Unix, Windows and
Learning { Mac platforms is to use the Java programming
Algorithm=BPTT(0.01); language. Unfortunately, this language is currently
NoMaxStep=50000; too slow, due to its nature and its youth.
MinStopError=0.001
} We decided to use the best of Java, its
ability to develop universal graphical interfaces, to
add an interface which operates on any platform with
a Java-compliant Web browser. But the kernel of
5 YANNS, implementing all the learning algorithms,
was developed in C++.
The Web-based interface also allows us to
have control the simulator from distant machines. In
the current version, the Web-based interface includes
some Java applets in HTML files but most of the
2 3 4 control is performed with the help of forms.
Nevertheless, this interface allows us to use all the
features of the kernel. Data files, network and
experiment description files, or resulting files can be
automatically transferred (an ftp access is required).
1
We are now developing a pure Java interface, with
increased flexibility.
Even without a Web browser, the user can
Fig. 4. A simple recurrent neural network described by the use the text interface to work with YANNS directly
language. or to set up batch learning sessions.
The kernel of YANNS was written in a
Some tests performed with the help of a panel of normalized version of C++. We compiled the kernel
students show that this language allows one to and the interpreter of the description language with
reduce significantly the developing time as compared numerous compilers and operating systems, with no
to some others simulation tools. With his high level trouble.
Taking into account the constraints of
network specification language or the soon available modularity and ease of use, it is necessary to keep
graphical user interface, YANNS provides a the code as efficient as possible. This can be
complete set of learning algorithms and ways to accomplished by using efficient data structures and
initialize the network and control the learning: numerical methods. These objectives are often
? Back-Propagation for layered networks inconsistent with modularity. However, our
composed of simple, FIR and IIR neurons comparisons show that the learning times obtained
? Back-Propagation Through Time for simple and with YANNS compare favorably to those of several
FIR neurons