(Return to ICNN97 Homepage) (Return to ICNN'97 Agenda)
ICNN'97 FINAL ABSTRACTS
EO: ELECTRONICS & OPTICAL IMPLEMENTATIONS
ICNN97 Electronics & Optical Implementations Session: EO1A Paper Number: 584 Oral
An analog VLSI front-end for auditory feature analysis
Nagendra Kumar, Wolfgang Himmelbauer, Gert Cauwenberghs and Andreas G. Andreou
Keywords: analog VLSI auditory feature analysis TI-DIGITS database
Abstract:
Several researchers have found that signal processing of speech based on the principles of human auditory system leads to noise-robust speech recognition systems. However, it is impractical to implement such signal processing algorithms on a general-purpose digital computer, due to the enormous computational complexity of these algorithms. Therefore, we have implemented a auditory-signal-processing system using low-power, real-time analog and mixed-mode circuits. This implementation attempts to minimize the potential device-mismatch limitations, that can affect the performance of the final speech recognition system. The analog VLSI chip will serve as the front-end of a speech-recognition system. The chip architecture is inspired by biological auditory models common to humans and primate vertebrates. We include experimental results for a 1.2um CMOS prototype. We also include speech recognition results obtained from software simulations of the hardware on the TI-DIGITS database, in which we have used linear discriminant analysis to reduce the feature dimension, and to interface the auditory-features to hidden Markov models.
_____
ICNN97 Electronics & Optical Implementations Session: EO1B Paper Number: 377 Oral
Accurate analog VLSI model of calcium-dependent bursting neurons
A. Laflaquiere, S. Le Masson, J.P. Dom; G. Le Masson
Keywords: VLSI Model neurophysiology biological model
Abstract:
Our paper deals with an electronic full-custom circuit designed to accurately model the calcium dependence function of biological neurons. This ASIC is one in a set of modules dedicated for the analog modeling of neural networks and more specifically their use in real-time running hybrid experiments. Measurements and behaviors of modeled neurons are presented in various configurations and compared with theoretical predictions.
_____
ICNN97 Electronics & Optical Implementations Session: EO1C Paper Number: 232 Oral
Efficient VLSI implementation of a 3-layer threshold network
Jung H. Kim, Sung-Kwon Park and Youngnam Han
Keywords: VLSI threshold network Expand-and-truncate learning
Abstract:
In this paper, the learning algorithm called Expand-and-Truncate Learning (ETL) is proposed to synthesize a three-layer threshold network (TLTN) with guaranteed convergence for an arbitrary switching function. To the best of our knowledge, ETL is the first algorithm to synthesize a threshold network for an arbitrary switching function, automatically determining a required number of threshold elements in the hidden layer. For example, it turns out that the required number of threshold elements in the hidden layer of TLTN for an $n$-bit parity function is equal to $n$. Utilizing the fact that the threshold element in the proposed TLTN employs only integer weights and an integer threshold, we propose an efficient method to implement the proposed TLTN using current CMOS VLSI technology. The positive weights are realized using pMOS gates and negative weights using nMOS gates. The weights themselves are realized by manipulating the W/L (width/length) ratio of the respective transistor's channel.
_____
ICNN97 Electronics & Optical Implementations Session: EO1D Paper Number: 555 Oral
Implementing resistive fuse with floating gate MOS transistors
H. Nagai, T. Sakai, T. Sawaji, T. Kunishima and T. Matsumoto
Keywords: resistive fuse floating gate MOS transistors weak string filter
Abstract:
Resistive fuse is the key element for weak string filter which smoothes out noise while it detects and preserves step edges inherent to original data. Resistive fuse is implemented by two pairs of floating gate MOS transistors in a chip by a standard double poly CMOS process.
_____
ICNN97 Electronics & Optical Implementations Session: EO1E Paper Number: 78 Oral
Parallel architecture for vector quantization
Fabio Ancona, Stefano Rovetta and Rodolfo Zunino
Keywords: Parallel architecture vector quantization toroidal-mesh topology
Abstract:
The paper describes a parallel implementation of neural networks based on vector quantization. A toroidal-mesh topology has been used to assess the overall approach. A theoretical analysis of the modular system's efficiency is presented. The final application goal is a lossy compression of high-dimensional data for low bit-rate communications. Experimental results on a significant testbed shows a remarkable encrease of the system's performances. In addition, the fit between predicted and measured efficiency values confirms the validity of the overall theoretical model.
_____
ICNN97 Electronics & Optical Implementations Session: EO1F Paper Number: 166 Oral
Mappings of SOM and LVQ on the partial tree shape neurocomputer
Pasi Kolinummi, Timo Hämäläinen and Kimmo Kaski
Keywords: Self-Organizing Map (SOM), Learning Vector Quatization (LVQ), neurocomputer
Abstract:
Mappings of Self-Organizing Map (SOM) and Learning Vector Quatization (LVQ) networks are presented for a parallel neurocomputer system called PARNEU (PARtial tree shape NEUrocomputer). The partial tree shape architecture offers many mapping possibilities at several levels of parallelism for both execution and learning mode. In this paper we present both neuron and weight parallel mapping with on-line updating scheme. Computational complexity and the time required in each step are considered in order to compare mappings and to find out expected performance. About 8 MCUPS can be achieved with four PUs operating at the frequency of 40 MHz.
_____
ICNN97 Electronics & Optical Implementations Session: EO2A Paper Number: 90 Oral
ANNSyS: an analog neural network synthesis system
I. Bayraktaroglu, S. Ogrenci, G. Dundar, S. Balkir and E. Alpaydin
Keywords: analog neural network mixed signal implementation analog hardware
Abstract:
We present an Analog Neural Network Synthesis System based on a circuit simulator and a silicon assembler for neural networks. The circuit simulator makes use of the fact that neural networks with multilayer perceptron architecture consist of many decoupled blocks if the blocks are designed in MOS technology. We implement on-chip training on the software by incorporating the Madaline Rule III into our simulator. The assembler generates the layout by reading the standard cells from a library once the architecture of the network is given.
_____
ICNN97 Electronics & Optical Implementations Session: EO2B Paper Number: 631 Oral
Analog hardware implementation of adaptive filter structures
Jui-Kuo Juan, John G. Harris and Jose C. Principe
Keywords: Analog hardware adaptive filter VLSI
Abstract:
We have implemented a four-tap adaptive filter in a continuous-time analog VLSI circuit. Since an ideal delay is impossible to implement in continuous-time hardware, we implemented the delay line as a cascade of low-pass filters (called the Gamma filter). Since many years of research in our lab has shown that the Gamma filter outperforms the delay line for a wide range of applications, the Gamma filter should not be considered merely a crude approximation of the ideal delay line. We show measured results from an analog chip that solves the problem of system identification--identifying an unknown linear circuit from its input/output relationship. Furthermore, we believe that a cascade of all-pass filters (called the Laguerre filter) may potentially outperform the Gamma filter and we demonstrate a feedforward Laguerre filter still without adaptive weights.
_____
ICNN97 Electronics & Optical Implementations Session: EO2C Paper Number: 253 Oral
VHDL-based design of biologically inspired pitch detection systems
S. C. Lim, A. R. Temple, S. Jones and R. Meddis
Keywords: VHDL pitch detection systems signal processing
Abstract:
Engineering researchers at Loughborough University and hearing researchers at Essex University are working together to produce a real-time digital electronic pitch extraction system based on neuromorphic principles and implemented in digital technology. This paper reports on the VHDL simulations of the system which demonstrates good and robust pitch tracking across the human voice range.
_____
ICNN97 Electronics & Optical Implementations Session: EO2D Paper Number: 534 Oral
Effects of analog multiplier offsets on on-chip learning
Y. K. Choi, K. H. Ahn and S. -Y. Lee
Keywords: analog multiplier on-chip learning backpropagation
Abstract:
paper no. : 534
Offsets inherent in analog circuits have been big obstacle in analog implementations of backpropagation algorithm. In this article the effects of analog multiplier offsets on on-chip learning are systematically analyzed. Offsets in a multiplier are mathematically modeled and incoorporated into backpropagation learning equations. The deformed equations are investigated to show how the offsets degrade learning performance and under which conditions the neuron's output fails to converge. Simulation results agree well with analytic calculations.
_____
ICNN97 Electronics & Optical Implementations Session: EO3A Paper Number: 280 Oral
On the Cost effectiveness of logarithmic arithmetic for back propagation training on SIMD Processors
Mark G. Arnold, Thomas A. Baily, Jerry J. Cupal, Mark D. Winkel
Keywords: Back propagation training SIMD Processor Cost effectiveness
Abstract:
We show that back propagation on an SIMD processor with logarithmic arithmetic uses less memory and may be up to 3.2 times more cost effective than on one with fixed point arithmetic when synthesized in the same technology.
_____
ICNN97 Electronics & Optical Implementations Session: EO3B Paper Number: 458 Oral
Parallel implementation of backpropagation neural network on a heterogenous RING processor topology
Shou King Foo, P. Saratchandran and N. Sundararajan
Keywords: backpropagation heterogenous RING processor Parallel implementation
Abstract:
_____
ICNN97 Electronics & Optical Implementations Session: EO3C Paper Number: 133 Oral
Penalty terms for fault tolerance
Peter J. Edwards and Alan F. Murray
Keywords: Penalty terms fault tolerance learning algorithm
Abstract:
This paper presents penalty terms for fault tolerance enhancement.
We argue that the use of conventional learning algorithms leads to networks that have solutions that are non-optimally distributed and hence susceptible to faults. In addition we assert that fault tolerance will become an increasingly important factor in practical applications of neural networks. To address these issues we present a realistic model of hardware error and go on to propose a new method for optimizing fault tolerance via penalty terms. The penalty terms are incorporated into the learning to optimize the network for the smoothness of the solution locus. Such smoothness can be thought of as low average weight saliency and optimally distributed computation. We compare two roughness penalty terms with our previous work with weight-noise. Results from MLPs trained on two problems, one artificial and the other a real world task, show that fault tolerance can be achieved for a realistic fault model via the use of penalty terms.
_____
ICNN97 Electronics & Optical Implementations Session: EO3D Paper Number: 346 Oral
Ising Model calculation using PDM neural network hardware: Boltzmann statistical mechanics embedded in the hardware
Moritoshi Yasunaga and Yuzo Hirai
Keywords: Pulse Density Modulating NN. Boltzmann Statistical mechanics
Abstract:
We have proposed the network configuration for the Ising model calculation using a pulse-density-modulating neural network hardware.
The hardware consists of 54 asynchronous digital neurons with full interconnections. One logical spin in the Ising model is composed of two physical neuron circuits. We have analyzed the computation results of the hardware and it was demonstrated that the statistical behavior of the hardware was subject to Boltzmann statistical mechanics.
Utilizing this intrinsic dynamics, we calculated the magnetization and phase-transfer temperature of the Ising model without any programs and additional circuits, and the results closely agreed with theoretical ones.
_____
ICNN97 Electronics & Optical Implementations Session: EO3E Paper Number: 303 Oral
Spectral Noise-shaping in integrate-and fire neural network
Robert W. Adams
Keywords: Low-noise transmission Analog signals
Abstract:
_____
ICNN97 Electronics & Optical Implementations Session: EO3F Paper Number: 343 Oral
Digital Implementation of discrete-time cellular neural networks with distributed arithmetic
Sungjun Park, Joonho Lim Soo-Ik Chae
Keywords: Cellular NN Digital implementation
Abstract:
In this paper, we propose an efficient digital architecture for the discrete-time cellular neural networks (DTCNN's). That is based on the combination of the bit-serial computation of distributed arithmetic (DA) with the characteristics of the DTCNN: the local connectivity and the translation invariance in the templates. Implementation of the DTCNN with the proposed architecture requires a reduced hardware complexity and a small number of bus lines. It consumes less silicon area because of the bit serial computation of DA and offers higher speed operation than the analog implementations of the DTCNN. A DTCNN cell was implemented in a 0.8 um CMOS technology. The experimental results show that the maximum operation frequency of the chip is 30 MHz.
_____
ICNN97 Electronics & Optical Implementations Session: EOP2 Paper Number: 609 Poster
Pulse density neural network system using simultaneous perturbation learning rule
Yutaka Maeda, Atsushi Nakazawa and Yakichi Kanata
Keywords: Pulse density neural network simultaneous perturbation learning rule backpropagation
Abstract:
Learning scheme is very important in implementation of neural networks to take advantage of their learning ability. Usually, the back-propagation method is widely used as a learning rule of neural networks. Since the back-propagation needs so-called error back propagation to update weights, realizing it in a form of hardware is relatively difficult.
In this paper, we present a pulse density neural network system with learning ability. As a learning rules, the simultaneous perturbation method is used. The learning rules need only one forward operation of networks. Thus, without complicated circuit which calculates gradients of an error function, we could construct the network system with learning ability. Pulse density is used to represent basic quantities in this system. A result for the exclusive OR problem is shown.
_____
ICNN97 Electronics & Optical Implementations Session: EOP2 Paper Number: 453 Poster
VLSI architecture for analog bidirectional pulse-coupled neural networks
Yasuhiro Ota and Bogdan M. Wilamowski
Keywords: VLSI architecture analog bidirectional pulse-coupled neural net CMOS
Abstract:
This paper presents a compact architecture for analog CMOS VLSI implementation of voltage-mode pulse-coupled neural networks (PCNN).
The hardware implementation method shows inherent fault tolerance specialties and high speed, which is usually more than an order over the software counterpart. Pulse-stream encoding technique uses pulse streams to carry information and control analog circuitry, while storing further analog information on the time axis. The main feature of the proposed neuron circuit is that the structure is compact, yet exhibiting all the basic properties of natural biological neurons.
These properties are: (1) threshold of excitation, (2) refractory period, (3) constant pulse propagation velocity, (4) pulse-shaping action, (5) annihilation of pulses in case of their collision. Another unique feature of the proposed neuron cell is that one node serves as both input and output, mimicking a natural biological neuron, and the neuron cell uses frequency modulated bidirectional pulse-streams. Functionality of the proposed PCNN circuit is verified with SPICE simulations.
_____
ICNN97 Electronics & Optical Implementations Session: EOP2 Paper Number: 646 Poster
A digital neuron realization for the random neural network model
Cuneyt Cerkez, Isik Aybay and Ugur Halici
Keywords: digital neuron random neural network routing module
Abstract:
In this study, the neuron of the random neural network (RNN) model (Gelenbe 1989) is designed using digital circuitry. In the RNN model, each neuron accumulates arriving pulses and can fire if its potential at a given instant of time is strictly positive. Firing occurs at random, the intervals between successive firing instants following an exponential distribution of constant rate. When a neuron fires, it routes the generated pulses to the appropriate output lines in accordance with the connection probabilities. In the digital circuitry the fundamental parts of the neuron are simulated by realizing Input Module, Neuron Potential Module, Firing Module and Routing Module. The neuron potential module accumulates incoming signals collected by the input Module at the input site. The Firing Module generates random pulses with an exponential distribution of fixed rate. The pulses generated by the firing Module are distributed to the other neurons through the Routing Module at the output side. A network of neurons can be constructed by using the digital circuitry presented for the single neuron. All the parts of the random neuron circuit are simulated by using Circuitmaker and Pspice digital simulation packages and the neuron is realized digitally by using LS-TTL IC's.
_____
ICNN97 Electronics & Optical Implementations Session: EOP2 Paper Number: 258 Poster
Sensitivity analysis of an analog circuit model of Lamprey unit pattern generator
Elizabeth J. Brauer, Ranu Jung, Denise Wilson and James J. Abbas
Keywords: analog circuit Lamprey unit pattern generator locomotion
Abstract:
Neural circuitry within the spinal cord of the lamprey, a primitive vertebrate, can generate self-sustained oscillations for locomotion (swimming). This pattern generator can be modeled as a chain of oscillatory unit pattern generator segments which exhibit behavior depending on the parameter values in the network. In this paper, we present the results of a simulation study of an analog electronic circuit which mimics the behavior of the biological lamprey unit pattern generator. The circuitry mimics a neural network containing 6 neurons with simplified biophysical properties. The analog circuit exhibits symmetric oscillations, asymmetric oscillations, and fixed points, similar to the behavior of the mathematical model of the lamprey. This work is the first in a series of circuits designed to have possible applications in neuroscience research and in the development of artificial locomotor systems.
_____
ICNN97 Electronics & Optical Implementations Session: EOP2 Paper Number: 173 Poster
Memory based processor array for artificial neural networks
Youngsik Kim, Mi-Jung Noh, Tack-Don Han, Shin-Dug Kim and Sung-Bong Yang
Keywords: Memory based processor array artificial neural networks memory interface
Abstract:
In this paper an effective memory-processor integrated architecture, called memory_based processor array for artificial neural networks (MPAA), is proposed. The MPAA can be easily integrated into any host system via memory interface. Specifically, the MPAA system provides an efficient mechanism for its local memory accesses allowed by the row basis and the column basis using the hybrid row and column decoding, which is suitable for the computation model of ANNs such as the accessing and alignment patterns given for matrix-by-vector operations. Mapping algorithms to implement the multilayer perceptron with backpropagation learning on the MPAA system are also provided. The proposed algorithms support both neuron and layer level parallelisms which allow the MPAA system to operate the learning phase as well as the recall phase in the pipelined fashion. Performance evaluation is provided by detailed comparison in terms of two metrics such as the cost and the number of computation steps.
KEYWORDS: parallel processing, memory--processor integration, multilayer perceptron, backpropagation learning, algorithmic mapping.
_____
ICNN97 Electronics & Optical Implementations Session: EOP2 Paper Number: 162 Poster
On the implementation pf backpropagation on Alex AVX-2 parallel system
Hazem M. Abbas and Mohamed M. Bayoumi
Keywords: backpropagation Alex AVX-2 parallel system parallel architecture
Abstract:
Training backpropagation (BP) networks is a time-consuming process especially on sequential machines. This has motivated the use of parallel architectures to decrease the processing time required for training.
In this paper the implementation of the BP algorithm on the Alex AVX-2 MIMD machine is investigated. Due to the high communication time caused by sending and receiving network information and due to the overhead of the message passing process, the conventional use of block-BP is not appropriate for this particular machine. Increasing the processing load of the workers with respect to the communication load will definitely increase the speedup factor. Here, we propose a block-update learning method for BP which reduces the communication time and produces results similar to those obtained with parallel block-BP.
_____
ICNN97 Electronics & Optical Implementations Session: EOP2 Paper Number: 147 Poster
A pipelined speculative SIMD architecture for SOM ANN
O. Hammami and D. Suzuki
Keywords: SIMD architecture pipelining SOM ANN
Abstract:
Hardware implementations of the SOM ANN have often favored a standard SIMD type of architecture. Although the SOM ANN have some features to support this choice, the standard SIMD model does not sufficiently exploit temporal parallelism and does not exploit the idle time of the idle neurons outside of the winning neuron neighborhood during weights updating. We propose to pipeline the SIMD model of execution and to add speculative features for weights update. Some architectural proposals are made and analyzed in this context based on digital synchronous design techniques.
_____
ICNN97 Electronics & Optical Implementations Session: EOP2 Paper Number: 89 Poster
Improving the resolution of Lazzaro winner-take-all circuit
Barbaros Sekerkiran and Ugur Cilingiroglu
Keywords: Lazzaro winner-take-all circuit CMOS resolution
Abstract:
This paper describes an improvement on Lazzaro winner-take-all (WTA) circuit which is one of the most widely used architectures among all CMOS WTA circuits.
A salient drawback of the original circuit is its limited resolution which stems from low gain of its stages. Unlike previously reported improvements based on positive feedback, the output impedance of the current source providing the negative feedback have been increased by inserting a cascode connected transistor. This simple modification confers a significant resolution improvement on the circuit at the expense of a small reduction in the output voltage range and a minute increase in the real estate consumption. The new configuration was analyzed in detail, the main design issues were discussed and the fundamental limitations were identified.
The simple structure of the circuit makes it highly suitable for implementation of WTA process especially where a high discrimination sensibility is required.
_____
ICNN97 Electronics & Optical Implementations Session: EOP2 Paper Number: 79 Poster
Hardware implementation of the neural gas
Fabio Ancona, Stefano Rovetta and Rodolfo Zunino
Keywords: neural gas vector quantization video compression
Abstract:
The paper presents a hardware implementation of the Neural Gas (NGAS) algorithm. The NGAS is based on vector quantization and is applied to very low bit-rate video compression. The algorithm exhibits interesting properties that can be exploited in an HW realization. The modular structure provides inherent parallelism and can therefore be regarded as an open architecture. The neuro-board interfaces to a PC through a standard ISA bus. The novelty of the proposed solution lies in providing a PC-based configurable HW support for VQ training joining affordable costs with satisfactory effectiveness. Simplicity and easy control for HW tests and SW development represent the basic advantages of the overall approach.
_____
ICNN97 Electronics & Optical Implementations Session: EOP2 Paper Number: 276 Poster
Do we really need multiplier-based synapses for neuro-fuzzy classifiers ?
R. Dogaru, A. T. Murgan and L. O. Chua
Keywords: Multiplier based synapse Neuro-fuzzy classifiers
Abstract:
The purpose of this paper is to show that the standard, multiplier-based synapse, may be replaced with a more convenient to implement synaptic model, while maintaining the overall classification performances of a neuro-fuzzy network. The new synaptic model was called a "comparative synapse", since computati is based mainly on comparisons. The incremental learning rule derived for the new synaptic model has also implementation advantages over the learning rule used by the multiplier-based synapses.
Classification performances were investigated for different problems when both synaptic models (multiplier-based and comparative) were employed, showing very small dependence of the overall neural network system performance on the choice of the synaptic model.