This is a collection of research projects and applications I worked on using neural networks. Most of the video demos linked below are no longer available — the servers they lived on have been dead for years. I’m keeping the descriptions because they capture what the work was about.

What is a Neural Network?

An artificial neural network (ANN) is an information processing system inspired by biological nervous systems. It’s made up of a large number of interconnected processing elements — neurons — that work together to solve problems. Like people, ANNs learn by example. You configure them for a task, whether that’s pattern recognition or data classification, and they figure out the solution through a learning process. In biological systems, learning means adjusting synaptic connections between neurons. Neural networks do the same thing.

Why use neural networks?

They can extract patterns and detect trends from data that’s too complicated for humans or conventional computing to handle. A trained network acts like an expert in whatever category of information it was given to analyse. You can then use it to make projections and answer “what if” questions.

The advantages:

  • Adaptive learning — they learn from the data you give them, not from explicit instructions.
  • Self-organisation — they create their own internal representation of the information they receive.
  • Parallel computation — neural network computations can run in parallel, and special hardware has been designed to take advantage of this.
  • Fault tolerance — damage to part of the network degrades performance gradually rather than causing total failure. Some capabilities survive even significant damage.

Neural networks versus conventional computers

Conventional computers solve problems algorithmically — they follow a set of known instructions. They can only solve problems we already understand and know how to break down into steps. Neural networks take a different approach. They process information the way the brain does: a large number of interconnected elements working in parallel.

They can’t be programmed for a specific task. You show them examples and they find the solution themselves. That makes them unpredictable — you don’t always know how they arrived at an answer. Conventional computers, by contrast, are totally predictable. If something goes wrong, it’s a software or hardware fault.

The two approaches aren’t in competition. They complement each other. Arithmetic is better suited to algorithms; pattern recognition is better suited to neural networks. Many tasks need both — usually a conventional computer supervising the network.

Neural networks don’t perform miracles. But used sensibly, they produce some genuinely interesting results.

Research projects

Co-evolution of neural networks for pursuit and evasion

Recurrent neural network controllers were co-evolved for pursuit and evasion. Starting from a random population of network designs, successful controllers in each generation were selected for reproduction with recombination, mutation, and gene duplication. Selection was based on how well each controller performed in pursuit-evasion contests — a pursuer and an evader, each with limited energy, pitted against each other in a 2D environment.

The time-series data from each contest was fed into a 3D movie generator. The chase behaviours were real data; the 3D structures and shading were purely illustrative.

The progression showed the evolution of skill: early generations produced pursuers that couldn’t chase and evaders that couldn’t evade. Over time, the pursuer learned to manage its energy, eventually catching the evader after a series of close runs.

Learning object trajectories for event recognition

This work modelled object behaviour using statistical models learned from continuous observation of long video sequences. The goal was automated surveillance and event recognition — identifying unusual behaviour without needing high-level scene knowledge upfront.

The model was learned unsupervised by tracking objects over long sequences, combining neural network vector quantization with a type of neuron that has short-term memory. Once trained, it could assess the typicality of new trajectories, predict future movement, and even generate realistic synthetic behaviour for virtual environments.

Radiosity for virtual reality (ROVER)

Radiosity models the actual interaction between light and environment, producing some of the most realistic rendered images possible. But traditional radiosity is computationally expensive — impractical for the real-time feedback that virtual reality demands.

The ROVER project introduced new methods for using radiosity in VR, including neural network-based approaches to approximate the lighting calculations fast enough for interactive use. The work covered flybys, walkthroughs, and full virtual spaces.

Autonomous walker and swimming lamprey

This research combined biology, mechanical engineering, and information technology to build a dynamically stable legged vehicle controlled by a neural network. The system incorporated command signals, sensory feedback, and reflex circuitry to produce natural movement.

A separate simulation modelled the swimming motion of a lamprey (an eel-like creature), also driven by a neural network.

RoboCup: Robot World Cup

The RoboCup competition pits robots — real and virtual — against each other in a simulated soccer tournament. The aim is to foster interdisciplinary robotics and agent-based AI in a domain that requires large-scale cooperation in a dynamic, noisy, complex environment.

This work focused on building softbots for the Simulation League. Machine learning was applied to train players in decision-making (dribble, pass, or shoot), train goalkeepers in predicting opponent behaviour, and evolve cooperative team strategies. The methods used were variants of neural networks and genetic algorithms.

Audio-to-visual conversion using hidden Markov models

Speech-driven facial animation synthesises realistic video from acoustic speech. Previous research had used vector quantization and neural networks for this. This project examined whether hidden Markov models (HMM) could accomplish the conversion more effectively.

The system included a face tracker that extracted visual parameters from video frames and a morphing module that synthesised facial movements from a single image, with parameters obtained manually.

Artificial life: Galapagos

Galapagos is a game featuring a synthetic organism called Mendel, who can sense infrared radiation and tactile stimulus. His mind is an adaptive controller using Non-stationary Entropic Reduction Mapping — a form of artificial life technology developed by Anark. Mendel can learn and adapt to hostile environments but can’t solve the puzzles that block his escape.

The game featured 3D texture-mapped worlds with continuous-motion graphics, dramatic camera movement, and 3D stereo sound.

Speechreading (lipreading)

As part of the Neuroinformatik research programme, a neural speechreading system was developed as part of a workstation user interface. The system had three components: a face tracker, lip modelling and speech processing, and neural network training using SNNS.

Automatic speechreading relied on robust lip image analysis from true-colour video — no special lighting or lip make-up required. The system tracked and stored the lip region in real time, then used a neural classifier to detect teeth visibility and other attributes. A model description language allowed flexible definition of edge models based on Bezier curves and control knots.

Detection and tracking of moving targets

These methods used a “track before detect” approach, correlating sensor data across time and location based on the nature of actual tracks. The track statistics were learned through neural network training with prior real or simulated data.

Compared to conventional matched filter techniques, the neural network approach reduced false alarm rates by up to a factor of 1000 in simulations involving very weak ICBM targets against cloud and nuclear backgrounds. The methods were designed to overcome limitations of 3D+ matched filtering, dynamic programming, and multi-hypothesis tracking.

Real-time target identification for security

The system localised and tracked faces as people moved through a scene, integrating motion detection, motion-based person tracking, and appearance-model-based face tracking. Faces were tracked robustly by combining motion and model-based approaches, maintaining lock even in low resolution and poor lighting.

Facial animation

Facial animations were created using hierarchical B-splines as the underlying surface representation, with neural networks used to learn variations in facial expressions. The work included an early prototype for the character “Mouse” in the YTV/ABC television series ReBoot — though they didn’t end up using hierarchical splines in the final production.

Artificial life for graphics and animation

Some graphics researchers began exploring objects of enormously greater complexity than physical modelling alone can produce — objects that are alive. The modelling and simulation of living systems for computer graphics resonates with the field of artificial life.

The demos demonstrated models that emulate a broad variety of living things, from simple organisms to humans. These models inhabited virtual worlds governed by physical laws and simulated natural processes: birth and death, growth, natural selection, evolution, perception, locomotion, adaptive behaviour, intelligence, and learning.

Creatures

Creatures was described as the most advanced commercial application of artificial life technology at the time. You raised, taught, bred, and cared for computer pets that were genuinely alive — if neglected, they would die. The game used neural network-based adaptive controllers that had impressed scientists worldwide.