Artificial Neural Networks : Interview Questions

What is a neural network (NN) ?

First of all, when we are talking about a neural network, we *should* usually better say "artificial neural network" (ANN), because that is what we mean most of the time. Biological neural networks are much more complicated in their elementary structures than the mathematical models we use for ANNs. A neural network is a powerful data modeling tool to capture and represent complex input/output relationships.

A vague description is as follows:: An ANN is a network of many very simple processors ("units"), each possibly having a (small amount of) local memory. The units are connected by unidirectional communication channels ("connections"), which carry numeric (as opposed to symbolic) data. The units operate only on their local data and on the inputs they receive via the connections.
The design motivation is what distinguishes neural networks from other mathematical techniques:: A neural network is a processing device, either an algorithm, or actual hardware, whose design was motivated by the design and functioning of human brains and components thereof.

Most neural networks have some sort of "training" rule whereby the weights of connections are adjusted on the basis of presented patterns. In other words, neural networks "learn" from examples, just like children learn to recognize dogs from examples of dogs, and exhibit some structural capability for generalization.

Neural networks normally have great potential for parallelism, since the computations of the components are independent of each other.

The true power and advantage of neural networks lies in their ability to represent both linear and non-linear relationships and in their ability to learn these relationships directly from the data being modeled. Traditional linear models are simply inadequate when it comes to modeling data that contains non-linear characteristics.
The most common neural network model is the ultilayer perceptron (MLP). This type of neural network is known as a supervised network because it requires a desired output in order to learn.

What kind of real-world problems can neural networks solve?
A. Neural networks can be applied to almost any problem where you have
1) historical data and
2) a need to create a model for that data.
Neural networks have been successfully applied to broad spectrum of data-intensive applications, such as:

Process Modeling and Control - Creating a neural network model for a physical plant then using that model to determine the best control settings for the plant.
Machine Diagnostics - Detect when a machine has failed so that the system can automatically shut down the machine when this occurs.
Portfolio Management - Allocate the assets in a portfolio in a way that maximizes return and minimizes risk.
Target Recognition - Military application which uses video and/or infrared image data to determine if an enemy target is present.
Medical Diagnosis - Assisting doctors with their diagnosis by analyzing the reported symptoms and/or image data such as MRIs or X-rays.
Credit Rating - Automatically assigning a company's or individuals credit rating based on their financial condition.
Targeted Marketing - Finding the set of demographics which have the highest response rate for a particular marketing campaign.
Voice Recognition - Transcribing spoken words into ASCII text.
Financial Forecasting - Using the historical data of a security to predict the future movement of that security.
Quality Control - Attaching a camera or sensor to the end of a production process to automatically inspect for defects.
Intelligent Searching - An internet search engine that provides the most relevant content and banner ads based on the users' past behavior.
Fraud Detection - Detect fraudulent credit card transactions and automatically decline the charge.

Optical Character Recognition (OCR) - Example shown above where a printed document is scanned and converted to an electronic format such as ASCII text, which can then be manipulated and/or stored more efficiently.

What are the activation functions in NN?

The most common activation functions used are:

Step Function: is simplest of all the activation functions. If x_i represents the inputs to a neuron and w_i represents the weight of each edge the for all inputs to the node:

        The activation function, f, is:
                                                            f(Net) = 1 if Net > 0
                                                            f(Net) = 0 if Net <= 0
        The graph for the step function is

Step Function

Sigmoid Function: Unlike the step function, which is a linear function, the Sigmoid function a non-linear function, and is best suited for Backpropagation. The function is given by the following expression:

f(Net) =

The graph for the Sigmoid function is

Sigmoid Function

Tell about the the two types of learning methods used in neural networks: supervised and unsupervised learning.

In supervised learning, the concept of a teacher exists. The role of the teacher is to tell the network when its output is incorrect by showing it what the output should have been.

In unsupervised learning, there is no teacher and therefore no rules by which the network can learn. Here the network must learn by examining the input data and eventually discovering any inherent properties that exist.

Explain Back Propagation?

The operations of the Backpropagation neural networks can be divided into two steps: feedforward and Backpropagation.

In the feedforward step, an input pattern is applied to the input layer and its effect propagates, layer by layer, through the network until an output is produced. The network's actual output value is then compared to the expected output, and an error signal is computed for each of the output nodes. Since all the hidden nodes have, to some degree, contributed to the errors evident in the output layer, the output error signals are transmitted backwards from the output layer to each node in the hidden layer that immediately contributed to the output layer. This process is then repeated, layer by layer, until each node in the network has received an error signal that describes its relative contribution to the overall error.

Once the error signal for each node has been determined, the errors are then used by the nodes to update the values for each connection weights until the network converges to a state that allows all the training patterns to be encoded. The Backpropagation algorithm looks for the minimum value of the error function in weight space using a technique called the delta rule or gradient descent[2]. The weights that minimize the error function is then considered to be a solution to the learning problem.

Explain different learning methods of NN's.

UNSUPERVISED LEARNING (i.e. without a "teacher"):
- Feedback Nets:
  - Additive Grossberg (AG)
  - Shunting Grossberg (SG)
  - Binary Adaptive Resonance Theory (ART1)
  - Analog Adaptive Resonance Theory (ART2, ART2a)
  - Discrete Hopfield (DH)
  - Continuous Hopfield (CH)
  - Discrete Bidirectional Associative Memory (BAM)
  - Temporal Associative Memory (TAM)
  - Adaptive Bidirectional Associative Memory (ABAM)
  - Kohonen Self-organizing Map (SOM)
  - Kohonen Topology-preserving Map (TPM)
- Feedforward-only Nets:
  - Learning Matrix (LM)
  - Driver-Reinforcement Learning (DR)
  - Linear Associative Memory (LAM)
  - Optimal Linear Associative Memory (OLAM)
  - Sparse Distributed Associative Memory (SDM)
  - Fuzzy Associative Memory (FAM)
  - Counterprogation (CPN)
SUPERVISED LEARNING (i.e. with a "teacher"):
- Feedback Nets:
  - Brain-State-in-a-Box (BSB)
  - Fuzzy Congitive Map (FCM)
  - Boltzmann Machine (BM)
  - Mean Field Annealing (MFT)
  - Recurrent Cascade Correlation (RCC)
  - Learning Vector Quantization (LVQ)
- Feedforward-only Nets:
  - Perceptron
  - Adaline, Madaline
  - Backpropagation (BP)
  - Cauchy Machine (CM)
  - Adaptive Heuristic Critic (AHC)
  - Time Delay Neural Network (TDNN)
  - Associative Reward Penalty (ARP)
  - Avalanche Matched Filter (AMF)
  - Backpercolation (Perc)
  - Artmap
  - Adaptive Logic Network (ALN)
  - Cascade Correlation (CasCor)

Explain Under-fitting and Over-fitting training Data.

There are also issues regarding generalizing a neural network. Issues to consider are problems associated with under-training and over-training data. Under-training can occur when the neural network is not complex enough to detect a pattern in a complicated data set. This is usually the result of networks with so few hidden nodes that it cannot accurately represent the solution, therefore under-fitting the data (Figure 6).

Figure 6 Under-fitting data

On the other hand, over-training can result in a network that is too complex, resulting in predictions that are far beyond the range of the training data. Networks with too many hidden nodes will tend to over-fit the solution (Figure 7).

Figure 7 Over-fitting data

The aim is to create a neural network with the "right" number of hidden nodes that will lead to a good solution to the problem (Figure 8).

Figure 8 Good fit of the data

What are some of the types of neural networks that I can build with NeuroSolutions?A. NeuroSolutions includes two wizards to create neural networks for you based on your data and specifications. One of these wizards, the NeuralBuilder, centers the design specifications around the specific neural network architecture you wish to have built. Some of the most common neural network architectures include:

Multilayer Perceptron (MLP)
Generalized Feedforward
Modular
Jordan/Elman
Principal Component Analysis (PCA)
Radial Basis Function (RBF)
General Regression Neural Network (GRNN)
Probabilistic Neural Network (PNN)
Self-Organizing Map (SOM)
Time-Lag Recurrent Network (TLRN)
Recurrent Network
CANFIS Network (Fuzzy Logic)
Support Vector Machine (SVM)

Main Menu