A simple implementation

Here is a readable implementation using classes in Python. This implementation trades efficiency for understandability:

    import math    import random    BIAS = -1    """    To view the structure of the Neural Network, type    print network_name    """    class Neuron:        def __init__(self, n_inputs ):            self.n_inputs = n_inputs            self.set_weights( [random.uniform(0,1) for x in range(0,n_inputs+1)] ) # +1 for bias weight        def sum(self, inputs ):            # Does not include the bias            return sum(val*self.weights[i] for i,val in enumerate(inputs))        def set_weights(self, weights ):            self.weights = weights        def __str__(self):            return 'Weights: %s, Bias: %s' % ( str(self.weights[:-1]),str(self.weights[-1]) )    class NeuronLayer:        def __init__(self, n_neurons, n_inputs):            self.n_neurons = n_neurons            self.neurons = [Neuron( n_inputs ) for _ in range(0,self.n_neurons)]        def __str__(self):            return 'Layer:\n\t'+'\n\t'.join([str(neuron) for neuron in self.neurons])+''    class NeuralNetwork:        def __init__(self, n_inputs, n_outputs, n_neurons_to_hl, n_hidden_layers):            self.n_inputs = n_inputs            self.n_outputs = n_outputs            self.n_hidden_layers = n_hidden_layers            self.n_neurons_to_hl = n_neurons_to_hl                # Do not touch            self._create_network()            self._n_weights = None            # end        def _create_network(self):            if self.n_hidden_layers>0:                # create the first layer                self.layers = [NeuronLayer( self.n_neurons_to_hl,self.n_inputs )]                        # create hidden layers                self.layers += [NeuronLayer( self.n_neurons_to_hl,self.n_neurons_to_hl ) for _ in range(0,self.n_hidden_layers)]                        # hidden-to-output layer                self.layers += [NeuronLayer( self.n_outputs,self.n_neurons_to_hl )]            else:                # If we don't require hidden layers                self.layers = [NeuronLayer( self.n_outputs,self.n_inputs )]        def get_weights(self):            weights = []                for layer in self.layers:                for neuron in layer.neurons:                    weights += neuron.weights                return weights        @property        def n_weights(self):            if not self._n_weights:                self._n_weights = 0                for layer in self.layers:                    for neuron in layer.neurons:                        self._n_weights += neuron.n_inputs+1 # +1 for bias weight            return self._n_weights        def set_weights(self, weights ):            assert len(weights)==self.n_weights, "Incorrect amount of weights."                stop = 0            for layer in self.layers:                for neuron in layer.neurons:                    start, stop = stop, stop+(neuron.n_inputs+1)                    neuron.set_weights( weights[start:stop] )            return self        def update(self, inputs ):            assert len(inputs)==self.n_inputs, "Incorrect amount of inputs."                for layer in self.layers:                outputs = []                for neuron in layer.neurons:                    tot = neuron.sum(inputs) + neuron.weights[-1]*BIAS                    outputs.append( self.sigmoid(tot) )                inputs = outputs               return outputs        def sigmoid(self, activation,response=1 ):            # the activation function            try:                return 1/(1+math.e**(-activation/response))            except OverflowError:                return float("inf")        def __str__(self):            return '\n'.join([str(i+1)+' '+str(layer) for i,layer in enumerate(self.layers)])

A more efficient implementation (with learning)

If you are looking for a more efficient example of a neural network with learning (backpropagation), take a look at my neural network Github repository here.

Hmm this is tricky. I had the same problem before and I couldn't find anything between good but heavily math loaded explanation and ready to use implementations.

The problem with ready to use implementations like PyBrain is that they hide the details, so people interested in learning how to implement ANNs are in need of something else. Reading the code of such solutions can be challenging too because they often use heuristics to improve performance and that makes the code harder to follow for a starter.

However, there are a few of resources you could use:

http://msdn.microsoft.com/en-us/magazine/jj658979.aspx

http://itee.uq.edu.au/~cogs2010/cmc/chapters/BackProp/

http://www.codeproject.com/Articles/19323/Image-Recognition-with-Neural-Networks

http://freedelta.free.fr/r/php-code-samples/artificial-intelligence-neural-network-backpropagation/

Here is an example of how you can implement a feedforward neural network using numpy. First import numpy and specify the dimensions of your inputs and your targets.

import numpy as npinput_dim = 1000target_dim = 10

We will build the network structure now. As suggested in Bishop's great "Pattern Recognition and Machine Learning", you can simply consider the last row of your numpy matrices as bias weights and the last column of your activations as bias neurons. Input/output dimensions of the first/last weight matrix needs to be 1 greater, then.

dimensions = [input_dim+1, 500, 500, target_dim+1]weight_matrices = []for i in range(len(dimensions)-1):  weight_matrix = np.ones((dimensions[i], dimensions[i]))  weight_matrices.append(weight_matrix)

If your inputs are stored in a 2d numpy matrix, where each row corresponds to one sample and the columns correspond to the attributes of your samples, you can propagate through the network like this: (assuming logistic sigmoid function as activation function)

def activate_network(inputs):  activations = [] # we store the activations for each layer here  a = np.ones((inputs.shape[0], inputs.shape[1]+1)) #add the bias to the inputs  a[:,:-1] = inputs  for w in weight_matrices:    x = a.dot(w) # sum of weighted inputs    a = 1. / (1. - np.exp(-x)) # apply logistic sigmoid activation    a[:,-1] = 1. # bias for the next layer.    activations.append(a)  return activations

The last element in activations will be the output of your network, but be careful, you need to omit the additional column for the biases, so your output will be activations[-1][:,:-1].

To train a network, you need to implement backpropagation which takes a few additional lines of code. You need to loop from the last element of activations to the first, basically. Make sure to set the bias column in the error signal to zero for each layer before each backpropagation step.