How to calculate precision and recall in Keras

Python package keras-metrics could be useful for this (I'm the package's author).

import kerasimport keras_metricsmodel = models.Sequential()model.add(keras.layers.Dense(1, activation="sigmoid", input_dim=2))model.add(keras.layers.Dense(1, activation="softmax"))model.compile(optimizer="sgd",              loss="binary_crossentropy",              metrics=[keras_metrics.precision(), keras_metrics.recall()])

UPDATE: Starting with Keras version 2.3.0, such metrics as precision, recall, etc. are provided within library distribution package.

The usage is the following:

model.compile(optimizer="sgd",              loss="binary_crossentropy",              metrics=[keras.metrics.Precision(), keras.metrics.Recall()])

As of Keras 2.0, precision and recall were removed from the master branch. You will have to implement them yourself. Follow this guide to create custom metrics : Here.

Precision and recall equation can be found Here

Or reuse the code from keras before it was removed Here.

There metrics were remove because they were batch-wise so the value may or may not be correct.

My answer is based on the comment of Keras GH issue. It calculates validation precision and recall at every epoch for a onehot-encoded classification task. Also please look at this SO answer to see how it can be done with keras.backend functionality.

import keras as kerasimport numpy as npfrom keras.optimizers import SGDfrom sklearn.metrics import precision_score, recall_scoremodel = keras.models.Sequential()# ...sgd = SGD(lr=0.001, momentum=0.9)model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])class Metrics(keras.callbacks.Callback):    def on_train_begin(self, logs={}):        self._data = []    def on_epoch_end(self, batch, logs={}):        X_val, y_val = self.validation_data[0], self.validation_data[1]        y_predict = np.asarray(model.predict(X_val))        y_val = np.argmax(y_val, axis=1)        y_predict = np.argmax(y_predict, axis=1)        self._data.append({            'val_recall': recall_score(y_val, y_predict),            'val_precision': precision_score(y_val, y_predict),        })        return    def get_data(self):        return self._datametrics = Metrics()history = model.fit(X_train, y_train, epochs=100, validation_data=(X_val, y_val), callbacks=[metrics])metrics.get_data()