Jensen-Shannon Divergence

Note that the scipy entropy call below is the Kullback-Leibler divergence.

See: http://en.wikipedia.org/wiki/Jensen%E2%80%93Shannon_divergence

#!/usr/bin/env pythonfrom scipy.stats import entropyfrom numpy.linalg import normimport numpy as npdef JSD(P, Q):    _P = P / norm(P, ord=1)    _Q = Q / norm(Q, ord=1)    _M = 0.5 * (_P + _Q)    return 0.5 * (entropy(_P, _M) + entropy(_Q, _M))

Also note that the test case in the Question looks erred?? The sum of the p distribution does not add to 1.0.

See: http://www.itl.nist.gov/div898/handbook/eda/section3/eda361.htm

Since the Jensen-Shannon distance (distance.jensenshannon) has been included in Scipy 1.2, the Jensen-Shannon divergence can be obtained as the square of the Jensen-Shannon distance:

from scipy.spatial import distancedistance.jensenshannon([1.0/10, 9.0/10, 0], [0, 1.0/10, 9.0/10]) ** 2# 0.5306056938642212

Get some data for distributions with known divergence and compare your results against those known values.

BTW: the sum in KL_divergence may be rewritten using the zip built-in function like this:

sum(_p * log(_p / _q) for _p, _q in zip(p, q) if _p != 0)

This does away with lots of "noise" and is also much more "pythonic". The double comparison with 0.0 and 0 is not necessary.