RuntimeWarning: invalid value encountered in greater

python numpy softmax

Your problem is caused by the NaN or Inf elements in your out_vec array. You could use the following code to avoid this problem:

if np.isnan(np.sum(out_vec)):    out_vec = out_vec[~numpy.isnan(out_vec)] # just remove nan elements from vectorout_vec[out_vec > 709] = 709...

or you could use the following code to leave the NaN values in your array:

out_vec[ np.array([e > 709 if ~np.isnan(e) else False for e in out_vec], dtype=bool) ] = 709

python numpy softmax

In my case the warning did not show up when calling this before the comparison (I had NaN values getting compared)

np.warnings.filterwarnings('ignore')

python numpy softmax

IMO the better way would be to use a more numerically stable implementation of sum of exponentials.

from scipy.misc import logsumexpout_vec = np.exp(out_vec - logsumexp(out_vec))

CodeHunter