Programming function containing cut in negative imaginary axis

The branch cut of H1 in scipy is (-inf, 0) and not in (-1j*inf, 0) as expected in the paper you cite, which explains why you get incorrect results.

The problem can be worked about with some creative use of an argument transformation for Y_nu, as I point out above in the comments.

Let us assume integer order n. We have

hankel1(n, z) = jv(n, z) + 1j*yv(n, z)

jv has no branch cuts (integer order), but yv has. The transformation formula reads

yv(n, 1j*z) = -2/(pi*(1j)**n)*kv(n, z) + 2*(1j)**n/pi * (log(1j*z) - log(z))*iv(n,z)

or, in other words,

yv(n, z) = -2/(pi*(1j)**n)*kv(n, -1j*z) + 2*(1j)**n/pi * (log(z) - log(-1j*z))*iv(n,-1j*z)

kv(n,z) is defined in Scipy to have branch cut at (-inf, 0), and iv(n,z) has no branch cuts (integer order). Apart from the logarithms, the branch cuts on the RHS are therefore in (-1j*inf,0 ), exactly at where we want them to be. The only thing left to do is to choose the branch cut of the logarithm term appropriately.

The correct analytic continuation with branch cut in (-1j*inf, 0) is then

def yv_imcut(n, z):    return -2/(pi*(1j)**n)*kv(n, -1j*z) + 2*(1j)**n/pi * (0.5j*pi) * iv(n,-1j*z)

This coincides exactly with yv(n, z) in 3 of the 4 quadrants. It has a branch cut in (-1j*inf,0). Moreover, it is obviously an analytic function. Therefore, it is the same as yv, but with a different branch cut choice.

We then have

def hankel1_imcut(n, z):    return jv(n, z) + 1j*yv_imcut(n, z)

It is obviously then the Hankel function with branch cut in (-1j*inf, 0).

Based on this, you can also work out the derivatives.