Conda downgrade numpy version

You could simply install the correct version using the command

conda install -c conda-forge numpy=1.16.4

conda will automatically take care of downgrading to your version correctly

If downgrading to an specific version of numpy takes forever while conda is solving the environment, or conda is unable to resolve the conflicts, you can use conda-tree to inspect the dependences and then manually uninstall with conda (or attempt to downgrade) the incompatible packages. However note that creating a new environment with the correct numpy version could be faster if there are many dependences.

conda install -c conda-forge conda-treeconda-tree whoneeds -t numpy

This will display a tree with the supported numpy versions for each dependent package:

numpy==1.20.3  ├─ h5py 3.2.1 [required: >=1.16.6,<2.0a0]  │  └─ tensorflow-base 2.5.0 [required: >=3.1.0]  │     └─ tensorflow 2.5.0 [required: 2.5.0, gpu_py37hb3da07e_0]  │        └─ tensorflow-gpu 2.5.0 [required: 2.5.0]  ├─ keras-preprocessing 1.1.2 [required: >=1.9.1]  │  └─ tensorflow-base 2.5.0 [required: >=1.1.2]  │     └─ dependent packages of tensorflow-base displayed above  ├─ matplotlib-base 3.4.2 [required: >=1.17.5,<2.0a0]  │  └─ matplotlib 3.4.2 [required: >=3.4.2,<3.4.3.0a0]  ├─ opt_einsum 3.3.0 [required: any]  │  └─ tensorflow-base 2.5.0 [required: 3.3.0.*]  │     └─ dependent packages of tensorflow-base displayed above  ├─ pandas 1.2.5 [required: >=1.20.2,<2.0a0]  │  └─ statsmodels 0.12.2 [required: >=0.21]  ├─ patsy 0.5.1 [required: >=1.4.0]  │  └─ statsmodels 0.12.2 [required: >=0.5.1]  ├─ scipy 1.6.2 [required: >=1.16.6,<2.0a0]  │  ├─ keras-preprocessing 1.1.2 [required: >=0.14]  │  │  └─ dependent packages of keras-preprocessing displayed above  │  ├─ patsy 0.5.1 [required: any]  │  │  └─ dependent packages of patsy displayed above  │  ├─ statsmodels 0.12.2 [required: >=1.0]  │  └─ tensorflow-base 2.5.0 [required: >=1.6.2]  │     └─ dependent packages of tensorflow-base displayed above  ├─ statsmodels 0.12.2 [required: >=1.17.0,<2.0a0]  ├─ tensorboard 2.5.0 [required: >=1.12.0]  │  ├─ tensorflow 2.5.0 [required: >=2.5.0]  │  │  └─ dependent packages of tensorflow displayed above  │  └─ tensorflow-base 2.5.0 [required: >=2.5.0,<2.6]  │     └─ dependent packages of tensorflow-base displayed above  ├─ tensorflow-base 2.5.0 [required: >=1.20]  │  └─ dependent packages of tensorflow-base displayed above  └─ tensorflow-estimator 2.5.0 [required: >=1.16.1]     ├─ tensorflow 2.5.0 [required: >=2.5.0]     │  └─ dependent packages of tensorflow displayed above     └─ tensorflow-base 2.5.0 [required: >=2.5.0,<2.6]        └─ dependent packages of tensorflow-base displayed above