Is it possible to switch between BLAS libraries without recompiling program?

Sure, you have to have them installed, and on Ubuntu/Debian issue command

update-alternatives --config libblas.so.3

You'll get numbered list of alternatives and could switch between them easily

Link: https://wiki.debian.org/DebianScience/LinearAlgebraLibraries

You could also do it without changing system wide settings, for example by adding the library you want to use to the LD_PRELOAD or LD_LIBRARY_PATH environment variables. The first library on this path will be the one used to resolve the symbols against.

For example, you could run with

LD_PRELOAD=/path/to/blas/lib.so ./my_executable_using_caffe

You can see that this approach would be extremely useful as part of a benchmarking script for different implementations since it doesn't affect the benchmarking environment itself. For example (in bash):

my_libraries=/path/to/blas1.so /path/to/blas2.sofor lib in $my_librariesdo  LD_PRELOAD=${lib} ./my_executable_using_caffedone

This approach, based on dynamic linking, applies to any other shared library your program is compiled against.

Yes, you can. You just need to use dynamic library linking.The libblas.so.3 is a soft linking to /etc/alternatives/libblas.so.3Which in turn points to the BLAS library implementation

For example if you installing Atlas (sudo apt-get install libatlas3-base):

/usr/lib/libblas.so.3 -> /etc/alternatives/libblas.so.3/etc/alternatives/libblas.so.3 -> /usr/lib/atlas-base/atlas/libblas.so.3

while after installing Openblas (sudo apt-get install libopenblas-base):

/usr/lib/libblas.so.3 -> /etc/alternatives/libblas.so.3/etc/alternatives/libblas.so.3 -> /usr/lib/openblas-base/libblas.so.3

Of course you can use your own soft link to your library builds.