Can I use Snappy compression from a local client without a full Hadoop install?

Hadoop looks for libsnappy.so in $HADOOP_HOME/lib/native/. So you can try to do this:

1. Create a folder structure like /home/user/hadoop/lib/native
2. Put a libsnappy.so and libhadoop.so in this folder. You can copy it from your cluster.
3. Set $HADOOP_HOME to /home/user/hadoop
4. Run your app.

I finally got this sorted out.

As shutty expected, it was just a matter of the class loader looking in the wrong spot. (Or if you want to think of it the other way, I didn't put it in the right spot.)

Good old strace ended up saving the day. It revealed the following:

stat64("/usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/i386/libsnappy.so", 0xb68eef50) = -1 ENOENT (No such file or directory)stat64("/usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/i386/server/libsnappy.so", 0xb68eef50) = -1 ENOENT (No such file or directory)stat64("/usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/i386/libsnappy.so", 0xb68eef50) = -1 ENOENT (No such file or directory)stat64("/usr/lib/jvm/java-6-sun-1.6.0.26/jre/../lib/i386/libsnappy.so", 0xb68eef50) = -1 ENOENT (No such file or directory)stat64("/usr/lib/intellij-12.1/bin/libsnappy.so", 0xb68eef50) = -1 ENOENT (No such file or directory)stat64("./libsnappy.so", 0xb68eef50)    = -1 ENOENT (No such file or directory)stat64("/usr/java/packages/lib/i386/libsnappy.so", 0xb68eef50) = -1 ENOENT (No such file or directory)stat64("/lib/libsnappy.so", 0xb68eef50) = -1 ENOENT (No such file or directory)stat64("/usr/lib/libsnappy.so", 0xb68eef50) = -1 ENOENT (No such file or directory)

Once I just slapped the files into one of those spots, things were golden.

I hope this helps someone else out in the future.