HDFS vs GridFS: When to use which? HDFS vs GridFS: When to use which? hadoop hadoop

HDFS vs GridFS: When to use which?


HDFS intended for batch processing (you're know, when you running a query that will read many of your files one-by-one), but really suck when you doing random access operations and it is pain in the neck to maintain it or even deploy (you're know, all of these Zookepers, Namenodes and so on). On the other hand GridFS is slower at batches, but not in the case when you do a lot of random accesses, but have a bigger storage overhead compared to HDFS.

I would say that you should use HDFS for analitycs and GridFS for backing web-site.


Use HDFS if you are using Hadoop and use GridFS if you are using MongoDB. Neither are that great for just storing random files. They are built to work with the analytic platform.


I would recommend to use GridFS, if you are going only store your files without any analytic and map-reduce jobs. It's easier to customize and maintain. I used for file hosting application. HFDS in this case is overkill.