Database replication on Raspberry Pi mesh network Database replication on Raspberry Pi mesh network database database

Database replication on Raspberry Pi mesh network


For this scenario I would recommend the Hadoop Distributed File System (HDFS).

Features that make HDFS attractive to your scenario:

  • It is a distributed file system with variable replication factor (the default is 3 which is nearly impossible to lose data with).
  • Can scale up to thousands of different machines
  • Does not depend on high availability of individual nodes -- automatically handles node failure and replicates any data from downed nodes

As for the actual database... HBase, Mongo, or Cassandra are all good options here, pick whatever you are most comfortable with -- HDFS will take care of all of the replication for you.


According to this SO response:

https://stackoverflow.com/a/8787999/2020565

And upon cheking their website, maybe you should check Elliptics:http://www.ioremap.net/projects/elliptics/

The network does not use dedicated servers to maintain the metadata information, it supports redundant objects storage. Small to medium sized write benchmarks can be found on eblob page.


In my experience Elasticsearch has great and easy-to-use cluster management, it supports out of the box nice features such as node autodiscovery, data replication, auto-rebalancing etc., have a look at docs. Usually it is used to replicate data from an other database to make it searchable but I don't see why it couldn't be used in this context as well.

Basically when you create a "table" (called "index" in ES) you get to decide that in how many "partitions" (called "shards") the data should be partitioned, and ad-hoc set that how many replicas of that table you want to have (this doesn't 100% match the correct terminology since an "index" can consist of multiple "types" but I think this is the best analogy).

An example project with three Pis is here.

I have read a bit about Cassandra as well and I imagine it would have similar features, for example partitions and replicas are mentioned here.