How can I load balance reads on a master/slave redis setup?
SE.Redis has a CommandFlags parameter that is optional on every command. There are some useful and relevant options here:
DemandMasterPreferMasterDemandSlavePreferSlave
The default behaviour is PreferMaster; write operations bump that to DemandMaster, and there are a very few commands that actively prefer replicas (keyspace iteration, etc).
So: if you aren't specifying CommandFlags, then right now you're probably using the default: PreferMaster. Assuming a master exists and is reachable, then: it will use the master. And there can only be one master, so: it'll use one server.
A cheap option for today would be to add PreferSlave as a CommandFlags option on your high-volume read operations. This will push the work to the replicas if they can be resolved - or if no replicas can be found: the master. Since there can be multiple replicas, it applies a basic rotation-based load-balancing scheme, and you should start to see load on multiple replicas.
If you want to spread load over all nodes including masters and replicas... then I'll need to add new code for that. So if you want that, please log it as an issue on the github repo.
As per the docs it should automatically detect master/slaves. It may be that StackExchange.Redis is detecting that all your nodes are masters, thus just selecting one using its own tiebreaker rules.
I would also check the request logs on your redis-pods if there are some invalid commands that are being sent by StackExchange.Redis, maybe you don't have the right permissions for it to detect master/slaves.
Maybe you also have Sentinel enabled and StackExchange doesn't support sentinel
If you see something is not working you can open an issue on Github
Finally, you could also try twemproxy.