It is possible to make your Redis cluster Fault Tolerant and Highly Available by building a replica set and then monitor these nodes using sentinel for automatic failover.
I am going to give an example setup to explain it. The structure is built with three nodes running one as a master and two as slaves.
Master Node: (Centos 7.2) 192.168.1.11
Slave1 Node: (Centos 7.2) 192.168.1.12
Slave2 Node: (Centos 7.2) 192.168.1.13
Edit System settings on each node: /etc/sysctl.conf
Disable transparent hugepage (transparent_hugepage=never) on each node: /etc/default/grub
Apply grub config and reboot each node:
Master Node: /etc/redis/6379.conf
Slave1 Node: /etc/redis/6379.conf
Slave2 Node: /etc/redis/6379.conf
Master Node: /etc/redis/sentinel.conf
Slave1 Node: /etc/redis/sentinel.conf
Slave2 Node: /etc/redis/sentinel.conf
Each Node: /etc/systemd/system/multi-user.target.wants/redis-server.service
Each Node: /etc/systemd/system/multi-user.target.wants/redis-sentinel.service
After creating service files reload systemd and enable,start related services on each node:
Check replication and sentinel status from redis-cli Master and slave nodes:
Stop wait and start Redis Master and check logs on a slave:
Now redis master is failed over another node.