This section includes various troubleshooting tips and tricks for Redis Enterprise Software.
If you encounter any issues that you are not able to resolve yourself and need to contact Redis Labs support for assistance, you can create a support package that gathers all essential information to help us debug your issues. Note: The process of creating the support package can take several minutes and generates load on the system. Creating a support package To create a support package:
Because data is being moved around across shards during the resharding process, it is impossible to report the values of some of the metrics in an accurate manner. Therefore, several database metrics are not collected, and as a result, the graph is blank when you view the database or shard metrics. Examples of metrics that are not collected during the resharding process are: Used memory Memory usage Total keys Write misses/sec Read misses/sec Expired objects/sec Evicted objects/sec Incoming traffic Outgoing traffic
The Redis Enterprise Software cluster nodes host a range of services that support the cluster processes. In most deployments, either all of these services are required, or there are enough memory resources on the nodes for the database requirements. In a deployment with limited memory resources, certain services can be disabled from API endpoint to free system memory or using the rladmin command. Before you disable a service, make sure that your deployment does not depend on that service.
Nodes in the same cluster must reside on the same VLAN. If you cannot host the nodes on the same VLAN then you must configure all ports between them to be open.
When a Redis Enterprise Software (RS) cluster fails, you must use the cluster configuration file and database data to recover the cluster. Note: For cluster recovery in a Kubernetes Operator deployment, go to: Redis Enterprise Cluster Recovery for Kubernetes. Cluster failure can be caused by: A hardware or software failure that causes the cluster to be unresponsive to client requests or administrative actions. More than half of the cluster nodes lose connection with the cluster, resulting in quorum loss.
When a cluster fails or a database is corrupted, you must: Restore the cluster configuration from the CCS files Recover the databases with their previous configuration and data To restore the data that was in the databases to databases in the new cluster you must restore the database persistence files (backup, AOF, or snapshot files) to the databases. These files are stored in the persistence storage location. The database recovery process includes:
Problem: The Replica Of process repeatedly fails and restarts Diagnostic: A log entry in the Redis log of the source database shows repeated failures and restarts. Cause: The Redis “client-output-buffer-limit” setting on the source database is configured to a relatively small value, which causes the connection drop. Resolution: Reconfigure the buffer on the source database to a bigger value: If the source is a Redis database on an RS cluster, increase the slave buffer size of the source database with:
In various scenarios, such as after creating a new cluster or upgrading the cluster, it is highly advisable to verify client connectivity to the database. To test client connectivity: Create a Redis database and get the database endpoint, which contains the cluster name (FQDN). Try to connect to the database endpoint from your client of choice, and execute commands against the database. If the database does not respond, try to connect to the database endpoint by using the IP address rather than the FQDN; if you succeed, it means that the DNS is not properly configured.