Getting Started with VMWare Tanzu Kubernetes Grid Integrated Edition (formerly Pivotal PKS)
These are the steps required to set up a Redis Enterprise Cluster with the Kubernetes Operator on VMWare Tanzu Kubernetes Grid Integrated Edition (formerly Pivotal PKS).
Prerequisites:
- A PKS environment installed on PCF version 2.4.6 or above and PKS version 1.3.3 or above.
- A PKS cluster with at least three nodes, each meeting the minimum requirements for a development installation in the case of a development environment. For production use-cases and large-scale POCs, use hardware recommendations for production environments.
- The kubectl package installed at version 1.8 or higher.
- The PKS cli installed.
Step 1: Login and prepare your PKS environment and PKS cluster
-
Log in to PKS and your PKS cluster:
pks login -a PKS-API -u USERNAME -k
-
Find the cluster you created by listing the available clusters:
pks clusters
Example of a response:
Name Plan Name UUID Status Action cluster1 dev d8g7s9g9-789a-789a-879a-ad8f798s7dfs succeeded CREATE cluster2 prod s7f9sadf-sfd9-as8d-45af-a9s8d7f3niuy succeeded CREATE
-
Change the context to your target cluster:
pks get-credentials CLUSTER-NAME
Example of a response:
Fetching credentials for cluster pks-re-cluster Context set for cluster pks-re-cluster. You can now switch between clusters by using: $kubectl config use-context <cluster-name>
-
Confirm you can access your cluster using the Kubernetes CLI:
kubectl cluster-info
Example of a response:
Kubernetes master is running at http...
Note -While you can use the Kubernetes default namespace, it is a best practice to use a separate namespace if you are sharing the cluster with others. The Operator deployment deploys and runs one Redis Enterprise Cluster in one Kubernetes namespace. In order to run multiple Redis Enterprise Clusters, deploy each one in its own namespace. -
Create a namespace for your deployment:
-
Review the current namespaces:
kubectl get namespaces
Example of a response:
NAME STATUS AGE default Active 14d kube-public Active 14d kube-system Active 14d pks-system Active 14d
-
Create a new namespace with a unique name:
kubectl create namespace redis-enterprise
Example of a response:
namespace/redis-enterprise created
-
Switch context to operate within the newly created namespace:
kubectl config set-context --current --namespace=redis-enterprise
Example of a response:
Context "pks-re-cluster" modified.
-
Step 2: Get and prepare deployment files
-
Clone this repository, which contains the deployment files:
git clone https://github.com/RedisLabs/redis-enterprise-k8s-docs
Example of a response:
Cloning into 'redis-enterprise-k8s-docs'... remote: Enumerating objects: 37, done. remote: Counting objects: 100% (37/37), done. remote: Compressing objects: 100% (30/30), done. remote: Total 168 (delta 19), reused 9 (delta 7), pack-reused 131 Receiving objects: 100% (168/168), 45.32 KiB | 7.55 MiB/s, done. Resolving deltas: 100% (94/94), done.
-
Edit the yaml files for your specific deployment, if necessary:
-
bundle.yaml - The bundle file includes several declarations:
rbac
(Role-Based Access Control) defines who can access specified resources. The Operator application requires these definitions to deploy and manage the entire Redis Enterprise deployment (all cluster resources within a namespace). These include declaration of rules, role and rolebinding.crd
creates a CustomResourceDefinition for your Redis Enterprise Cluster resource. This provides another API resource to be handled by the k8s API server and managed by the operator we will deploy next.operator
creates the operator deployment that is responsible for managing the k8s deployment and lifecycle of a Redis Enterprise Cluster. Among many other responsibilities, it creates a stateful set that runs the Redis Enterprise nodes, as pods. The yaml contains the latest image tag representing the latest Operator version available.
This yaml file is commonly not necessary to edit.
-
To apply this yaml file, run:
kubectl apply -f bundle.yaml
After the yaml is applied you receive the response:
role.rbac.authorization.k8s.io/redis-enterprise-operator created serviceaccount/redis-enterprise-operator created rolebinding.rbac.authorization.k8s.io/redis-enterprise-operator created customresourcedefinition.apiextensions.k8s.io/redisenterpriseclusters.app.redislabs.com configured deployment.apps/redis-enterprise-operator created
-
Verify that your redis-enterprise-operator deployment is running:
kubectl get deployment -l name=redis-enterprise-operator
A typical response looks like:
NAME READY UP-TO-DATE AVAILABLE AGE redis-enterprise-operator 1/1 1 1 0m36s
-
Create a storage class. The Redis Enterprise Cluster deployment dynamically provisions Persistent Volume Claims (PVCs) for use with cluster persistent storage needs. In order to create dynamic PVCs, the Kubernetes cluster must have a storage class defined. Determine whether a storage class is defined on your PKS cluster:
kubectl get storageclasses
Since PKS does not automatically provision storage classes, and if you, or the cluster administrator, did not provision storage classes then the response will be:
No resources found.
To create a storage class, determine the type of IaS your PKS cluster is running on and consult the table in the Kubernetes Storage Classes article to determine which provisioner to use.
Two examples of yaml files you can use for popular IaS providers are:
- AWS - gp2.yaml
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: gp2 mountOptions: - debug parameters: type: gp2 provisioner: kubernetes.io/aws-ebs reclaimPolicy: Retain
- GCP - standard.yaml
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: standard mountOptions: - debug parameters: type: pd-standard provisioner: kubernetes.io/gce-pd reclaimPolicy: Retain
-
Create the appropriate yaml file for your IaS and apply it:
kubectl apply -f <your-storage-class.yaml>
More information about persistent storage in Operator deployment.
Note -You can omit the reclaimPolicy declaration in the yaml file, in case of error, for testing and development environments. For production environments you must retain the Persistent Volume Claims (PVCs) when cluster persistent is used, in order to enable recovery.You will use the storage class name you have just created in the next step, editing the Redis Enterprise Cluster (REC) yaml.
-
app_v1_redisenterprisecluster_cr.yaml - Defines the configuration of the newly created resource: Redis Enterprise Cluster. This yaml could be renamed your_pks_cluster.yaml to keep things tidy, but this isn’t a mandatory step. This yaml must be edited, however, to reflect the specific configurations of your cluster. Here are the only fields you must review before you apply the REC yaml:
name
- “your_cluster_name” (e.g. “re-cluster”). You can keep the default name or choose your own in thespec:
section.nodes
- The number of nodes in the cluster, 3 by default (In order to evaluate cluster functionality, must be an uneven number of at least 3 or greater—here’s why)
username
- <your_email_address> - use an accessible email if evaluating alerting or use the default or any other properly formatted address. If not specified the default username is demo@redislabs.com
storageClassName
- Your storage class name from the previous step.redisEnterpriseNodeResources
- The compute resources required for each node. You can use the default or set your own. If your cluster is resource constrained, the minimum workable limits for basic testing are 2 CPU and 3GB. For development and production, see the minimum hardware requirements.- limits – specifies the max resources for a Redis node
- requests – specifies the minimum resources for a Redis node
enforceIPv4: true
- Add this line underspec:
at the same indentation (2 spaces) as ‘nodes’. This indicates to the REC deployment to not attempt to bind to IPv6, which is currently not supported on PKS clusters.redisEnterpriseImageSpec
- This configuration controls the Redis Enterprise version used, and where it is fetched from. The GitHub repository yaml contains the latest image tag from DockerHub. If omitted, the Operator will default to the compatible image version and pull it from DockerHub. This configuration should stay as-is in most circumstances, unless the image used is pulled from a private repository.
Here is an example of the edited REC yaml file:
apiVersion: "app.redislabs.com/v1" kind: "RedisEnterpriseCluster" metadata: name: "rec-pks" spec: enforceIPv4: true nodes: 3 persistentSpec: enabled: true storageClassName: "standard" # ! edit according to infrastructure uiServiceType: LoadBalancer username: "demo@redislabs.com" redisEnterpriseNodeResources: limits: cpu: "2000m" memory: 3Gi requests: cpu: "2000m" memory: 3Gi redisEnterpriseImageSpec: imagePullPolicy: IfNotPresent repository: redislabs/redis versionTag: 5.4.10-22
-
Step 3: Create your cluster
-
Once you have
your_pks_cluster.yaml
file set, you need to apply it to create your Redis Enterprise Cluster:$ kubectl apply -f your_cluster_name.yaml
A typical response looks like:
redisenterprisecluster.app.redislabs.com/rec-pks created
-
To track the creation of the cluster nodes, track the creation of the StatefulSet, which will be names the same as the cluster name you provided in the
your_pks_cluster.yaml
file. In the example above it isrec-pks
:kubectl rollout status sts/rec-pks
A typical response looks like:
Waiting for 3 pods to be ready... Waiting for 2 pods to be ready... Waiting for 1 pods to be ready... statefulset rolling update complete 3 pods at revision rec-pks-808w0973...
-
Verify
rec
creation was successful:kubectl get rec
A typical response looks like:
NAME AGE rec-pks 7m
-
Verify all the pods and deployments are available and running:
kubectl get all
A typical response looks like:
NAME READY STATUS RESTARTS AGE pod/rec-pks-0 1/1 Running 0 16m pod/rec-pks-1 1/1 Running 0 14m pod/rec-pks-2 1/1 Running 0 13m pod/rec-pks-services-rigger-585cbf5ff-5f2z5 1/1 Running 0 16m pod/redis-enterprise-operator-954b6c68c-bgwpr 1/1 Running 0 18m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/rec-pks ClusterIP None <none> 9443/TCP,8001/TCP,8070/TCP 16m service/rec-pks-ui ClusterIP 10.100.200.101 <none> 8443:31459/TCP 16m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/rec-pks-services-rigger 1/1 1 1 16m deployment.apps/redis-enterprise-operator 1/1 1 1 18m NAME DESIRED CURRENT READY AGE replicaset.apps/rec-pks-services-rigger-585f5bcff 1 1 1 16m replicaset.apps/redis-enterprise-operator-9546cb68c 1 1 1 18m NAME READY AGE statefulset.apps/rec-pks 3/3 16m
Step 4: Create a database
In order to create your database, you will log in to the Redis Enterprise UI.
-
First, determine you administrator password. It is stored in an opaque k8s secret named after the REC name. In this example:
kubectl get secret/rec-pks -o yaml
A typical response will include the following lines:
apiVersion: v1 data: license: "" password: ZGdlaWw3Cg== username: YWRtaW5AcmVkaXNsYWJzLmNvbQ== kind: Secret
-
Decode the password:
echo 'ZGdlaWw3Cg==' | base64 --decode
A typical response will include the following lines:
dgeil7
-
There are two primary options for accessing the Web UI:
-
If your PKS cluster has a load balancer service setup with a public IP you have access to or otherwise a routable IP address from your machine:
-
Determine that IP address:
kubectl get service/rec-pks-ui
A typical response will include the following lines:
service/rec-pks-ui LoadBalancer 10.100.200.101 53.128.131.29 8443:31459/TCP 16m
-
Enter the IP address followed by port number 8443 into your browser address bar:
https://53.128.131.29:8443
-
-
If your PKS cluster does not have a routable IP address from your machine:
-
Setup port forwarding for port 8443 to one of you cluster pods:
kubectl port-forward rec-pks-0 8443
A typical response will include the following lines:
Forwarding from 127.0.0.1:8443 -> 8443 Forwarding from [::1]:8443 -> 8443
-
Use
localhost
followed by port number 8443 in your browser address bar:https://localhost:8443
-
-
-
Log in to the Web UI with the username defined in your REC yaml and the password.
-
Follow the instructions to create your database.
For example:
-
In databases, click
.
If you do not have any databases on the node, you are prompted to create a database.
-
Click Next to create a single-region deployment on RAM.
-
Enter the mandatory details of the new :
- Name - enter
pks-test
or another database name. - Memory limit - use the default 0.10GB or whatever value within the available memory.
- Password -enter a password and record it for the next steps.
- Click Activate.
- Name - enter
-
Test the database connectivity test using Telnet:
-
Find the Kubernetes services automatically created for your Redis Enterprise database:
kubectl get service -l app=redis-enterprise-bdb
A typical response will list all database services in the cluster, for example:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE pks-test ClusterIP 10.100.200.52 <none> 14771/TCP 22m pks-test-headless ClusterIP None <none> 14771/TCP 22m
-
Set up port forwarding for the database port to one of your database services:
kubectl port-forward service/pks-test 14771
A typical response will list all database services in the cluster, for example:
Forwarding from 127.0.0.1:14771 -> 14771 Forwarding from [::1]:14771 -> 14771
-
Connect to your database via Telnet and test some basic Redis commands with the database password:
$ telnet 127.0.0.1 14771 Connected to localhost. Escape character is '^]'. auth cdef2843d3e0 +OK ping +PONG get * $-1 set foo bar +OK get foo $3 bar ^] telnet> ^C $
-
-
Step 5: Cleaning up
To remove the Redis Enterprise Cluster from your PKS deployment:
-
Delete any databases you created.
-
Delete the REC custom resource. This removes the pods on the cluster nodes and all related services and deployments, except for the Operator deployment itself.
kubectl delete rec rec-pks
A typical response looks like:
redisenterprisecluster.app.redislabs.com "rec-pks" deleted
-
Delete the Operator deployment:
kubectl delete deployment -l name=redis-enterprise-operator
A typical response looks like:
deployment.extensions "redis-enterprise-operator" deleted
-
Verify that all resources have been removed:
kubectl get all
A typical response looks like:
No resources found.
-
Delete the namespace you’ve created in step 1-5:
kubectl delete namespace redis-enterprise
A typical response looks like:
namespace "redis-enterprise" deleted.
-
Optionally, switch to the default namespace:
kubectl config set-context --current --namespace default