This tutorial describes how to configure LDAP-based authentication for a Redis Enterprise cluster on Kubernetes. Throughout this tutorial, we will assume that there is a single namespace called “db”.

LDAP overview

At minimum, you’ll need the following information about your LDAP server:

  • The base name for where to find users (e.g., ou=users,dc=example,dc=org)
  • The filter to identify a particular user (e.g., uid=%u)
  • The bind DN for the account that can search (e.g., cn=admin,dc=example,dc=org)
  • The password for the bind DN
  • The server host and port
  • Whether you are using SSL

This tutorial uses a test LDAP server deployed on the k8s cluster hosting Redis Enterprise.

Setting up an LDAP server

If you don’t already have an LDAP server, you can easily deploy one in the same K8s cluster for testing using a helm chart for OpenLDAP:

  1. Create a file called ldap-values.yaml containing the following:

    env:
  LDAP_ORGANISATION: "ACME Inc."
  LDAP_DOMAIN: "example.org"
adminPassword: admin
configPassword: config

  2. Deploy OpenLDAP:

    helm install ldap -f ldap-values.yaml stable/openldap

  3. Forward the pod to your localhost so you can configure the users. The ‘xxx’ should be replace with the actual pod hash:

    kubectl port-forward pod/ldap-openldap-xxx 3889:389

  4. Create a file called users.ldif for the users Organizational Unit:

    dn: ou=users,dc=example,dc=org
objectClass: organizationalUnit
ou: users

  5. Create a file called user.ldif for your test user:

    dn: uid=tester,ou=users,dc=example,dc=org
objectClass: top
objectClass: account
objectClass: posixAccount
cn: tester
uid: tester
uidNumber: 1001
gidNumber: 1001
homeDirectory: /home/tester
userPassword: {SSHA}5IcQ2zo5wkCCohXHGWteDMBDbJElbChP

    The password for the user is “tester”. If you’d like to change it, run the following command:

    ldappasswd  -h localhost -p 3889 -s newpassword -W -D "cn=admin,dc=example,dc=org" -x "uid=tester,ou=users,dc=example,dc=org"

  6. Create the OU and user:

    ldapadd -h localhost -p 3889 -x -W -D "cn=admin,dc=example,dc=org" -f users.ldif
ldapadd -h localhost -p 3889 -x -W -D "cn=admin,dc=example,dc=org" -f user.ldif

Deploying a Cluster with LDAP Support

Overview

A cluster must be configured to use an external LDAP server. This can be done via the rladmin command or via the REST API by setting the “saslauthd_ldap_conf” cluster parameter with the saslauthd configuration information. Internally, the saslauthd ademon handles LDAP-based authentication.

Once you have enabled LDAP authentication, you can test connectivity using the testsaslauthd command. Log in to any Redis Enterprise Node’s pod, and then run the following:

testsaslauthd -u [USERNAME] -p [PASSWORD]

Once the saslauthd daemon can successfully authenticate users, you need to add the user to the list of allowed users in the Redis Enterprise cluster.

You can add new users using the administrative UI or the REST API.

A user that is not added is not allowed to authenticate. If you can authenticate with testsaslauthd but not via the REST API or the UI, then you need to verify the user has been added to the Redis Enterprise cluster.

If you want to programatically add your LDAP user, you can simply use the REST API. You’ll need to have access to the API. In this example, the API has been port-forwarded to the local host:

cat << EOF > add-user.json
{
    "name":"tester",
    "email":"tester@example.org",
    "role":"admin",
    "email_alerts":false,
    "auth_method":"external"
}
EOF
curl -v -k -u "demo@redislabs.com:xxx" -X POST -d @add-user.json -H "Content-Type: application/json" https://localhost:9443/v1/users

Deploying a Redis Enterprise cluster

A Redis Enterprise cluster requires no special setup other to be configured with LDAP as the configuration is outside of the scope of the Redis Enterprise custom resource.

You can create a simple test cluster that can be used with the examples below by using the following custom resource:

apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: ldap
spec:
  nodes: 3
  redisEnterpriseNodeResources:
    limits:
      cpu: 1
      memory: 3Gi
    requests:
      cpu: 1
      memory: 3Gi

LDAP configuration

A new Redis Enterprise cluster does not have any LDAP configuration but is ready to accept them. The saslauthd daemon is already running and configured to use LDAP as a mechanism. All you need to do is to provide the cluster with the LDAP configuration information (see saslauthd’s configuration reference).

The following configuration parameters for saslauthd reference the LDAP server we just configured:

ldap_servers: ldap://ldap-openldap.bdb.svc:389
ldap_search_base: ou=users,dc=example,dc=org
ldap_filter: (uid=%u)
ldap_bind_dn: cn=admin,dc=example,dc=org
ldap_password: admin

Using rladmin to update the configuration

If this information was in a file called “ldap.conf” stored in “/tmp”, you can connect to a Redis Enterprise Node pod and configure the server via:

rladmin cluster config saslauthd_ldap_conf /tmp/ldap.conf

The saslauthd daemon is configured to use the file /etc/opt/redislabs/saslauthd.conf on each node. You cannot edit this file directly on the node. Instead, you just need a local copy of the configuration you desire and rladmin command will update the cluster configuration. The cluster will update all the node’s configuration and preserve the setting after any pod restarts.

Using the REST API to update the configuration

Alternatively, you can post the same configuration data in a request to the REST API for the cluster. First you forward the cluster API from a pod:

kubectl port-forward pod/ldap-0 9443

Then you change the LDAP configuration with a JSON version of the ldap configuration.

cat << EOF > tmp.json
{"saslauthd_ldap_conf": "ldap_servers: ldap://ldap-openldap.bdb.svc:389\nldap_search_base: ou=users,dc=example,dc=org\nldap_filter: (uid=%u)\nldap_bind_dn: cn=admin,dc=example,dc=org\nldap_password: admin\n"}
EOF
curl -f -k -u "demo@redislabs.com:xxx" -X PUT -d @tmp.json -H "Content-Type: application/json" https://localhost:9443/v1/cluster

The key value for “saslauthd_ldap_conf” is the configuration file verbatim as it would be in the saslauthd configuration file. As such, it can be relatively tricky to get the syntax right and generating this programmatically is helpful. A little python program (make_request.py) can help:

import sys
import json

with open(sys.argv[2]) as data:
   d = {}
   d[sys.argv[1]] = data.read()
   print(json.dumps(d))

And so we can do:

python make_request.py saslauthd_ldap_conf ldap.conf > tmp.json
curl -f -k -u "demo@redislabs.com:xxx" -X PUT -d @tmp.json -H "Content-Type: application/json" https://localhost:9443/v1/cluster

Automating LDAP Configuration

If you are automating deployments with the operator, you can automate this configuration change too with a simple K8s job.

  1. Create a ConfigMap that contains the LDAP configuration:

    kubectl create configmap config-job --from-file=saslauthd_ldap_conf=ldap.conf

  2. Submit this job:

    apiVersion: batch/v1
kind: Job
metadata:
  name: config-ldap
spec:
  backoffLimit: 4
  template:
    spec:
      serviceAccountName: redis-enterprise-operator
      restartPolicy: Never
      containers:
        - name: config
          image: you/job-config-ldap:1
          env:
            - name: MY_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: CLUSTER_NAME
              value: ldap
            - name: CLUSTER_USER
              valueFrom:
               secretKeyRef:
                 name: ldap
                 key: username
            - name: CLUSTER_PASSWORD
              valueFrom:
               secretKeyRef:
                 name: ldap
                 key: password
          volumeMounts:
            - name: config
              mountPath: "/config"
              readOnly: true
      volumes:
        - name: config
          configMap:
            name: config-job

    In the above job, the cluster name is “ldap”. You will have to change this to the name of your cluster in all three of the CLUSTER_NAME, CLUSTER_USER, and CLUSTER_PASSWORD environment variables.

This job waits for the cluster to be in the “Running” state. You can submit this job at the same time as the Redis Enterprise custom resource. It will wait for a period of time for the cluster to spin up and then change the configuration. This time period is tunable.

You’ll need to create the Docker image (i.e., in the example as “you/job-config-ldap:1”) and make it available to your K8s cluster.

You can build this Docker image yourself with the following:

  • Dockerfile

    FROM python:3.8-slim

RUN apt-get update && apt-get install -y curl
RUN pip install kubernetes jsonpath_ng
RUN mkdir -p /app
COPY config.sh /app/
COPY make_request.py /app/
COPY waitfor.py /app/
WORKDIR /app

ENTRYPOINT ["/bin/sh", "/app/config.sh"]

  • config.sh

    #!/bin/sh
CLUSTER_HOST=${CLUSTER_NAME}.${MY_NAMESPACE}.svc.cluster.local
CLUSTER_PORT=9443
if python waitfor.py --namespace $MY_NAMESPACE --group app.redislabs.com redisenterpriseclusters $CLUSTER_NAME "$.status.state"  --value "Running" --wait --period 30 --limit 20
then
   python make_request.py saslauthd_ldap_conf /config/saslauthd_ldap_conf > tmp.json
   curl -f -k -u "$CLUSTER_USER:$CLUSTER_PASSWORD" -X PUT -d @tmp.json -H "Content-Type: application/json" https://${CLUSTER_HOST}:${CLUSTER_PORT}/v1/cluster
else
   echo "Cluster is not ready within required period."
   exit 1
fi

  • make_request.py

    import sys
import json

with open(sys.argv[2]) as data:
   d = {}
   d[sys.argv[1]] = data.read()
   print(json.dumps(d))

  • waitfor.py

    from kubernetes import client, config
import argparse
import pprint
from jsonpath_ng import jsonpath, parse as parse_jsonpath
import sys
from time import sleep

if __name__ == '__main__':

   argparser = argparse.ArgumentParser(description='k8s-waitfor')
   argparser.add_argument('--use-config',help='Use the .kubeconfig file.',action='store_true',default=False)
   argparser.add_argument('--wait',help='Wait for success',action='store_true',default=False)
   argparser.add_argument('--limit',help='The limit of interations of waiting',type=int,default=5)
   argparser.add_argument('--period',help='The period of time to wait (seconds)',type=int,default=12)
   argparser.add_argument('--verbose',help='Verbose output',action='store_true',default=False)
   argparser.add_argument('--group',help='The API group',default='app.redislabs.com')
   argparser.add_argument('--version',help='The API version',default='v1')
   argparser.add_argument('--namespace',help='The namespace',default='default')
   argparser.add_argument('--value',help='A value to compare')
   argparser.add_argument('--value-type',help='A value type',default='string',choices=['integer','float','string'])
   argparser.add_argument('--compare',help='A comparison operator',default='eq',choices=['eq','neq','gt','gte','lt','lte'])
   argparser.add_argument('--cluster',help='Check a cluster scoped object',action='store_true',default=False)
   argparser.add_argument('plural',help='The object kind')
   argparser.add_argument('name',help='The object name')
   argparser.add_argument('expr',help='The jsonpath expression')

   args = argparser.parse_args()

   expr = parse_jsonpath(args.expr)

   if args.use_config:
      config.load_kube_config()
   else:
      config.load_incluster_config()

   api = client.CustomObjectsApi()

   pp = pprint.PrettyPrinter(indent=2)


   if not args.wait:
      args.limit = 1

   iteration = 0
   count = 0

   while count==0 and iteration < args.limit:

      if args.cluster:
         # /apis/{group}/{version}/{plural}/{name}
         # group = apiextensions.k8s.io
         # version = v1
         # plural = customresourcedefinitions
         # name = redisenterpriseclusters.app.redislabs.com
         obj = api.get_cluster_custom_object(args.group,args.version,args.plural,args.name)
      else:
         #  /apis/{group}/{version}/namespaces/{namespace}/{plural}/{name}
         # e.g.
         # group = app.redislabs.com
         # version = v1
         # namespace = default
         # plural = redisenterpriseclusters
         # name = test
         obj = api.get_namespaced_custom_object(args.group,args.version,args.namespace,args.plural,args.name)

      if args.value is not None:
         if args.value_type=='integer':
            args.value = int(args.value)
         if args.value_type=='float':
            args.value = float(args.value)
      for match in expr.find(obj):
         if args.verbose:
            pp.pprint(match.value)
         if args.value is not None:
            if (args.compare=='eq' and args.value==match.value) or \
               (args.compare=='neq' and args.value!=match.value) or \
               (args.compare=='gt' and match.value>args.value) or \
               (args.compare=='gte' and match.value>=args.value) or \
               (args.compare=='lt' and match.value<args.value) or \
               (args.compare=='lte' and match.value<=args.value):
               count += 1
         else:
            count += 1
      if args.wait and count==0:
         if args.verbose:
            print('Waiting: {}'.format(iteration))
         sleep(args.period)

      iteration += 1

   sys.exit(0 if count>0 else 1)

Using volumes for TLS keys and certificate files

There are four parameters for TLS that require a local file:

  • ldap_tls_cacert_file - a file containing the CA certificates
  • ldap_tls_cacert_dir - a directory containing CA certificates
  • ldap_tls_cert - the client certificate file
  • ldap_tls_key - the client private key

You can provide these files and directories using a volume mount. On the cluster specification, you can specify the extra volumes via “volumes” and the extra volume mounts via “redisEnterpriseVolumeMounts”.

One way to handle the certificate files is put them into a ConfigMap and mount the volume from it. For example, if we have a special certificate authority in a file a called “ca.crt”, we can create a ConfigMap:

kubectl create configmap ldap-certs --from-file=ca.crt=ca.crt

Then in our cluster, we add the volume and volume mount:

apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: ldap
spec:
  nodes: 3
  redisEnterpriseNodeResources:
    limits:
      cpu: 1
      memory: 3Gi
    requests:
      cpu: 1
      memory: 3Gi
  volumes:
   - name: ldap-volume
     configMap:
       name: ldap-certs
  redisEnterpriseVolumeMounts:
   - name: ldap-volume
     mountPath: /opt/ldap

The result is we can use the file path “/opt/ldap/ca.crt” for the “ldap_tls_cert_file” parameter.

You can use the usual techniques to map a set of files into a directory for the “ldap_tls_cert_dir” parameter if you have multiple CA certificates:

apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: ldap
spec:
  nodes: 3
  redisEnterpriseNodeResources:
    limits:
      cpu: 1
      memory: 3Gi
    requests:
      cpu: 1
      memory: 3Gi
  volumes:
   - name: ldap-volume
     configMap:
       name: ldap-certs
       items:
        - key: A.crt
          path: ca/A.crt
        - key: B.crt
          path: ca/B.crt
        - key: client.crt
          path: client.crt
        - key: client.key
          path: client.key
  redisEnterpriseVolumeMounts:
   - name: ldap-volume
     mountPath: /opt/ldap

which allows you to use:

ldap_tls_cacert_dir: /opt/ldap/ca
ldap_tls_cert: /opt/ldap/client.crt
ldap_tls_key: /opt/ldap/client.key

Known Issues

  1. The saslauthd daemon is not restarted when the configuration changes. If you have never configured LDAP, it will still work. The configuration file does not exist until you do and so it will get read. Unfortunately, if you change the configuration and have previously tried to authenticate with an external user, the configuration will not get read again until the saslauthd daemon is restarted. There is currently no way to do that via K8s other than to restart the pods because K8s administrators don’t have access to root in the Redis Enterprise container.

    You can proceed with caution and restart the pods by running:

    kubectl rollout restart statefulset name-of-cluster

    where “name-of-cluster” is the name of the Redis Enterprise cluster in your CR.

  2. For cluster-deployed LDAP servers, the suffix “cluster.local” will not resolve properly. You can remove the “cluster.local” suffix for LDAP deployments in the same K8s cluster.