Kubernetes部署Redis集群-小果冻之家

在Kubernetes上部署Redis集群，需要用到PersistentVolume，持久卷类型我选用NFS，当然用local-storage也是可以的，不过在自动化上没有NFS方便。

整个过程步骤较多。

NFS配置

首先Kubernetes主节点来部署并充当NFS服务器。

在主节点上执行如下操作

apt install -y nfs-kernel-server
systemctl start nfs-server

创建6个共享文件夹，我放在/data/redis下，分别是pv1到pv6（后面的redis-trib要求至少有6个节点）。

修改目录权限

chown -R nobody:nogroup /data

编辑/etc/exports文件，追加内容

/data/redis/pv1 *(insecure,rw,async,no_root_squash,no_subtree_check)
/data/redis/pv2 *(insecure,rw,async,no_root_squash,no_subtree_check)
/data/redis/pv3 *(insecure,rw,async,no_root_squash,no_subtree_check)
/data/redis/pv4 *(insecure,rw,async,no_root_squash,no_subtree_check)
/data/redis/pv5 *(insecure,rw,async,no_root_squash,no_subtree_check)
/data/redis/pv6 *(insecure,rw,async,no_root_squash,no_subtree_check)

然后执行

exportfs -arv

或重启nfs-server服务。

主节点操作完成后，还需要在各个子节点上安装nfs客户端：

apt install -y nfs-common

yaml文件

下面是接下来要用到的4个yaml文件。

1-pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv1
spec:
  capacity:
    storage: 200M      #磁盘大小200M
  accessModes:
    - ReadWriteMany    #多客户可读写
  nfs:
    server: rhonin-vm1
    path: /data/redis/pv1

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-vp2
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteMany
  nfs:
    server: rhonin-vm1
    path: /data/redis/pv2

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv3
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteMany
  nfs:
    server: rhonin-vm1
    path: /data/redis/pv3

---

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv4
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteMany
  nfs:
    server: rhonin-vm1
    path: /data/redis/pv4

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-vp5
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteMany
  nfs:
    server: rhonin-vm1
    path: /data/redis/pv5

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv6
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteMany
  nfs:
    server: rhonin-vm1
    path: /data/redis/pv6

server我直接写的域名而不是ip，需要在各个子节点上添加hosts解析，也可以直接用主节点的IP。

2-configmap.yaml，我直接把redis.conf写进yaml里了，注释要拿掉。

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-conf
  labels:
    app: redis
data:
  redis.conf: |
    appendonly yes
    cluster-enabled yes
    cluster-config-file /var/lib/redis/nodes.conf
    cluster-node-timeout 5000
    dir /var/lib/redis
    port 6379

3-headless-services.yaml

apiVersion: v1
kind: Service
metadata:
  name: redis-service
  labels:
    app: redis
spec:
  ports:
    - name: redis-port
      port: 6379
  clusterIP: None
  selector:
    app: redis
    appCluster: redis-cluster

4-stateufl-set.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-set
spec:
  selector:
    matchLabels:
      app: redis
  serviceName: redis-service
  replicas: 6
  template:
    metadata:
      labels:
        app: redis
        appCluster: redis-cluster
    spec:
      terminationGracePeriodSeconds: 20
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                    - key: app
                      operator: In
                      values:
                        - redis
                topologyKey: kubernetes.io/hostname
      containers:
        - name: redis
          image: redis
          imagePullPolicy: IfNotPresent
          command:
            - "redis-server"
          args:
            - "/etc/redis/redis.conf"
            - "--protected-mode"
            - "no"
          # command: redis-server /etc/redis/redis.conf --protected-mode no
          resources:
            requests:
              cpu: 100m
              memory: 100Mi
          ports:
            - name: redis
              containerPort: 6379
              protocol: TCP
            - name: cluster
              containerPort: 16379
              protocol: TCP
          volumeMounts:
            - name: redis-conf
              mountPath: /etc/redis
            - name: redis-data
              mountPath: /var/lib/redis
      volumes:
        - name: redis-conf
          configMap:
            name: redis-conf
            items:
              - key: redis.conf
                path: redis.conf
  volumeClaimTemplates:
    - metadata:
        name: redis-data
      spec:
        accessModes:
          - ReadWriteMany
        resources:
          requests:
            storage: 200M

部署

kubectl apply -f 1-pv.yaml
kubectl apply -f 2-configmap.yaml
kubectl apply -f 3-headless-services.yaml
kubectl apply -f 4-stateful-set.yaml

每个Pod在K8S的域名格式如下

$(pod name).$(service name).$(namespace).svc.cluster.local

初始化集群

部署完成后，需要初始化一下集群。

先启动一个CentOS的Pod

kubectl run -i --tty centos --image=centos:7 --restart=Never /bin/bash

启动成功会自动进入容器。

在/etc/yum.repo.d目录新建一个文件epel.repo，内容如下

[epel]
name=Extra Packages for Enterprise Linux 7 - $basearch
baseurl=https://mirrors.tuna.tsinghua.edu.cn/epel/7/$basearch
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch
failovermethod=priority
enabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

安装必要的软件

yum install -y redis-trib bind-utils

现在，可以进行集群初始化了

redis-trib create --replicas 1 \
`dig +short redis-set-0.redis-service.default.svc.cluster.local`:6379 \
`dig +short redis-set-1.redis-service.default.svc.cluster.local`:6379 \
`dig +short redis-set-2.redis-service.default.svc.cluster.local`:6379 \
`dig +short redis-set-3.redis-service.default.svc.cluster.local`:6379 \
`dig +short redis-set-4.redis-service.default.svc.cluster.local`:6379 \
`dig +short redis-set-5.redis-service.default.svc.cluster.local`:6379

验证集群

随便进入一个Pod，执行redis-cli连接Redis Server，在其命令行下执行

cluster info
cluster nodes

使用集群

由于我们在yaml中配置的Service为redis-service，所以通过它就可以连接Redis集群了。

参考链接

https://cloud.tencent.com/developer/article/1392872

这篇教程有问题，未经过充分验证，需要修改不少地方，比如开头的NFS服务器就没讲明白，到后面Pod初始化时就会报错mount不上。

点击量: 407