一、Volume介绍

容器和Pod是短暂的，它们的生命周期可能很短，会被频繁的销毁和创建，存在容器中的数据会被清除，为了持久化保存容器的数据，k8s提供了Volume。
Volume的生命周期独立于容器，它是一个目录，Volume会被mount到Pod，Pod中的所有容器都可以访问这个Volume，和Docker Volume类似。
Volume支持的类型：emptyDir、hostPath、AWS Elastic Block Store、NFS、Ceph等。

1、emptyDir

emptyDir是最基础的Volume类型，它对于容器来说是持久的，对于Pod刚不是，当Pod从节点删除时Volume中的数据也会被删除，如果只是容器被销毁而Pod还在，则Volume不受影响。

①创建示例应用

apiVersion: v1

kind: Pod

metadata:

  name: hello-world

spec:

  containers:

  - image: busybox

    name: hello

    volumeMounts:

    - mountPath: /hello_dir   #②hello容器将shared-volume mount到/hello_dir目录

      name: shared-volume

    args:

    - /bin/sh       #③hello通过echo将数据写到文件hello里

    - -c

    - echo "hello world" >/hello_dir/hello; sleep 

  - image: busybox

    name: world    #④world容器将 shared-volume mount到 /world_dir目录

    volumeMounts:

    - mountPath: /world_dir

      name: shared-volume

    args:

    - /bin/sh     #⑤world通过cat从文件hello读取数据

    - -c

    - cat /world_dir/hello; sleep 

  volumes:

  - name: shared-volume   #①定义了一个emptyDir类型的Volume shared-volume

    emptyDir: {}

②查看验证

[root@k8s-node1 volume]# kubectl apply -f emptyDir.yaml

pod "hello-world" created

[root@k8s-node1 volume]# kubectl get pod

NAME          READY     STATUS    RESTARTS   AGE

hello-world   /       Running             36s

[root@k8s-node1 volume]# kubectl logs hello-world world

hello world

上述显示容器world成功读到了hello容器写入的数据，验证了两个容器共享emptyDir Volume,其效果相当于执行了docker run -v /hello_dir 和/docker run -v /world_dir

emptyDir是Host主机上创建的临时目录，其优点能够方便地为Pod中的容器提供共享存储，但不具有持久性，如果Pod不存在了，emptyDir也就没有了，适合Pod中容器需要临时共享存储空间的应用场景

2、HostPath

hostPath Volume的作用是将Docker Host文件系统中已经存在的目录mount给Pod的容器，大部分应用不会使用此类型，因为这实际上增加上了Pod与节点的耦合，有些需要访问k8s内部数据，比如配置文件和二进制库这样的应用需要使用hostPath，例如k8s自身的应用kube-apiserver和kube-controller-manager。

3、外部Storage Provider

如果k8s部署在AWS、GCE、Azure等公有云上，可以直接使用云盘作为Volume。因资源有限，此处无法演示。

二、PV & PVC

PV是外部存储系统中的一块存储空间，由管理员创建，与Volume一样具有持久性，生命周期独立于Pod。
PVC刚是对PV的申请，PVC通常由普通用户创建，需要为Pod分配存储资源时，用户可以创建一个PVC，指明存储资源的容量大小和访问权限等，k8s会查找并提供满足条件的PV。
k8s支持多种类型的PV，比如AWS EBS、Ceph、NFS等

1、NFS PV

①在k8s-master节点搭建一个NFS服务，目录为/nfsdata

[root@k8s-node1 ~]# showmount -e

Export list for k8s-node1:

/nfsdata *

NFS服务具体部署请参照：https://www.cnblogs.com/yanxinjiang/p/7419469.html

②创建应用示例

apiVersion: v1

kind: PersistentVolume

metadata:

  name: mypv

spec:

  capacity:

    storage: 1Gi    #指定PV的容量为1GB

  accessModes:

    - ReadWriteOnce  #表示PV能为read-write模式mount到单个节点，ReadOnlyMany表示只读，mount到多个节点，ReadWriteMany表示读写可以mount到多节点

  persistentVolumeReclaimPolicy: Recycle  #PV的回收策略，清除PV中的数据，Retain表示管理员手工回收

  storageClassName: nfs    #指定PV的class为nfs，相当于为PV设置了一个分类，PVC可以指定class申请相应的PV

  nfs:

    path: /nfsdata/pv1     #指定PV在NFS服务器上对应的目录

    server: 192.168.56.11

[root@k8s-node1 nfs-pv]# kubectl apply -f nfs-pv1.yaml

persistentvolume "mypv1" created

[root@k8s-node1 nfs-pv]# kubectl get pv

NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM     STORAGECLASS   REASON    AGE   #STATUS为Available，表示mypv1就绪，可以被PVC申请

mypv1     1Gi        RWO            Recycle          Available             nfs                      4s

③创建PVC

kind: PersistentVolumeClaim

apiVersion: v1

metadata:

  name: mypv

spec:

  accessModes:

    - ReadWriteOnce

  resources:

    requests:

      storage: 1Gi

  storageClassName: nfs

[root@k8s-node1 nfs-pv]# kubectl apply -f nfs-pvc1.yaml

persistentvolumeclaim "mypvc1" created

[root@k8s-node1 nfs-pv]# kubectl get pvc

NAME      STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE

mypvc1    Bound     mypv1     1Gi        RWO            nfs            13s

[root@k8s-node1 nfs-pv]# kubectl get pv

NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM            STORAGECLASS   REASON    AGE

mypv1     1Gi        RWO            Recycle          Bound     default/mypvc1   nfs                      22m

结果：mypvc1已经成功绑定到mypv1上面

④在Pod中使用存储

kind: Pod

apiVersion: v1

metadata:

  name: mypod1

spec:

  containers:

    - name: mypod1

      image: busybox

      args:

      - /bin/sh

      - -c

      - sleep

      volumeMounts:

      - mountPath: "/mydata"

        name: mypv

  volumes:

    - name: mypv

      persistentVolumeClaim:

        claimName: mypv

⑤测试验证

[root@k8s-node1 nfs-pv]# kubectl apply -f pod1.yaml

[root@k8s-node1 nfs-pv]# kubectl get pod -o wide

NAME      READY     STATUS    RESTARTS   AGE       IP           NODE

mypod1    /       Running             11m       10.2.83.14   192.168.56.12

[root@k8s-node1 nfs-pv]# kubectl exec mypod1 touch /mydata/readme.md

[root@k8s-node1 nfs-pv]# ls /nfsdata/pv1/

readme.md

在Pod中创建的文件/mydata已经存储到了NFS服务器目录/nfsdata/pv1中。

⑥回收PV

当不需要使用PV时，可用删除PVC回收PV

[root@k8s-node1 nfs-pv]# kubectl delete pvc mypvc1

persistentvolumeclaim "mypvc1" deleted

因PV的回收策略设置为Recycle，所以数据会被清除，如想保存数据，可以将策略设置为Retain

apiVersion: v1

kind: PersistentVolume

metadata:

  name: mypv1

spec:

  capacity:

    storage: 1Gi

  accessModes:

    - ReadWriteOnce

  persistentVolumeReclaimPolicy: Retain  #Recycle

  storageClassName: nfs

  nfs:

    path: /nfsdata/pv1

    server: 192.168.56.11

[root@k8s-node1 nfs-pv]# kubectl apply -f nfs-pv1.yaml

persistentvolume "mypv1" configured

[root@k8s-node1 nfs-pv]# kubectl get pv

NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM            STORAGECLASS   REASON    AGE

mypv1     1Gi        RWO            Retain           Bound     default/mypvc1   nfs

⑦验证

[root@k8s-node1 nfs-pv]# kubectl apply -f nfs-pvc1.yaml

[root@k8s-node1 nfs-pv]# kubectl exec mypod1 touch /mydata/hello.world

[root@k8s-node1 nfs-pv]# ls /nfsdata/pv1/

hello.world

[root@k8s-node1 nfs-pv]# kubectl delete pvc mypvc1

persistentvolumeclaim "mypvc1" deleted

[root@k8s-node1 nfs-pv]# ls /nfsdata/pv1/

hello.world

2、PV动态供给

静态供给：我们提前创建PV，通过PVC申请PV并在Pod中使用
动态供给：如果没有满足PVC条件的PV，会动态创建PV，不需要提前创建PV，效率高，通过StorageClass实现。

3、k8s中MySQL数据持久化存储

①创建PV和PVC

apiVersion: v1

kind: PersistentVolume

metadata:

  name: mysql-pv

spec:

  capacity:

    storage: 2Gi

  accessModes:

    - ReadWriteOnce

  persistentVolumeReclaimPolicy: Retain

  storageClassName: nfs

  nfs:

    server: 192.168.56.11

    path: /data/k8s/mysql-pv

---

kind: PersistentVolumeClaim

apiVersion: v1

metadata:

  name: mysql-pvc

spec:

  accessModes:

    - ReadWriteOnce

  resources:

    requests:

      storage: 2Gi

  storageClassName: nfs

查看pv,pvc状态

[root@k8s-node1 mysql]# kubectl apply -f mysql_pv.yaml

persistentvolume "mysql-pv" created

persistentvolumeclaim "mysql-pvc" created

[root@k8s-node1 mysql]# kubectl get pv,pvc

NAME                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM               STORAGECLASS   REASON    AGE

persistentvolume/mysql-pv   2Gi        RWO            Retain           Bound     default/mysql-pvc   nfs                      4s

NAME                              STATUS    VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE

persistentvolumeclaim/mysql-pvc   Bound     mysql-pv   2Gi        RWO            nfs            4s

②部署MySQL

apiVersion: apps/v1beta1

kind: Deployment

metadata:

  name: mysql

spec:

  selector:

    matchLabels:

      app: mysql

  template:

    metadata:

      labels:

        app: mysql

    spec:

      containers:

      - name: mysql

        image: mysql:5.6

        imagePullPolicy: IfNotPresent

        env:

        - name: MYSQL_ROOT_PASSWORD

          value: password

        ports:

        - name: mysql

          containerPort:

        volumeMounts:

        - mountPath: /var/lib/mysql

          name: mysql-persistent-storage

      volumes:

      - name: mysql-persistent-storage

        persistentVolumeClaim:

          claimName: mysql-pvc

---

apiVersion: v1

kind: Service

metadata:

  name: mysql

spec:

  ports:

  - port:

  selector:

    app: mysql

创建应用查看

[root@k8s-node1 mysql]# kubectl apply -f mysql.yaml

deployment.apps "mysql" created

[root@k8s-node1 k8s]# kubectl get pod -o wide

NAME                     READY     STATUS    RESTARTS   AGE       IP          NODE

mysql-57c44d9d5f-5v7qc   /       Running             7m        10.2.8.25   192.168.56.13

③进入MySQL数据库创建测试数据

kubectl exec -it mysql-57c44d9d5f-5v7qc /bin/bash

root@mysql-57c44d9d5f-5v7qc:/# mysql -uroot -p

Enter password:

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is

Server version: 5.6. MySQL Community Server (GPL)

Copyright (c) , , Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

mysql>

mysql>

mysql> show databases;

+--------------------+

| Database           |

+--------------------+

| information_schema |

| mysql              |

| performance_schema |

mysql> create database test;

Query OK,  row affected (0.01 sec)

mysql> use test;

Database changed

mysql> create table my_id(id int());

Query OK,  rows affected (0.03 sec)

mysql> insert my_id values();

Query OK,  row affected (0.01 sec)

mysql> select * from my_id;

+------+

| id   |

+------+

|   |

+------+

 row in set (0.00 sec)

④模拟k8s-node3节点故障

此时node3节点变为不可用

[root@k8s-node1 mysql]# kubectl get node

NAME            STATUS     ROLES     AGE       VERSION

192.168.56.12   Ready      <none>    6d        v1.10.3

192.168.56.13   NotReady   <none>    6d        v1.10.3

⑤等待一段时间后，k8s会将MySQL迁移至node2节点

[root@k8s-node1 mysql]# kubectl get pod -o wide

NAME                     READY     STATUS    RESTARTS   AGE       IP           NODE

mysql-57c44d9d5f-5v7qc   /       Unknown             47m       10.2.8.25    192.168.56.13

mysql-57c44d9d5f-6gvh5   /       Running             18m       10.2.93.74   192.168.56.12

[root@k8s-node1 mysql]# kubectl exec -it mysql-57c44d9d5f-6gvh5 /bin/bash

⑥再次登录MySQL验证数据的一致性

root@mysql-57c44d9d5f-6gvh5:/# mysql -uroot -ppassword

Warning: Using a password on the command line interface can be insecure.

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is

Server version: 5.6. MySQL Community Server (GPL)

Copyright (c) , , Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;

+--------------------+

| Database           |

+--------------------+

| information_schema |

| mysql              |

| performance_schema |

| test               |

+--------------------+

 rows in set (0.02 sec)

mysql> use test;

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

mysql> show tables;

+----------------+

| Tables_in_test |

+----------------+

| my_id          |

+----------------+

 row in set (0.00 sec)

mysql> select * from my_id;

+------+

| id   |

+------+

|   |

+------+

 row in set (0.00 sec)

结果：MySQL服务恢复，数据无丢失

巴特西

linux运维、架构之路-K8s数据管理