011.Kubernetes二进制部署kube-scheduler

一部署高可用kube-scheduler

1.1 高可用kube-scheduler介绍

本实验部署一个三实例 kube-scheduler 的集群，启动后将通过竞争选举机制产生一个 leader 节点，其它节点为阻塞状态。当 leader 节点不可用时，阻塞的节点将再次进行选举产生新的 leader 节点，从而保证服务的可用性。

为保证通信安全，本文档先生成 x509 证书和私钥，kube-controller-manager 在如下两种情况下使用该证书：

与 kube-apiserver 的安全端口通信;
在安全端口(https，10251) 输出 prometheus 格式的 metrics。

1.2 创建kube-scheduler证书和私钥

  1 [root@k8smaster01 ~]# cd /opt/k8s/work

  2 [root@k8smaster01 work]# cat > kube-scheduler-csr.json <<EOF

  3 {

  4   "CN": "system:kube-scheduler",

  5   "hosts": [

  6     "127.0.0.1",

  7     "172.24.8.71",

  8     "172.24.8.72",

  9     "172.24.8.73"

 10   ],

 11   "key": {

 12     "algo": "rsa",

 13     "size": 2048

 14   },

 15   "names": [

 16     {

 17       "C": "CN",

 18       "ST": "Shanghai",

 19       "L": "Shanghai",

 20       "O": "system:kube-scheduler",

 21       "OU": "System"

 22     }

 23   ]

 24 }

 25 EOF

 26 #创建kube-scheduler的CA证书请求文件

解释：

hosts 列表包含所有 kube-scheduler 节点 IP；

CN 和 O 均为 system:kube-scheduler，kubernetes 内置的 ClusterRoleBindings system:kube-scheduler 将赋予 kube-scheduler 工作所需的权限。

  1 [root@k8smaster01 ~]# cd /opt/k8s/work

  2 [root@k8smaster01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem \

  3 -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json \

  4 -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler	#生成CA密钥（ca-key.pem）和证书（ca.pem）

1.3 分发证书和私钥

  1 [root@k8smaster01 ~]# cd /opt/k8s/work

  2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh

  3 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]}

  4   do

  5     echo ">>> ${master_ip}"

  6     scp kube-scheduler*.pem root@${master_ip}:/etc/kubernetes/cert/

  7   done

1.4 创建和分发kubeconfig

kube-scheduler 使用 kubeconfig 文件访问 apiserver，该文件提供了 apiserver 地址、嵌入的 CA 证书和 kube-scheduler 证书：

  1 [root@k8smaster01 ~]# cd /opt/k8s/work

  2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh

  3 [root@k8smaster01 work]# kubectl config set-cluster kubernetes \

  4   --certificate-authority=/opt/k8s/work/ca.pem \

  5   --embed-certs=true \

  6   --server=${KUBE_APISERVER} \

  7   --kubeconfig=kube-scheduler.kubeconfig

  8

  9 [root@k8smaster01 work]# kubectl config set-credentials system:kube-scheduler \

 10   --client-certificate=kube-scheduler.pem \

 11   --client-key=kube-scheduler-key.pem \

 12   --embed-certs=true \

 13   --kubeconfig=kube-scheduler.kubeconfig

 14

 15 [root@k8smaster01 work]# kubectl config set-context system:kube-scheduler \

 16   --cluster=kubernetes \

 17   --user=system:kube-scheduler \

 18   --kubeconfig=kube-scheduler.kubeconfig

 19

 20 [root@k8smaster01 work]# kubectl config use-context system:kube-scheduler --kubeconfig=kube-scheduler.kubeconfig

 21

 22 [root@k8smaster01 ~]# cd /opt/k8s/work

 23 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh

 24 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]}

 25   do

 26     echo ">>> ${master_ip}"

 27     scp kube-scheduler.kubeconfig root@${master_ip}:/etc/kubernetes/

 28   done

1.5 创建kube-scheduler 配置文件

  1 [root@k8smaster01 ~]# cd /opt/k8s/work

  2 [root@k8smaster01 work]# cat >kube-scheduler.yaml.template <<EOF

  3 apiVersion: kubescheduler.config.k8s.io/v1alpha1

  4 kind: KubeSchedulerConfiguration

  5 bindTimeoutSeconds: 600

  6 clientConnection:

  7   burst: 200

  8   kubeconfig: "/etc/kubernetes/kube-scheduler.kubeconfig"

  9   qps: 100

 10 enableContentionProfiling: false

 11 enableProfiling: true

 12 hardPodAffinitySymmetricWeight: 1

 13 healthzBindAddress: ##MASTER_IP##:10251

 14 leaderElection:

 15   leaderElect: true

 16 metricsBindAddress: ##MASTER_IP##:10251

 17 EOF

解释：

--kubeconfig：指定 kubeconfig 文件路径，kube-scheduler 使用它连接和验证 kube-apiserver；

--leader-elect=true：集群运行模式，启用选举功能；被选为 leader 的节点负责处理工作，其它节点为阻塞状态。

  1 [root@k8smaster ~]# cd /opt/k8s/work

  2 [root@k8smaster work]# source /opt/k8s/bin/environment.sh

  3 [root@k8smaster work]# for (( i=0; i < 3; i++ ))

  4   do

  5     sed -e "s/##MASTER_NAME##/${MASTER_NAMES[i]}/" -e "s/##MASTER_IP##/${MASTER_IPS[i]}/" kube-scheduler.yaml.template > kube-scheduler-${MASTER_IPS[i]}.yaml

  6   done					#替换模板文件中的变量

  7 [root@k8smaster01 work]# ls kube-scheduler*.yaml

  8 #MASTER_NAMES 和 MASTER_IPS 为相同长度的 bash 数组，分别为节点名称和对应的 IP

1.6 分发配置文件

  1 [root@k8smaster01 ~]# cd /opt/k8s/work

  2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh

  3 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]}

  4   do

  5     echo ">>> ${master_ip}"

  6     scp kube-scheduler-${master_ip}.yaml root@${master_ip}:/etc/kubernetes/kube-scheduler.yaml

  7   done

1.7 创建kube-scheduler的systemd

  1 [root@k8smaster01 ~]# cd /opt/k8s/work

  2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh

  3 [root@k8smaster01 work]# cat > kube-scheduler.service.template <<EOF

  4 [Unit]

  5 Description=Kubernetes Scheduler

  6 Documentation=https://github.com/GoogleCloudPlatform/kubernetes

  7

  8 [Service]

  9 WorkingDirectory=${K8S_DIR}/kube-scheduler

 10 ExecStart=/opt/k8s/bin/kube-scheduler \\

 11   --config=/etc/kubernetes/kube-scheduler.yaml \\

 12   --bind-address=##MASTER_IP## \\

 13   --secure-port=10259 \\

 14   --port=0 \\

 15   --tls-cert-file=/etc/kubernetes/cert/kube-scheduler.pem \\

 16   --tls-private-key-file=/etc/kubernetes/cert/kube-scheduler-key.pem \\

 17   --authentication-kubeconfig=/etc/kubernetes/kube-scheduler.kubeconfig \\

 18   --client-ca-file=/etc/kubernetes/cert/ca.pem \\

 19   --requestheader-allowed-names="" \\

 20   --requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem \\

 21   --requestheader-extra-headers-prefix="X-Remote-Extra-" \\

 22   --requestheader-group-headers=X-Remote-Group \\

 23   --requestheader-username-headers=X-Remote-User \\

 24   --authorization-kubeconfig=/etc/kubernetes/kube-scheduler.kubeconfig \\

 25   --logtostderr=true \\

 26   --v=2

 27 Restart=always

 28 RestartSec=5

 29 StartLimitInterval=0

 30

 31 [Install]

 32 WantedBy=multi-user.target

 33 EOF

1.8 分发systemd

  1 [root@k8smaster01 ~]# cd /opt/k8s/work

  2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh

  3 [root@k8smaster01 work]# for (( i=0; i < 3; i++ ))

  4   do

  5     sed -e "s/##MASTER_NAME##/${MASTER_NAMES[i]}/" -e "s/##MASTER_IP##/${MASTER_IPS[i]}/" kube-scheduler.service.template > kube-scheduler-${MASTER_IPS[i]}.service

  6   done					#修正相应IP

  7 [root@k8smaster01 work]# ls kube-scheduler*.service

  8 [root@k8smaster01 ~]# cd /opt/k8s/work

  9 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh

 10 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]}

 11   do

 12     echo ">>> ${master_ip}"

 13     scp kube-scheduler-${master_ip}.service root@${master_ip}:/etc/systemd/system/kube-scheduler.service

 14   done						#分发system

二启动并验证

2.1 启动kube-scheduler 服务

  1 [root@k8smaster01 ~]# cd /opt/k8s/work

  2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh

  3 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]}

  4   do

  5     echo ">>> ${master_ip}"

  6     ssh root@${master_ip} "mkdir -p ${K8S_DIR}/kube-scheduler"

  7     ssh root@${master_ip} "systemctl daemon-reload && systemctl enable kube-scheduler && systemctl restart kube-scheduler"

  8   done						#启动服务前必须先创建工作目录

2.2 检查kube-scheduler 服务

  1 [root@k8smaster01 ~]# source /opt/k8s/bin/environment.sh

  2 [root@k8smaster01 ~]# for master_ip in ${MASTER_IPS[@]}

  3   do

  4     echo ">>> ${master_ip}"

  5     ssh root@${master_ip} "systemctl status kube-scheduler|grep Active"

  6   done

2.3 查看输出的 metrics

kube-scheduler 监听 10251 和 10251 端口：

10251：接收 http 请求，非安全端口，不需要认证授权；
10259：接收 https 请求，安全端口，需要认证授权。
两个接口都对外提供 /metrics 和 /healthz 的访问。

  1 [root@k8smaster01 ~]# sudo netstat -lnpt |grep kube-sch

  1 [root@k8smaster01 ~]# curl -s http://127.0.0.1:10251/metrics |head

  2 [root@k8smaster01 ~]# curl -s --cacert /opt/k8s/work/ca.pem --cert /opt/k8s/work/admin.pem --key /opt/k8s/work/admin-key.pem https://172.24.8.71:10259/metrics |head

注意：以上命令在 kube-controller-manager 节点上执行。

2.4 查看当前leader

  1 [root@k8smaster01 ~]# kubectl get endpoints kube-scheduler --namespace=kube-system  -o yaml

当前leader为k8smaster01。

巴特西

011.Kubernetes二进制部署kube-scheduler

一部署高可用kube-scheduler

1.1 高可用kube-scheduler介绍

1.2 创建kube-scheduler证书和私钥

1.3 分发证书和私钥

1.4 创建和分发kubeconfig

1.5 创建kube-scheduler 配置文件

1.6 分发配置文件

1.7 创建kube-scheduler的systemd

1.8 分发systemd

二启动并验证

2.1 启动kube-scheduler 服务

2.2 检查kube-scheduler 服务

2.3 查看输出的 metrics

2.4 查看当前leader

最新文章

热门文章

巴特西

011.Kubernetes二进制部署kube-scheduler

一 部署高可用kube-scheduler

1.1 高可用kube-scheduler介绍

1.2 创建kube-scheduler证书和私钥

1.3 分发证书和私钥

1.4 创建和分发kubeconfig

1.5 创建kube-scheduler 配置文件

1.6 分发配置文件

1.7 创建kube-scheduler的systemd

1.8 分发systemd

二 启动并验证

2.1 启动kube-scheduler 服务

2.2 检查kube-scheduler 服务

2.3 查看输出的 metrics

2.4 查看当前leader

最新文章

热门文章

一部署高可用kube-scheduler

二启动并验证