kubernetes部署kube-controller-manager服务

本文档介绍部署高可用 kube-controller-manager 集群的步骤。

该集群包含 3 个节点，启动后将通过竞争选举机制产生一个 leader 节点，其它节点为阻塞状态。当 leader 节点不可用后，剩余节点将再次进行选举产生新的 leader 节点，从而保证服务的可用性。

以下是非认证的的配置文件（用127.0.0.1连接kube-apiserver）：

cat > /lib/systemd/system/kube-controller-manager.service <<"EOF"

[Unit]

Description=Kubernetes Controller Manager

Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]

ExecStart=/usr/local/bin/kube-controller-manager \

  --address=127.0.0.1 \

  --master=http://127.0.0.1:8080 \

  --allocate-node-cidrs=true \

  --service-cluster-ip-range=10.254.0.0/16 \

  --cluster-cidr=172.30.0.0/16 \

  --cluster-name=kubernetes \

  --leader-elect=true \

  --cluster-signing-cert-file=/etc/kubernetes/ca/ca.pem \

  --cluster-signing-key-file=/etc/kubernetes/ca/ca-key.pem \

  --service-account-private-key-file=/etc/kubernetes/ca/ca-key.pem \

  --root-ca-file=/etc/kubernetes/ca/ca.pem \

  --v=2

Restart=on-failure

RestartSec=5

[Install]

WantedBy=multi-user.target

EOF

这个简单很多，但是以后可能会取消，如果要使用认证授权的方式看下面：

------------------------------------------------------------------

为保证通信安全，本文档先生成 x509 证书和私钥，kube-controller-manager 在如下两种情况下使用该证书：

与 kube-apiserver 的安全端口通信时;
在安全端口(https，10252) 输出 prometheus 格式的 metrics；

创建 kube-controller-manager 证书和私钥

cat > kube-controller-manager-csr.json <<EOF

{

    "CN": "system:kube-controller-manager",

    "key": {

        "algo": "rsa",

        "size": 2048

    },

    "hosts": [

      "127.0.0.1",

      "192.168.111.10",

      "192.168.111.11",

      "192.168.111.12"

    ],

    "names": [

      {

        "C": "CN",

        "ST": "ChongQing",

        "L": "ChongQing",

        "O": "system:kube-controller-manager",

        "OU": "yunwei"

      }

    ]

}

EOF

cfssl gencert -ca=/etc/kubernetes/ca/ca.pem \

  -ca-key=/etc/kubernetes/ca/ca-key.pem \

  -config=/etc/kubernetes/ca/ca-config.json \

  -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager

创建和分发 kubeconfig 文件

kubeconfig 文件包含访问 apiserver 的所有信息，如 apiserver 地址、CA 证书和自身使用的证书；

kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/ca/ca.pem \
--embed-certs=true \
--server=https://192.168.111.9:8443 \
--kubeconfig=kube-controller-manager.kubeconfig

kubectl config set-credentials system:kube-controller-manager \
--client-certificate=/etc/kubernetes/ca/kube-controller-manager.pem \
--client-key=/etc/kubernetes/ca/kube-controller-manager-key.pem \
--embed-certs=true \
--kubeconfig=kube-controller-manager.kubeconfig

kubectl config set-context system:kube-controller-manager \
--cluster=kubernetes \
--user=system:kube-controller-manager \
--kubeconfig=kube-controller-manager.kubeconfig

kubectl config use-context system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig

分发 kubeconfig 到所有 master 节点：

# scp kube-controller-manager.kubeconfig 192.168.111.11:/etc/kubernetes/
# scp kube-controller-manager.kubeconfig 192.168.111.12:/etc/kubernetes/

创建和分发 kube-controller-manager systemd unit 文件：

cat > /lib/systemd/system/kube-controller-manager.service <<"EOF"

[Unit]

Description=Kubernetes Controller Manager

Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]

ExecStart=/usr/local/bin/kube-controller-manager \

  --port=0 \

  --secure-port=10252 \

  --kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \

  --service-cluster-ip-range=10.254.0.0/16 \

  --cluster-signing-cert-file=/etc/kubernetes/ca/ca.pem \

  --cluster-signing-key-file=/etc/kubernetes/ca/ca-key.pem \

  --root-ca-file=/etc/kubernetes/ca/ca.pem \

  --service-account-private-key-file=/etc/kubernetes/ca/ca-key.pem \

  --leader-elect=true \

  --feature-gates=RotateKubeletServerCertificate=true \

  --controllers=*,bootstrapsigner,tokencleaner \

  --horizontal-pod-autoscaler-use-rest-clients=true \

  --horizontal-pod-autoscaler-sync-period=10s \

  --tls-cert-file=/etc/kubernetes/ca/kube-controller-manager.pem \

  --tls-private-key-file=/etc/kubernetes/ca/kube-controller-manager-key.pem \

  --use-service-account-credentials=true \

  --alsologtostderr=true \

  --logtostderr=false \

  --log-dir=/var/log/kubernetes \

  --v=2

Restart=on

Restart=on-failure

RestartSec=5

User=k8s

[Install]

WantedBy=multi-user.target

EOF

-------------------------------------------------

启动kube-controller-manager

# systemctl daemon-reload&&for SERVICES in kube-controller-manager;do systemctl enable $SERVICES; systemctl start $SERVICES; systemctl status $SERVICES; done

查看输出的 metric
注意：以下命令在 kube-controller-manager 节点上执行。

kube-controller-manager 监听 10252 端口，接收 https 请求：

# ss -lnpt|grep kube-controll

# curl http://127.0.0.1:10252/metrics |head

测试 kube-controller-manager 集群的高可用
停掉一个或两个节点的 kube-controller-manager 服务，观察其它节点的日志，看是否获取了 leader 权限。
查看当前的 leader
kubectl get endpoints kube-controller-manager --namespace=kube-system -o yaml

巴特西

kubernetes部署kube-controller-manager服务

最新文章

热门文章