一 glusterfs存储集群部署

1.1 架构示意

1.2 相关规划

1.3 安装glusterfs

1.4 添加信任池

1.5 安装heketi

1.6 配置heketi

 {  " "",   " "  " " " " " },  " " " }  },   " " " " " " " " " " ],  "  " " " " "",  " },   " "  " " " " ],  " }  }

1.7 配置免秘钥

1.8 启动heketi

1.9 配置Heketi拓扑

 {  " {  " {  " " " " ],  " " ]  },  " },  " " ]  },  {  " " " " ],  " " ]  },  " },  " " ]  },  {  " " " " ],  " " ]  },  " },  " " ]  }  ]  }  ]  }

1.10 集群管理及测试

1.11 创建StorageClass

 apiVersion: v1  kind: Secret  metadata:   namespace: heketi   key: YWRtaW4xMjM=  type: kubernetes.io/glusterfs

 StorageClass  apiVersion: storage.k8s.io/v1  kind: StorageClass  metadata:   parameters:  resturl: " clusterid: " restauthenabled: " restuser: " secretName: " secretNamespace: " volumetype: " provisioner: kubernetes.io/glusterfs  reclaimPolicy: Delete

二 集群监控Metrics

2.1 开启聚合层

2.2 获取部署文件

 ……  image: mirrorgooglecontainers/metrics-server-amd64:v0.3.6  command:  - /metrics-server  - --metric-resolution=30s  - --kubelet-insecure-tls  - --kubelet-preferred-address- ……

2.3 正式部署

2.4 确认验证

三 Prometheus部署

3.1 获取部署文件

3.2 创建命名空间

 apiVersion: v1  kind: Namespace  metadata:   

3.3 创建RBAC

 apiVersion: rbac.authorization.k8s.io/v1beta1  kind: ClusterRole  metadata:   rules:  - apiGroups: [""]  resources:  - nodes  - nodes/ - services  - endpoints  - pods  verbs: [" - apiGroups:  - extensions  resources:  - ingresses  verbs: [" - nonResourceURLs: [" verbs: [" ---  apiVersion: v1  kind: ServiceAccount  metadata:   namespace: monitoring  ---  apiVersion: rbac.authorization.k8s.io/v1beta1  kind: ClusterRoleBinding  metadata:   roleRef:  apiGroup: rbac.authorization.k8s.io  kind: ClusterRole   subjects:  - kind: ServiceAccount   namespace: monitoring #仅需修改命名空间

3.4 创建Prometheus ConfigMap

 apiVersion: v1  kind: ConfigMap  metadata:   labels:   namespace: monitoring   prometheus.yml: |-  global:  scrape_interval: 10s  evaluation_interval: 10s   scrape_configs:  - job_name: 'kubernetes-apiservers'  kubernetes_sd_configs:  - role: endpoints  scheme: https  tls_config:  ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token  relabel_configs:  - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]  action: keep  regex: default;kubernetes;https   - job_name: 'kubernetes-nodes'  scheme: https  tls_config:  ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token  kubernetes_sd_configs:  - role: node  relabel_configs:  - action: labelmap  regex: __meta_kubernetes_node_label_(.+)  - target_label: __address__  replacement: kubernetes.default.svc:443  - source_labels: [__meta_kubernetes_node_name]  regex: (.+)  target_label: __metrics_path__  replacement: /api/v1/nodes/${1}/  - job_name: 'kubernetes-cadvisor'  scheme: https  tls_config:  ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token  kubernetes_sd_configs:  - role: node  relabel_configs:  - action: labelmap  regex: __meta_kubernetes_node_label_(.+)  - target_label: __address__  replacement: kubernetes.default.svc:443  - source_labels: [__meta_kubernetes_node_name]  regex: (.+)  target_label: __metrics_path__  replacement: /api/v1/nodes/${1}/  - job_name: 'kubernetes-service-endpoints'  kubernetes_sd_configs:  - role: endpoints  relabel_configs:  - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]  action: keep  regex: true  - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]  action:  target_label: __scheme__  regex: (https?)  - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]  action:  target_label: __metrics_path__  regex: (.+)  - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]  action:  target_label: __address__  regex: ([^:]+)(?::\d+)?;(\d+)  replacement: $1:$2  - action: labelmap  regex: __meta_kubernetes_service_label_(.+)  - source_labels: [__meta_kubernetes_namespace]  action:  target_label: kubernetes_namespace  - source_labels: [__meta_kubernetes_service_name]  action:  target_label: kubernetes_name   - job_name: 'kubernetes-services'  metrics_path: /probe  params:   kubernetes_sd_configs:  - role: service  relabel_configs:  - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]  action: keep  regex: true  - source_labels: [__address__]  target_label: __param_target  - target_label: __address__  replacement: blackbox-exporter.example.com:9115  - source_labels: [__param_target]  target_label:  - action: labelmap  regex: __meta_kubernetes_service_label_(.+)  - source_labels: [__meta_kubernetes_namespace]  target_label: kubernetes_namespace  - source_labels: [__meta_kubernetes_service_name]  target_label: kubernetes_name   - job_name: 'kubernetes-ingresses'  kubernetes_sd_configs:  - role: ingress  relabel_configs:  - source_labels: [__meta_kubernetes_ingress_annotation_prometheus_io_probe]  action: keep  regex: true  - source_labels: [__meta_kubernetes_ingress_scheme,__address__,__meta_kubernetes_ingress_path]  regex: (.+);(.+);(.+)  replacement: ${1}://${2}${3}  target_label: __param_target  - target_label: __address__  replacement: blackbox-exporter.example.com:9115  - source_labels: [__param_target]  target_label:  - action: labelmap  regex: __meta_kubernetes_ingress_label_(.+)  - source_labels: [__meta_kubernetes_namespace]  target_label: kubernetes_namespace  - source_labels: [__meta_kubernetes_ingress_name]  target_label: kubernetes_name   - job_name: 'kubernetes-pods'  kubernetes_sd_configs:  - role: pod  relabel_configs:  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]  action: keep  regex: true  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]  action:  target_label: __metrics_path__  regex: (.+)  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]  action:  regex: ([^:]+)(?::\d+)?;(\d+)  replacement: $1:$2  target_label: __address__  - action: labelmap  regex: __meta_kubernetes_pod_label_(.+)  - source_labels: [__meta_kubernetes_namespace]  action:  target_label: kubernetes_namespace  - source_labels: [__meta_kubernetes_pod_name]  action:  target_label: kubernetes_pod_name

3.5 创建持久PVC

 apiVersion: v1  kind: PersistentVolumeClaim  metadata:   namespace: monitoring  annotations:  volume.beta.kubernetes.io/storage- spec:  accessModes:  - ReadWriteMany  resources:  requests:  storage: 5Gi

3.6 Prometheus部署

 apiVersion: apps/v1beta2  kind: Deployment  metadata:  labels:    namespace: monitoring  spec:  replicas: 1  selector:  matchLabels:  app: prometheus-server  template:  metadata:  labels:  app: prometheus-server  spec:  containers:  -  image: prom/prometheus:v2.14.0  command:  - " args:  - " - " - " ports:  - containerPort: 9090  protocol: TCP  volumeMounts:  -  mountPath: /etc/prometheus/  -  mountPath: /prometheus/  serviceAccountName: prometheus  imagePullSecrets:  -  volumes:  -  configMap:  defaultMode: 420   -  persistentVolumeClaim:  claimName: prometheus-pvc

3.7 创建Prometheus Service

 apiVersion: v1  kind: Service  metadata:  labels:  app: prometheus-service   namespace: monitoring  spec:  type: NodePort  selector:  app: prometheus-server  ports:  - port: 9090  targetPort: 9090  nodePort: 30001

3.8 确认验证Prometheus

四 部署grafana

4.1 获取部署文件

4.2 创建持久PVC

 apiVersion: v1  kind: PersistentVolumeClaim  metadata:   namespace: monitoring  annotations:  volume.beta.kubernetes.io/storage- spec:  accessModes:  - ReadWriteOnce  resources:  requests:  storage: 5Gi

4.3 grafana部署

 apiVersion: extensions/v1beta1  kind: Deployment  metadata:   namespace: monitoring  spec:  replicas: 1  template:  metadata:  labels:  task: monitoring  k8s-app: grafana  spec:  containers:  -  image: grafana/grafana:6.5.0  imagePullPolicy: IfNotPresent  ports:  - containerPort: 3000  protocol: TCP  volumeMounts:  - mountPath: /var/lib/grafana   env:  -  value: monitoring-influxdb  -  value: ""  -  value: " -  value: " -  value: Admin  -  value: /  readinessProbe:  httpGet:   port: 3000  volumes:  -  persistentVolumeClaim:  claimName: grafana- nodeSelector:  node-role.kubernetes.io/master: " tolerations:  - key: " effect: " ---  apiVersion: v1  kind: Service  metadata:  labels:  kubernetes.io/cluster-service: 'true'  kubernetes.io/ annotations:  prometheus.io/scrape: 'true'  prometheus.io/tcp-probe: 'true'  prometheus.io/tcp-probe-port: '80'   namespace: monitoring  spec:  type: NodePort  ports:  - port: 80  targetPort: 3000  nodePort: 30002  selector:  k8s-app: grafana

4.4 确认验证Prometheus

4.4 grafana配置

4.5 查看监控

五 日志管理

5.1 获取部署文件

5.2 修改相关源

 ……  - image: quay-mirror.qiniu.com/fluentd_elasticsearch/elasticsearch:v7.3.2   imagePullPolicy: IfNotPresent  ……

 ……  image: quay-mirror.qiniu.com/fluentd_elasticsearch/fluentd:v2.7.0  imagePullPolicy: IfNotPresent  ……

 ……  image: docker.elastic.co/kibana/kibana-oss:7.3.2  imagePullPolicy: IfNotPresent  ……

5.3 创建持久PVC

 apiVersion: v1  kind: PersistentVolumeClaim  metadata:   namespace: kube-system  annotations:  volume.beta.kubernetes.io/storage- spec:  accessModes:  - ReadWriteMany  resources:  requests:  storage: 5Gi

 apiVersion: v1
 kind: ServiceAccount
 metadata:
   name: elasticsearch-logging
   namespace: kube-system
   labels:
     k8s-app: elasticsearch-logging
     addonmanager.kubernetes.io/mode: Reconcile
 ---
 kind: ClusterRole
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
   name: elasticsearch-logging
   labels:
     k8s-app: elasticsearch-logging
     addonmanager.kubernetes.io/mode: Reconcile
 rules:
 - apiGroups:
   - ""
   resources:
   - "services"
   - "namespaces"
   - "endpoints"
   verbs:
   - "get"
 ---
 kind: ClusterRoleBinding
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
   namespace: kube-system
   name: elasticsearch-logging
   labels:
     k8s-app: elasticsearch-logging
     addonmanager.kubernetes.io/mode: Reconcile
 subjects:
 - kind: ServiceAccount
   name: elasticsearch-logging
   namespace: kube-system
   apiGroup: ""
 roleRef:
   kind: ClusterRole
   name: elasticsearch-logging
   apiGroup: ""
 ---
 apiVersion: apps/v1
 kind: StatefulSet
 metadata:
   name: elasticsearch-logging
   namespace: kube-system
   labels:
     k8s-app: elasticsearch-logging
     version: v7.3.2
     addonmanager.kubernetes.io/mode: Reconcile
 spec:
   serviceName: elasticsearch-logging
   replicas: 1
   selector:
     matchLabels:
       k8s-app: elasticsearch-logging
       version: v7.3.2
   template:
     metadata:
       labels:
         k8s-app: elasticsearch-logging
         version: v7.3.2
     spec:
       serviceAccountName: elasticsearch-logging
       containers:
       - image: quay-mirror.qiniu.com/fluentd_elasticsearch/elasticsearch:v7.3.2
         name: elasticsearch-logging
         imagePullPolicy: IfNotPresent
         resources:
           limits:
             cpu: 1000m
             memory: 3Gi
           requests:
             cpu: 100m
             memory: 3Gi
         ports:
         - containerPort: 9200
           name: db
           protocol: TCP
         - containerPort: 9300
           name: transport
           protocol: TCP
         volumeMounts:
         - name: elasticsearch-logging
           mountPath: /data
         env:
         - name: "NAMESPACE"
           valueFrom:
             fieldRef:
               fieldPath: metadata.namespace
       volumes:
       - name: elasticsearch-logging			#挂载永久存储PVC
         persistentVolumeClaim:
           claimName: elasticsearch-pvc
       initContainers:
       - image: alpine:3.6
         command: ["/sbin/sysctl", "-w", "vm.max_map_count=262144"]
         name: elasticsearch-logging-init
         securityContext:
           privileged: true

5.5 部署Elasticsearch SVC

 apiVersion: v1
 kind: Service
 metadata:
   name: elasticsearch-logging
   namespace: kube-system
   labels:
     k8s-app: elasticsearch-logging
     kubernetes.io/cluster-service: "true"
     addonmanager.kubernetes.io/mode: Reconcile
     kubernetes.io/name: "Elasticsearch"
 spec:
   ports:
   - port: 9200
     protocol: TCP
     targetPort: db
   selector:
     k8s-app: elasticsearch-logging

5.6 部署fluentd

[root@k8smaster01 fluentd-elasticsearch]# kubectl create -f fluentd-es-ds.yaml                     #部署fluentd

5.7 部署Kibana

 apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: kibana-logging
   namespace: kube-system
   labels:
     k8s-app: kibana-logging
     addonmanager.kubernetes.io/mode: Reconcile
 spec:
   replicas: 1
   selector:
     matchLabels:
       k8s-app: kibana-logging
   template:
     metadata:
       labels:
         k8s-app: kibana-logging
       annotations:
         seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
     spec:
       containers:
       - name: kibana-logging
         image: docker.elastic.co/kibana/kibana-oss:7.3.2
         imagePullPolicy: IfNotPresent
         resources:
           limits:
             cpu: 1000m
           requests:
             cpu: 100m
         env:
           - name: ELASTICSEARCH_HOSTS
             value: http://elasticsearch-logging:9200
         ports:
         - containerPort: 5601
           name: ui
           protocol: TCP

5.8 部署Kibana SVC

 apiVersion: v1
 kind: Service
 metadata:
   name: kibana-logging
   namespace: kube-system
   labels:
     k8s-app: kibana-logging
     kubernetes.io/cluster-service: "true"
     addonmanager.kubernetes.io/mode: Reconcile
     kubernetes.io/name: "Kibana"
 spec:
   type: NodePort
   ports:
   - port: 5601
     protocol: TCP
     nodePort: 30003
     targetPort: ui
   selector:
     k8s-app: kibana-logging

5.9 确认验证

最新文章

1. SHA-1算法
2. 前端开发与SEO
3. ubuntu 14.04安装右键打开终端功能
4. gdb 调试程序
5. Gradle学习系列之八——构建多个Project
6. javascript对象转化为基本数据类型规则
7. 原创:分享asp.net伪静态成目录形式iis如何设置
8. lnmp 下安装yaf
9. 谈谈python 中__name__ = '__main__' 的作用
10. udelay、mdelay、ndelay、msleep使用比较说明
11. Android 源码编译环境搭建(64位Ubuntu)各种依赖包安装
12. Openjudge-计算概论（A）-1的个数
13. UOJ188 Sanrd Min_25筛
14. 【Eclipse】-NO.163.Eclipse.1 -【Eclipse springboot 1.x 创建maven工程初始化报错】
15. ES6 let const 声明变量 块级作用域
16. 对于maven的一些命令
17. Lombok快速上手（安装、使用与注解参数）
18. C#窗体读取EXCEL存入SQL数据库
19. [译]C语言实现一个简易的Hash table(6)
20. centos开机提示CPU无法识别的问题

热门文章

1. 浏览器证书问题，chorm,ie,edge,safari都会去读系统证书，firefox例外
2. mongodb Map/reduce测试代码
3. flutter实践 - plsy
4. iOS多线程开发之GCD（基础篇）
5. 安卓权威编程指南 挑战练习 25章 深度优化 PhotoGallery 应用
6. textarea 实现高度自动增长
7. c#百度IP定位API使用方法
8. 有了这个开源 Java 项目，开发出炫酷的小游戏好像不难？
9. meterpreter会话渗透利用常用的32个命令归纳小结
10. SQL Server 最小日志记录