ansible playbook部署ELK集群系统

一、介绍

总共4台机器，分别为

192.168.1.99

192.168.1.100

192.168.1.210

192.168.1.211

服务所在机器为：

redis：192.168.1.211

logstash: 192.168.210 192.168.1.211

elasticsearch: 分别布置在四台（211不为主、不存储数据）

kibana：192.168.1.211

nginx：192.168.1.211

二、elk集群架构图

三、为每一个软件或集群创建一个单独的角色。

cd /etc/ansible

mkdir -pv elk/roles/{jdk,redis,logstash,es,es_head,kibana,nginx}/{tasks,files,templates,meta,handlers,vars}

四、配置hosts

在/etc/ansible/hosts添加如下内容：

[jdk_servers]

192.168.1.99

192.168.1.100

192.168.1.210

192.168.1.211

[redis_servers]

192.168.1.211

[es_servers]

192.168.1.99  node_name=node-99  node_master=true node_data=true network_host=192.168.1.99

192.168.1.100 node_name=node-100 node_master=true node_data=true network_host=192.168.1.100

192.168.1.210 node_name=node-210 node_master=true node_data=true network_host=192.168.1.210

192.168.1.211 node_name=node-211 node_master=true node_data=true network_host=192.168.1.211

[es_plugin_head]

192.168.1.211

[logstash_servers]

192.168.1.210

192.168.1.211

[kibana_servers]

192.168.1.211

[nginx_servers]

192.168.1.211

PS：es_servers组里面，后面定义了变量，因为es配置文件中，会根据服务器的不同，对应的配置也不同，在这里定义，可以直接在tasks里面去引用

五、JDK roles

1.把jdk-8u91-linux-x64.tar.gz和Java环境变量脚本传到files目录下

2.编写vars

[root@master jdk]# cat vars/main.yml

jdk_package_name: jdk-8u91-linux-x64.tar.gz

env_dir: /etc/profile.d

3.编写tasks

[root@master jdk]# cat tasks/main.yml

- name: copy jdk software

  copy: src={{ jdk_package_name }} dest=/tmp/ owner=root group=root

- name: uncompression jdk software

  shell: tar -zxf /tmp/{{ jdk_package_name }} -C /usr/local/

- name: copy jdk env

  copy: src=java.sh dest={{ env_dir }}/ owner=root group=root mode=0644

- name: enforce env

  shell: source {{ env_dir }}/java.sh

- name: delete jdk compressin files

  shell: rm -f /tmp/{{ jdk_package_name }}

4.使用角色

在roles同级目录，创建一个jdk.yml文件，里面定义好你的playbook。

[root@master elk]# cat jdk.yml

- hosts: 127.0.0.1

  remote_user: root

  roles:

    - jdk

5. 运行playbook安装jdk

[root@master elk]# ansible-playbook jdk.yml

六、redis roles

1.把redis.tar.gz传到files目录下，redis配置文件和启动脚本传到templates目录下

2.编写vars

[root@master redis]# cat vars/main.yml

redis_package_file: redis.tar.gz

redis_dir: /usr/local/redis

exec_dir: /etc/init.d

3.编写tasks

[root@master redis]# cat tasks/main.yml

- name: copy redis software

  copy: src={{ redis_package_file }} dest=/tmp/ owner=root group=root

- name: uncompression redis software

  shell: tar zxf /tmp/{{ redis_package_file }} -C /usr/local

- name: copy redis start script

  template: src=redis dest={{ exec_dir }}/ owner=root group=root mode=0755

- name: copy redis config

  template: src=redis.conf dest={{ redis_dir }}/etc/ owner=redis group=redis

- name: create redis user

  user: name=redis shell=/sbin/nologin

- name: change owner and group

  file: path={{ redis_dir }} owner=redis group=redis #recurse=yes

- name: mkdir directory for redis data

  file: dest=/data/redis mode=0755 state=directory owner=redis group=redis

- name: start redis service

  shell: /etc/init.d/redis start

- name: add boot start redis start

  shell: chkconfig --level 345 redis on

- name: delete redis compression files

  shell: rm -f /tmp/{{ redis_package_file }}

4.使用角色

在roles同级目录，创建一个redis.yml文件，里面定义好你的playbook

[root@master elk]# cat redis.yml

- hosts: redis_servers

  remote_user: root

  roles:

    - redis

5. 运行playbook安装redis

[root@master elk]# ansible-playbook redis.yml

七、logstash roles

1.把logstash.tar.gz传到files目录下，logstash自定义配置文件和启动脚本传到templates目录下

2.编写vars

[root@master logstash]# cat vars/main.yml

logstash_package_file: logstash.tar.gz

logstash_dir: /usr/local/logstash

3.编写tasks

[root@master logstash]# cat tasks/main.yml

- name: copy logstash software

  copy: src={{ logstash_package_file }} dest=/tmp/ owner=root group=root

- name: uncompression logstash software

  shell: tar zxf /tmp/{{ logstash_package_file }} -C /usr/local

- name: mkdir config files dir

  file: dest={{ logstash_dir }}/conf.d mode=0755 state=directory owner=es group=es

- name: copy logstash start script

  template: src=start.sh dest={{ logstash_dir }}/ owner=es group=es mode=0755

- name: copy logstash config

  template: src=visit.conf dest={{ logstash_dir }}/conf.d/ owner=es group=es

- name: create es user

  user: name=es

- name: change owner and group

  file: path={{ logstash_dir }} owner=es group=es

- name: start logstash service

  shell: su - es -c "{{ logstash_dir }}/start.sh"

- name: delete logstash compression files

  shell: rm -f /tmp/{{ logstash_package_file }}

4.使用角色

在roles同级目录，创建一个logstash.yml文件，里面定义好你的playbook

[root@master elk]# cat logstash.yml

- hosts: logstash_servers

  remote_user: root

  roles:

    - logstash

5. 运行playbook安装logstash

[root@master elk]# ansible-playbook logstash.yml

八、es roles

1.把elasticsearch.tar.gz传到files目录下，es配置文件和启动脚本传到templates目录下

2.编写vars

[root@master es]# cat vars/main.yml

es_package_file: elasticsearch.tar.gz

es_dir: /usr/local/elasticsearch

config_name: elasticsearch.yml

3.编写tasks

[root@master es]# cat tasks/main.yml

- name: copy es software

  copy: src={{ es_package_file }} dest=/tmp/ owner=root group=root

- name: uncompression es software

  shell: tar zxf /tmp/{{ es_package_file }} -C /usr/local

- name: copy es start script

  template: src=start.sh dest={{ es_dir }}/ owner=es group=es mode=0755

- name: copy es config

  template: src=elasticsearch.yml dest={{ es_dir }}/config/ owner=es group=es

- name: update es config

  shell: sed -i "{{ item }}" {{ es_dir }}/config/{{ config_name }}

  with_items:

    - 's/#node.name/node.name: {{ node_name }}/'

    - 's/#node.master/node.master: {{ node_master }}/'

    - 's/#node.data/node.data: {{ node_data }}/'

    - 's/#network.host/network.host: {{ network_host }}/'

- name: create es user

  user: name=es

- name: change owner and group

  file: path={{ es_dir }} owner=es group=es #recurse=yes

- name: mkdir directory for es data

  file: dest=/data/elasticsearch mode=0755 state=directory owner=es group=es

  notify: mkdir Subdirectory

- name: start es service

  shell: su es -c 'cd {{ es_dir }} && ./start.sh'

- name: delete es compression files

  shell: rm -f /tmp/{{ es_package_file }}

4.编写handlers

[root@master es]# cat handlers/main.yml

- name: mkdir Subdirectory

  file: dest=/data/elasticsearch/{{ item }} mode=0755 state=directory owner=es group=es

  with_items:

    - logs

    - data

5.使用角色

在roles同级目录，创建一个es.yml文件，里面定义好你的playbook

[root@master elk]# cat es.yml

- hosts: es_servers

  remote_user: root

  roles:

    - es

5. 运行playbook安装es

[root@master elk]# ansible-playbook es.yml

九、es-head roles

1.把node-v4.4.7-linux-x64.tar.gz和elasticsearch-head.tar.gz传到files目录下，把_site/app.js传到templates目录下

2.编写vars

[root@master es_head]# cat vars/main.yml

node_package_file: node-v4.4.7-linux-x64.tar.gz

node_dir: /usr/local/node-v4.4.7-linux-x64

head_package_file: elasticsearch-head.tar.gz

head_dir: /usr/local/elasticsearch/elasticsearch-head

3.编写tasks

[root@master es_head]# cat tasks/main.yml

- name: copy node software

  copy: src={{ node_package_file }} dest=/tmp/ owner=root group=root

- name: uncompression node software

  shell: tar zxf /tmp/{{ node_package_file }} -C /usr/local/

- name: copy node to env

  shell: rsync -a {{ node_dir  }}/bin/* /usr/bin/

- name: copy head plugin software

  copy: src={{ head_package_file }} dest=/tmp/ owner=root group=root

- name: uncopmression head plugin software

  shell: tar zxf /tmp/{{ head_package_file }} -C /usr/local/elasticsearch/

- name: copy head config

  template: src=app.js dest={{ head_dir }}/_site/ owner=es group=es

- name: change owner and group

  file: path={{ head_dir }} owner=es group=es #recurse=yes

- name: start es service

  shell: su es -c 'cd {{ head_dir }} && npm run start > {{ head_dir }}/run.log &'

- name: delete node compression files

  shell: rm -f /tmp/{{ node_package_file }}

- name: delete head compression files

  shell: rm -f /tmp/{{ head_package_file }}

4.使用角色

在roles同级目录，创建一个es_head.yml文件，里面定义好你的playbook

[root@master elk]# cat es_head.yml

- hosts: es_plugin_head

  remote_user: root

  roles:

    - es_head

5. 运行playbook安装es-head

ansible-playbook es_head.yml

十、kibana roles

1.把kibana.tar.gz传到files目录下，kibana配置文件和启动脚本传到templates目录下

2.编写vars

[root@master kibana]# cat vars/main.yml

kibana_package_file: kibana.tar.gz

kibana_dir: /usr/local/kibana

3.编写tasks

[root@master kibana]# cat tasks/main.yml

- name: copy kibana software

  copy: src={{ kibana_package_file }} dest=/tmp/ owner=root group=root

- name: uncompression kibana software

  shell: tar zxf /tmp/{{ kibana_package_file }} -C /usr/local

- name: copy kibana start script

  template: src=start.sh dest={{ kibana_dir }}/ owner=es group=es mode=0755

- name: copy kibana config

  template: src=kibana.yml dest={{ kibana_dir }}/config/ owner=es group=es

- name: change owner and group

  file: path={{ kibana_dir }} owner=es group=es

- name: start kibana service

  shell: su - es -c "{{ kibana_dir }}/start.sh"

- name: delete kibana compression files

  shell: rm -f /tmp/{{ kibana_package_file }}

4.使用角色

在roles同级目录，创建一个kibana.yml文件，里面定义好你的playbook

[root@master elk]# cat kibana.yml

- hosts: kibana_servers

  remote_user: root

  roles:

    - kibana

5. 运行playbook安装kibana

ansible-playbook kibana.yml

十一、nginx roles（211本身就已经安装nginx,如果目标机器没安装，请参考我之前的文章）

1.把htpasswd生成密码的对应文件传到files目录下，nginx配置文件传到templates目录下

2.编写vars

[root@master nginx]# cat vars/main.yml

nginx_basedir: /usr/local/nginx

3.编写tasks

[root@master nginx]# cat tasks/main.yml

- name: copy conf file

  template: src={{ item.src }} dest={{ nginx_basedir }}/{{ item.dest }} backup=yes owner=root group=root mode=0644

  with_items:

    - { src: es.conf, dest: conf/vhosts/ }

    - { src: kibana.conf, dest: conf/vhosts/ }

- name: copy passwd file

  copy: src={{ item }} dest={{ nginx_basedir }}/conf/htpasswd/ owner=root group=root mode=0644

  with_items:

    - es

    - kibana

  notify: restart nginx

4.编写handlers

[root@master nginx]# cat handlers/main.yml

- name: restart nginx

  shell: /usr/local/nginx/sbin/nginx -s reload

5.使用角色

在roles同级目录，创建一个nginx.yml文件，里面定义好你的playbook

[root@master elk]# cat nginx.yml

---

- hosts: nginx_servers

  user: root

  roles:

    - nginx

5. 运行playbook加载nginx

ansible-playbook nginx.yml

巴特西

ansible playbook部署ELK集群系统

最新文章

热门文章