搭建spring boot+elasticsearch+activemq服务
目前时间是:2017-01-24
本文不涉及activemq的安装
需求
activemq实时传递数据至服务 elasticsearch做索引 对外开放查询接口 完成全文检索
环境
jdk:1.8
spirng boot:1.4.3.RELEASE
elasticsearch:2.4.3
activemq:5.13.2
ES插件
head:版本好像无太大差别 能查数据就行
analysis-ik:1.10.3
search-guard-2:2.4.3.9
search-guard-ssl:2.4.3.19
注意
作者遇到的最大问题就是版本兼容 网上资料少且版本较低 故列出以下版本矩阵 方便查阅
spring boot与elasticsearch:
Spring Boot Version (x) | Spring Data Elasticsearch Version (y) | Elasticsearch Version (z) |
x <= 1.3.5 | y <= 1.3.4 | z <= 1.7.2* |
x >= 1.4.x | 2.0.0 <=y < 5.0.0** | 2.0.0 <= z < 5.0.0** |
矩阵来源以及更多版本兼容:https://github.com/spring-projects/spring-data-elasticsearch
elasticsearch与ik:我的ES版本为2.x 对应如下
IK version | ES version |
---|---|
master | 2.4.0 -> master |
1.10.3 | 2.4.3 |
1.9.5 | 2.3.5 |
1.9.4 | 2.3.4 |
1.9.3 | 2.3.3 |
1.9.0 | 2.3.0 |
1.8.1 | 2.2.1 |
1.7.0 | 2.1.1 |
1.5.0 | 2.0.0 |
1.2.6 | 1.0.0 |
1.2.5 | 0.90.x |
1.1.3 | 0.20.x |
1.0.0 | 0.16.2 -> 0.19.0 |
矩阵来源以及更多版本兼容:https://github.com/medcl/elasticsearch-analysis-ik
elasticsearch与search-guard以及search-guard-ssl:
Elasticsearch Version | Latest Search Guard Version | Search Guard SSL Version | Commercial support available |
---|---|---|---|
1.x.y | not available | - | - |
2.0.x | not available | - | - |
2.1.x | not available | - | - |
2.2.0 | 2.2.0.7 | 2.2.0.16 | Yes |
2.3.1 | available upon request | - | - |
2.3.2 | available upon request | - | - |
2.3.3 | 2.3.3.10 | 2.3.3.19 | YES |
2.3.4 | 2.3.4.10 | 2.3.4.19 | YES |
2.3.5 | 2.3.5.10 | 2.3.5.19 | YES |
2.4.0 | 2.4.0.10 | 2.4.0.19 | YES |
2.4.1 | 2.4.1.10 | 2.4.1.19 | YES |
2.4.2 | 2.4.2.10 | 2.4.2.19 | YES |
2.4.3 | 2.4.3.10 | 2.4.3.19 | YES |
2.4.4 | 2.4.4.10 | 2.4.4.19 | YES |
5.0.0 | 5.0.0-10 | (comes bundled since SG 5) | YES |
5.0.1 | 5.0.1-10 | (comes bundled since SG 5) | YES |
5.0.2 | 5.0.2-10 | (comes bundled since SG 5) | YES |
5.1.1 | 5.1.1-10 | (comes bundled since SG 5) | YES |
5.1.2 | 5.1.2-10 | (comes bundled since SG 5) | YES |
矩阵来源以及更多版本兼容:https://github.com/floragunncom/search-guard/wiki
开始
安装elasticsearch
我的安装目录:/usr/local
注意:elasticsearch不能用root用户运行 所以创建你的用户组和用户 切换到新用户再安装 如何创建切换 请自行搜索
wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.4.3/elasticsearch-2.4.3.tar.gz
tar -zxvf elasticsearch-2.4.3.tar.gz
cd elasticsearch-2.4.3/config/
vim elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please see the documentation for further information on configuration options:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: 你的集群名称
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
# node.name: node-1
#
# Add custom attributes to the node:
#
# node.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
# path.data: /path/to/data
#
# Path to log files:
#
# path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
# bootstrap.memory_lock: true
#
# Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
# available on the system and that the owner of the process is allowed to use this limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
# http.port: 9200
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
#
# discovery.zen.minimum_master_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
# gateway.recover_after_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
#
# ---------------------------------- Various -----------------------------------
#
# Disable starting multiple nodes on a single system:
#
# node.max_local_storage_nodes: 1
#
# Require explicit names when deleting indices:
#
# action.destructive_requires_name: true
我修改了两个地方 cluster.name和network.host
cd ../bin/
./elasticsearch
[2017-01-24 10:02:49,627][INFO ][node ] [Ariel] version[2.4.3], pid[23274], build[d38a34e/2016-12-07T16:28:56Z]
[2017-01-24 10:02:49,628][INFO ][node ] [Ariel] initializing ...
[2017-01-24 10:02:50,259][INFO ][plugins ] [Ariel] modules [reindex, lang-expression, lang-groovy], plugins [], sites []
[2017-01-24 10:02:50,279][INFO ][env ] [Ariel] using [1] data paths, mounts [[/ (overlay)]], net usable_space [75.3gb], net total_space [113.9gb], spins? [possibly], types [overlay]
[2017-01-24 10:02:50,279][INFO ][env ] [Ariel] heap size [990.7mb], compressed ordinary object pointers [true]
[2017-01-24 10:02:52,051][INFO ][node ] [Ariel] initialized
[2017-01-24 10:02:52,051][INFO ][node ] [Ariel] starting ...
[2017-01-24 10:02:52,110][INFO ][transport ] [Ariel] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2017-01-24 10:02:52,116][INFO ][discovery ] [Ariel] elasticsearch/MI21JVBWSbKfj9nC1V6N9w
[2017-01-24 10:02:55,166][INFO ][cluster.service ] [Ariel] new_master {Ariel}{MI21JVBWSbKfj9nC1V6N9w}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2017-01-24 10:02:55,197][INFO ][http ] [Ariel] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2017-01-24 10:02:55,197][INFO ][node ] [Ariel] started
出现类似这样的信息 说明安装成功
安装head插件
elasticsearch根目录执行
bin/plugin install mobz/elasticsearch-head
注意:2.0以下版本应该是 -install
访问地址:http://ip:9200/_plugin/head/
安装analysis-ik插件
下载地址:https://github.com/medcl/elasticsearch-analysis-ik/tree/v1.10.3
使用maven打包:mvn clean package
生成的zip包在target/releases下
在elasticsearch-2.4.3/plugins下创建ik目录
将zip包放到该目录下并解压 解压出如下文件
编辑elasticsearch-2.4.3/config/elasticsearch.yml配置文件 添加如下内容
index:
analysis:
analyzer:
ik:
alias: [ik_analyzer]
type: org.elasticsearch.index.analysis.IkAnalyzerProvider
ik_max_word:
type: ik
use_smart: false
ik_smart:
type: ik
use_smart: true
或者
index.analysis.analyzer.ik.type : “ik”
测试:http://ip:9200/_analyze?analyzer=ik&pretty=true&text=我是中国人
安装searchguard
elasticsearch根目录执行
bin/plugin install -b com.floragunn/search-guard-2/2.4.3.9
bin/plugin install -b com.floragunn/search-guard-ssl/2.4.3.19
下载 searchguard-ssl 的包,里面包含自动创建证书的脚本:
wget https://github.com/floragunncom/search-guard-ssl/archive/v2.4.3.19.zip
unzip v2.4.3.19.zip
cd search-guard-ssl-2.4.3.19/example-pki-scripts/
有三个脚本
gen_client_node_cert.sh 创建客户端证书
gen_node_cert.sh 创建节点证书
gen_root_ca.sh 创建根证书
编辑脚本 vim example.sh
#!/bin/bash
set -e
./clean.sh
./gen_root_ca.sh password password
./gen_node_cert.sh 0 password password
./gen_node_cert.sh 1 password password
./gen_client_node_cert.sh admin password password
cp truststore.jks node-0-keystore.jks /usr/local/elasticsearch-2.4.3/config/
cp truststore.jks admin-keystore.jks /usr/local/elasticsearch-2.4.3/plugins/search-guard-2/sgconfig/
./example.sh
可以发现 最后两句就是将证书cp到相应目录
编辑elasticsearch-2.4.3/config/elasticsearch.yml配置文件 添加如下内容
#############################################################################################
# SEARCH GUARD #
# Configuration #
#############################################################################################
security.manager.enabled: false
searchguard.authcz.admin_dn:
- "CN=admin, OU=client, O=client, L=Test, C=DE" #############################################################################################
# SEARCH GUARD SSL #
# Configuration #
############################################################################################# #############################################################################################
# Transport layer SSL #
# #
#############################################################################################
# Enable or disable node-to-node ssl encryption (default: true)
# searchguard.ssl.transport.enabled: true
# JKS or PKCS12 (default: JKS)
#searchguard.ssl.transport.keystore_type: PKCS12
# Relative path to the keystore file (mandatory, this stores the server certificates), must be placed under the config/ dir
searchguard.ssl.transport.keystore_filepath: node-0-keystore.jks
# Alias name (default: first alias which could be found)
#searchguard.ssl.transport.keystore_alias: my_alias
# Keystore password (default: changeit)
searchguard.ssl.transport.keystore_password: password
# JKS or PKCS12 (default: JKS)
#searchguard.ssl.transport.truststore_type: PKCS12
# Relative path to the truststore file (mandatory, this stores the client/root certificates), must be placed under the config/ dir
searchguard.ssl.transport.truststore_filepath: truststore.jks
# Alias name (default: first alias which could be found)
#searchguard.ssl.transport.truststore_alias: my_alias
# Truststore password (default: changeit)
searchguard.ssl.transport.truststore_password: password
# Enforce hostname verification (default: true)
# searchguard.ssl.transport.enforce_hostname_verification: true
# If hostname verification specify if hostname should be resolved (default: true)
# searchguard.ssl.transport.resolve_hostname: true
# Use native Open SSL instead of JDK SSL if available (default: true)
# searchguard.ssl.transport.enable_openssl_if_available: false
在elasticsearch根目录 执行命令 将配置插入
./plugins/search-guard-2/tools/sgadmin.sh -cn 集群名称 -h hostname -cd plugins/search-guard-2/sgconfig -ks plugins/search-guard-2/sgconfig/admin-keystore.jks -kspass password -ts plugins/search-guard-2/sgconfig/truststore.jks -tspass password -nhnv
注意:elasticsearch的服务必须是运行状态
elasticsearch-2.4.3/plugins/search-guard-2/sgconfig下的配置文件是管理用户角色的
安装配置成功后 任何客户端访问elasticsearch 需提供用户名及密码
至此服务端安装结束
客户端将以源码方式提供 为公司信息安全着想 仅提供关键性代码供参考 无法运行
最新文章
- BZOJ 1031: [JSOI2007]字符加密Cipher 后缀数组
- selenium使用actions.moveToElement处理菜单
- 使用jQuery Mobile的注意事项(译)
- 魅蓝Note2 在Android Studio 与 Eclipse中无法被检测到
- Java学习-015-CSV 文件写入实例源代码
- lintcode:交换链表当中两个节点
- poi实现将数据输出到Excel表格当中
- 用 Graphviz画神经网络图
- 学习MongoDB 二:MongoDB加入、删除、改动
- mac 终端常用目录跳转命令
- 英语口语练习系列-C05-水电
- android开发学习 ------- 关于getSupportFragmentManager()不可用的问题
- 树莓派3 Raspberry系统安装samba
- aspx 页面中 js 引用与页面后台的数据交互 --【 js 调后台】
- hihocoder1696 折线中点(几何)
- LINQ之路13:LINQ Operators之连接(Joining)
- sqlserver乱码问题解决
- 【托业】托业(TOEIC)成绩 &; 等级划分以及评分标准
- <;亲测>;CentOS 7.3下Node.js 8.6安装配置(含NPM以及PM2)
- AC自动机算法学习