生产环境elasticsearch5.0报错IllegalArgumentException: number of documents in the index cannot exceed 2147483519的处理

最近几天的push:user:req数据写不到集群，报错如下：

[--13T09::,][DEBUG][o.e.a.b.TransportShardBulkAction] [yunva_etl_es8] [push:user:req][] failed to execute bulk item (index) index {[push:user:req][/push/user/req][AVyfE_JChh2DyKB0G1Kj], source[{"appId":"","yunvaId":"362b103623aa4f75911b374615c6a682","providerId":"","proviceId":"","mac":"B0:C4:E7:61:E6:E7","imsi":"","imei":"","factory":"samsung","model":"GT-S7568","osType":"android","osVersion":,"networkType":"wifi","ip":"14.152.80.239","sessionId":"","time":"2017-06-13 09:11:39.061.+0800"}]}

java.lang.IllegalArgumentException: number of documents in the index cannot exceed

at org.apache.lucene.index.DocumentsWriterPerThread.reserveOneDoc(DocumentsWriterPerThread.java:) ~[lucene-core-6.2..jar:6.2. 43ab70147eb494324a1410f7a9f16a896a59bc6f - shalin - -- ::]

at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:) ~[lucene-core-6.2..jar:6.2. 43ab70147eb494324a1410f7a9f16a896a59bc6f - shalin - -- ::]

at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:) ~[lucene-core-6.2..jar:6.2. 43ab70147eb494324a1410f7a9f16a896a59bc6f - shalin - -- ::]

at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:) ~[lucene-core-6.2..jar:6.2. 43ab70147eb494324a1410f7a9f16a896a59bc6f - shalin - -- ::]

at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:) ~[lucene-core-6.2..jar:6.2. 43ab70147eb494324a1410f7a9f16a896a59bc6f - shalin - -- ::]

at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:) ~[elasticsearch-5.0..jar:5.0.]

搜索了下：
是lucene的文档数限制，每个分片最大支持2的31次方个文档数量
https://discuss.elastic.co/t/lucene-max-documents-limit/34761
Lucene max documents limit，Yep, there is a 2^31 limit per shard in Lucene.

查询当前es分片文档：

# curl 10.26.241.237:/_cat/shards?v

index shard prirep state docs store ip node

push:user:req  p STARTED  .7gb 10.30.136.143 yunva_etl_es8

push:user:req  r STARTED  .7gb 10.25.135.215 yunva_etl_es2

push:user:req  p STARTED  .7gb 10.45.150.115 yunva_etl_es9

push:user:req  r STARTED  .7gb 10.30.136.143 yunva_etl_es8

push:user:req  r STARTED  .9gb 10.30.136.143 yunva_etl_es8

push:user:req  p STARTED  .9gb 10.174.12.230 yunva_etl_es10

push:user:req  p STARTED  884gb 10.174.12.230 yunva_etl_es10

push:user:req  r STARTED  884gb 10.27.78.228 yunva_etl_es5

push:user:req  r STARTED  883gb 10.45.150.115 yunva_etl_es9

push:user:req  p STARTED  883gb 10.174.12.230 yunva_etl_es10

查询出来的2147483519和2的31次方2147483648接近，所以新增加的数据无法插入

临时的解决办法：先把kafka的磁盘加到数据多保留一段时间20天

原来的集群数据抽取到hbase中作为冷数据处理，再把kafka里面的数据导入数据到新的集群中

巴特西

生产环境elasticsearch5.0报错IllegalArgumentException: number of documents in the index cannot exceed 2147483519的处理

最新文章

热门文章