解决异常断电导致的: CorruptSSTableException: java.io.EOFException
2024-09-02 09:07:56
问题产生
服务器重启,导致cassandra损坏,整个集群不可用。所使用的cassandra为2.1.9版本。
问题描述
运行启动命令,报错如下:
DEBUG :: All segments have been unmapped successfully
INFO :: Opening ./../data/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/system-size_estimates-ka- ( bytes)
ERROR :: Exiting forcefully due to file system exception on startup, disk failure policy "stop"
org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.EOFException
at org.apache.cassandra.io.compress.CompressionMetadata.<init>(CompressionMetadata.java:) ~[apache-cassandra-2.1..jar:2.1.]
at org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:) ~[apache-cassandra-2.1..jar:2.1.]
at org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:) ~[apache-cassandra-2.1..jar:2.1.]
at org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:) ~[apache-cassandra-2.1..jar:2.1.]
at org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:) ~[apache-cassandra-2.1..jar:2.1.]
at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:) ~[apache-cassandra-2.1..jar:2.1.]
at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:) ~[apache-cassandra-2.1..jar:2.1.]
at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:) ~[apache-cassandra-2.1..jar:2.1.]
at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:) ~[apache-cassandra-2.1..jar:2.1.]
at org.apache.cassandra.io.sstable.SSTableReader$.run(SSTableReader.java:) ~[apache-cassandra-2.1..jar:2.1.]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:) [na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:) [na:1.8.0_45]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:) [na:1.8.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:) [na:1.8.0_45]
at java.lang.Thread.run(Thread.java:) [na:1.8.0_45]
Caused by: java.io.EOFException: null
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:) ~[na:1.8.0_45]
at java.io.DataInputStream.readUTF(DataInputStream.java:) ~[na:1.8.0_45]
at java.io.DataInputStream.readUTF(DataInputStream.java:) ~[na:1.8.0_45]
at org.apache.cassandra.io.compress.CompressionMetadata.<init>(CompressionMetadata.java:) ~[apache-cassandra-2.1..jar:2.1.]
... common frames omitted
DEBUG :: INDEX LOAD TIME for ./../data/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/system-size_estimates-ka-: ms.
DEBUG :: Load metadata for ./../data/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/system-size_estimates-ka-
INFO :: Opening ./../data/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/system-size_estimates-ka- ( bytes)
DEBUG :: INDEX LOAD TIME for ./../data/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/system-size_estimates-ka-: ms.
解决方案
1. 在正常节点上执行(节点需要启动)
./nodetool ring | grep 192.168.66.149 | awk '{print $NF ","}' | xargs
返回如下:
-, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -, -,
2. 在损坏节点上修改配置文件:cassandra.yaml,并加入如下配置项:
auto_bootstrap: false initial_token: -, -, -, -, -, -, -, -, -, -,......(后面的省略...)
注意:initial_token的取值为上一步骤的返回值。
3. 删除数据盘下system目录
如/usr/local/cassandra2/apache-cassandra-2.1.9/data/data/system
4. 启动cassandra
./cassandra
./cassandra
启动过程可能会报错,但会继续重建system库,只要能启动成功加入集群就算正常。
5. 修复数据
运行nodetool工具:
nodetool repair
6. 将配置项改回原样并重启
本解决方案参考自:/usr/local/cassandra2/apache-cassandra-2.1.9/data/data/system
最新文章
- 父进程等待子进程结束 waitpid wait
- centos 7 python2.7.5升级到3.5.2
- gitgub利用客户端实现简单的上传和同步
- 翻译: TypeScript 1.8 Beta 发布
- DevExpress Crack
- Struts2的国际化
- 用css控制cellspacing、cellpadding
- Intellj idea 安装JUnit
- java代码中 路径符号的写法
- Codeforces 845 简要题解
- centos6.5新增加硬盘挂载并实现开机自动挂载
- 简述HttpSession的作用、使用方法,可用代码说明
- vijos p1027休息中的小呆
- sqlTransaction 简单的应用
- C# worksheet设置Excel样式
- python_94_类变量实例变量
- windowsServer2008搭建域服务器
- ORA-16009 remote archive log destination must be a STANDBY database
- [LeetCode] 13. Roman to Integer 罗马数字转化成整数
- 微信小程序开发--flex详细解读