You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "C. Scott Andreas (JIRA)" <ji...@apache.org> on 2018/11/19 01:40:00 UTC
[jira] [Resolved] (CASSANDRA-13757) Cassandra 3.5.0 JVM Segfault
Problem While Repair Job is Running
[ https://issues.apache.org/jira/browse/CASSANDRA-13757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
C. Scott Andreas resolved CASSANDRA-13757.
------------------------------------------
Resolution: Duplicate
> Cassandra 3.5.0 JVM Segfault Problem While Repair Job is Running
> ----------------------------------------------------------------
>
> Key: CASSANDRA-13757
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13757
> Project: Cassandra
> Issue Type: Bug
> Environment: Operation System: Debian Jessie
> Java: Oracle JDK 1.8.0_131
> Cassandra: 3.5.0
> Reporter: Serhat Rıfat Demircan
> Priority: Major
>
> We got following error while repair job running on our cluster. One of the nodes stop due to segmantation fault in JVM and repair job fails.
> We could not reproduce this problem on our test and staging enviroment (main difference is data size).
> {code:java}
> #
> # SIGSEGV (0xb) at pc=0x00007fd80a399e70, pid=1305, tid=0x00007fd7ee7c4700
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_131-b11) (build 1.8.0_131-b11)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.131-b11 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # C [liblz4-java3580121503903465201.so+0x5e70] LZ4_decompress_fast+0xd0
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> --------------- T H R E A D ---------------
> Current thread (0x00007fce32dad1b0): JavaThread "CompactionExecutor:9798" daemon [_thread_in_native, id=16879, stack(0x00007fd7ee784000,0x00007fd7ee7c5000)]
> siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x00007fd450c4d000
> Registers:
> RAX=0x00007fcde6560d32, RBX=0x00007fd450c4cff9, RCX=0x00007fcde6560c7a, RDX=0x00007fcde6560d3e
> RSP=0x00007fd7ee7c3160, RBP=0x00007fd450c44ae6, RSI=0x00007fcde6562ff8, RDI=0x00000000000000c2
> R8 =0x00007fcde6562ff4, R9 =0x00007fcde6563000, R10=0x0000000000000000, R11=0x0000000000000000
> R12=0x000000000000000c, R13=0x00007fd4501cd000, R14=0x00007fcde6562ff7, R15=0x00007fcde6562ffb
> RIP=0x00007fd80a399e70, EFLAGS=0x0000000000010283, CSGSFS=0x0000000000000033, ERR=0x0000000000000004
> TRAPNO=0x000000000000000e
> Top of Stack: (sp=0x00007fd7ee7c3160)
> 0x00007fd7ee7c3160: 0000000000000008 00007fd81e21c3d0
> 0x00007fd7ee7c3170: 0000000000000004 0000000000000001
> 0x00007fd7ee7c3180: 0000000000000002 0000000000000001
> 0x00007fd7ee7c3190: 0000000000000004 0000000000000004
> 0x00007fd7ee7c31a0: 0000000000000004 0000000000000004
> 0x00007fd7ee7c31b0: 0000000000000000 0000000000000000
> 0x00007fd7ee7c31c0: 0000000000000000 ffffffffffffffff
> 0x00007fd7ee7c31d0: 0000000000000000 0000000000000001
> 0x00007fd7ee7c31e0: 0000000000000002 0000000000000003
> 0x00007fd7ee7c31f0: 00007fd7ee7c32b8 00007fce32dad3a8
> 0x00007fd7ee7c3200: 0000000000000000 0000000000000000
> 0x00007fd7ee7c3210: 00007fd4501cd000 00007fcde6553000
> 0x00007fd7ee7c3220: 0000000000a77ae6 00007fd80a39659d
> 0x00007fd7ee7c3230: 0000000000000000 00000000dcb8fc9b
> 0x00007fd7ee7c3240: 00007fd7ee7c32d0 0000000000000000
> 0x00007fd7ee7c3250: 00000006e5c7e4d8 00007fd7ee7c32b8
> 0x00007fd7ee7c3260: 00007fce32dad1b0 00007fd81df2099d
> 0x00007fd7ee7c3270: 00007fd7ee7c32a8 0000000000000000
> 0x00007fd7ee7c3280: 0000000000010000 0000000000000000
> 0x00007fd7ee7c3290: 00000006e5c7e528 00007fd81d74df10
> 0x00007fd7ee7c32a0: 0000000000000000 00000006e5c7e4d8
> 0x00007fd7ee7c32b0: 00000006f6c7fbf8 00000006f6e957f0
> 0x00007fd7ee7c32c0: 00000006e5c7e350 00007fd87fffffff
> 0x00007fd7ee7c32d0: 00000006e5c7e528 00007fd81fa867e0
> 0x00007fd7ee7c32e0: 00a77ae200010000 0000000000a77ae2
> 0x00007fd7ee7c32f0: 00000006e5c7e488 000000000112d5f1
> 0x00007fd7ee7c3300: dcb8fc9b000099ce 0001000000a77ae6
> 0x00007fd7ee7c3310: 00a814b000a814b4 00000006e5c7e4d8
> 0x00007fd7ee7c3320: 00000006e5c7e4d8 00000006f6a4df38
> 0x00007fd7ee7c3330: 0000000600010000 000000067fffffff
> 0x00007fd7ee7c3340: 0089715800002c8a 0006189d87852057
> 0x00007fd7ee7c3350: 0000000000000000 00000000e5244e71
> Instructions: (pc=0x00007fd80a399e70)
> 0x00007fd80a399e50: e4 0f 49 83 fc 0f 0f 84 94 00 00 00 4a 8d 14 20
> 0x00007fd80a399e60: 48 39 f2 0f 87 c0 00 00 00 0f 1f 80 00 00 00 00
> 0x00007fd80a399e70: 48 8b 0b 48 83 c3 08 48 89 08 48 83 c0 08 48 39
> 0x00007fd80a399e80: c2 77 ed 48 29 d0 48 89 d1 48 29 c3 0f b7 03 48
> Register to memory mapping:
> RAX=0x00007fcde6560d32 is an unknown value
> RBX=0x00007fd450c4cff9 is an unknown value
> RCX=0x00007fcde6560c7a is an unknown value
> RDX=0x00007fcde6560d3e is an unknown value
> RSP=0x00007fd7ee7c3160 is pointing into the stack for thread: 0x00007fce32dad1b0
> RBP=0x00007fd450c44ae6 is an unknown value
> RSI=0x00007fcde6562ff8 is an unknown value
> RDI=0x00000000000000c2 is an unknown value
> R8 =0x00007fcde6562ff4 is an unknown value
> R9 =0x00007fcde6563000 is an unknown value
> R10=0x0000000000000000 is an unknown value
> R11=0x0000000000000000 is an unknown value
> R12=0x000000000000000c is an unknown value
> R13=0x00007fd4501cd000 is an unknown value
> R14=0x00007fcde6562ff7 is an unknown value
> R15=0x00007fcde6562ffb is an unknown value
> Stack: [0x00007fd7ee784000,0x00007fd7ee7c5000], sp=0x00007fd7ee7c3160, free space=252k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> C [liblz4-java3580121503903465201.so+0x5e70] LZ4_decompress_fast+0xd0
> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
> J 5084 net.jpountz.lz4.LZ4JNI.LZ4_decompress_fast([BLjava/nio/ByteBuffer;I[BLjava/nio/ByteBuffer;II)I (0 bytes) @ 0x00007fd81df20923 [0x00007fd81df20880+0xa3]
> J 20311 C2 org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBufferMmap()V (354 bytes) @ 0x00007fd81fa867e0 [0x00007fd81fa86160+0x680]
> J 10706 C2 org.apache.cassandra.io.util.RandomAccessReader.reBuffer()V (78 bytes) @ 0x00007fd81ea59c58 [0x00007fd81ea59b20+0x138]
> J 19996 C2 org.apache.cassandra.io.util.SegmentedFile.createReader(J)Lorg/apache/cassandra/io/util/FileDataInput; (12 bytes) @ 0x00007fd81fa32934 [0x00007fd81fa32540+0x3f4]
> J 20645 C2 org.apache.cassandra.db.columniterator.AbstractSSTableIterator.<init>(Lorg/apache/cassandra/io/sstable/format/SSTableReader;Lorg/apache/cassandra/io/util/FileDataInput;Lorg/apache/cassandra/db/DecoratedKey;Lorg/apache/cassandra/db/RowIndexEntry;Lorg/apache/cassandra/db/Slices;Lorg/apache/cassandra/db/filter/ColumnFilter;Z)V (423 bytes) @ 0x00007fd81fbf8cec [0x00007fd81fbf8a20+0x2cc]
> J 20644 C2 org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext()Lorg/apache/cassandra/db/rows/Unfiltered; (120 bytes) @ 0x00007fd81e61b5d0 [0x00007fd81e61b140+0x490]
> J 9597 C2 org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext()Ljava/lang/Object; (5 bytes) @ 0x00007fd81da8dad4 [0x00007fd81da8daa0+0x34]
> J 3432 C2 org.apache.cassandra.utils.AbstractIterator.hasNext()Z (80 bytes) @ 0x00007fd81da28f20 [0x00007fd81da28e80+0xa0]
> J 19755 C2 org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext()Ljava/lang/Object; (9 bytes) @ 0x00007fd81f81d00c [0x00007fd81f81cb60+0x4ac]
> J 21810 C2 org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext()Lorg/apache/cassandra/db/rows/Unfiltered; (42 bytes) @ 0x00007fd81ef929fc [0x00007fd81ef927a0+0x25c]
> J 15842 C2 org.apache.cassandra.db.SinglePartitionReadCommand.withSSTablesIterated(Ljava/util/List;Lorg/apache/cassandra/metrics/TableMetrics;)Lorg/apache/cassandra/db/rows/UnfilteredRowIterator; (71 bytes) @ 0x00007fd81ece33b4 [0x00007fd81ece3200+0x1b4]
> J 21162 C2 org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(Lorg/apache/cassandra/db/ColumnFamilyStore;)Lorg/apache/cassandra/db/rows/UnfilteredRowIterator; (635 bytes) @ 0x00007fd81fdb2ff0 [0x00007fd81fdb2100+0xef0]
> J 27194 C2 org.apache.cassandra.db.Keyspace.indexPartition(Lorg/apache/cassandra/db/DecoratedKey;Lorg/apache/cassandra/db/ColumnFamilyStore;Ljava/util/Set;)V (274 bytes) @ 0x00007fd820c8b044 [0x00007fd820c8aae0+0x564]
> J 27422 C2 org.apache.cassandra.index.internal.CollatedViewIndexBuilder.build()V (73 bytes) @ 0x00007fd81f0b5a00 [0x00007fd81f0b5940+0xc0]
> J 27619 C1 org.apache.cassandra.db.compaction.CompactionManager$12.run()V (56 bytes) @ 0x00007fd82051313c [0x00007fd820512a00+0x73c]
> J 24483 C2 java.util.concurrent.FutureTask.run()V (126 bytes) @ 0x00007fd81d50faf4 [0x00007fd81d50fa20+0xd4]
> J 23783 C2 java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (225 bytes) @ 0x00007fd81fca71a8 [0x00007fd81fca70a0+0x108]
> J 21390 C2 java.lang.Thread.run()V (17 bytes) @ 0x00007fd81e6d3cb8 [0x00007fd81e6d3c60+0x58]
> v ~StubRoutines::call_stub
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org