You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brandon Williams (JIRA)" <ji...@apache.org> on 2011/09/12 17:24:08 UTC
[jira] [Created] (CASSANDRA-3179) JVM segfaults
JVM segfaults
-------------
Key: CASSANDRA-3179
URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.0
Environment: java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
Reporter: Brandon Williams
Assignee: Sylvain Lebresne
Fix For: 1.0.0
Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: (was: 3179-performance-test.txt)
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3179:
----------------------------------------
Attachment: hs_err_pid7031.log
hs_err_pid28971.log
hs_err_pid12074.log
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: 3179-performance-test.txt
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Yang Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104035#comment-13104035 ]
Yang Yang commented on CASSANDRA-3179:
--------------------------------------
thanks.
unfortunately I got the SEGV again, after updating to the latest version. I tried to confirm so ran it again, so far seen 2 times.
INFO 14:49:57,806 Compacting Minor: [SSTableReader(path='/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h
-76-Data.db'), SSTableReader(path='/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-77-Data.db'), SSTableR
eader(path='/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-79-Data.db'), SSTableReader(path='/usr/scratc
h/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-78-Data.db')]
9135.963: [GC 9135.963: [ParNew: 2550601K->133528K(2764800K), 1.2821340 secs] 5163440K->2756799K(5836800K) icms_dc=0 , 1.2824040
secs] [Times: user=1.06 sys=0.04, real=1.28 secs]
INFO 14:50:07,017 Completed flushing /usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/measuredSession-h-88-Data.db (
21435483 bytes)
INFO 14:50:07,018 Writing Memtable-session_limit_filter@839347940(21205810/281849841 serialized/live bytes, 286565 ops)
INFO 14:50:15,987 Completed flushing /usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/session_limit_filter-h-88-Data
.db (23326502 bytes)
INFO 14:50:15,999 Writing Memtable-ad_impression_session@100750437(21205662/296528055 serialized/live bytes, 286563 ops)
9154.456: [GC 9154.456: [ParNew: 2591128K->125067K(2764800K), 0.2846910 secs] 5214399K->2756607K(5836800K) icms_dc=0 , 0.2849470 secs] [Times: user=0.71 sys=0.01, real=0.29 secs]
INFO 14:50:26,220 Completed flushing /usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/ad_impression_session-h-89-Data.db (52154466 bytes)
INFO 14:50:26,221 Writing Memtable-ad_ip_agent@1095480823(21205662/286042295 serialized/live bytes, 286563 ops)
INFO 14:50:26,463 Compacted to [/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-80-Data.db,]. 94,782,508 to 83,577,108 (~88% of original) bytes for 6,368 keys at 2.783688MBPS. Time: 28,633ms.
INFO 14:50:26,464 CF Total Bytes Compacted: 3,494,202,400
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002aaaab7b9c88, pid=6461, tid=1253022016
#
# JRE version: 7.0-b147
# Java VM: Java HotSpot(TM) 64-Bit Server VM (21.0-b17 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# J org.apache.cassandra.io.util.AbstractDataInput.readByte()B
#
# Core dump written. Default location: /usr/scratch/yyang/whisky/core or core.6461
attaching the err file separately
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103985#comment-13103985 ]
Jonathan Ellis commented on CASSANDRA-3179:
-------------------------------------------
bq. SSTableReader.releaseReference() could see a refcount of 2, and not call the dfile.cleanup()
That means a read thread has a reference still, so cleanup will run when THAT thread releases it.
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102746#comment-13102746 ]
Brandon Williams commented on CASSANDRA-3179:
---------------------------------------------
I am unable to repro without compaction running.
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: 3179-performance-test.txt
Also a patch that removes refcounting (and omits the copy above) to check what the performance hit is over pre-CASSANDRA-2521 sstable deletions.
Note that this is NOT intended as a "real" patch; it does not actually add back in GC-based sstable deletion. No sstables will be removed w/o a server restart, but that should be adequate for testing read performance.
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Yang Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103419#comment-13103419 ]
Yang Yang commented on CASSANDRA-3179:
--------------------------------------
btw, as I was trying to going through the code to understand the problem, I found it seems that the MMapedSegmentedFile.cleanup() code could be skipped in some sequences of mixed read and compact:
read increases the refcount, then DataTracker.replaceCompactedSSTables()---->....---> SSTableReader.releaseReference() could see a refcount of 2, and not call the dfile.cleanup()
it's not a big problem ( code seems to run fine with the cleanup() lines commented out), but that kind of thwarts the purpose of adding them in
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-3179.
---------------------------------------
Resolution: Fixed
Reviewer: brandon.williams
Assignee: Jonathan Ellis (was: Sylvain Lebresne)
committed v2
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104055#comment-13104055 ]
Brandon Williams commented on CASSANDRA-3179:
---------------------------------------------
Can you reproduce w/o compressed oops?
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103960#comment-13103960 ]
Brandon Williams commented on CASSANDRA-3179:
---------------------------------------------
I'm unable to discern any read performance difference between the performance test and v2 patches.
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Yang Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104141#comment-13104141 ]
Yang Yang commented on CASSANDRA-3179:
--------------------------------------
very interesting, I switched from mmap mode to "standard", and go the following errors on compaction,
I believe they follow the same path as the mmap route, but since it gave an exception here, instead of siliently SEGV, this could provide a useful hint to what caused the SEGV
Caused by: java.nio.channels.ClosedChannelException
at org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:268)
at java.io.RandomAccessFile.readByte(RandomAccessFile.java:640)
at org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:356)
at org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:367)
at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:87)
at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:82)
at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:72)
at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:36)
at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
... 21 more
ERROR 01:19:22,415 Fatal exception in thread Thread[ReadStage:246,5,main]
java.lang.RuntimeException: java.lang.RuntimeException: error reading 1 of 1
at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1165)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.RuntimeException: error reading 1 of 1
at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:83)
at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:40)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:107)
at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:194)
at org.apache.cassandra.utils.MergeIterator.<init>(MergeIterator.java:47)
at org.apache.cassandra.utils.MergeIterator$ManyToOne.<init>(MergeIterator.java:142)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:66)
at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:96)
at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:249)
at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:61)
at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1276)
at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1171)
at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1127)
at org.apache.cassandra.db.Table.getRow(Table.java:388)
at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:61)
at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:694)
at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1161)
... 3 more
Caused by: java.nio.channels.ClosedChannelException
at org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:268)
at java.io.RandomAccessFile.readByte(RandomAccessFile.java:640)
at org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:356)
at org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:367)
at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:87)
at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:82)
at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:72)
at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:36)
at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
... 21 more
INFO 01:19:23,738 Compacted to [/mnt/cass/lib/cassandra/data/testBudget_items/measuredSession-h-10-Data.db,]. 16,840,264 to 16,840,036 (~99% of original) bytes for 22,531 keys at 3.247707MBPS. Time: 4,945ms.
INFO 01:19:23,738 CF Total Bytes Compacted: 157,135,410
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Yang Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yang Yang updated CASSANDRA-3179:
---------------------------------
Attachment: hs_err_pid6461.log
failure log after using updated code
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: 3179-v2.txt
v2 adds missing flip()
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102777#comment-13102777 ]
Jonathan Ellis commented on CASSANDRA-3179:
-------------------------------------------
The problem is that our whole "zero copy" mmap'd read path is broken by "unmap sstables as soon as they are no longer referenced." Specifically, we release references during reads in CollationController or getRangeSlice as soon as we have generated CF objects from the sstables, but we will continue to reference the buffer contents later on when we send the results back to the user or the coordinator (or, as in the 7031 log, just use the results internally).
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103174#comment-13103174 ]
Brandon Williams commented on CASSANDRA-3179:
---------------------------------------------
+1
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Yang Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104637#comment-13104637 ]
Yang Yang commented on CASSANDRA-3179:
--------------------------------------
with 1.0.0 HEAD, no more SEGV after a night of stress tests.
thanks
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104140#comment-13104140 ]
Jonathan Ellis commented on CASSANDRA-3179:
-------------------------------------------
You're sure you're on 1.0.0 >= r1170342?
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Yang Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104143#comment-13104143 ]
Yang Yang commented on CASSANDRA-3179:
--------------------------------------
sorry for the misunderstanding. I applied the v2 patch on
4dbda612f49c97fd5e3f66e7875a20ec9a0dc829 (Aug 26 version)
I could test the latest one if you think that's going to be different.
On Tue, Sep 13, 2011 at 6:22 PM, Jonathan Ellis (JIRA) <ji...@apache.org> wrote:
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: (was: 3179-v2.txt)
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: 3179-performance-test.txt
rebased performance-test
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Yang Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104106#comment-13104106 ]
Yang Yang commented on CASSANDRA-3179:
--------------------------------------
confirmed with the -XX:-UseCompressedOops arg,
has generated 1 SEGV so far
btw, it would be really helpful to add a simple dumb stress test testcase,
some bugs are not easily exposed with the unit tests.
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: 3179.txt
patch to copy buffers read from an mmap'd source.
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104148#comment-13104148 ]
Jonathan Ellis commented on CASSANDRA-3179:
-------------------------------------------
Please test 1.0.0 branch to make sure we are doing the same thing.
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: 3179.txt
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Yang Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104061#comment-13104061 ]
Yang Yang commented on CASSANDRA-3179:
--------------------------------------
let me try ...
On Tue, Sep 13, 2011 at 3:47 PM, Brandon Williams (JIRA)
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: 3179-v2.txt
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Yang Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104119#comment-13104119 ]
Yang Yang commented on CASSANDRA-3179:
--------------------------------------
I'm just writing + reading on the cassandra server, and doing this on the side:
while : ;do nodetool flush ; sleep 20;done
(disclaimer: the cassandra server is a modified one, where I have a
thread that takes in client request through a custom AVRO server, and
then call the storageProxy.batch_mutate() and StorageProxy.get()
directly. it should not be materially different from the pure
cassandra server as far as this bug is concerned, since it's basically
just swapping out the thrift server
)
On Tue, Sep 13, 2011 at 5:42 PM, Brandon Williams (JIRA)
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Yang Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104333#comment-13104333 ]
Yang Yang commented on CASSANDRA-3179:
--------------------------------------
I know now, it must be due to
https://issues.apache.org/jira/browse/CASSANDRA-3110
let me verify with the latest code
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: (was: 3179.txt)
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104110#comment-13104110 ]
Brandon Williams commented on CASSANDRA-3179:
---------------------------------------------
What process are you following to reproduce? I can't do it with compaction+reads, but I'm also not using JRE 7.
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3179) JVM segfaults
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3179:
--------------------------------------
Attachment: (was: 3179-performance-test.txt)
> JVM segfaults
> -------------
>
> Key: CASSANDRA-3179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3179
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
> Reporter: Brandon Williams
> Assignee: Sylvain Lebresne
> Fix For: 1.0.0
>
> Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log
>
>
> Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira