You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Chris Goffinet (JIRA)" <ji...@apache.org> on 2009/12/06 00:31:22 UTC
[jira] Created: (CASSANDRA-605) Corruption in CommitLog
Corruption in CommitLog
-----------------------
Key: CASSANDRA-605
URL: https://issues.apache.org/jira/browse/CASSANDRA-605
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.5
Reporter: Chris Goffinet
Fix For: 0.5
We are seeing corruption in commit log files when cassandra gets shutdown sometimes. I can attach the commitlogs privately. Here is a stacktrace:
INFO [main] 2009-12-05 14:59:25,946 RecoveryManager.java (line 64) Replaying /mnt/var/cassandra/commitlog/CommitLog-1259972135241.log, /mnt/var/cassandra/commitlog/CommitLog-1260050162791.log, /mnt/var/cassandra/commitlog/CommitLog-1260047770958.log, /mnt/var/cassandra/commitlog/CommitLog-1260052237605.log
ERROR [COMPACTION-POOL:1] 2009-12-05 14:59:37,436 DebuggableThreadPoolExecutor.java (line 120) Error in executor futuretask
java.util.concurrent.ExecutionException: java.lang.NumberFormatException: For input string: "nan"
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:112)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NumberFormatException: For input string: "nan"
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
at java.lang.Float.valueOf(Float.java:388)
at java.lang.Float.<init>(Float.java:489)
at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:72)
at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:19)
at java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606)
at java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:797)
at java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1640)
at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:530)
at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:515)
at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:111)
at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:38)
at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130)
at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:919)
at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:861)
at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:663)
at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:180)
at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:177)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
... 2 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-605) Corruption in CommitLog
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-605:
-------------------------------------
Component/s: Core
Affects Version/s: (was: 0.5)
> Corruption in CommitLog
> -----------------------
>
> Key: CASSANDRA-605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-605
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Chris Goffinet
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 605.patch
>
>
> We are seeing corruption in commit log files when cassandra gets shutdown sometimes. I can attach the commitlogs privately. Here is a stacktrace:
> INFO [main] 2009-12-05 14:59:25,946 RecoveryManager.java (line 64) Replaying /mnt/var/cassandra/commitlog/CommitLog-1259972135241.log, /mnt/var/cassandra/commitlog/CommitLog-1260050162791.log, /mnt/var/cassandra/commitlog/CommitLog-1260047770958.log, /mnt/var/cassandra/commitlog/CommitLog-1260052237605.log
> ERROR [COMPACTION-POOL:1] 2009-12-05 14:59:37,436 DebuggableThreadPoolExecutor.java (line 120) Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.NumberFormatException: For input string: "nan"
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:112)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.NumberFormatException: For input string: "nan"
> at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
> at java.lang.Float.valueOf(Float.java:388)
> at java.lang.Float.<init>(Float.java:489)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:72)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:19)
> at java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606)
> at java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:797)
> at java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1640)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:530)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:515)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:111)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:38)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130)
> at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
> at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:919)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:861)
> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:663)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:180)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:177)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> ... 2 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-605) Corruption in CommitLog
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-605.
--------------------------------------
Resolution: Fixed
committed to 0.5 and trunk
> Corruption in CommitLog
> -----------------------
>
> Key: CASSANDRA-605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-605
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Chris Goffinet
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 605.patch
>
>
> We are seeing corruption in commit log files when cassandra gets shutdown sometimes. I can attach the commitlogs privately. Here is a stacktrace:
> INFO [main] 2009-12-05 14:59:25,946 RecoveryManager.java (line 64) Replaying /mnt/var/cassandra/commitlog/CommitLog-1259972135241.log, /mnt/var/cassandra/commitlog/CommitLog-1260050162791.log, /mnt/var/cassandra/commitlog/CommitLog-1260047770958.log, /mnt/var/cassandra/commitlog/CommitLog-1260052237605.log
> ERROR [COMPACTION-POOL:1] 2009-12-05 14:59:37,436 DebuggableThreadPoolExecutor.java (line 120) Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.NumberFormatException: For input string: "nan"
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:112)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.NumberFormatException: For input string: "nan"
> at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
> at java.lang.Float.valueOf(Float.java:388)
> at java.lang.Float.<init>(Float.java:489)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:72)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:19)
> at java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606)
> at java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:797)
> at java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1640)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:530)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:515)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:111)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:38)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130)
> at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
> at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:919)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:861)
> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:663)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:180)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:177)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> ... 2 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-605) Corruption in CommitLog
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786996#action_12786996 ]
Jonathan Ellis commented on CASSANDRA-605:
------------------------------------------
Chris indicated in IRC that this does look like it only occurs near the end of the commitlog, where fsync hasn't yet guaranteed that data is safe.
> Corruption in CommitLog
> -----------------------
>
> Key: CASSANDRA-605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-605
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Reporter: Chris Goffinet
> Fix For: 0.5
>
>
> We are seeing corruption in commit log files when cassandra gets shutdown sometimes. I can attach the commitlogs privately. Here is a stacktrace:
> INFO [main] 2009-12-05 14:59:25,946 RecoveryManager.java (line 64) Replaying /mnt/var/cassandra/commitlog/CommitLog-1259972135241.log, /mnt/var/cassandra/commitlog/CommitLog-1260050162791.log, /mnt/var/cassandra/commitlog/CommitLog-1260047770958.log, /mnt/var/cassandra/commitlog/CommitLog-1260052237605.log
> ERROR [COMPACTION-POOL:1] 2009-12-05 14:59:37,436 DebuggableThreadPoolExecutor.java (line 120) Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.NumberFormatException: For input string: "nan"
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:112)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.NumberFormatException: For input string: "nan"
> at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
> at java.lang.Float.valueOf(Float.java:388)
> at java.lang.Float.<init>(Float.java:489)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:72)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:19)
> at java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606)
> at java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:797)
> at java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1640)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:530)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:515)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:111)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:38)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130)
> at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
> at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:919)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:861)
> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:663)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:180)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:177)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> ... 2 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-605) Corruption in CommitLog
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788523#action_12788523 ]
Jonathan Ellis commented on CASSANDRA-605:
------------------------------------------
adding a CRC to each log record should fix this.
> Corruption in CommitLog
> -----------------------
>
> Key: CASSANDRA-605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-605
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Reporter: Chris Goffinet
> Fix For: 0.5
>
>
> We are seeing corruption in commit log files when cassandra gets shutdown sometimes. I can attach the commitlogs privately. Here is a stacktrace:
> INFO [main] 2009-12-05 14:59:25,946 RecoveryManager.java (line 64) Replaying /mnt/var/cassandra/commitlog/CommitLog-1259972135241.log, /mnt/var/cassandra/commitlog/CommitLog-1260050162791.log, /mnt/var/cassandra/commitlog/CommitLog-1260047770958.log, /mnt/var/cassandra/commitlog/CommitLog-1260052237605.log
> ERROR [COMPACTION-POOL:1] 2009-12-05 14:59:37,436 DebuggableThreadPoolExecutor.java (line 120) Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.NumberFormatException: For input string: "nan"
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:112)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.NumberFormatException: For input string: "nan"
> at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
> at java.lang.Float.valueOf(Float.java:388)
> at java.lang.Float.<init>(Float.java:489)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:72)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:19)
> at java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606)
> at java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:797)
> at java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1640)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:530)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:515)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:111)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:38)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130)
> at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
> at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:919)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:861)
> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:663)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:180)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:177)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> ... 2 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-605) Corruption in CommitLog
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-605:
-------------------------------------
Attachment: 605.patch
patch to add CRC
> Corruption in CommitLog
> -----------------------
>
> Key: CASSANDRA-605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-605
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Reporter: Chris Goffinet
> Fix For: 0.5
>
> Attachments: 605.patch
>
>
> We are seeing corruption in commit log files when cassandra gets shutdown sometimes. I can attach the commitlogs privately. Here is a stacktrace:
> INFO [main] 2009-12-05 14:59:25,946 RecoveryManager.java (line 64) Replaying /mnt/var/cassandra/commitlog/CommitLog-1259972135241.log, /mnt/var/cassandra/commitlog/CommitLog-1260050162791.log, /mnt/var/cassandra/commitlog/CommitLog-1260047770958.log, /mnt/var/cassandra/commitlog/CommitLog-1260052237605.log
> ERROR [COMPACTION-POOL:1] 2009-12-05 14:59:37,436 DebuggableThreadPoolExecutor.java (line 120) Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.NumberFormatException: For input string: "nan"
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:112)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.NumberFormatException: For input string: "nan"
> at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
> at java.lang.Float.valueOf(Float.java:388)
> at java.lang.Float.<init>(Float.java:489)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:72)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:19)
> at java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606)
> at java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:797)
> at java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1640)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:530)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:515)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:111)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:38)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130)
> at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
> at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:919)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:861)
> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:663)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:180)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:177)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> ... 2 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-605) Corruption in CommitLog
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789127#action_12789127 ]
Jonathan Ellis commented on CASSANDRA-605:
------------------------------------------
>From IRC:
junrao: if that doesn't add much overhead during writes, then it's ok
driftx: if there's a [performance] difference, it's negligible
> Corruption in CommitLog
> -----------------------
>
> Key: CASSANDRA-605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-605
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Reporter: Chris Goffinet
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 605.patch
>
>
> We are seeing corruption in commit log files when cassandra gets shutdown sometimes. I can attach the commitlogs privately. Here is a stacktrace:
> INFO [main] 2009-12-05 14:59:25,946 RecoveryManager.java (line 64) Replaying /mnt/var/cassandra/commitlog/CommitLog-1259972135241.log, /mnt/var/cassandra/commitlog/CommitLog-1260050162791.log, /mnt/var/cassandra/commitlog/CommitLog-1260047770958.log, /mnt/var/cassandra/commitlog/CommitLog-1260052237605.log
> ERROR [COMPACTION-POOL:1] 2009-12-05 14:59:37,436 DebuggableThreadPoolExecutor.java (line 120) Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.NumberFormatException: For input string: "nan"
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:112)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.NumberFormatException: For input string: "nan"
> at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
> at java.lang.Float.valueOf(Float.java:388)
> at java.lang.Float.<init>(Float.java:489)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:72)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:19)
> at java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606)
> at java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:797)
> at java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1640)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:530)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:515)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:111)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:38)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130)
> at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
> at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:919)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:861)
> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:663)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:180)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:177)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> ... 2 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CASSANDRA-605) Corruption in CommitLog
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-605:
----------------------------------------
Assignee: Jonathan Ellis
> Corruption in CommitLog
> -----------------------
>
> Key: CASSANDRA-605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-605
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Reporter: Chris Goffinet
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 605.patch
>
>
> We are seeing corruption in commit log files when cassandra gets shutdown sometimes. I can attach the commitlogs privately. Here is a stacktrace:
> INFO [main] 2009-12-05 14:59:25,946 RecoveryManager.java (line 64) Replaying /mnt/var/cassandra/commitlog/CommitLog-1259972135241.log, /mnt/var/cassandra/commitlog/CommitLog-1260050162791.log, /mnt/var/cassandra/commitlog/CommitLog-1260047770958.log, /mnt/var/cassandra/commitlog/CommitLog-1260052237605.log
> ERROR [COMPACTION-POOL:1] 2009-12-05 14:59:37,436 DebuggableThreadPoolExecutor.java (line 120) Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.NumberFormatException: For input string: "nan"
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:112)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.NumberFormatException: For input string: "nan"
> at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
> at java.lang.Float.valueOf(Float.java:388)
> at java.lang.Float.<init>(Float.java:489)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:72)
> at com.digg.cassandra.db.marshal.FloatStringType.compare(FloatStringType.java:19)
> at java.util.concurrent.ConcurrentSkipListMap$ComparableUsingComparator.compareTo(ConcurrentSkipListMap.java:606)
> at java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:797)
> at java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1640)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:530)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:515)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:111)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:38)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130)
> at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
> at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:919)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:861)
> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:663)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:180)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:177)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> ... 2 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.