You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Ralph Romanos (JIRA)" <ji...@apache.org> on 2012/10/17 15:52:03 UTC
[jira] [Comment Edited] (CASSANDRA-4813) Problem using
BulkOutputFormat while streaming several SSTables simultaneously from a
given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477678#comment-13477678 ]
Ralph Romanos edited comment on CASSANDRA-4813 at 10/17/12 1:50 PM:
--------------------------------------------------------------------
I get the following error in the tasktracker's logs when SSTables
are streamed into the Cassandra cluster:
Exception in thread "Streaming to /172.16.110.79:1" java.lang.RuntimeException: java.io.EOFException
at org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:628)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(Unknown Source)
at org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:194)
at org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:181)
at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:94)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
Exception in thread "Streaming to /172.16.110.92:1" java.lang.RuntimeException: java.io.EOFException
at org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:628)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(Unknown Source)
at org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:194)
at org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:181)
at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:94)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
was (Author: ralph.romanos):
I get the following error in the tasktracker's logs when they
are streamed into the Cassandra cluster:
Exception in thread "Streaming to /172.16.110.79:1" java.lang.RuntimeException: java.io.EOFException
at org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:628)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(Unknown Source)
at org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:194)
at org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:181)
at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:94)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
Exception in thread "Streaming to /172.16.110.92:1" java.lang.RuntimeException: java.io.EOFException
at org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:628)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(Unknown Source)
at org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:194)
at org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:181)
at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:94)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
> Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
> -------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-4813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4813
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 1.1.3, 1.1.5
> Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using Hadoop 0.20.2.
> Reporter: Ralph Romanos
> Labels: Bulkoutputformat, Hadoop, SSTables
>
> The issue occurs when streaming simultaneously SSTables from the same node to a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot handle receiving simultaneously SSTables from the same node. However, when it receives simultaneously SSTables from two different nodes, everything works fine. As a consequence, when using BulkOutputFormat to generate SSTables and stream them to a cassandra cluster, I cannot use more than one reducer per node otherwise I get a java.io.EOFException in the tasktracker's logs and a java.io.IOException: Broken pipe in the Cassandra logs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira