You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by chandra Varahala <ha...@gmail.com> on 2013/01/17 19:39:38 UTC
BulkOutputFormat
Hello,
I am facing issues with Bulkoutputformat loading data from hadoop to
cassandra.
Cluster details :
we have 15 nodes in Hadoop cluster, 2 nodes in cassandra - QA and we
have
150 hadoop, 10 nodes in Cassandra production environment
Two cassandra clusters Random order and Byte order on same machine with
different ports.
issues 1
i can load small amount(1G) of data in random cluster. But more than 1GB
throwing below error
:
java.io.IOException: Too many hosts failed: [/172.20.128.48, /172.20.128.49]
at
org.apache.cassandra.hadoop.BulkRecordWriter.close(BulkRecordWriter.java:243)
at
org.apache.cassandra.hadoop.BulkRecordWriter.close(BulkRecordWriter.java:208)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:540)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:649)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
attempt_201301100944_4262_m_000000_0: Exception in thread "Streaming to /
172.20.138.48:1" java.lang.RuntimeException: java.io.EOFException
ISSUE -2:
I am not able to load data in Byte order cluster:
same above error.
Please help.
Chandra
Re: BulkOutputFormat
Posted by Michael Kjellman <mk...@barracuda.com>.
It was primarily a streaming issue not a Hadoop component issue. Seems very similar to not be related IMHO
On Jan 17, 2013, at 10:59 AM, "chandra Varahala" <ha...@gmail.com>> wrote:
I am not reducers, just Map only job still same kind issue ?
thanks
chandra
On Thu, Jan 17, 2013 at 1:50 PM, Michael Kjellman <mk...@barracuda.com>> wrote:
https://issues.apache.org/jira/browse/CASSANDRA-4813
Fixed in 1.2.0
Best,
michael
From: chandra Varahala <ha...@gmail.com>>
Reply-To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Date: Thursday, January 17, 2013 10:39 AM
To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Subject: BulkOutputFormat
Hello,
I am facing issues with Bulkoutputformat loading data from hadoop to cassandra.
Cluster details :
we have 15 nodes in Hadoop cluster, 2 nodes in cassandra - QA and we have
150 hadoop, 10 nodes in Cassandra production environment
Two cassandra clusters Random order and Byte order on same machine with different ports.
issues 1
i can load small amount(1G) of data in random cluster. But more than 1GB throwing below error
:
java.io.IOException: Too many hosts failed: [/172.20.128.48<http://172.20.128.48>, /172.20.128.49<http://172.20.128.49>]
at org.apache.cassandra.hadoop.BulkRecordWriter.close(BulkRecordWriter.java:243)
at org.apache.cassandra.hadoop.BulkRecordWriter.close(BulkRecordWriter.java:208)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:540)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:649)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
attempt_201301100944_4262_m_000000_0: Exception in thread "Streaming to /172.20.138.48:1<http://172.20.138.48:1>" java.lang.RuntimeException: java.io.EOFException
ISSUE -2:
I am not able to load data in Byte order cluster:
same above error.
Please help.
Chandra
Re: BulkOutputFormat
Posted by chandra Varahala <ha...@gmail.com>.
I am not reducers, just Map only job still same kind issue ?
thanks
chandra
On Thu, Jan 17, 2013 at 1:50 PM, Michael Kjellman
<mk...@barracuda.com>wrote:
> https://issues.apache.org/jira/browse/CASSANDRA-4813
>
> Fixed in 1.2.0
>
> Best,
> michael
>
> From: chandra Varahala <ha...@gmail.com>
> Reply-To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
> Date: Thursday, January 17, 2013 10:39 AM
> To: "user@cassandra.apache.org" <us...@cassandra.apache.org>
> Subject: BulkOutputFormat
>
> Hello,
>
> I am facing issues with Bulkoutputformat loading data from hadoop to
> cassandra.
>
> Cluster details :
>
> we have 15 nodes in Hadoop cluster, 2 nodes in cassandra - QA and we
> have
> 150 hadoop, 10 nodes in Cassandra production environment
>
> Two cassandra clusters Random order and Byte order on same machine with
> different ports.
>
> issues 1
>
> i can load small amount(1G) of data in random cluster. But more than 1GB
> throwing below error
> :
> java.io.IOException: Too many hosts failed: [/172.20.128.48, /
> 172.20.128.49]
> at
> org.apache.cassandra.hadoop.BulkRecordWriter.close(BulkRecordWriter.java:243)
> at
> org.apache.cassandra.hadoop.BulkRecordWriter.close(BulkRecordWriter.java:208)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:540)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:649)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.Child.main(Child.java:264)
>
> attempt_201301100944_4262_m_000000_0: Exception in thread "Streaming to /
> 172.20.138.48:1" java.lang.RuntimeException: java.io.EOFException
>
>
> ISSUE -2:
>
> I am not able to load data in Byte order cluster:
>
> same above error.
>
>
> Please help.
>
> Chandra
>
>
>
>
>
Re: BulkOutputFormat
Posted by Michael Kjellman <mk...@barracuda.com>.
https://issues.apache.org/jira/browse/CASSANDRA-4813
Fixed in 1.2.0
Best,
michael
From: chandra Varahala <ha...@gmail.com>>
Reply-To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Date: Thursday, January 17, 2013 10:39 AM
To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Subject: BulkOutputFormat
Hello,
I am facing issues with Bulkoutputformat loading data from hadoop to cassandra.
Cluster details :
we have 15 nodes in Hadoop cluster, 2 nodes in cassandra - QA and we have
150 hadoop, 10 nodes in Cassandra production environment
Two cassandra clusters Random order and Byte order on same machine with different ports.
issues 1
i can load small amount(1G) of data in random cluster. But more than 1GB throwing below error
:
java.io.IOException: Too many hosts failed: [/172.20.128.48<http://172.20.128.48>, /172.20.128.49<http://172.20.128.49>]
at org.apache.cassandra.hadoop.BulkRecordWriter.close(BulkRecordWriter.java:243)
at org.apache.cassandra.hadoop.BulkRecordWriter.close(BulkRecordWriter.java:208)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:540)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:649)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
attempt_201301100944_4262_m_000000_0: Exception in thread "Streaming to /172.20.138.48:1<http://172.20.138.48:1>" java.lang.RuntimeException: java.io.EOFException
ISSUE -2:
I am not able to load data in Byte order cluster:
same above error.
Please help.
Chandra