You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Mark Lewandowski <ma...@gmail.com> on 2013/06/08 00:02:48 UTC

Cassandra (1.2.5) + Pig (0.11.1) Errors with large column families

I'm currently trying to get Cassandra (1.2.5) and Pig (0.11.1) to play nice
together.  I'm running a basic script:

rows = LOAD 'cassandra://keyspace/colfam' USING CassandraStorage();
dump rows;

This fails for my column family which has ~100,000 rows.  However, if I
modify the script to this:

rows = LOAD 'cassandra://betable_games/bets' USING CassandraStorage();
rows = limit rows 7000;
dump rows;

Then it seems to work.  7000 is about as high as I've been able to get it
before it fails.  The error I keep getting is:

2013-06-07 14:58:49,119 [Thread-4] WARN
 org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
java.lang.RuntimeException: org.apache.thrift.TException: Message length
exceeded: 4480
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:384)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:390)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:313)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.getProgress(ColumnFamilyRecordReader.java:103)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.getProgress(PigRecordReader.java:169)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.getProgress(MapTask.java:514)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:539)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:214)
Caused by: org.apache.thrift.TException: Message length exceeded: 4480
at
org.apache.thrift.protocol.TBinaryProtocol.checkReadLength(TBinaryProtocol.java:393)
at
org.apache.thrift.protocol.TBinaryProtocol.readBinary(TBinaryProtocol.java:363)
at org.apache.cassandra.thrift.Column.read(Column.java:535)
at
org.apache.cassandra.thrift.ColumnOrSuperColumn.read(ColumnOrSuperColumn.java:507)
at org.apache.cassandra.thrift.KeySlice.read(KeySlice.java:408)
at
org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12905)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:734)
at
org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:718)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:346)
... 13 more


I've seen a similar problem on this mailing list using Cassandra-1.2.3,
however the fixes on that thread of increasing
thrift_framed_transport_size_in_mb, thrift_max_message_length_in_mb in
cassandra.yaml did not appear to have any effect.  Has anyone else seen
this issue, and how can I fix it?

Thanks,

-Mark

Re: Cassandra (1.2.5) + Pig (0.11.1) Errors with large column families

Posted by Arthur Zubarev <ar...@aol.com>.
On 06/07/2013 06:02 PM, Mark Lewandowski wrote:
> I'm currently trying to get Cassandra (1.2.5) and Pig (0.11.1) to play 
> nice together.  I'm running a basic script:
>
> rows = LOAD 'cassandra://keyspace/colfam' USING CassandraStorage();
> dump rows;
>
> This fails for my column family which has ~100,000 rows.  However, if 
> I modify the script to this:
>
> rows = LOAD 'cassandra://betable_games/bets' USING CassandraStorage();
> rows = limit rows 7000;
> dump rows;
>
> Then it seems to work.  7000 is about as high as I've been able to get 
> it before it fails.  The error I keep getting is:
>
> 2013-06-07 14:58:49,119 [Thread-4] WARN 
>  org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
> java.lang.RuntimeException: org.apache.thrift.TException: Message 
> length exceeded: 4480
> at 
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:384)
> at 
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:390)
> at 
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:313)
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at 
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader.getProgress(ColumnFamilyRecordReader.java:103)
> at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.getProgress(PigRecordReader.java:169)
> at 
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.getProgress(MapTask.java:514)
> at 
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:539)
> at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:214)
> Caused by: org.apache.thrift.TException: Message length exceeded: 4480
> at 
> org.apache.thrift.protocol.TBinaryProtocol.checkReadLength(TBinaryProtocol.java:393)
> at 
> org.apache.thrift.protocol.TBinaryProtocol.readBinary(TBinaryProtocol.java:363)
> at org.apache.cassandra.thrift.Column.read(Column.java:535)
> at 
> org.apache.cassandra.thrift.ColumnOrSuperColumn.read(ColumnOrSuperColumn.java:507)
> at org.apache.cassandra.thrift.KeySlice.read(KeySlice.java:408)
> at 
> org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12905)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
> at 
> org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:734)
> at 
> org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:718)
> at 
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:346)
> ... 13 more
>
>
> I've seen a similar problem on this mailing list using 
> Cassandra-1.2.3, however the fixes on that thread of increasing 
> thrift_framed_transport_size_in_mb, thrift_max_message_length_in_mb in 
> cassandra.yaml did not appear to have any effect.  Has anyone else 
> seen this issue, and how can I fix it?
>
> Thanks,
>
> -Mark
Mark,

Reading your email made me wonder if your CF needs the compact storage 
directive applied as in the post about the Bulk Loader, in short, 
defining your CF

WITH COMPACT STORAGE

and compaction = {'class' : 'LeveledCompactionStrategy' }


Hopefully enables you to read the data in full.

-- 

Regards,

Arthur

Re: Cassandra (1.2.5) + Pig (0.11.1) Errors with large column families

Posted by Franc Carter <fr...@sirca.org.au>.
---------- Forwarded message ----------
From: "Mark Lewandowski" <ma...@gmail.com>
Date: Jun 8, 2013 8:03 AM
Subject: Cassandra (1.2.5) + Pig (0.11.1) Errors with large column families
To: <us...@cassandra.apache.org>
Cc:

> I'm cur.rently trying to get Cassandra (1.2.5) and Pig (0.11.1) to play
nice together.  I'm running a basic script:
>
> rows = LOAD 'cassandra://keyspace/colfam' USING CassandraStorage();
> dump rows;
>
> This fails for my column family which has ~100,000 rows.  However, if I
modify the script to this:
>
> rows = LOAD 'cassandra://betable_games/bets' USING CassandraStorage();
> rows = limit rows 7000;
> dump rows;
>
> Then it seems to work.  7000 is about as high as I've been able to get it
before it fails.  The error I keep getting is:
>
> 2013-06-07 14:58:49,119 [Thread-4] WARN
 org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
> java.lang.RuntimeException: org.apache.thrift.TException: Message length
exceeded: 4480
> at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:384)
> at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:390)
> at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:313)
> at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.getProgress(ColumnFamilyRecordReader.java:103)
> at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.getProgress(PigRecordReader.java:169)
> at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.getProgress(MapTask.java:514)
> at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:539)
> at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:214)
> Caused by: org.apache.thrift.TException: Message length exceeded: 4480
> at
org.apache.thrift.protocol.TBinaryProtocol.checkReadLength(TBinaryProtocol.java:393)
> at
org.apache.thrift.protocol.TBinaryProtocol.readBinary(TBinaryProtocol.java:363)
> at org.apache.cassandra.thrift.Column.read(Column.java:535)
> at
org.apache.cassandra.thrift.ColumnOrSuperColumn.read(ColumnOrSuperColumn.java:507)
> at org.apache.cassandra.thrift.KeySlice.read(KeySlice.java:408)
> at
org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12905)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
> at
org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:734)
> at
org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:718)
> at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:346)
> ... 13 more
>
>
> I've seen a similar problem on this mailing list using Cassandra-1.2.3,
however the fixes on that thread of
increasing thrift_framed_transport_size_in_mb, thrift_max_message_length_in_mb
in cassandra.yaml did not appear to have any effect.  Has anyone else seen
this issue, and how can I fix it?
>
> Thanks,
>
> -Mark