You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jacek Gerbszt (JIRA)" <ji...@apache.org> on 2011/07/27 14:23:10 UTC
[jira] [Created] (CASSANDRA-2955) Hadoop cannot read CounterColumn
Hadoop cannot read CounterColumn
--------------------------------
Key: CASSANDRA-2955
URL: https://issues.apache.org/jira/browse/CASSANDRA-2955
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 0.8.1
Reporter: Jacek Gerbszt
Accessing CounterColumn from Hadoop ends up with an exception:
{noformat}
java.lang.NullPointerException
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown Source)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
{noformat}
If you look into ColumnFamilyRecordRerader, you will quickly realize that it doesn't know about CounterColumn's existence:
{code:title=ColumnFamilyRecordRerader.java}
private IColumn unthriftify(ColumnOrSuperColumn cosc)
{
if (cosc.column == null)
return unthriftifySuper(cosc.super_column);
return unthriftifySimple(cosc.column);
}
{code}
It uses only column and super_column fields from ColumnOrSuperColumn class, while it keeps CounterColumn in one of two additional fields - counter_column or counter_super_column:
{code:title=ColumnOrSuperColumn.java}
public class ColumnOrSuperColumn {
public org.apache.cassandra.thrift.Column column;
public org.apache.cassandra.thrift.SuperColumn super_column;
public org.apache.cassandra.thrift.CounterColumn counter_column;
public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
}
{code}
Please add CounterColumn support to ColumnFamilyRecordRerader class.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2955) Hadoop cannot read CounterColumn
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2955:
--------------------------------------
Priority: Minor (was: Major)
Affects Version/s: (was: 0.8.1)
0.8.0
Fix Version/s: 0.8.3
Labels: lhf (was: )
Issue Type: New Feature (was: Bug)
> Hadoop cannot read CounterColumn
> --------------------------------
>
> Key: CASSANDRA-2955
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2955
> Project: Cassandra
> Issue Type: New Feature
> Components: Hadoop
> Affects Versions: 0.8.0
> Reporter: Jacek Gerbszt
> Priority: Minor
> Labels: lhf
> Fix For: 0.8.3
>
>
> Accessing CounterColumn from Hadoop ends up with an exception:
> {noformat}
> java.lang.NullPointerException
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
> at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown Source)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
> at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> {noformat}
> If you look into ColumnFamilyRecordRerader, you will quickly realize that it doesn't know about CounterColumn's existence:
> {code:title=ColumnFamilyRecordRerader.java}
> private IColumn unthriftify(ColumnOrSuperColumn cosc)
> {
> if (cosc.column == null)
> return unthriftifySuper(cosc.super_column);
> return unthriftifySimple(cosc.column);
> }
> {code}
> It uses only column and super_column fields from ColumnOrSuperColumn class, while this class keeps CounterColumn in one of two additional fields - counter_column or counter_super_column:
> {code:title=ColumnOrSuperColumn.java}
> public class ColumnOrSuperColumn {
> public org.apache.cassandra.thrift.Column column;
> public org.apache.cassandra.thrift.SuperColumn super_column;
> public org.apache.cassandra.thrift.CounterColumn counter_column;
> public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
> }
> {code}
> Please add CounterColumn support to ColumnFamilyRecordReader class.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2955) Hadoop cannot read CounterColumn
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-2955.
---------------------------------------
Resolution: Duplicate
Fix Version/s: (was: 0.8.3)
addressed in CASSANDRA-2981
> Hadoop cannot read CounterColumn
> --------------------------------
>
> Key: CASSANDRA-2955
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2955
> Project: Cassandra
> Issue Type: New Feature
> Components: Hadoop
> Affects Versions: 0.8.0
> Reporter: Jacek Gerbszt
> Priority: Minor
> Labels: lhf
>
> Accessing CounterColumn from Hadoop ends up with an exception:
> {noformat}
> java.lang.NullPointerException
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
> at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown Source)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
> at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> {noformat}
> If you look into ColumnFamilyRecordRerader, you will quickly realize that it doesn't know about CounterColumn's existence:
> {code:title=ColumnFamilyRecordRerader.java}
> private IColumn unthriftify(ColumnOrSuperColumn cosc)
> {
> if (cosc.column == null)
> return unthriftifySuper(cosc.super_column);
> return unthriftifySimple(cosc.column);
> }
> {code}
> It uses only column and super_column fields from ColumnOrSuperColumn class, while this class keeps CounterColumn in one of two additional fields - counter_column or counter_super_column:
> {code:title=ColumnOrSuperColumn.java}
> public class ColumnOrSuperColumn {
> public org.apache.cassandra.thrift.Column column;
> public org.apache.cassandra.thrift.SuperColumn super_column;
> public org.apache.cassandra.thrift.CounterColumn counter_column;
> public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
> }
> {code}
> Please add CounterColumn support to ColumnFamilyRecordReader class.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2955) Hadoop cannot read CounterColumn
Posted by "Jacek Gerbszt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jacek Gerbszt updated CASSANDRA-2955:
-------------------------------------
Description:
Accessing CounterColumn from Hadoop ends up with an exception:
{noformat}
java.lang.NullPointerException
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown Source)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
{noformat}
If you look into ColumnFamilyRecordRerader, you will quickly realize that it doesn't know about CounterColumn's existence:
{code:title=ColumnFamilyRecordRerader.java}
private IColumn unthriftify(ColumnOrSuperColumn cosc)
{
if (cosc.column == null)
return unthriftifySuper(cosc.super_column);
return unthriftifySimple(cosc.column);
}
{code}
It uses only column and super_column fields from ColumnOrSuperColumn class, while this class keeps CounterColumn in one of two additional fields - counter_column or counter_super_column:
{code:title=ColumnOrSuperColumn.java}
public class ColumnOrSuperColumn {
public org.apache.cassandra.thrift.Column column;
public org.apache.cassandra.thrift.SuperColumn super_column;
public org.apache.cassandra.thrift.CounterColumn counter_column;
public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
}
{code}
Please add CounterColumn support to ColumnFamilyRecordReader class.
was:
Accessing CounterColumn from Hadoop ends up with an exception:
{noformat}
java.lang.NullPointerException
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown Source)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
{noformat}
If you look into ColumnFamilyRecordRerader, you will quickly realize that it doesn't know about CounterColumn's existence:
{code:title=ColumnFamilyRecordRerader.java}
private IColumn unthriftify(ColumnOrSuperColumn cosc)
{
if (cosc.column == null)
return unthriftifySuper(cosc.super_column);
return unthriftifySimple(cosc.column);
}
{code}
It uses only column and super_column fields from ColumnOrSuperColumn class, while this class keeps CounterColumn in one of two additional fields - counter_column or counter_super_column:
{code:title=ColumnOrSuperColumn.java}
public class ColumnOrSuperColumn {
public org.apache.cassandra.thrift.Column column;
public org.apache.cassandra.thrift.SuperColumn super_column;
public org.apache.cassandra.thrift.CounterColumn counter_column;
public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
}
{code}
Please add CounterColumn support to ColumnFamilyRecordRerader class.
> Hadoop cannot read CounterColumn
> --------------------------------
>
> Key: CASSANDRA-2955
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2955
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 0.8.1
> Reporter: Jacek Gerbszt
>
> Accessing CounterColumn from Hadoop ends up with an exception:
> {noformat}
> java.lang.NullPointerException
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
> at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown Source)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
> at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> {noformat}
> If you look into ColumnFamilyRecordRerader, you will quickly realize that it doesn't know about CounterColumn's existence:
> {code:title=ColumnFamilyRecordRerader.java}
> private IColumn unthriftify(ColumnOrSuperColumn cosc)
> {
> if (cosc.column == null)
> return unthriftifySuper(cosc.super_column);
> return unthriftifySimple(cosc.column);
> }
> {code}
> It uses only column and super_column fields from ColumnOrSuperColumn class, while this class keeps CounterColumn in one of two additional fields - counter_column or counter_super_column:
> {code:title=ColumnOrSuperColumn.java}
> public class ColumnOrSuperColumn {
> public org.apache.cassandra.thrift.Column column;
> public org.apache.cassandra.thrift.SuperColumn super_column;
> public org.apache.cassandra.thrift.CounterColumn counter_column;
> public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
> }
> {code}
> Please add CounterColumn support to ColumnFamilyRecordReader class.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2955) Hadoop cannot read CounterColumn
Posted by "Jacek Gerbszt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jacek Gerbszt updated CASSANDRA-2955:
-------------------------------------
Description:
Accessing CounterColumn from Hadoop ends up with an exception:
{noformat}
java.lang.NullPointerException
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown Source)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
{noformat}
If you look into ColumnFamilyRecordRerader, you will quickly realize that it doesn't know about CounterColumn's existence:
{code:title=ColumnFamilyRecordRerader.java}
private IColumn unthriftify(ColumnOrSuperColumn cosc)
{
if (cosc.column == null)
return unthriftifySuper(cosc.super_column);
return unthriftifySimple(cosc.column);
}
{code}
It uses only column and super_column fields from ColumnOrSuperColumn class, while this class keeps CounterColumn in one of two additional fields - counter_column or counter_super_column:
{code:title=ColumnOrSuperColumn.java}
public class ColumnOrSuperColumn {
public org.apache.cassandra.thrift.Column column;
public org.apache.cassandra.thrift.SuperColumn super_column;
public org.apache.cassandra.thrift.CounterColumn counter_column;
public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
}
{code}
Please add CounterColumn support to ColumnFamilyRecordRerader class.
was:
Accessing CounterColumn from Hadoop ends up with an exception:
{noformat}
java.lang.NullPointerException
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown Source)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
{noformat}
If you look into ColumnFamilyRecordRerader, you will quickly realize that it doesn't know about CounterColumn's existence:
{code:title=ColumnFamilyRecordRerader.java}
private IColumn unthriftify(ColumnOrSuperColumn cosc)
{
if (cosc.column == null)
return unthriftifySuper(cosc.super_column);
return unthriftifySimple(cosc.column);
}
{code}
It uses only column and super_column fields from ColumnOrSuperColumn class, while it keeps CounterColumn in one of two additional fields - counter_column or counter_super_column:
{code:title=ColumnOrSuperColumn.java}
public class ColumnOrSuperColumn {
public org.apache.cassandra.thrift.Column column;
public org.apache.cassandra.thrift.SuperColumn super_column;
public org.apache.cassandra.thrift.CounterColumn counter_column;
public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
}
{code}
Please add CounterColumn support to ColumnFamilyRecordRerader class.
> Hadoop cannot read CounterColumn
> --------------------------------
>
> Key: CASSANDRA-2955
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2955
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 0.8.1
> Reporter: Jacek Gerbszt
>
> Accessing CounterColumn from Hadoop ends up with an exception:
> {noformat}
> java.lang.NullPointerException
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
> at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown Source)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
> at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> {noformat}
> If you look into ColumnFamilyRecordRerader, you will quickly realize that it doesn't know about CounterColumn's existence:
> {code:title=ColumnFamilyRecordRerader.java}
> private IColumn unthriftify(ColumnOrSuperColumn cosc)
> {
> if (cosc.column == null)
> return unthriftifySuper(cosc.super_column);
> return unthriftifySimple(cosc.column);
> }
> {code}
> It uses only column and super_column fields from ColumnOrSuperColumn class, while this class keeps CounterColumn in one of two additional fields - counter_column or counter_super_column:
> {code:title=ColumnOrSuperColumn.java}
> public class ColumnOrSuperColumn {
> public org.apache.cassandra.thrift.Column column;
> public org.apache.cassandra.thrift.SuperColumn super_column;
> public org.apache.cassandra.thrift.CounterColumn counter_column;
> public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
> }
> {code}
> Please add CounterColumn support to ColumnFamilyRecordRerader class.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira