You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Niel Drummond (JIRA)" <ji...@apache.org> on 2012/07/27 12:27:34 UTC
[jira] [Created] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Niel Drummond created CASSANDRA-4466:
----------------------------------------
Summary: ColumnFamilyRecordReader hadoop integration fails with ghost keys
Key: CASSANDRA-4466
URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.2
Reporter: Niel Drummond
Priority: Minor
When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
This seems to be related to changes made in #2855 ( https://issues.apache.org/jira/browse/CASSANDRA-2855 ), to prevent ghost keys appearing in a hadoop map.
The stack trace I get is:
java.lang.RuntimeException: java.util.NoSuchElementException
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:392) ~[cassandra-all-1.1.2.jar:1.1.2]
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:398) ~[cassandra-all-1.1.2.jar:1.1.2]
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:324) ~[cassandra-all-1.1.2.jar:1.1.2]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) ~[guava-r09.jar:na]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) ~[guava-r09.jar:na]
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:189) ~[cassandra-all-1.1.2.jar:1.1.2]
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532) ~[hadoop-core-0.20.205.0.jar:na]
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) ~[hadoop-core-0.20.205.0.jar:na]
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) ~[hadoop-core-0.20.205.0.jar:na]
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) ~[hadoop-core-0.20.205.0.jar:na]
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) ~[hadoop-core-0.20.205.0.jar:na]
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) ~[hadoop-core-0.20.205.0.jar:na]
Caused by: java.util.NoSuchElementException: null
at com.google.common.collect.Iterables.getLast(Iterables.java:713) ~[guava-r09.jar:na]
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:342) ~[cassandra-all-1.1.2.jar:1.1.2]
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:382) ~[cassandra-all-1.1.2.jar:1.1.2]
... 11 common frames omitted
I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4466:
--------------------------------------
Attachment: 4466.txt
I think we can just add back the last-seen row. Patch attached.
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Priority: Minor
> Fix For: 1.1.4
>
> Attachments: 4466-stack, 4466-v0.patch, 4466.txt
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in CASSANDRA-2855 to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4466:
--------------------------------------
Description:
When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
This seems to be related to changes made in CASSANDRA-2855 to prevent ghost keys appearing in a hadoop map.
The call stack trace is attached.
I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
was:
When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
This seems to be related to changes made in #2855 ( https://issues.apache.org/jira/browse/CASSANDRA-2855 ), to prevent ghost keys appearing in a hadoop map.
The call stack trace is attached.
I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Priority: Minor
> Fix For: 1.1.4
>
> Attachments: 4466-stack, 4466-v0.patch
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in CASSANDRA-2855 to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424089#comment-13424089 ]
Jonathan Ellis commented on CASSANDRA-4466:
-------------------------------------------
setting rows=null is the signal to stop looping over that inputsplit, so that's not going to work very well in the general case.
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Priority: Minor
> Fix For: 1.1.4
>
> Attachments: 4466-stack, 4466-v0.patch
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in #2855 ( https://issues.apache.org/jira/browse/CASSANDRA-2855 ), to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4466:
--------------------------------------
Fix Version/s: (was: 1.1.4)
1.1.3
Assignee: Brandon Williams
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Assignee: Brandon Williams
> Priority: Minor
> Fix For: 1.1.3
>
> Attachments: 4466-stack, 4466-v0.patch, 4466.txt
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in CASSANDRA-2855 to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Niel Drummond (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424753#comment-13424753 ]
Niel Drummond commented on CASSANDRA-4466:
------------------------------------------
Ah ok, thanks for the input. What's the best approach to solving this ?
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Priority: Minor
> Fix For: 1.1.4
>
> Attachments: 4466-stack, 4466-v0.patch
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in #2855 ( https://issues.apache.org/jira/browse/CASSANDRA-2855 ), to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Niel Drummond (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Niel Drummond updated CASSANDRA-4466:
-------------------------------------
Description:
When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
This seems to be related to changes made in #2855 ( https://issues.apache.org/jira/browse/CASSANDRA-2855 ), to prevent ghost keys appearing in a hadoop map.
The call stack trace is attached.
I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
was:
When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
This seems to be related to changes made in #2855 ( https://issues.apache.org/jira/browse/CASSANDRA-2855 ), to prevent ghost keys appearing in a hadoop map.
The stack trace I get is:
java.lang.RuntimeException: java.util.NoSuchElementException
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:392) ~[cassandra-all-1.1.2.jar:1.1.2]
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:398) ~[cassandra-all-1.1.2.jar:1.1.2]
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:324) ~[cassandra-all-1.1.2.jar:1.1.2]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) ~[guava-r09.jar:na]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) ~[guava-r09.jar:na]
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:189) ~[cassandra-all-1.1.2.jar:1.1.2]
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532) ~[hadoop-core-0.20.205.0.jar:na]
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) ~[hadoop-core-0.20.205.0.jar:na]
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) ~[hadoop-core-0.20.205.0.jar:na]
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) ~[hadoop-core-0.20.205.0.jar:na]
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) ~[hadoop-core-0.20.205.0.jar:na]
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) ~[hadoop-core-0.20.205.0.jar:na]
Caused by: java.util.NoSuchElementException: null
at com.google.common.collect.Iterables.getLast(Iterables.java:713) ~[guava-r09.jar:na]
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:342) ~[cassandra-all-1.1.2.jar:1.1.2]
at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:382) ~[cassandra-all-1.1.2.jar:1.1.2]
... 11 common frames omitted
I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Priority: Minor
> Attachments: 4466-stack, 4466-v0.patch
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in #2855 ( https://issues.apache.org/jira/browse/CASSANDRA-2855 ), to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425802#comment-13425802 ]
Brandon Williams commented on CASSANDRA-4466:
---------------------------------------------
+1
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.1.3
>
> Attachments: 4466-stack, 4466-v0.patch, 4466.txt
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in CASSANDRA-2855 to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-4466.
---------------------------------------
Resolution: Fixed
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 0.8.8
> Reporter: Niel Drummond
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.0.12, 1.1.4
>
> Attachments: 4466-stack, 4466-v0.patch, 4466.txt
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in CASSANDRA-2855 to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Niel Drummond (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425661#comment-13425661 ]
Niel Drummond commented on CASSANDRA-4466:
------------------------------------------
I can confirm the patch works for our dataset. Thanks for this.
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.1.3
>
> Attachments: 4466-stack, 4466-v0.patch, 4466.txt
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in CASSANDRA-2855 to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4466:
--------------------------------------
Affects Version/s: (was: 1.1.2)
0.8.8
Fix Version/s: (was: 1.1.3)
1.1.4
1.0.12
committed
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 0.8.8
> Reporter: Niel Drummond
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.0.12, 1.1.4
>
> Attachments: 4466-stack, 4466-v0.patch, 4466.txt
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in CASSANDRA-2855 to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4466:
--------------------------------------
Reviewer: ndrummond@rim.com
Assignee: Jonathan Ellis (was: Brandon Williams)
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.1.3
>
> Attachments: 4466-stack, 4466-v0.patch, 4466.txt
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in CASSANDRA-2855 to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4466:
--------------------------------------
Fix Version/s: 1.1.4
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Priority: Minor
> Fix For: 1.1.4
>
> Attachments: 4466-stack, 4466-v0.patch
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in #2855 ( https://issues.apache.org/jira/browse/CASSANDRA-2855 ), to prevent ghost keys appearing in a hadoop map.
> The call stack trace is attached.
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Niel Drummond (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Niel Drummond updated CASSANDRA-4466:
-------------------------------------
Attachment: 4466-stack
exception call stack
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Priority: Minor
> Attachments: 4466-stack, 4466-v0.patch
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in #2855 ( https://issues.apache.org/jira/browse/CASSANDRA-2855 ), to prevent ghost keys appearing in a hadoop map.
> The stack trace I get is:
> java.lang.RuntimeException: java.util.NoSuchElementException
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:392) ~[cassandra-all-1.1.2.jar:1.1.2]
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:398) ~[cassandra-all-1.1.2.jar:1.1.2]
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:324) ~[cassandra-all-1.1.2.jar:1.1.2]
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) ~[guava-r09.jar:na]
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) ~[guava-r09.jar:na]
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:189) ~[cassandra-all-1.1.2.jar:1.1.2]
> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532) ~[hadoop-core-0.20.205.0.jar:na]
> at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) ~[hadoop-core-0.20.205.0.jar:na]
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) ~[hadoop-core-0.20.205.0.jar:na]
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) ~[hadoop-core-0.20.205.0.jar:na]
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) ~[hadoop-core-0.20.205.0.jar:na]
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) ~[hadoop-core-0.20.205.0.jar:na]
> Caused by: java.util.NoSuchElementException: null
> at com.google.common.collect.Iterables.getLast(Iterables.java:713) ~[guava-r09.jar:na]
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:342) ~[cassandra-all-1.1.2.jar:1.1.2]
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:382) ~[cassandra-all-1.1.2.jar:1.1.2]
> ... 11 common frames omitted
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4466) ColumnFamilyRecordReader hadoop
integration fails with ghost keys
Posted by "Niel Drummond (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Niel Drummond updated CASSANDRA-4466:
-------------------------------------
Attachment: 4466-v0.patch
> ColumnFamilyRecordReader hadoop integration fails with ghost keys
> -----------------------------------------------------------------
>
> Key: CASSANDRA-4466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4466
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.2
> Reporter: Niel Drummond
> Priority: Minor
> Attachments: 4466-v0.patch
>
>
> When running hadoop-cassandra jobs with range queries over ghost keys, the ColumnFamilyRecordReader throws an exception if the last key in a slice_range query is a ghost key.
> This seems to be related to changes made in #2855 ( https://issues.apache.org/jira/browse/CASSANDRA-2855 ), to prevent ghost keys appearing in a hadoop map.
> The stack trace I get is:
> java.lang.RuntimeException: java.util.NoSuchElementException
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:392) ~[cassandra-all-1.1.2.jar:1.1.2]
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:398) ~[cassandra-all-1.1.2.jar:1.1.2]
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:324) ~[cassandra-all-1.1.2.jar:1.1.2]
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) ~[guava-r09.jar:na]
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) ~[guava-r09.jar:na]
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:189) ~[cassandra-all-1.1.2.jar:1.1.2]
> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532) ~[hadoop-core-0.20.205.0.jar:na]
> at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) ~[hadoop-core-0.20.205.0.jar:na]
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) ~[hadoop-core-0.20.205.0.jar:na]
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) ~[hadoop-core-0.20.205.0.jar:na]
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) ~[hadoop-core-0.20.205.0.jar:na]
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) ~[hadoop-core-0.20.205.0.jar:na]
> Caused by: java.util.NoSuchElementException: null
> at com.google.common.collect.Iterables.getLast(Iterables.java:713) ~[guava-r09.jar:na]
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:342) ~[cassandra-all-1.1.2.jar:1.1.2]
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:382) ~[cassandra-all-1.1.2.jar:1.1.2]
> ... 11 common frames omitted
> I made a one-line change to ColumnFamilyRecordReader.java, which seems to solve this issue for us.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira