You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brandon Williams (Created) (JIRA)" <ji...@apache.org> on 2012/02/14 15:29:59 UTC
[jira] [Created] (CASSANDRA-3909) Pig should handle wide rows
Pig should handle wide rows
---------------------------
Key: CASSANDRA-3909
URL: https://issues.apache.org/jira/browse/CASSANDRA-3909
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 1.1.0
Pig should be able to use the wide row support in CFIF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3909) Pig should handle wide rows
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3909:
----------------------------------------
Reviewer: xedin
> Pig should handle wide rows
> ---------------------------
>
> Key: CASSANDRA-3909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3909
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1.1
>
> Attachments: 3909.txt
>
>
> Pig should be able to use the wide row support in CFIF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3909) Pig should handle wide rows
Posted by "Sylvain Lebresne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257455#comment-13257455 ]
Sylvain Lebresne commented on CASSANDRA-3909:
---------------------------------------------
Is that a big deal if it's only in 1.1.1? I mean, personally I do trust you on that "this can't break anything" and I don't object on putting it in 1.1.0. I do however think that in general there would be some merit to stick to more strict rules. But that's not a debate related to this issue in particular so let's leave that discussing to some other venue.
> Pig should handle wide rows
> ---------------------------
>
> Key: CASSANDRA-3909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3909
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1.1
>
> Attachments: 3909.txt
>
>
> Pig should be able to use the wide row support in CFIF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3909) Pig should handle
wide rows
Posted by "Brandon Williams (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254712#comment-13254712 ]
Brandon Williams edited comment on CASSANDRA-3909 at 4/16/12 2:17 PM:
----------------------------------------------------------------------
CASSANDRA-3264 (and subsequently CASSANDRA-3883) added wide row support to hadoop, by returning one column in the row in every call. Pig, however, is fancy enough that it could handle a wide row in a bag, since bags spill to disk; it just needs the pagination for transport since thrift doesn't stream. Also, if we returned what CFIF gave us, a user wanting to work within the row would need another costly M/R job to join the row back to its original state, so we essentially need to 'undo' the pagination and rebuild the row as a bag. This patch does that, with the caveat that you cannot access any indexes (and frankly if you have indexes on a wide row you're probably doing something wrong) since it's impossible for us to order the indexes correctly ahead of time in a wide row.
was (Author: brandon.williams):
CASSANDRA-3264 (and subsequently CASSANDRA-3883) added wide row support to hadoop, by returning one column in the row in every call. Pig, however, is fancy enough that it could handle a wide row in a bag, since bags spill to disk; it just needs the pagination to for transport since thrift doesn't stream. Also, if we returned what CFIF gave us, a user wanting to work within the row would need another costly M/R job to join the row back to its original state, so we essentially need to 'undo' the pagination and rebuild the row as a bag. This patch does that, with the caveat that you cannot access any indexes (and frankly if you have indexes on a wide row you're probably doing something wrong) since it's impossible for us to order the indexes correctly ahead of time in a wide row.
> Pig should handle wide rows
> ---------------------------
>
> Key: CASSANDRA-3909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3909
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1.1
>
> Attachments: 3909.txt
>
>
> Pig should be able to use the wide row support in CFIF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3909) Pig should handle wide rows
Posted by "Matthew F. Dennis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255956#comment-13255956 ]
Matthew F. Dennis commented on CASSANDRA-3909:
----------------------------------------------
+1 on inclusion in 1.1.0 (and if not, ASAP after 1.1.0)
> Pig should handle wide rows
> ---------------------------
>
> Key: CASSANDRA-3909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3909
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1.1
>
> Attachments: 3909.txt
>
>
> Pig should be able to use the wide row support in CFIF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3909) Pig should handle wide rows
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255954#comment-13255954 ]
Brandon Williams commented on CASSANDRA-3909:
---------------------------------------------
Sylvain, any reason we can't put this in 1.1.0? It has to be explicitly enabled so it can't break anything existing, and it goes well with the hadoop wide row support we already put in 1.1.0.
> Pig should handle wide rows
> ---------------------------
>
> Key: CASSANDRA-3909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3909
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1.1
>
> Attachments: 3909.txt
>
>
> Pig should be able to use the wide row support in CFIF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3909) Pig should handle wide rows
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3909:
----------------------------------------
Attachment: 3909.txt
CASSANDRA-3264 (and subsequently CASSANDRA-3883) added wide row support to hadoop, by returning one column in the row in every call. Pig, however, is fancy enough that it could handle a wide row in a bag, since bags spill to disk; it just needs the pagination to for transport since thrift doesn't stream. Also, if we returned what CFIF gave us, a user wanting to work within the row would need another costly M/R job to join the row back to its original state, so we essentially need to 'undo' the pagination and rebuild the row as a bag. This patch does that, with the caveat that you cannot access any indexes (and frankly if you have indexes on a wide row you're probably doing something wrong) since it's impossible for us to order the indexes correctly ahead of time in a wide row.
> Pig should handle wide rows
> ---------------------------
>
> Key: CASSANDRA-3909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3909
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1.1
>
> Attachments: 3909.txt
>
>
> Pig should be able to use the wide row support in CFIF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3909) Pig should handle wide rows
Posted by "Brandon Williams (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams resolved CASSANDRA-3909.
-----------------------------------------
Resolution: Fixed
Committed.
> Pig should handle wide rows
> ---------------------------
>
> Key: CASSANDRA-3909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3909
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1.1
>
> Attachments: 3909.txt
>
>
> Pig should be able to use the wide row support in CFIF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3909) Pig should handle wide rows
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257556#comment-13257556 ]
Brandon Williams commented on CASSANDRA-3909:
---------------------------------------------
bq. personally I do trust you on that "this can't break anything"
<3
bq. I do however think that in general there would be some merit to stick to more strict rules.
I agree, however my reasoning is thus: if we support wide rows in 1.1.0 (and we do) then why not pig?
> Pig should handle wide rows
> ---------------------------
>
> Key: CASSANDRA-3909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3909
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1.1
>
> Attachments: 3909.txt
>
>
> Pig should be able to use the wide row support in CFIF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3909) Pig should handle wide rows
Posted by "Pavel Yaskevich (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255134#comment-13255134 ]
Pavel Yaskevich commented on CASSANDRA-3909:
--------------------------------------------
+1
> Pig should handle wide rows
> ---------------------------
>
> Key: CASSANDRA-3909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3909
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1.1
>
> Attachments: 3909.txt
>
>
> Pig should be able to use the wide row support in CFIF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira