You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brandon Williams (Created) (JIRA)" <ji...@apache.org> on 2012/02/01 21:19:02 UTC
[jira] [Created] (CASSANDRA-3826) Pig cannot use output formats
other than CFOF
Pig cannot use output formats other than CFOF
---------------------------------------------
Key: CASSANDRA-3826
URL: https://issues.apache.org/jira/browse/CASSANDRA-3826
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 1.0.8
Pig has ColumnFamilyOutputFormat hard coded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3826) Pig cannot use output formats
other than CFOF
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3826:
----------------------------------------
Attachment: (was: 3826-trunk.txt)
> Pig cannot use output formats other than CFOF
> ---------------------------------------------
>
> Key: CASSANDRA-3826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3826
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.0.8
>
>
> Pig has ColumnFamilyOutputFormat hard coded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3826) Pig cannot use output formats
other than CFOF
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3826:
----------------------------------------
Attachment: 3826-trunk.txt
3826.txt
Patch to keep CFIF/OF as defaults, but allow overriding both with environment variables.
> Pig cannot use output formats other than CFOF
> ---------------------------------------------
>
> Key: CASSANDRA-3826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3826
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.0.8
>
> Attachments: 3826-trunk.txt, 3826.txt
>
>
> Pig has ColumnFamilyOutputFormat hard coded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3826) Pig cannot use output formats
other than CFOF
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3826:
----------------------------------------
Reviewer: xedin
> Pig cannot use output formats other than CFOF
> ---------------------------------------------
>
> Key: CASSANDRA-3826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3826
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.0.8
>
> Attachments: 3826-trunk.txt, 3826.txt
>
>
> Pig has ColumnFamilyOutputFormat hard coded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3826) Pig cannot use output formats
other than CFOF
Posted by "Pavel Yaskevich (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198182#comment-13198182 ]
Pavel Yaskevich commented on CASSANDRA-3826:
--------------------------------------------
I guess we better do `if (format.contains("."))` at the time when {input,output}format is set instead of getter methods? I also can suggest to make "org.apache.cassandra.hadoop.ColumnFamilyInputFormat" and "org.apache.cassandra.hadoop.ColumnFamilyOutputFormat" as DEFAULT_{INPUT, OUTPUT}_FORMAT and just set them to {input,output}format variables when user didn't give any by System.env(...), what do you think?
> Pig cannot use output formats other than CFOF
> ---------------------------------------------
>
> Key: CASSANDRA-3826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3826
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.0.8
>
> Attachments: 3826-trunk.txt, 3826.txt
>
>
> Pig has ColumnFamilyOutputFormat hard coded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3826) Pig cannot use output formats
other than CFOF
Posted by "Pavel Yaskevich (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199087#comment-13199087 ]
Pavel Yaskevich commented on CASSANDRA-3826:
--------------------------------------------
+1 with following nit:
{code}
private String getFullyQualifiedClassName(String classname)
{
String fqcn = classname.contains(".") ? classname : "org.apache.cassandra.hadoop." + classname;
return fqcn;
}
{code}
can be changed to
{code}
private String getFullyQualifiedClassName(String classname)
{
return classname.contains(".") ? classname : "org.apache.cassandra.hadoop." + classname;
}
{code}
> Pig cannot use output formats other than CFOF
> ---------------------------------------------
>
> Key: CASSANDRA-3826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3826
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1
>
> Attachments: 3826.txt
>
>
> Pig has ColumnFamilyOutputFormat hard coded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3826) Pig cannot use output formats
other than CFOF
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3826:
----------------------------------------
Attachment: (was: 3826.txt)
> Pig cannot use output formats other than CFOF
> ---------------------------------------------
>
> Key: CASSANDRA-3826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3826
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.0.8
>
>
> Pig has ColumnFamilyOutputFormat hard coded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3826) Pig cannot use output formats
other than CFOF
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3826:
----------------------------------------
Fix Version/s: (was: 1.0.8)
1.1
> Pig cannot use output formats other than CFOF
> ---------------------------------------------
>
> Key: CASSANDRA-3826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3826
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1
>
> Attachments: 3826.txt
>
>
> Pig has ColumnFamilyOutputFormat hard coded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3826) Pig cannot use output formats
other than CFOF
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3826:
----------------------------------------
Attachment: 3826.txt
Updated patch. I wanted to do some of this in ConfigHelper, but it doesn't really make sense because we can't cleanly return instance like getOutputPartitioner does since we have both the old and new hadoop interfaces to comply with, but also because Configuration is not how hadoop determines the input/output formats, those are set on the Job directly. So it's probably best to keep this pig-specific, since other M/R jobs can already control these classes that way.
> Pig cannot use output formats other than CFOF
> ---------------------------------------------
>
> Key: CASSANDRA-3826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3826
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.0.8
>
> Attachments: 3826.txt
>
>
> Pig has ColumnFamilyOutputFormat hard coded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3826) Pig cannot use output formats
other than CFOF
Posted by "Brandon Williams (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams resolved CASSANDRA-3826.
-----------------------------------------
Resolution: Fixed
Committed w/ternary change.
> Pig cannot use output formats other than CFOF
> ---------------------------------------------
>
> Key: CASSANDRA-3826
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3826
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.1
>
> Attachments: 3826.txt
>
>
> Pig has ColumnFamilyOutputFormat hard coded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira