You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org> on 2011/06/13 18:07:51 UTC
[jira] [Created] (PIG-2120) HBase storage does not respect
pig.hbase.caster property
HBase storage does not respect pig.hbase.caster property
--------------------------------------------------------
Key: PIG-2120
URL: https://issues.apache.org/jira/browse/PIG-2120
Project: Pig
Issue Type: Bug
Affects Versions: 0.8.1, 0.8.0, 0.9.0, 0.10
Reporter: Dmitriy V. Ryaboy
Assignee: Dmitriy V. Ryaboy
HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2120) HBase storage does not respect
pig.hbase.caster property
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048686#comment-13048686 ]
Alan Gates commented on PIG-2120:
---------------------------------
The UDFContext.setClientSystemProps() should just be changed to setClientSystemProps(Properties), rather than a second method added. If it's left in there and called with no Properties it would override the properties from the file, causing confusion for the users. This is not a public part of the API (specifically called out in the comments), so we are free to change it.
You could write an automated test to make sure that properties passed in pig.properties and on the command line still make it to the backend via UDFContext. This is an important thing to get right.
> HBase storage does not respect pig.hbase.caster property
> --------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2120) UDFContext.getClientSystemProps() does
not respect pig.properties
Posted by "Dmitriy V. Ryaboy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2120:
-----------------------------------
Fix Version/s: 0.9.2
Applied to 0.9
> UDFContext.getClientSystemProps() does not respect pig.properties
> -----------------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.10, 0.9.2
>
> Attachments: PIG-2120.2.patch, PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2120) HBase storage does not respect
pig.hbase.caster property
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2120:
-----------------------------------
Attachment: PIG-2120.patch
While fixing this, I discovered a bigger problem that's not HBaseStorage specific -- UDFContext.getUDFContext().getClientSystemProps() does NOT respect properties set in pig.properties.
The attached patch fixes both issues.
Tested manually, not sure how to write a proper unit test for this..
> HBase storage does not respect pig.hbase.caster property
> --------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2120) UDFContext.getClientSystemProps() does
not respect pig.properties
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2120:
-----------------------------------
Summary: UDFContext.getClientSystemProps() does not respect pig.properties (was: HBase storage does not respect pig.hbase.caster property)
Changing bug title to more accurately reflect the nature of the change
> UDFContext.getClientSystemProps() does not respect pig.properties
> -----------------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2120.2.patch, PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2120) HBase storage does not respect
pig.hbase.caster property
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2120:
-----------------------------------
Status: Patch Available (was: Open)
> HBase storage does not respect pig.hbase.caster property
> --------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1, 0.8.0, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2120.2.patch, PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2120) HBase storage does not respect
pig.hbase.caster property
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2120:
-----------------------------------
Attachment: PIG-2120.2.patch
Patch attached.
Switched the UDFContext tests to local mode while I was in there.
> HBase storage does not respect pig.hbase.caster property
> --------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2120.2.patch, PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2120) HBase storage does not respect
pig.hbase.caster property
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2120:
-----------------------------------
Status: Open (was: Patch Available)
> HBase storage does not respect pig.hbase.caster property
> --------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1, 0.8.0, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2120.2.patch, PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2120) HBase storage does not respect
pig.hbase.caster property
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2120:
-----------------------------------
Status: Patch Available (was: Open)
> HBase storage does not respect pig.hbase.caster property
> --------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1, 0.8.0, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2120) HBase storage does not respect
pig.hbase.caster property
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049313#comment-13049313 ]
Dmitriy V. Ryaboy commented on PIG-2120:
----------------------------------------
I'd have to shell out or something for that test -- the change is in Main.java...
Which probably points out yet another bug. Is it just me or do we need to set UDFContext clientSystemProps in PigServer? Something like
{code}
Index: src/org/apache/pig/PigServer.java
===================================================================
--- src/org/apache/pig/PigServer.java (revision 1135694)
+++ src/org/apache/pig/PigServer.java (working copy)
@@ -1632,6 +1632,8 @@
*/
private void parseQuery() throws FrontendException {
UDFContext.getUDFContext().reset();
+ UDFContext.getUDFContext().setClientSystemProps(pigContext.getProperties());
+
String query = buildQuery();
if( query.isEmpty() ) {
{code}
> HBase storage does not respect pig.hbase.caster property
> --------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2120) UDFContext.getClientSystemProps() does
not respect pig.properties
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-2120:
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.10
Status: Resolved (was: Patch Available)
Fixed in trunk.
Should I apply this to 0.9?
> UDFContext.getClientSystemProps() does not respect pig.properties
> -----------------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.10
>
> Attachments: PIG-2120.2.patch, PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2120) HBase storage does not respect
pig.hbase.caster property
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13052121#comment-13052121 ]
Alan Gates commented on PIG-2120:
---------------------------------
+1
> HBase storage does not respect pig.hbase.caster property
> --------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2120.2.patch, PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2120) UDFContext.getClientSystemProps()
does not respect pig.properties
Posted by "Dmitriy V. Ryaboy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182822#comment-13182822 ]
Dmitriy V. Ryaboy commented on PIG-2120:
----------------------------------------
Answer to self: yes, you should.
> UDFContext.getClientSystemProps() does not respect pig.properties
> -----------------------------------------------------------------
>
> Key: PIG-2120
> URL: https://issues.apache.org/jira/browse/PIG-2120
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Fix For: 0.10
>
> Attachments: PIG-2120.2.patch, PIG-2120.patch
>
>
> HBaseStorage is supposed to allow changing the default Caster via the pig.hbase.caster property. This does not work.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira