You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Johannes Schwenk (JIRA)" <ji...@apache.org> on 2012/07/20 18:21:33 UTC
[jira] [Created] (PIG-2832) org.apache.pig.pigunit.pig.PigServer
does not initialize udf.import.list of PigContext
Johannes Schwenk created PIG-2832:
-------------------------------------
Summary: org.apache.pig.pigunit.pig.PigServer does not initialize udf.import.list of PigContext
Key: PIG-2832
URL: https://issues.apache.org/jira/browse/PIG-2832
Project: Pig
Issue Type: Bug
Affects Versions: 0.10.0
Environment: pig-0.10.0, Hadoop 2.0.0-cdh4.0.1 on Kubuntu 12.04 64Bit.
Reporter: Johannes Schwenk
PigServer does not initialize udf.import.list.
So, if you have a pig script that uses UDFs and want to pass the udf.import.list via a property file you can do so using the -propertyFile command line to pig. But you should also be able to do it using pigunits PigServer class that already has the corresponding contructor, e.g. doing something similar to :
{code}
Properties props = new Properties();
props.load(new FileInputStream("./testdata/test.properties"));
pig = new PigServer(ExecType.LOCAL, props);
String[] params = {"data_dir=testdata"};
test = new PigTest("test.pig", params, pig, cluster);
test.assertSortedOutput("aggregated", new File("./testdata/expected.out"));
{code}
While udf.import.list is defined in test.properties and test.pig uses names of UDFs which should be resolved using that list.
This does not work!
I'd say the org.apache.pig.PigServer class is the problem. It should initialize the import list of the PigContext.
{code}
if(properties.get("udf.import.list") != null) {
PigContext.initializeImportList((String)properties.get("udf.import.list"));
}{code}
Right now this is done in org.apache.pig.Main.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2832) org.apache.pig.pigunit.pig.PigServer
does not initialize udf.import.list of PigContext
Posted by "Prashant Kommireddi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Kommireddi updated PIG-2832:
-------------------------------------
Patch Info: Patch Available
> org.apache.pig.pigunit.pig.PigServer does not initialize udf.import.list of PigContext
> --------------------------------------------------------------------------------------
>
> Key: PIG-2832
> URL: https://issues.apache.org/jira/browse/PIG-2832
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Environment: pig-0.10.0, Hadoop 2.0.0-cdh4.0.1 on Kubuntu 12.04 64Bit.
> Reporter: Johannes Schwenk
> Assignee: Prashant Kommireddi
> Attachments: PIG-2832.patch
>
>
> PigServer does not initialize udf.import.list.
> So, if you have a pig script that uses UDFs and want to pass the udf.import.list via a property file you can do so using the -propertyFile command line to pig. But you should also be able to do it using pigunits PigServer class that already has the corresponding contructor, e.g. doing something similar to :
> {code}
> Properties props = new Properties();
> props.load(new FileInputStream("./testdata/test.properties"));
> pig = new PigServer(ExecType.LOCAL, props);
> String[] params = {"data_dir=testdata"};
> test = new PigTest("test.pig", params, pig, cluster);
> test.assertSortedOutput("aggregated", new File("./testdata/expected.out"));
> {code}
> While udf.import.list is defined in test.properties and test.pig uses names of UDFs which should be resolved using that list.
> This does not work!
> I'd say the org.apache.pig.PigServer class is the problem. It should initialize the import list of the PigContext.
> {code}
> if(properties.get("udf.import.list") != null) {
> PigContext.initializeImportList((String)properties.get("udf.import.list"));
> }{code}
> Right now this is done in org.apache.pig.Main.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2832) org.apache.pig.pigunit.pig.PigServer
does not initialize udf.import.list of PigContext
Posted by "Prashant Kommireddi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482198#comment-13482198 ]
Prashant Kommireddi commented on PIG-2832:
------------------------------------------
Thanks Johannes from reporting this. It seems like there is no reason for this initialization to exist on Main (grunt) or on PigServer. This is more of a PigContext behavior.
Adding a patch that contains the necessary initialization moved to PigContext and removed from Main. PigServer benefits from this directly.
> org.apache.pig.pigunit.pig.PigServer does not initialize udf.import.list of PigContext
> --------------------------------------------------------------------------------------
>
> Key: PIG-2832
> URL: https://issues.apache.org/jira/browse/PIG-2832
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Environment: pig-0.10.0, Hadoop 2.0.0-cdh4.0.1 on Kubuntu 12.04 64Bit.
> Reporter: Johannes Schwenk
> Attachments: PIG-2832.patch
>
>
> PigServer does not initialize udf.import.list.
> So, if you have a pig script that uses UDFs and want to pass the udf.import.list via a property file you can do so using the -propertyFile command line to pig. But you should also be able to do it using pigunits PigServer class that already has the corresponding contructor, e.g. doing something similar to :
> {code}
> Properties props = new Properties();
> props.load(new FileInputStream("./testdata/test.properties"));
> pig = new PigServer(ExecType.LOCAL, props);
> String[] params = {"data_dir=testdata"};
> test = new PigTest("test.pig", params, pig, cluster);
> test.assertSortedOutput("aggregated", new File("./testdata/expected.out"));
> {code}
> While udf.import.list is defined in test.properties and test.pig uses names of UDFs which should be resolved using that list.
> This does not work!
> I'd say the org.apache.pig.PigServer class is the problem. It should initialize the import list of the PigContext.
> {code}
> if(properties.get("udf.import.list") != null) {
> PigContext.initializeImportList((String)properties.get("udf.import.list"));
> }{code}
> Right now this is done in org.apache.pig.Main.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2832) org.apache.pig.pigunit.pig.PigServer
does not initialize udf.import.list of PigContext
Posted by "Rohini Palaniswamy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485712#comment-13485712 ]
Rohini Palaniswamy commented on PIG-2832:
-----------------------------------------
+1. Will commit this soon.
> org.apache.pig.pigunit.pig.PigServer does not initialize udf.import.list of PigContext
> --------------------------------------------------------------------------------------
>
> Key: PIG-2832
> URL: https://issues.apache.org/jira/browse/PIG-2832
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Environment: pig-0.10.0, Hadoop 2.0.0-cdh4.0.1 on Kubuntu 12.04 64Bit.
> Reporter: Johannes Schwenk
> Assignee: Prashant Kommireddi
> Attachments: PIG-2832.patch
>
>
> PigServer does not initialize udf.import.list.
> So, if you have a pig script that uses UDFs and want to pass the udf.import.list via a property file you can do so using the -propertyFile command line to pig. But you should also be able to do it using pigunits PigServer class that already has the corresponding contructor, e.g. doing something similar to :
> {code}
> Properties props = new Properties();
> props.load(new FileInputStream("./testdata/test.properties"));
> pig = new PigServer(ExecType.LOCAL, props);
> String[] params = {"data_dir=testdata"};
> test = new PigTest("test.pig", params, pig, cluster);
> test.assertSortedOutput("aggregated", new File("./testdata/expected.out"));
> {code}
> While udf.import.list is defined in test.properties and test.pig uses names of UDFs which should be resolved using that list.
> This does not work!
> I'd say the org.apache.pig.PigServer class is the problem. It should initialize the import list of the PigContext.
> {code}
> if(properties.get("udf.import.list") != null) {
> PigContext.initializeImportList((String)properties.get("udf.import.list"));
> }{code}
> Right now this is done in org.apache.pig.Main.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (PIG-2832) org.apache.pig.pigunit.pig.PigServer
does not initialize udf.import.list of PigContext
Posted by "Prashant Kommireddi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Kommireddi reassigned PIG-2832:
----------------------------------------
Assignee: Prashant Kommireddi
> org.apache.pig.pigunit.pig.PigServer does not initialize udf.import.list of PigContext
> --------------------------------------------------------------------------------------
>
> Key: PIG-2832
> URL: https://issues.apache.org/jira/browse/PIG-2832
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Environment: pig-0.10.0, Hadoop 2.0.0-cdh4.0.1 on Kubuntu 12.04 64Bit.
> Reporter: Johannes Schwenk
> Assignee: Prashant Kommireddi
> Attachments: PIG-2832.patch
>
>
> PigServer does not initialize udf.import.list.
> So, if you have a pig script that uses UDFs and want to pass the udf.import.list via a property file you can do so using the -propertyFile command line to pig. But you should also be able to do it using pigunits PigServer class that already has the corresponding contructor, e.g. doing something similar to :
> {code}
> Properties props = new Properties();
> props.load(new FileInputStream("./testdata/test.properties"));
> pig = new PigServer(ExecType.LOCAL, props);
> String[] params = {"data_dir=testdata"};
> test = new PigTest("test.pig", params, pig, cluster);
> test.assertSortedOutput("aggregated", new File("./testdata/expected.out"));
> {code}
> While udf.import.list is defined in test.properties and test.pig uses names of UDFs which should be resolved using that list.
> This does not work!
> I'd say the org.apache.pig.PigServer class is the problem. It should initialize the import list of the PigContext.
> {code}
> if(properties.get("udf.import.list") != null) {
> PigContext.initializeImportList((String)properties.get("udf.import.list"));
> }{code}
> Right now this is done in org.apache.pig.Main.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2832) org.apache.pig.pigunit.pig.PigServer
does not initialize udf.import.list of PigContext
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2832:
----------------------------------
Fix Version/s: 0.11
Status: Patch Available (was: Open)
> org.apache.pig.pigunit.pig.PigServer does not initialize udf.import.list of PigContext
> --------------------------------------------------------------------------------------
>
> Key: PIG-2832
> URL: https://issues.apache.org/jira/browse/PIG-2832
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Environment: pig-0.10.0, Hadoop 2.0.0-cdh4.0.1 on Kubuntu 12.04 64Bit.
> Reporter: Johannes Schwenk
> Assignee: Prashant Kommireddi
> Fix For: 0.11
>
> Attachments: PIG-2832.patch
>
>
> PigServer does not initialize udf.import.list.
> So, if you have a pig script that uses UDFs and want to pass the udf.import.list via a property file you can do so using the -propertyFile command line to pig. But you should also be able to do it using pigunits PigServer class that already has the corresponding contructor, e.g. doing something similar to :
> {code}
> Properties props = new Properties();
> props.load(new FileInputStream("./testdata/test.properties"));
> pig = new PigServer(ExecType.LOCAL, props);
> String[] params = {"data_dir=testdata"};
> test = new PigTest("test.pig", params, pig, cluster);
> test.assertSortedOutput("aggregated", new File("./testdata/expected.out"));
> {code}
> While udf.import.list is defined in test.properties and test.pig uses names of UDFs which should be resolved using that list.
> This does not work!
> I'd say the org.apache.pig.PigServer class is the problem. It should initialize the import list of the PigContext.
> {code}
> if(properties.get("udf.import.list") != null) {
> PigContext.initializeImportList((String)properties.get("udf.import.list"));
> }{code}
> Right now this is done in org.apache.pig.Main.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2832) org.apache.pig.pigunit.pig.PigServer
does not initialize udf.import.list of PigContext
Posted by "Prashant Kommireddi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Kommireddi updated PIG-2832:
-------------------------------------
Attachment: PIG-2832.patch
> org.apache.pig.pigunit.pig.PigServer does not initialize udf.import.list of PigContext
> --------------------------------------------------------------------------------------
>
> Key: PIG-2832
> URL: https://issues.apache.org/jira/browse/PIG-2832
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Environment: pig-0.10.0, Hadoop 2.0.0-cdh4.0.1 on Kubuntu 12.04 64Bit.
> Reporter: Johannes Schwenk
> Assignee: Prashant Kommireddi
> Attachments: PIG-2832.patch
>
>
> PigServer does not initialize udf.import.list.
> So, if you have a pig script that uses UDFs and want to pass the udf.import.list via a property file you can do so using the -propertyFile command line to pig. But you should also be able to do it using pigunits PigServer class that already has the corresponding contructor, e.g. doing something similar to :
> {code}
> Properties props = new Properties();
> props.load(new FileInputStream("./testdata/test.properties"));
> pig = new PigServer(ExecType.LOCAL, props);
> String[] params = {"data_dir=testdata"};
> test = new PigTest("test.pig", params, pig, cluster);
> test.assertSortedOutput("aggregated", new File("./testdata/expected.out"));
> {code}
> While udf.import.list is defined in test.properties and test.pig uses names of UDFs which should be resolved using that list.
> This does not work!
> I'd say the org.apache.pig.PigServer class is the problem. It should initialize the import list of the PigContext.
> {code}
> if(properties.get("udf.import.list") != null) {
> PigContext.initializeImportList((String)properties.get("udf.import.list"));
> }{code}
> Right now this is done in org.apache.pig.Main.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2832) org.apache.pig.pigunit.pig.PigServer
does not initialize udf.import.list of PigContext
Posted by "Rohini Palaniswamy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rohini Palaniswamy updated PIG-2832:
------------------------------------
Resolution: Fixed
Fix Version/s: 0.12
Status: Resolved (was: Patch Available)
Committed to 0.11 and trunk. Thanks Prashant.
> org.apache.pig.pigunit.pig.PigServer does not initialize udf.import.list of PigContext
> --------------------------------------------------------------------------------------
>
> Key: PIG-2832
> URL: https://issues.apache.org/jira/browse/PIG-2832
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Environment: pig-0.10.0, Hadoop 2.0.0-cdh4.0.1 on Kubuntu 12.04 64Bit.
> Reporter: Johannes Schwenk
> Assignee: Prashant Kommireddi
> Fix For: 0.11, 0.12
>
> Attachments: PIG-2832.patch
>
>
> PigServer does not initialize udf.import.list.
> So, if you have a pig script that uses UDFs and want to pass the udf.import.list via a property file you can do so using the -propertyFile command line to pig. But you should also be able to do it using pigunits PigServer class that already has the corresponding contructor, e.g. doing something similar to :
> {code}
> Properties props = new Properties();
> props.load(new FileInputStream("./testdata/test.properties"));
> pig = new PigServer(ExecType.LOCAL, props);
> String[] params = {"data_dir=testdata"};
> test = new PigTest("test.pig", params, pig, cluster);
> test.assertSortedOutput("aggregated", new File("./testdata/expected.out"));
> {code}
> While udf.import.list is defined in test.properties and test.pig uses names of UDFs which should be resolved using that list.
> This does not work!
> I'd say the org.apache.pig.PigServer class is the problem. It should initialize the import list of the PigContext.
> {code}
> if(properties.get("udf.import.list") != null) {
> PigContext.initializeImportList((String)properties.get("udf.import.list"));
> }{code}
> Right now this is done in org.apache.pig.Main.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira