You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Chetan (JIRA)" <ji...@apache.org> on 2011/08/03 23:27:27 UTC
[jira] [Created] (PIG-2203) Pig does not allow loading resources
with the same name
Pig does not allow loading resources with the same name
-------------------------------------------------------
Key: PIG-2203
URL: https://issues.apache.org/jira/browse/PIG-2203
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.8.1
Reporter: Chetan
I am trying to load some resources in my UDF using MyClass.getClassLoader().getResources("myResource"). I have a bunch of different files called "myResource" which are scattered across my library files. In a non-Pig invocation, the above method returns an Enumeration containing all the resource files called "myResource". When I call this from a UDF, I can see only that the returned Enumeration only contains a single "myResource". It looks like Pig's JarManager is combining all the jar's in the class path into a single jar and the last entry wins. This kind of code and resource layout is used for loading resource bundles in i18n/l10n. In my opinion, at the very least, Pig shouldn't merge out resource files without a warning. Also why should jar files have to be merged like that? Can they not be added to Hadoop's distributed cache individually?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2203) Pig does not allow loading resources
with the same name
Posted by "Chetan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13079106#comment-13079106 ]
Chetan commented on PIG-2203:
-----------------------------
Sounds like PIG-2010 is the right approach. This issue can then be closed.
> Pig does not allow loading resources with the same name
> -------------------------------------------------------
>
> Key: PIG-2203
> URL: https://issues.apache.org/jira/browse/PIG-2203
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.1
> Reporter: Chetan
>
> I am trying to load some resources in my UDF using MyClass.getClassLoader().getResources("myResource"). I have a bunch of different files called "myResource" which are scattered across my library files. In a non-Pig invocation, the above method returns an Enumeration containing all the resource files called "myResource". When I call this from a UDF, I can see only that the returned Enumeration only contains a single "myResource". It looks like Pig's JarManager is combining all the jar's in the class path into a single jar and the last entry wins. This kind of code and resource layout is used for loading resource bundles in i18n/l10n. In my opinion, at the very least, Pig shouldn't merge out resource files without a warning. Also why should jar files have to be merged like that? Can they not be added to Hadoop's distributed cache individually?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-2203) Pig does not allow loading resources
with the same name
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates resolved PIG-2203.
-----------------------------
Resolution: Duplicate
Duplicates PIG-2010
> Pig does not allow loading resources with the same name
> -------------------------------------------------------
>
> Key: PIG-2203
> URL: https://issues.apache.org/jira/browse/PIG-2203
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.1
> Reporter: Chetan
>
> I am trying to load some resources in my UDF using MyClass.getClassLoader().getResources("myResource"). I have a bunch of different files called "myResource" which are scattered across my library files. In a non-Pig invocation, the above method returns an Enumeration containing all the resource files called "myResource". When I call this from a UDF, I can see only that the returned Enumeration only contains a single "myResource". It looks like Pig's JarManager is combining all the jar's in the class path into a single jar and the last entry wins. This kind of code and resource layout is used for loading resource bundles in i18n/l10n. In my opinion, at the very least, Pig shouldn't merge out resource files without a warning. Also why should jar files have to be merged like that? Can they not be added to Hadoop's distributed cache individually?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2203) Pig does not allow loading resources
with the same name
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13079093#comment-13079093 ]
Dmitriy V. Ryaboy commented on PIG-2203:
----------------------------------------
There's a patch for the dist cache issue: https://issues.apache.org/jira/browse/PIG-2010
Sadly there is an issue with Hadoop 0.20.2 that prevents this from being turned on by default; we need to add a property to control this behavior. It's a nice performance win on job startup, too.
> Pig does not allow loading resources with the same name
> -------------------------------------------------------
>
> Key: PIG-2203
> URL: https://issues.apache.org/jira/browse/PIG-2203
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.1
> Reporter: Chetan
>
> I am trying to load some resources in my UDF using MyClass.getClassLoader().getResources("myResource"). I have a bunch of different files called "myResource" which are scattered across my library files. In a non-Pig invocation, the above method returns an Enumeration containing all the resource files called "myResource". When I call this from a UDF, I can see only that the returned Enumeration only contains a single "myResource". It looks like Pig's JarManager is combining all the jar's in the class path into a single jar and the last entry wins. This kind of code and resource layout is used for loading resource bundles in i18n/l10n. In my opinion, at the very least, Pig shouldn't merge out resource files without a warning. Also why should jar files have to be merged like that? Can they not be added to Hadoop's distributed cache individually?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira