You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Matthew Hayes (JIRA)" <ji...@apache.org> on 2012/07/31 00:26:34 UTC
[jira] [Created] (PIG-2850) Pig should support loading macro files
as resources stored in JAR files
Matthew Hayes created PIG-2850:
----------------------------------
Summary: Pig should support loading macro files as resources stored in JAR files
Key: PIG-2850
URL: https://issues.apache.org/jira/browse/PIG-2850
Project: Pig
Issue Type: Improvement
Reporter: Matthew Hayes
Priority: Minor
Attachments: import_macros_from_jar.diff
A file containing macros can be imported in pig like so:
{code}
IMPORT 'some_path/my_macros.pig';
{code}
It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
{code}
REGISTER my_macros.jar;
IMPORT 'some_path/my_macros.pig';
{code}
Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2850) Pig should support loading macro
files as resources stored in JAR files
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440764#comment-13440764 ]
Dmitriy V. Ryaboy commented on PIG-2850:
----------------------------------------
Looks good. In fetchResource, you check if resourceStream is not null; if it is null, the whole function will return null. I suspect that would be surprising and cause NPEs downstream. Perhaps an exception-throwing assert would be better here? Is there a legit reason for this function to ever return null instead of throwing?
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-2850) Pig should support loading macro files
as resources stored in JAR files
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy resolved PIG-2850.
------------------------------------
Resolution: Fixed
Fix Version/s: 0.11
Committed to trunk.
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Assignee: Matthew Hayes
> Priority: Minor
> Fix For: 0.11
>
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar_4.diff, import_macros_from_jar_5.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2850) Pig should support loading macro
files as resources stored in JAR files
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425974#comment-13425974 ]
Dmitriy V. Ryaboy commented on PIG-2850:
----------------------------------------
Nice!
couple of quick notes:
most of the LOG.info calls should be LOG.debug
OutputStream should probably be buffered
The test involves creating a jar, writing it out to FS, and registering it.. how about we just check in a jar in test/resources ?
It's a little weird to be writing out to a temp file.. would be cleaner to allow pig to pick up from a jar without intermediate storage. Ideas on how to do that?
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2850) Pig should support loading macro
files as resources stored in JAR files
Posted by "Matthew Hayes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440488#comment-13440488 ]
Matthew Hayes commented on PIG-2850:
------------------------------------
Is there anything else necessary for me to change?
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2850) Pig should support loading macro
files as resources stored in JAR files
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440890#comment-13440890 ]
Dmitriy V. Ryaboy commented on PIG-2850:
----------------------------------------
Fails to compile. Did you forget to add ResourceNotFoundException ?
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar_4.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2850) Pig should support loading macro
files as resources stored in JAR files
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440628#comment-13440628 ]
Dmitriy V. Ryaboy commented on PIG-2850:
----------------------------------------
Sorry, didn't notice you updated. Will review.
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2850) Pig should support loading macro
files as resources stored in JAR files
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440862#comment-13440862 ]
Dmitriy V. Ryaboy commented on PIG-2850:
----------------------------------------
Ok.. it's actually a legit conflict, not just a merge issue. This code was changed in PIG-2866. I'll ask Bill to take a look.
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2850) Pig should support loading macro files
as resources stored in JAR files
Posted by "Matthew Hayes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew Hayes updated PIG-2850:
-------------------------------
Attachment: import_macros_from_jar.diff
Sample implementation with unit test
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2850) Pig should support loading macro files
as resources stored in JAR files
Posted by "Matthew Hayes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew Hayes updated PIG-2850:
-------------------------------
Attachment: import_macros_from_jar_5.diff
yep i did, sorry, now fixed
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar_4.diff, import_macros_from_jar_5.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2850) Pig should support loading macro
files as resources stored in JAR files
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440853#comment-13440853 ]
Dmitriy V. Ryaboy commented on PIG-2850:
----------------------------------------
Perfect, thanks for the explanation. I like patch 3, will commit.
Sorry about the slow turnaround, feel free to ping if there are no comments for more than a few days next time.. we shouldn't let tickets languish like that.
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2850) Pig should support loading macro
files as resources stored in JAR files
Posted by "Matthew Hayes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440824#comment-13440824 ]
Matthew Hayes commented on PIG-2850:
------------------------------------
The function could legitimately return null if the macro file is not found as a file or a resource. I thought an error about a resource not being found could be cryptic if the user isn't expecting this behavior when importing the macro file. So I checked for null in getMacroFile after attempting to fetch as a resource and then threw and exception about the file not being found.
Alternatively I could throw a ResourceNotFoundException in fetchResource and catch this in getMacroFile. See diff #3 where I am doing this instead.
Thanks!
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2850) Pig should support loading macro
files as resources stored in JAR files
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440857#comment-13440857 ]
Dmitriy V. Ryaboy commented on PIG-2850:
----------------------------------------
Patch isn't applying cleanly. I'll remediate and post rebased patch.
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2850) Pig should support loading macro files
as resources stored in JAR files
Posted by "Matthew Hayes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew Hayes updated PIG-2850:
-------------------------------
Attachment: import_macros_from_jar_3.diff
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2850) Pig should support loading macro files
as resources stored in JAR files
Posted by "Matthew Hayes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew Hayes updated PIG-2850:
-------------------------------
Attachment: import_macros_from_jar_4.diff
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar_4.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2850) Pig should support loading macro files
as resources stored in JAR files
Posted by "Matthew Hayes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew Hayes updated PIG-2850:
-------------------------------
Attachment: import_macros_from_jar_2.diff
Thanks for the feedback :)
Replaced LOG.info with LOG.debug.
Wrapped BufferedOutputStream around stream.
For the test, I like creating the JAR in the test so the test is totally self contained. You don't need to refer to a file in a separate JAR to understand it. What do you think?
Regarding writing to a temp file, I did it this way to mimic what happens with FileLocalizer. Other parts of QueryParserDriver are assuming the macro file is local and I didn't want to make significant changes. In this new diff I've moved the code to FileLocalizer.fetchResource(), which seems cleaner.
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar.diff, import_macros_from_jar_2.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2850) Pig should support loading macro
files as resources stored in JAR files
Posted by "Matthew Hayes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440876#comment-13440876 ]
Matthew Hayes commented on PIG-2850:
------------------------------------
Oops I should have done an svn up. I resolved the conflicts and submitted patch #4.
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar_4.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (PIG-2850) Pig should support loading macro files
as resources stored in JAR files
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy reassigned PIG-2850:
--------------------------------------
Assignee: Matthew Hayes
> Pig should support loading macro files as resources stored in JAR files
> -----------------------------------------------------------------------
>
> Key: PIG-2850
> URL: https://issues.apache.org/jira/browse/PIG-2850
> Project: Pig
> Issue Type: Improvement
> Reporter: Matthew Hayes
> Assignee: Matthew Hayes
> Priority: Minor
> Attachments: import_macros_from_jar_2.diff, import_macros_from_jar_3.diff, import_macros_from_jar_4.diff, import_macros_from_jar_5.diff, import_macros_from_jar.diff
>
>
> A file containing macros can be imported in pig like so:
> {code}
> IMPORT 'some_path/my_macros.pig';
> {code}
> It would be convenient if a macro file could be imported from a registered JAR as well. This would make it easier to distribute them. One could package a set of UDFs and macros in a single JAR. Once the JAR is registered any of the UDFs or macros can be used once.
> For example, support that {{some_path/my_macros.pig}} has been packaged in a JAR named {{my_macros.jar}}. The above code then becomes
> {code}
> REGISTER my_macros.jar;
> IMPORT 'some_path/my_macros.pig';
> {code}
> Pig would first check if the file is found at the path {{some_path/my_macros.pig}}, and failing that it would attempt to load a resource by that name. Since the JAR is registered it will find it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira