You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Venkatesh Seetharam (JIRA)" <ji...@apache.org> on 2012/11/01 22:08:12 UTC
[jira] [Created] (OOZIE-1048) Enable propagation of native
libraries as a VM argument using java.library.path
Venkatesh Seetharam created OOZIE-1048:
------------------------------------------
Summary: Enable propagation of native libraries as a VM argument using java.library.path
Key: OOZIE-1048
URL: https://issues.apache.org/jira/browse/OOZIE-1048
Project: Oozie
Issue Type: Bug
Components: scripts
Affects Versions: 3.2.0
Reporter: Venkatesh Seetharam
Assignee: Venkatesh Seetharam
Fix For: 3.3.0
Attachments: OOZIE-1048.patch.txt
If compression codecs are configured in hadoop core-site.xml, the codecs are loaded when splits are computed. Hence this requires configuring the required codecs in java.library.path and pass it as a VM argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1048) Enable propagation of native
libraries as a VM argument using java.library.path
Posted by "Venkatesh Seetharam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Venkatesh Seetharam updated OOZIE-1048:
---------------------------------------
Attachment: OOZIE-1048.patch.txt
> Enable propagation of native libraries as a VM argument using java.library.path
> -------------------------------------------------------------------------------
>
> Key: OOZIE-1048
> URL: https://issues.apache.org/jira/browse/OOZIE-1048
> Project: Oozie
> Issue Type: Bug
> Components: scripts
> Affects Versions: 3.2.0
> Reporter: Venkatesh Seetharam
> Assignee: Venkatesh Seetharam
> Fix For: 3.3.0
>
> Attachments: OOZIE-1048.patch.txt
>
>
> If compression codecs are configured in hadoop core-site.xml, the codecs are loaded when splits are computed. Hence this requires configuring the required codecs in java.library.path and pass it as a VM argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1048) Enable propagation of native
libraries as a VM argument using java.library.path
Posted by "Venkatesh Seetharam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489064#comment-13489064 ]
Venkatesh Seetharam commented on OOZIE-1048:
--------------------------------------------
The problem I faced is when using core-site.xml having the io.compression.codecs configured. Specifically, TextInputFormat's configure method finds the codecs specified in the config value io.compression.codecs and instantiates them which I think is not necessary.
Things work if I comment out the compression codecs in core-site.xml but than I'm having to maintain a separate copy for oozie.
Makes sense?
> Enable propagation of native libraries as a VM argument using java.library.path
> -------------------------------------------------------------------------------
>
> Key: OOZIE-1048
> URL: https://issues.apache.org/jira/browse/OOZIE-1048
> Project: Oozie
> Issue Type: Bug
> Components: scripts
> Affects Versions: 3.2.0
> Reporter: Venkatesh Seetharam
> Assignee: Venkatesh Seetharam
> Fix For: 3.3.0
>
> Attachments: OOZIE-1048.patch.txt
>
>
> If compression codecs are configured in hadoop core-site.xml, the codecs are loaded when splits are computed. Hence this requires configuring the required codecs in java.library.path and pass it as a VM argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1048) Enable propagation of native
libraries as a VM argument using java.library.path
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489073#comment-13489073 ]
Hadoop QA commented on OOZIE-1048:
----------------------------------
Testing JIRA OOZIE-1048
Cleaning local svn workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any line longer than 132
. {color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
. Tests run: 915
. Tests failed: 1
. Tests errors: 0
. The patch failed the following testcases:
. testClientWithAnonymous(org.apache.oozie.servlet.TestAuthFilterAuthOozieClient)
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/161/
> Enable propagation of native libraries as a VM argument using java.library.path
> -------------------------------------------------------------------------------
>
> Key: OOZIE-1048
> URL: https://issues.apache.org/jira/browse/OOZIE-1048
> Project: Oozie
> Issue Type: Bug
> Components: scripts
> Affects Versions: 3.2.0
> Reporter: Venkatesh Seetharam
> Assignee: Venkatesh Seetharam
> Fix For: 3.3.0
>
> Attachments: OOZIE-1048.patch.txt
>
>
> If compression codecs are configured in hadoop core-site.xml, the codecs are loaded when splits are computed. Hence this requires configuring the required codecs in java.library.path and pass it as a VM argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1048) Enable propagation of native
libraries as a VM argument using java.library.path
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489052#comment-13489052 ]
Alejandro Abdelnur commented on OOZIE-1048:
-------------------------------------------
Venkatesh, the patch looks good. I just wonder why we need this in the oozie server, oozie always runs launchers jobs and the splits are computed on the launcher job.
> Enable propagation of native libraries as a VM argument using java.library.path
> -------------------------------------------------------------------------------
>
> Key: OOZIE-1048
> URL: https://issues.apache.org/jira/browse/OOZIE-1048
> Project: Oozie
> Issue Type: Bug
> Components: scripts
> Affects Versions: 3.2.0
> Reporter: Venkatesh Seetharam
> Assignee: Venkatesh Seetharam
> Fix For: 3.3.0
>
> Attachments: OOZIE-1048.patch.txt
>
>
> If compression codecs are configured in hadoop core-site.xml, the codecs are loaded when splits are computed. Hence this requires configuring the required codecs in java.library.path and pass it as a VM argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1048) Enable propagation of native
libraries as a VM argument using java.library.path
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13490926#comment-13490926 ]
Alejandro Abdelnur commented on OOZIE-1048:
-------------------------------------------
Venkatesh, thanks for explaining this offline to me, that the idea is to leverage the Hadoop JAVA_LIBRARY_PATH env var when avail to avoid issues when Oozie loads its Hadoop confs.
+1
> Enable propagation of native libraries as a VM argument using java.library.path
> -------------------------------------------------------------------------------
>
> Key: OOZIE-1048
> URL: https://issues.apache.org/jira/browse/OOZIE-1048
> Project: Oozie
> Issue Type: Bug
> Components: scripts
> Affects Versions: 3.2.0
> Reporter: Venkatesh Seetharam
> Assignee: Venkatesh Seetharam
> Fix For: 3.3.0
>
> Attachments: OOZIE-1048.patch.txt
>
>
> If compression codecs are configured in hadoop core-site.xml, the codecs are loaded when splits are computed. Hence this requires configuring the required codecs in java.library.path and pass it as a VM argument.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira