You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Roman Shaposhnik (Created) (JIRA)" <ji...@apache.org> on 2012/03/20 23:57:38 UTC
[jira] [Created] (BIGTOP-474) improve Oozie integration with Hadoop
0.23
improve Oozie integration with Hadoop 0.23
------------------------------------------
Key: BIGTOP-474
URL: https://issues.apache.org/jira/browse/BIGTOP-474
Project: Bigtop
Issue Type: Bug
Components: General
Affects Versions: 0.4.0
Reporter: Roman Shaposhnik
Assignee: Roman Shaposhnik
Fix For: 0.4.0
Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
# renaming /usr/lib/oozie/oozie-server -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
# add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
# add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
# get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
# make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
I'll attach a patch shortly. But in the meantime, please let me know what do you think.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-474) improve Oozie integration with
Hadoop 0.23
Posted by "Roman Shaposhnik (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234461#comment-13234461 ]
Roman Shaposhnik commented on BIGTOP-474:
-----------------------------------------
Patch attached. Please let me know if sed is too much. The alternative is to have a private copy of the oozie-site.xml which is not that great since oozie is changing rapidly.
> improve Oozie integration with Hadoop 0.23
> ------------------------------------------
>
> Key: BIGTOP-474
> URL: https://issues.apache.org/jira/browse/BIGTOP-474
> Project: Bigtop
> Issue Type: Bug
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.4.0
>
> Attachments: BIGTOP-474.patch.txt
>
>
> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
> # renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-474) improve Oozie integration with
Hadoop 0.23
Posted by "Alejandro Abdelnur (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234064#comment-13234064 ]
Alejandro Abdelnur commented on BIGTOP-474:
-------------------------------------------
+1 on the proposal
> improve Oozie integration with Hadoop 0.23
> ------------------------------------------
>
> Key: BIGTOP-474
> URL: https://issues.apache.org/jira/browse/BIGTOP-474
> Project: Bigtop
> Issue Type: Bug
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.4.0
>
>
> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
> # renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-474) improve Oozie integration with Hadoop
0.23
Posted by "Roman Shaposhnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik updated BIGTOP-474:
------------------------------------
Attachment: BIGTOP-474.patch.txt
> improve Oozie integration with Hadoop 0.23
> ------------------------------------------
>
> Key: BIGTOP-474
> URL: https://issues.apache.org/jira/browse/BIGTOP-474
> Project: Bigtop
> Issue Type: Bug
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.4.0
>
> Attachments: BIGTOP-474.patch.txt
>
>
> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
> # renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Updated] (BIGTOP-474) improve Oozie integration with
Hadoop 0.23
Posted by Marcos Ortiz <ml...@uci.cu>.
+1
On 03/20/2012 07:21 PM, Roman Shaposhnik (Updated) (JIRA) wrote:
> [ https://issues.apache.org/jira/browse/BIGTOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Roman Shaposhnik updated BIGTOP-474:
> ------------------------------------
>
> Description:
> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
> # renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
>
> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
>
> was:
> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
> # renaming /usr/lib/oozie/oozie-server -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
>
> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
>
>
>> improve Oozie integration with Hadoop 0.23
>> ------------------------------------------
>>
>> Key: BIGTOP-474
>> URL: https://issues.apache.org/jira/browse/BIGTOP-474
>> Project: Bigtop
>> Issue Type: Bug
>> Components: General
>> Affects Versions: 0.4.0
>> Reporter: Roman Shaposhnik
>> Assignee: Roman Shaposhnik
>> Fix For: 0.4.0
>>
>>
>> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
>> # renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
>> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
>> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
>> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
>> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
>> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>
> http://www.uci.cu
> http://www.facebook.com/universidad.uci
> http://www.flickr.com/photos/universidad_uci
--
Marcos Luis Ortíz Valmaseda (@marcosluis2186)
Data Engineer at UCI
http://marcosluis2186.posterous.com
10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
[jira] [Updated] (BIGTOP-474) improve Oozie integration with Hadoop
0.23
Posted by "Roman Shaposhnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik updated BIGTOP-474:
------------------------------------
Description:
Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
# renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
# add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
# add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
# get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
# make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
I'll attach a patch shortly. But in the meantime, please let me know what do you think.
was:
Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
# renaming /usr/lib/oozie/oozie-server -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
# add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
# add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
# get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
# make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
I'll attach a patch shortly. But in the meantime, please let me know what do you think.
> improve Oozie integration with Hadoop 0.23
> ------------------------------------------
>
> Key: BIGTOP-474
> URL: https://issues.apache.org/jira/browse/BIGTOP-474
> Project: Bigtop
> Issue Type: Bug
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.4.0
>
>
> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
> # renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (BIGTOP-474) improve Oozie integration with
Hadoop 0.23
Posted by "Roman Shaposhnik (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik resolved BIGTOP-474.
-------------------------------------
Resolution: Fixed
> improve Oozie integration with Hadoop 0.23
> ------------------------------------------
>
> Key: BIGTOP-474
> URL: https://issues.apache.org/jira/browse/BIGTOP-474
> Project: Bigtop
> Issue Type: Bug
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.4.0
>
> Attachments: BIGTOP-474.patch.txt
>
>
> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
> # renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-474) improve Oozie integration with Hadoop
0.23
Posted by "Roman Shaposhnik (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik updated BIGTOP-474:
------------------------------------
Attachment: (was: BIGTOP-474.patch.txt)
> improve Oozie integration with Hadoop 0.23
> ------------------------------------------
>
> Key: BIGTOP-474
> URL: https://issues.apache.org/jira/browse/BIGTOP-474
> Project: Bigtop
> Issue Type: Bug
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.4.0
>
> Attachments: BIGTOP-474.patch.txt
>
>
> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
> # renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-474) improve Oozie integration with
Hadoop 0.23
Posted by "Roman Shaposhnik (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235041#comment-13235041 ]
Roman Shaposhnik commented on BIGTOP-474:
-----------------------------------------
@Bruno, not really. Currently client has a flat layout -- all the jar files are at the same level but they point to all sorts of places.
> improve Oozie integration with Hadoop 0.23
> ------------------------------------------
>
> Key: BIGTOP-474
> URL: https://issues.apache.org/jira/browse/BIGTOP-474
> Project: Bigtop
> Issue Type: Bug
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.4.0
>
> Attachments: BIGTOP-474.patch.txt
>
>
> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
> # renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-474) improve Oozie integration with
Hadoop 0.23
Posted by "Bruno Mahé (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234826#comment-13234826 ]
Bruno Mahé commented on BIGTOP-474:
-----------------------------------
* Don't you need /usr/lib/hadoop/client/lib/*.jar as well in your common loader?
If not +1
> improve Oozie integration with Hadoop 0.23
> ------------------------------------------
>
> Key: BIGTOP-474
> URL: https://issues.apache.org/jira/browse/BIGTOP-474
> Project: Bigtop
> Issue Type: Bug
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.4.0
>
> Attachments: BIGTOP-474.patch.txt
>
>
> Here's a bunch of improvements to the way Oozie is packaged in Bigtop that I would like to suggest based on my conversations so far:
> # renaming /usr/lib/oozie/oozie-core -> /usr/lib/oozie/libserver (that will give us lib, libtools and libserver and is pretty consistent)
> # add /var/lib/oozie to the catalina.properties classpath so that users can simply drop jar files there (e.g. for jdbc connectors)
> # add /etc/hadoop/conf to the oozie-site.xml oozie.service.HadoopAccessorService.hadoop.configurations
> # get rid of /etc/oozie/conf/hadoop-conf (once #3 is done)
> # make sure we don't ship docs twice (once in /usr/share/docs/oozie and the other time in webapps)
> I'll attach a patch shortly. But in the meantime, please let me know what do you think.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira