You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Bharat Viswanadham (JIRA)" <ji...@apache.org> on 2018/09/14 01:01:00 UTC
[jira] [Comment Edited] (HDDS-446) Provide shaded artifact to start
ozone service as a datanode plugin
[ https://issues.apache.org/jira/browse/HDDS-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614201#comment-16614201 ]
Bharat Viswanadham edited comment on HDDS-446 at 9/14/18 1:00 AM:
------------------------------------------------------------------
Hi [~elek]
Thanks for providing the patch.
I have few questions in trying to understand this patch. As I have not got complete context, just trying to understand the purpose of this patch.
# With this patch, now we can copy the hadoop-ozone-datanode-plugin.jar to hdfs cluster and make ozone work? (But still we require other ozone jars). So, what will be the advantage of this?
# In the patch in docker-compose, we have started datanode as hdfs datanode, and added org.apache.hadoop.ozone.web.OzoneHddsDatanodeService. So, this is for testing ozone running as a datanode plugin only right?
# And in pom.xml Line:87, instead of mentioning version 3.1.1, we can use maven-shade-plugin.version from hadoop-project pom.xml, but that is specified too old, 2.4.3, if we want we change this one.
# And in ozone-dist-layout-stitching, we are copying the hadoop-ozone-objectstore-service-${HDDS_VERSION}-plugin.jar as hadoop-ozone-datanode-plugin.jar with out any version. Is this intentional change, as I see same for hadoop-ozone-filesystem.jar also. If so, why this is done?
# This one is not related to this patch in ozone-dist-layout-stitching,
## why these empty folders needs to be created mkdir -p ./share/hadoop/mapreduce, mkdir -p ./share/hadoop/yarn and mkdir -p ./share/hadoop/hdfs ?
## Line 170: echo "Hadoop Ozone dist layout available at: ${BASEDIR}/ozone", I think this one should be changed to "Hadoop Ozone dist layout available at: ${BASEDIR}/ozone-${HDDS_VERSION}"
was (Author: bharatviswa):
Hi [~elek]
Thanks for providing the patch.
I have few questions in trying to understand this patch. As I have not got complete context, just trying to understand the purpose of this patch.
# With this patch, now we can copy the hadoop-ozone-datanode-plugin.jar to hdfs cluster and make ozone work? (But still we require other ozone jars). So, what will be the advantage of this?
# In the patch in docker-compose, we have started datanode as hdfs datanode, and added org.apache.hadoop.ozone.web.OzoneHddsDatanodeService. So, this is for testing ozone running as a datanode plugin only right?
# And in pom.xml Line:87, instead of mentioning version 3.1.1, we can use maven-shade-plugin.version from hadoop-project pom.xml, but that is specified too old, 2.4.3, if we want we change this one.
# And in ozone-dist-layout-stitching, we are copying the hadoop-ozone-objectstore-service-${HDDS_VERSION}-plugin.jar as hadoop-ozone-datanode-plugin.jar with out any version. Is this intentional change, as I see same for hadoop-ozone-filesystem.jar also. If so, why this is done?
> Provide shaded artifact to start ozone service as a datanode plugin
> -------------------------------------------------------------------
>
> Key: HDDS-446
> URL: https://issues.apache.org/jira/browse/HDDS-446
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Elek, Marton
> Assignee: Elek, Marton
> Priority: Major
> Fix For: 0.2.1
>
> Attachments: HDDS-446-ozone-0.2.001.patch
>
>
> Ozone datanode service could be started in two different way:
> # as a standalone process
> # as a datanode plugin
> We tested the second scenarion by compiling both ozone + hadoop trunk together and created a combined artifact. But we have no more full hadoop distribution in the ozone release package any more. And we had no answer how it could be started with an existing hadoop release.
> We need a well defined way to add the datanode-service to the classpath of the hadoop datanode.
> I propose here to create a lightweight shaded artifact (and here the shading only combines all the classes together without package name refactoring) to make it easier to extend an existing hadoop installation.
> In this patch I add the shade plugin execution to the ozone objectstore-service and the shaded file will be copied to the distribution.
> A new docker-compose based execution environment (ozone-hdfs) demonstrates/tests how it could work.
> Tested with hadoop 3.1.0 and datanode service could be started without any problem
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org