You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2020/04/07 11:05:00 UTC
[jira] [Commented] (HADOOP-16956) Tencent Cloud COS: Package it
into hadoop-dist
[ https://issues.apache.org/jira/browse/HADOOP-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077120#comment-17077120 ]
Steve Loughran commented on HADOOP-16956:
-----------------------------------------
it should end up in share/hadoop/tools/lib/ ; add it to hadoop-tools/hadoop-tools-dist/pom.xml
downstream releases of hadoop often pull cloud modules up from there to hadoop common lib as needed (e.g HDI does this for azure)
You should also add it as a dependency of hadoop-cloud-storage-project/hadoop-cloud-storage/pom.xml
That pom is intended to export all dependencies for cloud storage *excluding anything which comes from hadoop-common*; it is picked up by applications like spark, hive, druid so ends up in their packages/builds (usually some extra build options to get the cloud stuff in; look at the poms for the projects individually)
> Tencent Cloud COS: Package it into hadoop-dist
> ----------------------------------------------
>
> Key: HADOOP-16956
> URL: https://issues.apache.org/jira/browse/HADOOP-16956
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/cos
> Affects Versions: 3.3.0
> Reporter: YangY
> Assignee: YangY
> Priority: Major
>
> [~stevel@apache.org] Hi, Steve, the Hadoop-COS has been submitted to the trunk branch last year. (Related jira: https://issues.apache.org/jira/browse/HADOOP-15616).
> I am the contributor and the maintainer of this patch. Considering that you are the expert on the object storage support in Hadoop, I have some questions for you.
> # The Hadoop-cos patch is placed in the *hadoop-cloud-storage-project directory. How to package it into the hadoop-dist directory as the distribution package?*
> # How to manage the dependencies of the Hadoop-cos, such as cos-java-sdk? I use the maven to compile and package it, but only the hadoop-cos-3.4.0-SNAPSHOT.jar is generated. I compare the pom.xml between the hadoop-cos and hadoop-oss, and find the dependency scope is '*compile*' in their pom.xml. The hadoop-oss has oss-java-sdk.jar in dist directory, but the hadoop-cos does not.
> Can you guide me how to solve these two problems?
>
> Thanks for your time and attention. :)
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org