You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Siyao Meng (Jira)" <ji...@apache.org> on 2019/12/16 23:24:00 UTC
[jira] [Comment Edited] (HDDS-2709) Maven property skipShade should
not skip ozonefs compilation
[ https://issues.apache.org/jira/browse/HDDS-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997700#comment-16997700 ]
Siyao Meng edited comment on HDDS-2709 at 12/16/19 11:23 PM:
-------------------------------------------------------------
[~aengineer] Sorry I missed your comment earlier.
bq. The fix is to compile without skipShade, if you want to compile modules that depend on shading.
ozonefs doesn't really depend on shading (as I tested, check out my [comment|https://github.com/apache/hadoop-ozone/pull/342#issuecomment-565647833] in PR). So with {{skipShade}} to compile ONLY {{ozonefs}} but not the shaded FS jars I do save some time.
I also tested that: with the patch and {{-DskipShade}}, the resulting distribution won't contain {{hadoop-ozone-filesystem-lib-current}} or {{hadoop-ozone-filesystem-lib-legacy}} (as expected). And all functions still work (in docker-compose cluster).
The current behavior (without the patch) is: with {{-DskipShade}}, it skips compiling ozonefs module, but it would pick up the jar ({{hadoop-ozone-filesystem-0.5.0-SNAPSHOT.jar}}) from local maven cache (or from Apache repo if local cache expires or doesn't exist) when building {{dist}}.
{code:title=Build Ozone dist from fresh}
mv ~/.m2{,.bak}
git clone git@github.com:apache/hadoop-ozone.git ozone-fresh
cd ozone-fresh
mvn clean install -Pdist -DskipTests -e -Dmaven.javadoc.skip=true -DskipRecon -DskipShade
{code}
So a confusing fact I discovered (before I submit this patch) is that, I first compile it with all the shading stuff, later I made some change in ozonefs and decide to speed it up with {{-DskipShade}}. The resulting dist would actually contain the *previously compiled jar*. This is the "unexpected" behavior I am going after. As in Hadoop, {{skipShade}} would still compile the non-shaded client (and skip the shaded client jar, etc. to save time). I would want {{skipShade}} to at least have the same semantic.
Thanks.
was (Author: smeng):
[~aengineer] Sorry I missed your comment earlier.
?? The fix is to compile without skipShade, if you want to compile modules that depend on shading. ??
ozonefs doesn't really depend on shading (as I tested, check out my [comment|https://github.com/apache/hadoop-ozone/pull/342#issuecomment-565647833] in PR). So with {{skipShade}} to compile ONLY {{ozonefs}} but not the shaded FS jars I do save some time.
I also tested that: with the patch and {{-DskipShade}}, the resulting distribution won't contain {{hadoop-ozone-filesystem-lib-current}} or {{hadoop-ozone-filesystem-lib-legacy}} (as expected). And all functions still work (in docker-compose cluster).
The current behavior (without the patch) is: with {{-DskipShade}}, it skips compiling ozonefs module, but it would pick up the jar ({{hadoop-ozone-filesystem-0.5.0-SNAPSHOT.jar}}) from local maven cache (or from Apache repo if local cache expires or doesn't exist) when building {{dist}}.
{code:title=Build Ozone dist from fresh}
mv ~/.m2{,.bak}
git clone git@github.com:apache/hadoop-ozone.git ozone-fresh
cd ozone-fresh
mvn clean install -Pdist -DskipTests -e -Dmaven.javadoc.skip=true -DskipRecon -DskipShade
{code}
So a confusing fact I discovered (before I submit this patch) is that, I first compile it with all the shading stuff, later I made some change in ozonefs and decide to speed it up with {{-DskipShade}}. The resulting dist would actually contain the *previously compiled jar*. This is the "unexpected" behavior I am going after. As in Hadoop, {{skipShade}} would still compile the non-shaded client (and skip the shaded client jar, etc. to save time). I would want {{skipShade}} to have the at least same semantic.
Thanks.
> Maven property skipShade should not skip ozonefs compilation
> ------------------------------------------------------------
>
> Key: HDDS-2709
> URL: https://issues.apache.org/jira/browse/HDDS-2709
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, if {{-DskipShade}} is specified when running {{mvn}}, it will skip {{ozonefs}} module ({{hadoop-ozone-filesystem}} / Apache Hadoop Ozone FileSystem) compilation:
> {code:xml|title=hadoop-ozone/pom.xml}
> <profile>
> <id>build-with-ozonefs</id>
> <activation>
> <property>
> <name>!skipShade</name>
> </property>
> </activation>
> <modules>
> <module>ozonefs</module>
> <module>ozonefs-lib-current</module>
> <module>ozonefs-lib-legacy</module>
> </modules>
> </profile>
> {code}
> As result of this, when I make code change under {{./hadoop-ozone/ozonefs/}} then run {{mvn clean install -Pdist -DskipTests -e -Dmaven.javadoc.skip=true -DskipShade}}, the change won't be reflected in the dist. Property {{skipShade}} should not be expected to do this.
> We should compile {{ozonefs}} regardless of {{-DskipShade}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org