You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Meethu Mathew <me...@flytxt.com> on 2015/06/04 12:16:17 UTC
Anyone facing problem in incremental building of individual project
Hi all,
I added some new code to MLlib. When I am trying to build only the mllib
project using *mvn --projects mllib/ -DskipTests clean install*
* *after setting
export S
PARK_PREPEND_CLASSES=true
, the build is getting stuck with the following message.
> Excluding org.jpmml:pmml-schema:jar:1.1.15 from the shaded jar.
> [INFO] Excluding com.sun.xml.bind:jaxb-impl:jar:2.2.7 from the shaded jar.
> [INFO] Excluding com.sun.xml.bind:jaxb-core:jar:2.2.7 from the shaded jar.
> [INFO] Excluding javax.xml.bind:jaxb-api:jar:2.2.7 from the shaded jar.
> [INFO] Including org.spark-project.spark:unused:jar:1.0.0 in the shaded
> jar.
> [INFO] Excluding org.scala-lang:scala-reflect:jar:2.10.4 from the shaded
> jar.
> [INFO] Replacing original artifact with shaded artifact.
> [INFO] Replacing
> /home/meethu/git/FlytxtRnD/spark/mllib/target/spark-mllib_2.10-1.4.0-SNAPSHOT.jar
> with
> /home/meethu/git/FlytxtRnD/spark/mllib/target/spark-mllib_2.10-1.4.0-SNAPSHOT-shaded.jar
> [INFO] Dependency-reduced POM written at:
> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
> [INFO] Dependency-reduced POM written at:
> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
> [INFO] Dependency-reduced POM written at:
> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
> [INFO] Dependency-reduced POM written at:
> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
.....
But a full build completes as usual. Please help if anyone is facing the
same issue.
Regards,
Meethu Mathew
Senior Engineer
Flytxt
Re: Anyone facing problem in incremental building of individual project
Posted by Meethu Mathew <me...@flytxt.com>.
Hi,
I added
<createDependencyReducedPom> in my pom.xml and the problem is solved.
<!-- Work around MSHADE-148 -->
+
<createDependencyReducedPom>false</createDependencyReducedPom>
Thank you @Steve and @Ted
Regards,
Meethu Mathew
Senior Engineer
Flytxt
On Thu, Jun 4, 2015 at 9:51 PM, Ted Yu <yu...@gmail.com> wrote:
> Andrew Or put in this workaround :
>
> diff --git a/pom.xml b/pom.xml
> index 0b1aaad..d03d33b 100644
> --- a/pom.xml
> +++ b/pom.xml
> @@ -1438,6 +1438,8 @@
> <version>2.3</version>
> <configuration>
> <shadedArtifactAttached>false</shadedArtifactAttached>
> + <!-- Work around MSHADE-148 -->
> + <createDependencyReducedPom>false</createDependencyReducedPom>
> <artifactSet>
> <includes>
> <!-- At a minimum we must include this to force effective
> pom generation -->
>
> FYI
>
> On Thu, Jun 4, 2015 at 6:25 AM, Steve Loughran <st...@hortonworks.com>
> wrote:
>
>>
>> On 4 Jun 2015, at 11:16, Meethu Mathew <me...@flytxt.com> wrote:
>>
>> Hi all,
>>
>> I added some new code to MLlib. When I am trying to build only the
>> mllib project using *mvn --projects mllib/ -DskipTests clean install*
>> * *after setting
>> export S
>> PARK_PREPEND_CLASSES=true
>> , the build is getting stuck with the following message.
>>
>>
>>
>>> Excluding org.jpmml:pmml-schema:jar:1.1.15 from the shaded jar.
>>> [INFO] Excluding com.sun.xml.bind:jaxb-impl:jar:2.2.7 from the shaded
>>> jar.
>>> [INFO] Excluding com.sun.xml.bind:jaxb-core:jar:2.2.7 from the shaded
>>> jar.
>>> [INFO] Excluding javax.xml.bind:jaxb-api:jar:2.2.7 from the shaded jar.
>>> [INFO] Including org.spark-project.spark:unused:jar:1.0.0 in the shaded
>>> jar.
>>> [INFO] Excluding org.scala-lang:scala-reflect:jar:2.10.4 from the shaded
>>> jar.
>>> [INFO] Replacing original artifact with shaded artifact.
>>> [INFO] Replacing
>>> /home/meethu/git/FlytxtRnD/spark/mllib/target/spark-mllib_2.10-1.4.0-SNAPSHOT.jar
>>> with
>>> /home/meethu/git/FlytxtRnD/spark/mllib/target/spark-mllib_2.10-1.4.0-SNAPSHOT-shaded.jar
>>> [INFO] Dependency-reduced POM written at:
>>> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
>>> [INFO] Dependency-reduced POM written at:
>>> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
>>> [INFO] Dependency-reduced POM written at:
>>> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
>>> [INFO] Dependency-reduced POM written at:
>>> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
>>
>> .....
>>
>>
>>
>> I've seen something similar in a different build,
>>
>> It looks like MSHADE-148:
>> https://issues.apache.org/jira/browse/MSHADE-148
>> if you apply Tom White's patch, does your problem go away?
>>
>
>
Re: Anyone facing problem in incremental building of individual project
Posted by Ted Yu <yu...@gmail.com>.
Andrew Or put in this workaround :
diff --git a/pom.xml b/pom.xml
index 0b1aaad..d03d33b 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1438,6 +1438,8 @@
<version>2.3</version>
<configuration>
<shadedArtifactAttached>false</shadedArtifactAttached>
+ <!-- Work around MSHADE-148 -->
+ <createDependencyReducedPom>false</createDependencyReducedPom>
<artifactSet>
<includes>
<!-- At a minimum we must include this to force effective
pom generation -->
FYI
On Thu, Jun 4, 2015 at 6:25 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> On 4 Jun 2015, at 11:16, Meethu Mathew <me...@flytxt.com> wrote:
>
> Hi all,
>
> I added some new code to MLlib. When I am trying to build only the
> mllib project using *mvn --projects mllib/ -DskipTests clean install*
> * *after setting
> export S
> PARK_PREPEND_CLASSES=true
> , the build is getting stuck with the following message.
>
>
>
>> Excluding org.jpmml:pmml-schema:jar:1.1.15 from the shaded jar.
>> [INFO] Excluding com.sun.xml.bind:jaxb-impl:jar:2.2.7 from the shaded jar.
>> [INFO] Excluding com.sun.xml.bind:jaxb-core:jar:2.2.7 from the shaded jar.
>> [INFO] Excluding javax.xml.bind:jaxb-api:jar:2.2.7 from the shaded jar.
>> [INFO] Including org.spark-project.spark:unused:jar:1.0.0 in the shaded
>> jar.
>> [INFO] Excluding org.scala-lang:scala-reflect:jar:2.10.4 from the shaded
>> jar.
>> [INFO] Replacing original artifact with shaded artifact.
>> [INFO] Replacing
>> /home/meethu/git/FlytxtRnD/spark/mllib/target/spark-mllib_2.10-1.4.0-SNAPSHOT.jar
>> with
>> /home/meethu/git/FlytxtRnD/spark/mllib/target/spark-mllib_2.10-1.4.0-SNAPSHOT-shaded.jar
>> [INFO] Dependency-reduced POM written at:
>> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
>> [INFO] Dependency-reduced POM written at:
>> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
>> [INFO] Dependency-reduced POM written at:
>> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
>> [INFO] Dependency-reduced POM written at:
>> /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
>
> .....
>
>
>
> I've seen something similar in a different build,
>
> It looks like MSHADE-148:
> https://issues.apache.org/jira/browse/MSHADE-148
> if you apply Tom White's patch, does your problem go away?
>
Re: Anyone facing problem in incremental building of individual
project
Posted by Steve Loughran <st...@hortonworks.com>.
On 4 Jun 2015, at 11:16, Meethu Mathew <me...@flytxt.com>> wrote:
Hi all,
I added some new code to MLlib. When I am trying to build only the mllib project using mvn --projects mllib/ -DskipTests clean install
after setting
export S
PARK_PREPEND_CLASSES=true
, the build is getting stuck with the following message.
Excluding org.jpmml:pmml-schema:jar:1.1.15 from the shaded jar.
[INFO] Excluding com.sun.xml.bind:jaxb-impl:jar:2.2.7 from the shaded jar.
[INFO] Excluding com.sun.xml.bind:jaxb-core:jar:2.2.7 from the shaded jar.
[INFO] Excluding javax.xml.bind:jaxb-api:jar:2.2.7 from the shaded jar.
[INFO] Including org.spark-project.spark:unused:jar:1.0.0 in the shaded jar.
[INFO] Excluding org.scala-lang:scala-reflect:jar:2.10.4 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /home/meethu/git/FlytxtRnD/spark/mllib/target/spark-mllib_2.10-1.4.0-SNAPSHOT.jar with /home/meethu/git/FlytxtRnD/spark/mllib/target/spark-mllib_2.10-1.4.0-SNAPSHOT-shaded.jar
[INFO] Dependency-reduced POM written at: /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at: /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at: /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at: /home/meethu/git/FlytxtRnD/spark/mllib/dependency-reduced-pom.xml
.....
I've seen something similar in a different build,
It looks like MSHADE-148: https://issues.apache.org/jira/browse/MSHADE-148
if you apply Tom White's patch, does your problem go away?