You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by sp...@163.com on 2016/11/10 07:50:14 UTC

download-maven-plugin in module spark-dependencies failed to create file spark.xxx.tgz when spark-dependencies/target directory not created first

Hi zeppelin-dev,

I’m building the master branch, using following cmd:
mvn clean  install -Pbuild-distr -Pspark-1.6 -Dspark.version=1.6.2 -Phadoop-2.7 -Dhadoop.version=2.7.2 -Psparkr -Ppyspark -Dmaven.findbugs.enable=false -Drat.skip=true -Dcheckstyle.skip=true -Denforcer.skip=true -Dcobertura.skip=true -DskipTests

and got a error like this in module  spark-dependencies:
Failed to execute goal com.googlecode.maven-download-plugin:download-maven-plugin:1.2.1:wget (download-pyspark-files) on project zeppelin-spark-dependencies_2.10: IO Error: Destination ‘zeppelin.git/spark-dependencies/target/../../.spark-dist/spark-1.6.2.tgz' directory cannot be created

I find property definition in spark-dependencies/pom.xm like this:
<spark.dist.cache>${project.build.directory}/../../.spark-dist</spark.dist.cache>
and in profile pySpark find maven life cycle binding like this:
<plugins>
  <plugin>
    <groupId>com.googlecode.maven-download-plugin</groupId>
    <artifactId>download-maven-plugin</artifactId>
    <version>1.2.1</version>
    <executions>
      <execution>
        <id>download-pyspark-files</id>
        <phase>validate</phase>
        <goals>
          <goal>wget</goal>
        </goals>
        <configuration>
          <url>${spark.download.url}</url>
          <outputDirectory>${spark.dist.cache}</outputDirectory>
        </configuration>
      </execution>
    </executions>
  </plugin>

I think the cause could be this:
validate is the 1st phase in maven default life cycle https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#Built-in_Lifecycle_Bindings <https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#Built-in_Lifecycle_Bindings>,which in default binds no plugin.
 <spark.dist.cache> property defined like this: ${project.build.directory}/../../.spark-dist, which resolved to “target/../../.spark-dist”, may not by ready,when downloading happens, for directory “target” may not be created yet.

Does someone else came across the same problem? Or i miss something? 

FYI, i’m using: 
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)
Maven home: /opt/maven
Java version: 1.8.0_102, vendor: Oracle Corporation
OS name: "mac os x", version: "10.12.1", arch: "x86_64", family: “mac"

Any help would be appreciated.