You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by kpeng1 <kp...@gmail.com> on 2014/09/16 00:33:06 UTC
Invalid signature file digest for Manifest main attributes with
spark job built using maven
Hi All,
I am trying to submit a spark job that I have built in maven using the
following command:
/usr/bin/spark-submit --deploy-mode client --class com.spark.TheMain
--master local[1] /home/cloudera/myjar.jar 100
But I seem to be getting the following error:
Exception in thread "main" java.lang.SecurityException: Invalid signature
file digest for Manifest main attributes
at
sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:286)
at
sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:239)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:307)
at java.util.jar.JarVerifier.update(JarVerifier.java:218)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:345)
at java.util.jar.JarFile.getInputStream(JarFile.java:412)
at sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:775)
at sun.misc.Resource.cachedInputStream(Resource.java:77)
at sun.misc.Resource.getByteBuffer(Resource.java:160)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:436)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:289)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:55)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Here is the pom file I am using to build the jar:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.spark</groupId>
<artifactId>myjar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>${project.artifactId}</name>
<description>My wonderfull scala app</description>
<inceptionYear>2010</inceptionYear>
<licenses>
<license>
<name>My License</name>
<url>http://....</url>
<distribution>repo</distribution>
</license>
</licenses>
<properties>
<cdh.version>cdh5.1.0</cdh.version>
<maven.compiler.source>1.6</maven.compiler.source>
<maven.compiler.target>1.6</maven.compiler.target>
<encoding>UTF-8</encoding>
<scala.tools.version>2.10</scala.tools.version>
<scala.version>2.10.4</scala.version>
</properties>
<repositories>
<repository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
</repository>
<repository>
<id>maven-hadoop</id>
<name>Hadoop Releases</name>
<url>https://repository.cloudera.com/content/repositories/releases/</url>
</repository>
<repository>
<id>cloudera-repos</id>
<name>Cloudera Repos</name>
<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.0.0-${cdh.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-tools_2.10</artifactId>
<version>1.0.0-${cdh.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-flume_2.10</artifactId>
<version>1.0.0-${cdh.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>1.0.0-${cdh.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flume</groupId>
<artifactId>flume-ng-sdk</artifactId>
<version>1.5.0-${cdh.version}</version>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.flume</groupId>
<artifactId>flume-ng-core</artifactId>
<version>1.5.0-${cdh.version}</version>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>0.98.1-${cdh.version}</version>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.3.0-${cdh.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.specs2</groupId>
<artifactId>specs2_${scala.tools.version}</artifactId>
<version>1.13</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_${scala.tools.version}</artifactId>
<version>2.0.M6-SNAP8</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.1.3</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
<configuration>
<args>
<arg>-make:transitive</arg>
<arg>-dependencyfile</arg>
<arg>${project.build.directory}/.scala_dependencies</arg>
</args>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.13</version>
<configuration>
<useFile>false</useFile>
<disableXmlReport>true</disableXmlReport>
<includes>
<include>**/*Test.*</include>
<include>**/*Suite.*</include>
</includes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<configuration>
<transformers>
<transformer
implementation="org.apache..plugins.shade.resource.ApacheLicenseResourceTransformer">
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>reference.conf</resource>
</transformer>
</transformers>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Was wondering what I am missing, since the jar built and a MANIFEST.MF file
is created, not sure why it is looking for a signature file digest, am I
missing some exclusion clause maybe?
I saw a thread based on SBT and tried using the --deploy-mode client option;
even though, my understanding is by default it would use that mode anyway.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Invalid-signature-file-digest-for-Manifest-main-attributes-with-spark-job-built-using-maven-tp14299.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Invalid signature file digest for Manifest main attributes with
spark job built using maven
Posted by Kevin Peng <kp...@gmail.com>.
Sean,
Thanks. That worked.
Kevin
On Mon, Sep 15, 2014 at 3:37 PM, Sean Owen <so...@cloudera.com> wrote:
> This is more of a Java / Maven issue than Spark per se. I would use
> the shade plugin to remove signature files in your final META-INF/
> dir. As Spark does, in its <configuration>:
>
> <filters>
> <filter>
> <artifact>*:*</artifact>
> <excludes>
> <exclude>org/datanucleus/**</exclude>
> <exclude>META-INF/*.SF</exclude>
> <exclude>META-INF/*.DSA</exclude>
> <exclude>META-INF/*.RSA</exclude>
> </excludes>
> </filter>
> </filters>
>
> On Mon, Sep 15, 2014 at 11:33 PM, kpeng1 <kp...@gmail.com> wrote:
> > Hi All,
> >
> > I am trying to submit a spark job that I have built in maven using the
> > following command:
> > /usr/bin/spark-submit --deploy-mode client --class com.spark.TheMain
> > --master local[1] /home/cloudera/myjar.jar 100
> >
> > But I seem to be getting the following error:
> > Exception in thread "main" java.lang.SecurityException: Invalid signature
> > file digest for Manifest main attributes
> > at
> >
> sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:286)
> > at
> >
> sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:239)
> > at java.util.jar.JarVerifier.processEntry(JarVerifier.java:307)
> > at java.util.jar.JarVerifier.update(JarVerifier.java:218)
> > at java.util.jar.JarFile.initializeVerifier(JarFile.java:345)
> > at java.util.jar.JarFile.getInputStream(JarFile.java:412)
> > at
> sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:775)
> > at sun.misc.Resource.cachedInputStream(Resource.java:77)
> > at sun.misc.Resource.getByteBuffer(Resource.java:160)
> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:436)
> > at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:270)
> > at
> org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:289)
> > at
> org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:55)
> > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> >
> >
> > Here is the pom file I am using to build the jar:
> > <project xmlns="http://maven.apache.org/POM/4.0.0"
> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> > http://maven.apache.org/maven-v4_0_0.xsd">
> > <modelVersion>4.0.0</modelVersion>
> > <groupId>com.spark</groupId>
> > <artifactId>myjar</artifactId>
> > <version>0.0.1-SNAPSHOT</version>
> > <name>${project.artifactId}</name>
> > <description>My wonderfull scala app</description>
> > <inceptionYear>2010</inceptionYear>
> > <licenses>
> > <license>
> > <name>My License</name>
> > <url>http://....</url>
> > <distribution>repo</distribution>
> > </license>
> > </licenses>
> >
> > <properties>
> > <cdh.version>cdh5.1.0</cdh.version>
> > <maven.compiler.source>1.6</maven.compiler.source>
> > <maven.compiler.target>1.6</maven.compiler.target>
> > <encoding>UTF-8</encoding>
> > <scala.tools.version>2.10</scala.tools.version>
> > <scala.version>2.10.4</scala.version>
> > </properties>
> >
> > <repositories>
> > <repository>
> > <id>scala-tools.org</id>
> > <name>Scala-tools Maven2 Repository</name>
> > <url>https://oss.sonatype.org/content/repositories/snapshots/
> </url>
> > </repository>
> > <repository>
> > <id>maven-hadoop</id>
> > <name>Hadoop Releases</name>
> >
> > <url>https://repository.cloudera.com/content/repositories/releases/
> </url>
> > </repository>
> > <repository>
> > <id>cloudera-repos</id>
> > <name>Cloudera Repos</name>
> > <url>https://repository.cloudera.com/artifactory/cloudera-repos/
> </url>
> > </repository>
> > </repositories>
> > <pluginRepositories>
> > <pluginRepository>
> > <id>scala-tools.org</id>
> > <name>Scala-tools Maven2 Repository</name>
> > <url>https://oss.sonatype.org/content/repositories/snapshots/
> </url>
> > </pluginRepository>
> > </pluginRepositories>
> >
> > <dependencies>
> > <dependency>
> > <groupId>org.scala-lang</groupId>
> > <artifactId>scala-library</artifactId>
> > <version>${scala.version}</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.spark</groupId>
> > <artifactId>spark-core_2.10</artifactId>
> > <version>1.0.0-${cdh.version}</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.spark</groupId>
> > <artifactId>spark-tools_2.10</artifactId>
> > <version>1.0.0-${cdh.version}</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.spark</groupId>
> > <artifactId>spark-streaming-flume_2.10</artifactId>
> > <version>1.0.0-${cdh.version}</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.spark</groupId>
> > <artifactId>spark-streaming_2.10</artifactId>
> > <version>1.0.0-${cdh.version}</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.flume</groupId>
> > <artifactId>flume-ng-sdk</artifactId>
> > <version>1.5.0-${cdh.version}</version>
> >
> > <exclusions>
> > <exclusion>
> > <groupId>io.netty</groupId>
> > <artifactId>netty</artifactId>
> > </exclusion>
> > </exclusions>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.flume</groupId>
> > <artifactId>flume-ng-core</artifactId>
> > <version>1.5.0-${cdh.version}</version>
> >
> > <exclusions>
> > <exclusion>
> > <groupId>io.netty</groupId>
> > <artifactId>netty</artifactId>
> > </exclusion>
> > </exclusions>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.hbase</groupId>
> > <artifactId>hbase-client</artifactId>
> > <version>0.98.1-${cdh.version}</version>
> >
> > <exclusions>
> > <exclusion>
> > <groupId>io.netty</groupId>
> > <artifactId>netty</artifactId>
> > </exclusion>
> > </exclusions>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.hadoop</groupId>
> > <artifactId>hadoop-client</artifactId>
> > <version>2.3.0-${cdh.version}</version>
> >
> > </dependency>
> >
> >
> > <dependency>
> > <groupId>junit</groupId>
> > <artifactId>junit</artifactId>
> > <version>4.11</version>
> > <scope>test</scope>
> > </dependency>
> > <dependency>
> > <groupId>org.specs2</groupId>
> > <artifactId>specs2_${scala.tools.version}</artifactId>
> > <version>1.13</version>
> > <scope>test</scope>
> > </dependency>
> > <dependency>
> > <groupId>org.scalatest</groupId>
> > <artifactId>scalatest_${scala.tools.version}</artifactId>
> > <version>2.0.M6-SNAP8</version>
> > <scope>test</scope>
> > </dependency>
> > </dependencies>
> >
> > <build>
> > <sourceDirectory>src/main/scala</sourceDirectory>
> > <testSourceDirectory>src/test/scala</testSourceDirectory>
> > <plugins>
> > <plugin>
> >
> > <groupId>net.alchim31.maven</groupId>
> > <artifactId>scala-maven-plugin</artifactId>
> > <version>3.1.3</version>
> > <executions>
> > <execution>
> > <goals>
> > <goal>compile</goal>
> > <goal>testCompile</goal>
> > </goals>
> > <configuration>
> > <args>
> > <arg>-make:transitive</arg>
> > <arg>-dependencyfile</arg>
> > <arg>${project.build.directory}/.scala_dependencies</arg>
> > </args>
> > </configuration>
> > </execution>
> > </executions>
> > </plugin>
> > <plugin>
> > <groupId>org.apache.maven.plugins</groupId>
> > <artifactId>maven-surefire-plugin</artifactId>
> > <version>2.13</version>
> > <configuration>
> > <useFile>false</useFile>
> > <disableXmlReport>true</disableXmlReport>
> >
> >
> > <includes>
> > <include>**/*Test.*</include>
> > <include>**/*Suite.*</include>
> > </includes>
> > </configuration>
> > </plugin>
> > <plugin>
> > <groupId>org.apache.maven.plugins</groupId>
> > <artifactId>maven-shade-plugin</artifactId>
> > <version>2.2</version>
> > <configuration>
> >
> > <transformers>
> > <transformer
> >
> implementation="org.apache..plugins.shade.resource.ApacheLicenseResourceTransformer">
> > </transformer>
> > <transformer
> >
> implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
> > <resource>reference.conf</resource>
> > </transformer>
> > </transformers>
> > </configuration>
> > <executions>
> > <execution>
> > <phase>package</phase>
> > <goals>
> > <goal>shade</goal>
> > </goals>
> > </execution>
> > </executions>
> > </plugin>
> > </plugins>
> > </build>
> > </project>
> >
> > Was wondering what I am missing, since the jar built and a MANIFEST.MF
> file
> > is created, not sure why it is looking for a signature file digest, am I
> > missing some exclusion clause maybe?
> >
> > I saw a thread based on SBT and tried using the --deploy-mode client
> option;
> > even though, my understanding is by default it would use that mode
> anyway.
> >
> >
> >
> >
> > --
> > View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Invalid-signature-file-digest-for-Manifest-main-attributes-with-spark-job-built-using-maven-tp14299.html
> > Sent from the Apache Spark User List mailing list archive at Nabble.com.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> > For additional commands, e-mail: user-help@spark.apache.org
> >
>
Re: Invalid signature file digest for Manifest main attributes with
spark job built using maven
Posted by Sean Owen <so...@cloudera.com>.
This is more of a Java / Maven issue than Spark per se. I would use
the shade plugin to remove signature files in your final META-INF/
dir. As Spark does, in its <configuration>:
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>org/datanucleus/**</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
On Mon, Sep 15, 2014 at 11:33 PM, kpeng1 <kp...@gmail.com> wrote:
> Hi All,
>
> I am trying to submit a spark job that I have built in maven using the
> following command:
> /usr/bin/spark-submit --deploy-mode client --class com.spark.TheMain
> --master local[1] /home/cloudera/myjar.jar 100
>
> But I seem to be getting the following error:
> Exception in thread "main" java.lang.SecurityException: Invalid signature
> file digest for Manifest main attributes
> at
> sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:286)
> at
> sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:239)
> at java.util.jar.JarVerifier.processEntry(JarVerifier.java:307)
> at java.util.jar.JarVerifier.update(JarVerifier.java:218)
> at java.util.jar.JarFile.initializeVerifier(JarFile.java:345)
> at java.util.jar.JarFile.getInputStream(JarFile.java:412)
> at sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:775)
> at sun.misc.Resource.cachedInputStream(Resource.java:77)
> at sun.misc.Resource.getByteBuffer(Resource.java:160)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:436)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:289)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:55)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>
>
> Here is the pom file I am using to build the jar:
> <project xmlns="http://maven.apache.org/POM/4.0.0"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> http://maven.apache.org/maven-v4_0_0.xsd">
> <modelVersion>4.0.0</modelVersion>
> <groupId>com.spark</groupId>
> <artifactId>myjar</artifactId>
> <version>0.0.1-SNAPSHOT</version>
> <name>${project.artifactId}</name>
> <description>My wonderfull scala app</description>
> <inceptionYear>2010</inceptionYear>
> <licenses>
> <license>
> <name>My License</name>
> <url>http://....</url>
> <distribution>repo</distribution>
> </license>
> </licenses>
>
> <properties>
> <cdh.version>cdh5.1.0</cdh.version>
> <maven.compiler.source>1.6</maven.compiler.source>
> <maven.compiler.target>1.6</maven.compiler.target>
> <encoding>UTF-8</encoding>
> <scala.tools.version>2.10</scala.tools.version>
> <scala.version>2.10.4</scala.version>
> </properties>
>
> <repositories>
> <repository>
> <id>scala-tools.org</id>
> <name>Scala-tools Maven2 Repository</name>
> <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
> </repository>
> <repository>
> <id>maven-hadoop</id>
> <name>Hadoop Releases</name>
>
> <url>https://repository.cloudera.com/content/repositories/releases/</url>
> </repository>
> <repository>
> <id>cloudera-repos</id>
> <name>Cloudera Repos</name>
> <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
> </repository>
> </repositories>
> <pluginRepositories>
> <pluginRepository>
> <id>scala-tools.org</id>
> <name>Scala-tools Maven2 Repository</name>
> <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
> </pluginRepository>
> </pluginRepositories>
>
> <dependencies>
> <dependency>
> <groupId>org.scala-lang</groupId>
> <artifactId>scala-library</artifactId>
> <version>${scala.version}</version>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.10</artifactId>
> <version>1.0.0-${cdh.version}</version>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-tools_2.10</artifactId>
> <version>1.0.0-${cdh.version}</version>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming-flume_2.10</artifactId>
> <version>1.0.0-${cdh.version}</version>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming_2.10</artifactId>
> <version>1.0.0-${cdh.version}</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flume</groupId>
> <artifactId>flume-ng-sdk</artifactId>
> <version>1.5.0-${cdh.version}</version>
>
> <exclusions>
> <exclusion>
> <groupId>io.netty</groupId>
> <artifactId>netty</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
> <dependency>
> <groupId>org.apache.flume</groupId>
> <artifactId>flume-ng-core</artifactId>
> <version>1.5.0-${cdh.version}</version>
>
> <exclusions>
> <exclusion>
> <groupId>io.netty</groupId>
> <artifactId>netty</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
> <dependency>
> <groupId>org.apache.hbase</groupId>
> <artifactId>hbase-client</artifactId>
> <version>0.98.1-${cdh.version}</version>
>
> <exclusions>
> <exclusion>
> <groupId>io.netty</groupId>
> <artifactId>netty</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
> <dependency>
> <groupId>org.apache.hadoop</groupId>
> <artifactId>hadoop-client</artifactId>
> <version>2.3.0-${cdh.version}</version>
>
> </dependency>
>
>
> <dependency>
> <groupId>junit</groupId>
> <artifactId>junit</artifactId>
> <version>4.11</version>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.specs2</groupId>
> <artifactId>specs2_${scala.tools.version}</artifactId>
> <version>1.13</version>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.scalatest</groupId>
> <artifactId>scalatest_${scala.tools.version}</artifactId>
> <version>2.0.M6-SNAP8</version>
> <scope>test</scope>
> </dependency>
> </dependencies>
>
> <build>
> <sourceDirectory>src/main/scala</sourceDirectory>
> <testSourceDirectory>src/test/scala</testSourceDirectory>
> <plugins>
> <plugin>
>
> <groupId>net.alchim31.maven</groupId>
> <artifactId>scala-maven-plugin</artifactId>
> <version>3.1.3</version>
> <executions>
> <execution>
> <goals>
> <goal>compile</goal>
> <goal>testCompile</goal>
> </goals>
> <configuration>
> <args>
> <arg>-make:transitive</arg>
> <arg>-dependencyfile</arg>
> <arg>${project.build.directory}/.scala_dependencies</arg>
> </args>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>2.13</version>
> <configuration>
> <useFile>false</useFile>
> <disableXmlReport>true</disableXmlReport>
>
>
> <includes>
> <include>**/*Test.*</include>
> <include>**/*Suite.*</include>
> </includes>
> </configuration>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-shade-plugin</artifactId>
> <version>2.2</version>
> <configuration>
>
> <transformers>
> <transformer
> implementation="org.apache..plugins.shade.resource.ApacheLicenseResourceTransformer">
> </transformer>
> <transformer
> implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
> <resource>reference.conf</resource>
> </transformer>
> </transformers>
> </configuration>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>shade</goal>
> </goals>
> </execution>
> </executions>
> </plugin>
> </plugins>
> </build>
> </project>
>
> Was wondering what I am missing, since the jar built and a MANIFEST.MF file
> is created, not sure why it is looking for a signature file digest, am I
> missing some exclusion clause maybe?
>
> I saw a thread based on SBT and tried using the --deploy-mode client option;
> even though, my understanding is by default it would use that mode anyway.
>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Invalid-signature-file-digest-for-Manifest-main-attributes-with-spark-job-built-using-maven-tp14299.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org