You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Pala M Muthaia <mc...@rocketfuelinc.com> on 2015/03/26 23:48:20 UTC
Re: Building spark 1.2 from source requires more dependencies
+spark-dev
Yes, the dependencies are there. I guess my question is how come the build
is succeeding in the mainline then, without adding these dependencies?
On Thu, Mar 26, 2015 at 3:44 PM, Ted Yu <yu...@gmail.com> wrote:
> Looking at output from dependency:tree, servlet-api is brought in by the
> following:
>
> [INFO] +- org.apache.cassandra:cassandra-all:jar:1.2.6:compile
> [INFO] | +- org.antlr:antlr:jar:3.2:compile
> [INFO] | +- com.googlecode.json-simple:json-simple:jar:1.1:compile
> [INFO] | +- org.yaml:snakeyaml:jar:1.6:compile
> [INFO] | +- edu.stanford.ppl:snaptree:jar:0.1:compile
> [INFO] | +- org.mindrot:jbcrypt:jar:0.3m:compile
> [INFO] | +- org.apache.thrift:libthrift:jar:0.7.0:compile
> [INFO] | | \- javax.servlet:servlet-api:jar:2.5:compile
>
> FYI
>
> On Thu, Mar 26, 2015 at 3:36 PM, Pala M Muthaia <
> mchettiar@rocketfuelinc.com> wrote:
>
>> Hi,
>>
>> We are trying to build spark 1.2 from source (tip of the branch-1.2 at
>> the moment). I tried to build spark using the following command:
>>
>> mvn -U -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive
>> -Phive-thriftserver -DskipTests clean package
>>
>> I encountered various missing class definition exceptions (e.g: class
>> javax.servlet.ServletException not found).
>>
>> I eventually got the build to succeed after adding the following set of
>> dependencies to the spark-core's pom.xml:
>>
>> <dependency>
>> <groupId>javax.servlet</groupId>
>> <artifactId>*servlet-api*</artifactId>
>> <version>3.0</version>
>> </dependency>
>>
>> <dependency>
>> <groupId>org.eclipse.jetty</groupId>
>> <artifactId>*jetty-io*</artifactId>
>> </dependency>
>>
>> <dependency>
>> <groupId>org.eclipse.jetty</groupId>
>> <artifactId>*jetty-http*</artifactId>
>> </dependency>
>>
>> <dependency>
>> <groupId>org.eclipse.jetty</groupId>
>> <artifactId>*jetty-servlet*</artifactId>
>> </dependency>
>>
>> Pretty much all of the missing class definition errors came up while
>> building HttpServer.scala, and went away after the above dependencies were
>> included.
>>
>> My guess is official build for spark 1.2 is working already. My question
>> is what is wrong with my environment or setup, that requires me to add
>> dependencies to pom.xml in this manner, to get this build to succeed.
>>
>> Also, i am not sure if this build would work at runtime for us, i am
>> still testing this out.
>>
>>
>> Thanks,
>> pala
>>
>
>
Re: Building spark 1.2 from source requires more dependencies
Posted by Xi Shen <da...@gmail.com>.
It it bought in by another dependency, so you do not need to specify it
explicitly...I think this is what Ted mean.
On Fri, Mar 27, 2015 at 9:48 AM Pala M Muthaia <mc...@rocketfuelinc.com>
wrote:
> +spark-dev
>
> Yes, the dependencies are there. I guess my question is how come the build
> is succeeding in the mainline then, without adding these dependencies?
>
> On Thu, Mar 26, 2015 at 3:44 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Looking at output from dependency:tree, servlet-api is brought in by the
>> following:
>>
>> [INFO] +- org.apache.cassandra:cassandra-all:jar:1.2.6:compile
>> [INFO] | +- org.antlr:antlr:jar:3.2:compile
>> [INFO] | +- com.googlecode.json-simple:json-simple:jar:1.1:compile
>> [INFO] | +- org.yaml:snakeyaml:jar:1.6:compile
>> [INFO] | +- edu.stanford.ppl:snaptree:jar:0.1:compile
>> [INFO] | +- org.mindrot:jbcrypt:jar:0.3m:compile
>> [INFO] | +- org.apache.thrift:libthrift:jar:0.7.0:compile
>> [INFO] | | \- javax.servlet:servlet-api:jar:2.5:compile
>>
>> FYI
>>
>> On Thu, Mar 26, 2015 at 3:36 PM, Pala M Muthaia <
>> mchettiar@rocketfuelinc.com> wrote:
>>
>>> Hi,
>>>
>>> We are trying to build spark 1.2 from source (tip of the branch-1.2 at
>>> the moment). I tried to build spark using the following command:
>>>
>>> mvn -U -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive
>>> -Phive-thriftserver -DskipTests clean package
>>>
>>> I encountered various missing class definition exceptions (e.g: class
>>> javax.servlet.ServletException not found).
>>>
>>> I eventually got the build to succeed after adding the following set of
>>> dependencies to the spark-core's pom.xml:
>>>
>>> <dependency>
>>> <groupId>javax.servlet</groupId>
>>> <artifactId>*servlet-api*</artifactId>
>>> <version>3.0</version>
>>> </dependency>
>>>
>>> <dependency>
>>> <groupId>org.eclipse.jetty</groupId>
>>> <artifactId>*jetty-io*</artifactId>
>>> </dependency>
>>>
>>> <dependency>
>>> <groupId>org.eclipse.jetty</groupId>
>>> <artifactId>*jetty-http*</artifactId>
>>> </dependency>
>>>
>>> <dependency>
>>> <groupId>org.eclipse.jetty</groupId>
>>> <artifactId>*jetty-servlet*</artifactId>
>>> </dependency>
>>>
>>> Pretty much all of the missing class definition errors came up while
>>> building HttpServer.scala, and went away after the above dependencies were
>>> included.
>>>
>>> My guess is official build for spark 1.2 is working already. My question
>>> is what is wrong with my environment or setup, that requires me to add
>>> dependencies to pom.xml in this manner, to get this build to succeed.
>>>
>>> Also, i am not sure if this build would work at runtime for us, i am
>>> still testing this out.
>>>
>>>
>>> Thanks,
>>> pala
>>>
>>
>>
>