You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by rapelly kartheek <ka...@gmail.com> on 2014/09/11 17:27:48 UTC
compiling spark source code
HI,
Can someone please tell me how to compile the spark source code to effect
the changes in the source code. I was trying to ship the jars to all the
slaves, but in vain.
-Karthik
Re: compiling spark source code
Posted by kkptninja <kk...@gmail.com>.
Hi,
I took am having problem with compiling Spark from source. However, my
problem is different. I downloaded latest version (1.1.0) and ran ./sbt/sbt
assembly from the command line. I end up with the following error
[info] SHA-1: 20abd673d1e0690a6d5b64951868eef8d332d084
[info] Packaging
/home/kkpt/Desktop/spark-1.1.0/examples/target/scala-2.10/spark-examples-1.1.0-hadoop1.0.4.jar
...
[info] Done packaging.
[error] (repl/compile:compile) Compilation failed
[error] Total time: 207 s, completed Sep 13, 2014 2:57:45 PM
Can you tell how i could proceed forward? (I do not need hadoop support)
Thanks in advance and hear from you
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/compiling-spark-source-code-tp13980p14166.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: compiling spark source code
Posted by qihong <qc...@pivotal.io>.
follow the instruction here:
http://spark.apache.org/docs/latest/building-with-maven.html
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/compiling-spark-source-code-tp13980p14144.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: compiling spark source code
Posted by rapelly kartheek <ka...@gmail.com>.
I have been doing that. All the modifications to the code are not being
compiled.
On Thu, Sep 11, 2014 at 10:45 PM, Daniil Osipov <da...@shazam.com>
wrote:
> In the spark source folder, execute `sbt/sbt assembly`
>
> On Thu, Sep 11, 2014 at 8:27 AM, rapelly kartheek <kartheek.mbms@gmail.com
> > wrote:
>
>> HI,
>>
>>
>> Can someone please tell me how to compile the spark source code to effect
>> the changes in the source code. I was trying to ship the jars to all the
>> slaves, but in vain.
>>
>> -Karthik
>>
>
>
Re: compiling spark source code
Posted by Daniil Osipov <da...@shazam.com>.
In the spark source folder, execute `sbt/sbt assembly`
On Thu, Sep 11, 2014 at 8:27 AM, rapelly kartheek <ka...@gmail.com>
wrote:
> HI,
>
>
> Can someone please tell me how to compile the spark source code to effect
> the changes in the source code. I was trying to ship the jars to all the
> slaves, but in vain.
>
> -Karthik
>
Re: compiling spark source code
Posted by Matei Zaharia <ma...@gmail.com>.
I've seen the "file name too long" error when compiling on an encrypted Linux file system -- some of them have a limit on file name lengths. If you're on Linux, can you try compiling inside /tmp instead?
Matei
On September 13, 2014 at 10:03:14 PM, Yin Huai (huaiyin.thu@gmail.com) wrote:
Can you try "sbt/sbt clean" first?
On Sat, Sep 13, 2014 at 4:29 PM, Ted Yu <yu...@gmail.com> wrote:
bq. [error] File name too long
It is not clear which file(s) loadfiles was loading.
Is the filename in earlier part of the output ?
Cheers
On Sat, Sep 13, 2014 at 10:58 AM, kkptninja <kk...@gmail.com> wrote:
Hi Ted,
Thanks for the prompt reply :)
please find details of the issue at this url http://pastebin.com/Xt0hZ38q
<http://pastebin.com/Xt0hZ38q>
Kind Regards
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/compiling-spark-source-code-tp13980p14175.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: compiling spark source code
Posted by Yin Huai <hu...@gmail.com>.
Can you try "sbt/sbt clean" first?
On Sat, Sep 13, 2014 at 4:29 PM, Ted Yu <yu...@gmail.com> wrote:
> bq. [error] File name too long
>
> It is not clear which file(s) loadfiles was loading.
> Is the filename in earlier part of the output ?
>
> Cheers
>
> On Sat, Sep 13, 2014 at 10:58 AM, kkptninja <kk...@gmail.com> wrote:
>
>> Hi Ted,
>>
>> Thanks for the prompt reply :)
>>
>> please find details of the issue at this url
>> http://pastebin.com/Xt0hZ38q
>> <http://pastebin.com/Xt0hZ38q>
>>
>> Kind Regards
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/compiling-spark-source-code-tp13980p14175.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
Re: compiling spark source code
Posted by Ted Yu <yu...@gmail.com>.
bq. [error] File name too long
It is not clear which file(s) loadfiles was loading.
Is the filename in earlier part of the output ?
Cheers
On Sat, Sep 13, 2014 at 10:58 AM, kkptninja <kk...@gmail.com> wrote:
> Hi Ted,
>
> Thanks for the prompt reply :)
>
> please find details of the issue at this url http://pastebin.com/Xt0hZ38q
> <http://pastebin.com/Xt0hZ38q>
>
> Kind Regards
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/compiling-spark-source-code-tp13980p14175.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: compiling spark source code
Posted by kkptninja <kk...@gmail.com>.
Hi Ted,
Thanks for the prompt reply :)
please find details of the issue at this url http://pastebin.com/Xt0hZ38q
<http://pastebin.com/Xt0hZ38q>
Kind Regards
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/compiling-spark-source-code-tp13980p14175.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org