You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Stephen Boesch <ja...@gmail.com> on 2015/02/05 04:25:13 UTC
Broken record a bit here: building spark on intellij with sbt
For building in intellij with sbt my mileage has varied widely: it had
built as late as Monday (after the 1.3.0 release) - and with zero
'special' steps: just "import" as sbt project.
However I can not presently repeat the process. The wiki page has the
latest instructions on how to build with maven - but not with sbt. Is
there a resource for that?
https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark#ContributingtoSpark-IntelliJ
The error I see is same as from a post in July
http://apache-spark-user-list.1001560.n3.nabble.com/Build-spark-with-Intellij-IDEA-13-td9904.html
Here is an excerpt:
uncaught exception during compilation: java.lang.AssertionError
Error:scalac: Error: assertion failed:
com.google.protobuf.InvalidProtocolBufferException
java.lang.AssertionError: assertion failed:
com.google.protobuf.InvalidProtocolBufferException
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1212)
The answer in the mailing list to that thread was about using maven .. so
that is not useful here.
Re: Broken record a bit here: building spark on intellij with sbt
Posted by Deep Pradhan <pr...@gmail.com>.
Akhil, it is not able to find the SBT package when I tried the steps given
in the site.
On Thu, Feb 5, 2015 at 11:16 AM, Akhil Das <ak...@sigmoidanalytics.com>
wrote:
> Here's the sbt version
>
> https://docs.sigmoidanalytics.com/index.php/Step_by_Step_instructions_on_how_to_build_Spark_App_with_IntelliJ_IDEA
>
>
> Thanks
> Best Regards
>
> On Thu, Feb 5, 2015 at 8:55 AM, Stephen Boesch <ja...@gmail.com> wrote:
>
> > For building in intellij with sbt my mileage has varied widely: it had
> > built as late as Monday (after the 1.3.0 release) - and with zero
> > 'special' steps: just "import" as sbt project.
> >
> > However I can not presently repeat the process. The wiki page has the
> > latest instructions on how to build with maven - but not with sbt. Is
> > there a resource for that?
> >
> >
> >
> https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark#ContributingtoSpark-IntelliJ
> >
> > The error I see is same as from a post in July
> >
> >
> >
> http://apache-spark-user-list.1001560.n3.nabble.com/Build-spark-with-Intellij-IDEA-13-td9904.html
> >
> > Here is an excerpt:
> >
> > uncaught exception during compilation: java.lang.AssertionError
> > Error:scalac: Error: assertion failed:
> > com.google.protobuf.InvalidProtocolBufferException
> > java.lang.AssertionError: assertion failed:
> > com.google.protobuf.InvalidProtocolBufferException
> > at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1212)
> >
> > The answer in the mailing list to that thread was about using maven .. so
> > that is not useful here.
> >
>
Re: Broken record a bit here: building spark on intellij with sbt
Posted by Arush Kharbanda <ar...@sigmoidanalytics.com>.
I follow these ones to import sbt projects.
1. Install sbt plugins: Goto File -> Settings -> Plugins -> Install
IntelliJ Plugins -> Search for sbt and install it
2. File ->Import->browse the root of spark source code
I hope this helps
On Fri, Feb 6, 2015 at 1:41 AM, Stephen Boesch <ja...@gmail.com> wrote:
> Hi Akhil
> Those instructions you provided are showing how to manually build an sbt
> project that may include adding spark dependencies. Whereas my OP was
> about how to open the existing spark sbt project . These two are not
> similar tasks.
>
> 2015-02-04 21:46 GMT-08:00 Akhil Das <ak...@sigmoidanalytics.com>:
>
> > Here's the sbt version
> >
> https://docs.sigmoidanalytics.com/index.php/Step_by_Step_instructions_on_how_to_build_Spark_App_with_IntelliJ_IDEA
> >
> >
> > Thanks
> > Best Regards
> >
> > On Thu, Feb 5, 2015 at 8:55 AM, Stephen Boesch <ja...@gmail.com>
> wrote:
> >
> >> For building in intellij with sbt my mileage has varied widely: it had
> >> built as late as Monday (after the 1.3.0 release) - and with zero
> >> 'special' steps: just "import" as sbt project.
> >>
> >> However I can not presently repeat the process. The wiki page has the
> >> latest instructions on how to build with maven - but not with sbt. Is
> >> there a resource for that?
> >>
> >>
> >>
> https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark#ContributingtoSpark-IntelliJ
> >>
> >> The error I see is same as from a post in July
> >>
> >>
> >>
> http://apache-spark-user-list.1001560.n3.nabble.com/Build-spark-with-Intellij-IDEA-13-td9904.html
> >>
> >> Here is an excerpt:
> >>
> >> uncaught exception during compilation: java.lang.AssertionError
> >> Error:scalac: Error: assertion failed:
> >> com.google.protobuf.InvalidProtocolBufferException
> >> java.lang.AssertionError: assertion failed:
> >> com.google.protobuf.InvalidProtocolBufferException
> >> at
> scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1212)
> >>
> >> The answer in the mailing list to that thread was about using maven ..
> so
> >> that is not useful here.
> >>
> >
> >
>
--
[image: Sigmoid Analytics] <http://htmlsig.com/www.sigmoidanalytics.com>
Arush Kharbanda || Technical Teamlead
arush@sigmoidanalytics.com || www.sigmoidanalytics.com
Re: Broken record a bit here: building spark on intellij with sbt
Posted by Stephen Boesch <ja...@gmail.com>.
Hi Akhil
Those instructions you provided are showing how to manually build an sbt
project that may include adding spark dependencies. Whereas my OP was
about how to open the existing spark sbt project . These two are not
similar tasks.
2015-02-04 21:46 GMT-08:00 Akhil Das <ak...@sigmoidanalytics.com>:
> Here's the sbt version
> https://docs.sigmoidanalytics.com/index.php/Step_by_Step_instructions_on_how_to_build_Spark_App_with_IntelliJ_IDEA
>
>
> Thanks
> Best Regards
>
> On Thu, Feb 5, 2015 at 8:55 AM, Stephen Boesch <ja...@gmail.com> wrote:
>
>> For building in intellij with sbt my mileage has varied widely: it had
>> built as late as Monday (after the 1.3.0 release) - and with zero
>> 'special' steps: just "import" as sbt project.
>>
>> However I can not presently repeat the process. The wiki page has the
>> latest instructions on how to build with maven - but not with sbt. Is
>> there a resource for that?
>>
>>
>> https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark#ContributingtoSpark-IntelliJ
>>
>> The error I see is same as from a post in July
>>
>>
>> http://apache-spark-user-list.1001560.n3.nabble.com/Build-spark-with-Intellij-IDEA-13-td9904.html
>>
>> Here is an excerpt:
>>
>> uncaught exception during compilation: java.lang.AssertionError
>> Error:scalac: Error: assertion failed:
>> com.google.protobuf.InvalidProtocolBufferException
>> java.lang.AssertionError: assertion failed:
>> com.google.protobuf.InvalidProtocolBufferException
>> at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1212)
>>
>> The answer in the mailing list to that thread was about using maven .. so
>> that is not useful here.
>>
>
>
Re: Broken record a bit here: building spark on intellij with sbt
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Here's the sbt version
https://docs.sigmoidanalytics.com/index.php/Step_by_Step_instructions_on_how_to_build_Spark_App_with_IntelliJ_IDEA
Thanks
Best Regards
On Thu, Feb 5, 2015 at 8:55 AM, Stephen Boesch <ja...@gmail.com> wrote:
> For building in intellij with sbt my mileage has varied widely: it had
> built as late as Monday (after the 1.3.0 release) - and with zero
> 'special' steps: just "import" as sbt project.
>
> However I can not presently repeat the process. The wiki page has the
> latest instructions on how to build with maven - but not with sbt. Is
> there a resource for that?
>
>
> https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark#ContributingtoSpark-IntelliJ
>
> The error I see is same as from a post in July
>
>
> http://apache-spark-user-list.1001560.n3.nabble.com/Build-spark-with-Intellij-IDEA-13-td9904.html
>
> Here is an excerpt:
>
> uncaught exception during compilation: java.lang.AssertionError
> Error:scalac: Error: assertion failed:
> com.google.protobuf.InvalidProtocolBufferException
> java.lang.AssertionError: assertion failed:
> com.google.protobuf.InvalidProtocolBufferException
> at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1212)
>
> The answer in the mailing list to that thread was about using maven .. so
> that is not useful here.
>