You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Justin Uang <ju...@gmail.com> on 2015/09/08 06:02:11 UTC

Fast Iteration while developing

Hi,

What is the normal workflow for the core devs?

- Do we need to build the assembly jar to be able to run it from the spark
repo?
- Do you use sbt or maven to do the build?
- Is zinc only usuable for maven?

I'm asking because the current process I have right now is to do sbt build,
which means I'm stuck with about a 3-5 minute iteration cycle.

Thanks!

Justin

Re: Fast Iteration while developing

Posted by Michael Armbrust <mi...@databricks.com>.

+1 to reynolds suggestion.  This is probably the fastest way to iterate.

Another option for more ad-hoc debugging is `sbt/sbt sparkShell` which is
similar to bin/spark-shell but doesn't require you to rebuild the assembly
jar.

On Mon, Sep 7, 2015 at 9:03 PM, Reynold Xin <rx...@databricks.com> wrote:

> I usually write a test case for what I want to test, and then run
>
> sbt/sbt "~module/test:test-only *MyTestSuite"
>
>
>
> On Mon, Sep 7, 2015 at 6:02 PM, Justin Uang <ju...@gmail.com> wrote:
>
>> Hi,
>>
>> What is the normal workflow for the core devs?
>>
>> - Do we need to build the assembly jar to be able to run it from the
>> spark repo?
>> - Do you use sbt or maven to do the build?
>> - Is zinc only usuable for maven?
>>
>> I'm asking because the current process I have right now is to do sbt
>> build, which means I'm stuck with about a 3-5 minute iteration cycle.
>>
>> Thanks!
>>
>> Justin
>>
>
>

Re: Fast Iteration while developing

Posted by Reynold Xin <rx...@databricks.com>.

I usually write a test case for what I want to test, and then run

sbt/sbt "~module/test:test-only *MyTestSuite"



On Mon, Sep 7, 2015 at 6:02 PM, Justin Uang <ju...@gmail.com> wrote:

> Hi,
>
> What is the normal workflow for the core devs?
>
> - Do we need to build the assembly jar to be able to run it from the spark
> repo?
> - Do you use sbt or maven to do the build?
> - Is zinc only usuable for maven?
>
> I'm asking because the current process I have right now is to do sbt
> build, which means I'm stuck with about a 3-5 minute iteration cycle.
>
> Thanks!
>
> Justin
>