You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Umar Javed <um...@gmail.com> on 2013/10/25 11:29:11 UTC
help with sbt
Hi,
After making a change to a .scala file in the spark source code, how I
build it using sbt? Do I have to use 'sbt/sbt assembly' again?
cheers,
Umar
Re: help with sbt
Posted by Cesar Arevalo <ce...@cesararevalo.com>.
Yes. That's what Sergey meant.
-Cesar
On Oct 25, 2013, at 4:23 AM, Umar Javed <um...@gmail.com> wrote:
so assembly will be required if the master and worker sit on different
machines?
On Fri, Oct 25, 2013 at 4:17 AM, Sergey Soldatov
<se...@gmail.com>wrote:
> In this case package would be enough. If you run a cluster the assembly
> may be required.
>
>
> On Fri, Oct 25, 2013 at 3:08 PM, Umar Javed <um...@gmail.com> wrote:
>
>> I'm sorry I didn't quite understand that. I'm using and running spark on
>> my local machine. What did you mean by infrastructure?
>>
>>
>> On Fri, Oct 25, 2013 at 4:01 AM, Sergey Soldatov <
>> sergey.soldatov@gmail.com> wrote:
>>
>>> For building Spark only you may use sbt/sbt package. It's much faster.
>>> But if you use Spark in some infrastructure which requires the assembly,
>>> the only way is to use assembly target.
>>>
>>>
>>> On Fri, Oct 25, 2013 at 2:34 PM, Umar Javed <um...@gmail.com>wrote:
>>>
>>>> thanks. This takes a lot of time though. It takes me 10 mins for the
>>>> build to finish after changing a single line of code. Is there a quicker
>>>> way?
>>>>
>>>>
>>>> On Fri, Oct 25, 2013 at 2:35 AM, Sergey Soldatov <
>>>> sergey.soldatov@gmail.com> wrote:
>>>>
>>>>> Hi Umar,
>>>>> Exactly. You need to use sbt/sbt assembly
>>>>> It will compile only changed files and build the assembly.
>>>>>
>>>>> Thanks,
>>>>> Sergey
>>>>>
>>>>>
>>>>> On Fri, Oct 25, 2013 at 1:29 PM, Umar Javed <um...@gmail.com>wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> After making a change to a .scala file in the spark source code, how
>>>>>> I build it using sbt? Do I have to use 'sbt/sbt assembly' again?
>>>>>>
>>>>>> cheers,
>>>>>> Umar
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: help with sbt
Posted by Umar Javed <um...@gmail.com>.
so assembly will be required if the master and worker sit on different
machines?
On Fri, Oct 25, 2013 at 4:17 AM, Sergey Soldatov
<se...@gmail.com>wrote:
> In this case package would be enough. If you run a cluster the assembly
> may be required.
>
>
> On Fri, Oct 25, 2013 at 3:08 PM, Umar Javed <um...@gmail.com> wrote:
>
>> I'm sorry I didn't quite understand that. I'm using and running spark on
>> my local machine. What did you mean by infrastructure?
>>
>>
>> On Fri, Oct 25, 2013 at 4:01 AM, Sergey Soldatov <
>> sergey.soldatov@gmail.com> wrote:
>>
>>> For building Spark only you may use sbt/sbt package. It's much faster.
>>> But if you use Spark in some infrastructure which requires the assembly,
>>> the only way is to use assembly target.
>>>
>>>
>>> On Fri, Oct 25, 2013 at 2:34 PM, Umar Javed <um...@gmail.com>wrote:
>>>
>>>> thanks. This takes a lot of time though. It takes me 10 mins for the
>>>> build to finish after changing a single line of code. Is there a quicker
>>>> way?
>>>>
>>>>
>>>> On Fri, Oct 25, 2013 at 2:35 AM, Sergey Soldatov <
>>>> sergey.soldatov@gmail.com> wrote:
>>>>
>>>>> Hi Umar,
>>>>> Exactly. You need to use sbt/sbt assembly
>>>>> It will compile only changed files and build the assembly.
>>>>>
>>>>> Thanks,
>>>>> Sergey
>>>>>
>>>>>
>>>>> On Fri, Oct 25, 2013 at 1:29 PM, Umar Javed <um...@gmail.com>wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> After making a change to a .scala file in the spark source code, how
>>>>>> I build it using sbt? Do I have to use 'sbt/sbt assembly' again?
>>>>>>
>>>>>> cheers,
>>>>>> Umar
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: help with sbt
Posted by Sergey Soldatov <se...@gmail.com>.
In this case package would be enough. If you run a cluster the assembly may
be required.
On Fri, Oct 25, 2013 at 3:08 PM, Umar Javed <um...@gmail.com> wrote:
> I'm sorry I didn't quite understand that. I'm using and running spark on
> my local machine. What did you mean by infrastructure?
>
>
> On Fri, Oct 25, 2013 at 4:01 AM, Sergey Soldatov <
> sergey.soldatov@gmail.com> wrote:
>
>> For building Spark only you may use sbt/sbt package. It's much faster.
>> But if you use Spark in some infrastructure which requires the assembly,
>> the only way is to use assembly target.
>>
>>
>> On Fri, Oct 25, 2013 at 2:34 PM, Umar Javed <um...@gmail.com>wrote:
>>
>>> thanks. This takes a lot of time though. It takes me 10 mins for the
>>> build to finish after changing a single line of code. Is there a quicker
>>> way?
>>>
>>>
>>> On Fri, Oct 25, 2013 at 2:35 AM, Sergey Soldatov <
>>> sergey.soldatov@gmail.com> wrote:
>>>
>>>> Hi Umar,
>>>> Exactly. You need to use sbt/sbt assembly
>>>> It will compile only changed files and build the assembly.
>>>>
>>>> Thanks,
>>>> Sergey
>>>>
>>>>
>>>> On Fri, Oct 25, 2013 at 1:29 PM, Umar Javed <um...@gmail.com>wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> After making a change to a .scala file in the spark source code, how I
>>>>> build it using sbt? Do I have to use 'sbt/sbt assembly' again?
>>>>>
>>>>> cheers,
>>>>> Umar
>>>>>
>>>>
>>>>
>>>
>>
>
Re: help with sbt
Posted by Umar Javed <um...@gmail.com>.
I'm sorry I didn't quite understand that. I'm using and running spark on my
local machine. What did you mean by infrastructure?
On Fri, Oct 25, 2013 at 4:01 AM, Sergey Soldatov
<se...@gmail.com>wrote:
> For building Spark only you may use sbt/sbt package. It's much faster. But
> if you use Spark in some infrastructure which requires the assembly, the
> only way is to use assembly target.
>
>
> On Fri, Oct 25, 2013 at 2:34 PM, Umar Javed <um...@gmail.com> wrote:
>
>> thanks. This takes a lot of time though. It takes me 10 mins for the
>> build to finish after changing a single line of code. Is there a quicker
>> way?
>>
>>
>> On Fri, Oct 25, 2013 at 2:35 AM, Sergey Soldatov <
>> sergey.soldatov@gmail.com> wrote:
>>
>>> Hi Umar,
>>> Exactly. You need to use sbt/sbt assembly
>>> It will compile only changed files and build the assembly.
>>>
>>> Thanks,
>>> Sergey
>>>
>>>
>>> On Fri, Oct 25, 2013 at 1:29 PM, Umar Javed <um...@gmail.com>wrote:
>>>
>>>> Hi,
>>>>
>>>> After making a change to a .scala file in the spark source code, how I
>>>> build it using sbt? Do I have to use 'sbt/sbt assembly' again?
>>>>
>>>> cheers,
>>>> Umar
>>>>
>>>
>>>
>>
>
Re: help with sbt
Posted by Sergey Soldatov <se...@gmail.com>.
For building Spark only you may use sbt/sbt package. It's much faster. But
if you use Spark in some infrastructure which requires the assembly, the
only way is to use assembly target.
On Fri, Oct 25, 2013 at 2:34 PM, Umar Javed <um...@gmail.com> wrote:
> thanks. This takes a lot of time though. It takes me 10 mins for the build
> to finish after changing a single line of code. Is there a quicker way?
>
>
> On Fri, Oct 25, 2013 at 2:35 AM, Sergey Soldatov <
> sergey.soldatov@gmail.com> wrote:
>
>> Hi Umar,
>> Exactly. You need to use sbt/sbt assembly
>> It will compile only changed files and build the assembly.
>>
>> Thanks,
>> Sergey
>>
>>
>> On Fri, Oct 25, 2013 at 1:29 PM, Umar Javed <um...@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> After making a change to a .scala file in the spark source code, how I
>>> build it using sbt? Do I have to use 'sbt/sbt assembly' again?
>>>
>>> cheers,
>>> Umar
>>>
>>
>>
>
Re: help with sbt
Posted by Umar Javed <um...@gmail.com>.
thanks. This takes a lot of time though. It takes me 10 mins for the build
to finish after changing a single line of code. Is there a quicker way?
On Fri, Oct 25, 2013 at 2:35 AM, Sergey Soldatov
<se...@gmail.com>wrote:
> Hi Umar,
> Exactly. You need to use sbt/sbt assembly
> It will compile only changed files and build the assembly.
>
> Thanks,
> Sergey
>
>
> On Fri, Oct 25, 2013 at 1:29 PM, Umar Javed <um...@gmail.com> wrote:
>
>> Hi,
>>
>> After making a change to a .scala file in the spark source code, how I
>> build it using sbt? Do I have to use 'sbt/sbt assembly' again?
>>
>> cheers,
>> Umar
>>
>
>
Re: help with sbt
Posted by Sergey Soldatov <se...@gmail.com>.
Hi Umar,
Exactly. You need to use sbt/sbt assembly
It will compile only changed files and build the assembly.
Thanks,
Sergey
On Fri, Oct 25, 2013 at 1:29 PM, Umar Javed <um...@gmail.com> wrote:
> Hi,
>
> After making a change to a .scala file in the spark source code, how I
> build it using sbt? Do I have to use 'sbt/sbt assembly' again?
>
> cheers,
> Umar
>