You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Tyson Norris <tn...@adobe.com> on 2014/04/10 21:58:50 UTC
local debugging?
Hi -
Is there an easy way to run tasks locally all within the same jvm to ease debugging? Or else, is there a good way to attach a debugger to separate task processes?
I’m thinking something along the lines of Storm’s LocalCluster, where there is some emulation of separate processes but using threads within the same jvm.
Thanks
Tyson
Re: local debugging?
Posted by Chris Riccomini <cr...@linkedin.com>.
Hey Tyson,
There's also a remote debugging tutorial:
http://samza.incubator.apache.org/learn/tutorials/0.7.0/remote-debugging-sa
mza.html
Unfortunately, this tutorial is affected by this bug:
https://issues.apache.org/jira/browse/SAMZA-211
So you might run into some difficulty until it's resolved.
Cheers,
Chris
On 4/10/14 1:26 PM, "Yan Fang" <ya...@gmail.com> wrote:
>Hi Tyson,
>
>If you change the YarnJobFactory to LocalJobRunner in configuration file,
>Samza will run in local mode.
>
>http://samza.incubator.apache.org/learn/documentation/0.7.0/jobs/job-runne
>r.html
>
>Thanks,
>Fang, Yan
>yanfang724@gmail.com
>+1 (206) 849-4108
>
>
>On Thu, Apr 10, 2014 at 12:58 PM, Tyson Norris <tn...@adobe.com> wrote:
>
>> Hi -
>> Is there an easy way to run tasks locally all within the same jvm to
>>ease
>> debugging? Or else, is there a good way to attach a debugger to
>>separate
>> task processes?
>>
>> I¹m thinking something along the lines of Storm¹s LocalCluster, where
>> there is some emulation of separate processes but using threads within
>>the
>> same jvm.
>>
>> Thanks
>> Tyson
Re: local debugging?
Posted by Chris Riccomini <cr...@linkedin.com>.
Hey Tyson,
Correct. If you have two jobs you wish to debug, you start each one
independently in their own JVM. You then have to debug them independently.
Cheers,
Chris
On 4/10/14 1:58 PM, "Tyson Norris" <tn...@adobe.com> wrote:
>Hi Yan -
>
>I think this is slightly different. From the docs:
>"In the case of LocalJobRunner, it uses a run-container.sh script to
>execute the TaskRunner in a separate process, which will start one
>TaskRunner locally on the machine that you ran run-job.sh on.²
>
>which means it will be in a separate jvm, although it will be local to
>the host where the job is running afaik.
>
>It still means that I would not be able to debug multiple tasks in the
>same debugger session, I think?
>
>Thanks
>Tyson
>
>On Apr 10, 2014, at 1:26 PM, Yan Fang <ya...@gmail.com> wrote:
>
>> Hi Tyson,
>>
>> If you change the YarnJobFactory to LocalJobRunner in configuration
>>file,
>> Samza will run in local mode.
>>
>>
>>http://samza.incubator.apache.org/learn/documentation/0.7.0/jobs/job-runn
>>er.html
>>
>> Thanks,
>> Fang, Yan
>> yanfang724@gmail.com
>> +1 (206) 849-4108
>>
>>
>> On Thu, Apr 10, 2014 at 12:58 PM, Tyson Norris <tn...@adobe.com>
>>wrote:
>>
>>> Hi -
>>> Is there an easy way to run tasks locally all within the same jvm to
>>>ease
>>> debugging? Or else, is there a good way to attach a debugger to
>>>separate
>>> task processes?
>>>
>>> I¹m thinking something along the lines of Storm¹s LocalCluster, where
>>> there is some emulation of separate processes but using threads within
>>>the
>>> same jvm.
>>>
>>> Thanks
>>> Tyson
>
Re: local debugging?
Posted by Tyson Norris <tn...@adobe.com>.
Hi Yan -
I think this is slightly different. From the docs:
"In the case of LocalJobRunner, it uses a run-container.sh script to execute the TaskRunner in a separate process, which will start one TaskRunner locally on the machine that you ran run-job.sh on.”
which means it will be in a separate jvm, although it will be local to the host where the job is running afaik.
It still means that I would not be able to debug multiple tasks in the same debugger session, I think?
Thanks
Tyson
On Apr 10, 2014, at 1:26 PM, Yan Fang <ya...@gmail.com> wrote:
> Hi Tyson,
>
> If you change the YarnJobFactory to LocalJobRunner in configuration file,
> Samza will run in local mode.
>
> http://samza.incubator.apache.org/learn/documentation/0.7.0/jobs/job-runner.html
>
> Thanks,
> Fang, Yan
> yanfang724@gmail.com
> +1 (206) 849-4108
>
>
> On Thu, Apr 10, 2014 at 12:58 PM, Tyson Norris <tn...@adobe.com> wrote:
>
>> Hi -
>> Is there an easy way to run tasks locally all within the same jvm to ease
>> debugging? Or else, is there a good way to attach a debugger to separate
>> task processes?
>>
>> I’m thinking something along the lines of Storm’s LocalCluster, where
>> there is some emulation of separate processes but using threads within the
>> same jvm.
>>
>> Thanks
>> Tyson
Re: local debugging?
Posted by Yan Fang <ya...@gmail.com>.
Hi Tyson,
If you change the YarnJobFactory to LocalJobRunner in configuration file,
Samza will run in local mode.
http://samza.incubator.apache.org/learn/documentation/0.7.0/jobs/job-runner.html
Thanks,
Fang, Yan
yanfang724@gmail.com
+1 (206) 849-4108
On Thu, Apr 10, 2014 at 12:58 PM, Tyson Norris <tn...@adobe.com> wrote:
> Hi -
> Is there an easy way to run tasks locally all within the same jvm to ease
> debugging? Or else, is there a good way to attach a debugger to separate
> task processes?
>
> I’m thinking something along the lines of Storm’s LocalCluster, where
> there is some emulation of separate processes but using threads within the
> same jvm.
>
> Thanks
> Tyson