You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Shengkai Zhu <ge...@gmail.com> on 2008/07/14 15:19:20 UTC
Why is the task run in a child JVM?
What's the benefits from such design compared to multi-thread?
--
朱盛凯
Jash Zhu
复旦大学软件学院
Software School, Fudan University
Re: Why is the task run in a child JVM?
Posted by Torsten Curdt <tc...@apache.org>.
> On 7/14/08, Jason Venner <ja...@attributor.com> wrote:
>>
>> One benefit is that if your map or reduce behaves badly it can't
>> take down
>> the task tracker.
As the tracker jvm could also be monitored (and restarted) from
outside, the internal execution might still be worth looking into. At
least to have the option. We had a patch ...but it's terrible out of
date.
On the other hand starting up the second jvm is supposed to be much
faster on 1.6 ....no idea if that's true though.
cheers
--
Torsten
Re: Why is the task run in a child JVM?
Posted by Shengkai Zhu <ge...@gmail.com>.
Well, I got it.
On 7/14/08, Jason Venner <ja...@attributor.com> wrote:
>
> One benefit is that if your map or reduce behaves badly it can't take down
> the task tracker.
>
> In our case we have some poorly behaved external native libraries we use,
> and we have to forcibly ensure that the child vms are killed when the child
> main finishes (often by kill -9), so the fact the child (task) is a separate
> jvm process is very helpful.
>
> The downside is the jvm start time. Has anyone experimented with the jar
> freezing for more than the standard boot class path jars to speed up
> startup?
>
>
> Shengkai Zhu wrote:
>
>> What's the benefits from such design compared to multi-thread?
>>
>>
>>
> --
> Jason Venner
> Attributor - Program the Web <http://www.attributor.com/>
> Attributor is hiring Hadoop Wranglers and coding wizards, contact if
> interested
>
--
朱盛凯
Jash Zhu
复旦大学软件学院
Software School, Fudan University
Re: Why is the task run in a child JVM?
Posted by Jason Venner <ja...@attributor.com>.
One benefit is that if your map or reduce behaves badly it can't take
down the task tracker.
In our case we have some poorly behaved external native libraries we
use, and we have to forcibly ensure that the child vms are killed when
the child main finishes (often by kill -9), so the fact the child (task)
is a separate jvm process is very helpful.
The downside is the jvm start time. Has anyone experimented with the jar
freezing for more than the standard boot class path jars to speed up
startup?
Shengkai Zhu wrote:
> What's the benefits from such design compared to multi-thread?
>
>
--
Jason Venner
Attributor - Program the Web <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers and coding wizards, contact if
interested