You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by Ariel Rabkin <as...@gmail.com> on 2009/05/14 08:28:31 UTC
another rc4 issue?
Also, the demux mapred jobs all fail almost immediately with an
out-of-memory. The sort buffer sizes in chukwa-demux-conf are
incompatible with the Hadoop defaults.
I think we should dial down the default sizes in chukwa-demux-conf.
Outside Yahoo!, much smaller buffers should still be OK, since nobody
else will be running Chukwa at large scale for a while.
--Ari
--
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department
Re: another rc4 issue?
Posted by Mac Yang <ma...@yahoo-inc.com>.
Unfortunately that override / merging infrastructure is not completely up
yet. Once that's done then we will be able to handle this type of situations
more cleanly. And I do agree that this needs to get addressed before we make
the external release.
On 5/14/09 9:33 AM, "Ariel Rabkin" <as...@gmail.com> wrote:
> I just opened CHUKWA-239 for this. It should be a near-trivial patch.
>
> Am I right in assuming that Yahoo! has internal corporate settings
> which will override chukwa-demux-conf anyway?
>
> --Ari
>
> On Thu, May 14, 2009 at 8:04 AM, Cheng Zhang <ch...@yahoo-inc.com> wrote:
>> +1. As an open source project, the default settings should be workable.
>>
>>
>>
>>> -----Original Message-----
>>> From: Ariel Rabkin [mailto:asrabkin@gmail.com]
>>> Sent: Wednesday, May 13, 2009 11:29 PM
>>> To: chukwa-dev@hadoop.apache.org
>>> Subject: another rc4 issue?
>>>
>>> Also, the demux mapred jobs all fail almost immediately with an
>>> out-of-memory. The sort buffer sizes in chukwa-demux-conf are
>>> incompatible with the Hadoop defaults.
>>>
>>> I think we should dial down the default sizes in chukwa-demux-conf.
>>> Outside Yahoo!, much smaller buffers should still be OK, since nobody
>>> else will be running Chukwa at large scale for a while.
>>>
>>> --Ari
>>>
>>> --
>>> Ari Rabkin asrabkin@gmail.com
>>> UC Berkeley Computer Science Department
>>
>
>
Re: another rc4 issue?
Posted by Ariel Rabkin <as...@gmail.com>.
I just opened CHUKWA-239 for this. It should be a near-trivial patch.
Am I right in assuming that Yahoo! has internal corporate settings
which will override chukwa-demux-conf anyway?
--Ari
On Thu, May 14, 2009 at 8:04 AM, Cheng Zhang <ch...@yahoo-inc.com> wrote:
> +1. As an open source project, the default settings should be workable.
>
>
>
>> -----Original Message-----
>> From: Ariel Rabkin [mailto:asrabkin@gmail.com]
>> Sent: Wednesday, May 13, 2009 11:29 PM
>> To: chukwa-dev@hadoop.apache.org
>> Subject: another rc4 issue?
>>
>> Also, the demux mapred jobs all fail almost immediately with an
>> out-of-memory. The sort buffer sizes in chukwa-demux-conf are
>> incompatible with the Hadoop defaults.
>>
>> I think we should dial down the default sizes in chukwa-demux-conf.
>> Outside Yahoo!, much smaller buffers should still be OK, since nobody
>> else will be running Chukwa at large scale for a while.
>>
>> --Ari
>>
>> --
>> Ari Rabkin asrabkin@gmail.com
>> UC Berkeley Computer Science Department
>
--
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department
RE: another rc4 issue?
Posted by Cheng Zhang <ch...@yahoo-inc.com>.
+1. As an open source project, the default settings should be workable.
> -----Original Message-----
> From: Ariel Rabkin [mailto:asrabkin@gmail.com]
> Sent: Wednesday, May 13, 2009 11:29 PM
> To: chukwa-dev@hadoop.apache.org
> Subject: another rc4 issue?
>
> Also, the demux mapred jobs all fail almost immediately with an
> out-of-memory. The sort buffer sizes in chukwa-demux-conf are
> incompatible with the Hadoop defaults.
>
> I think we should dial down the default sizes in chukwa-demux-conf.
> Outside Yahoo!, much smaller buffers should still be OK, since nobody
> else will be running Chukwa at large scale for a while.
>
> --Ari
>
> --
> Ari Rabkin asrabkin@gmail.com
> UC Berkeley Computer Science Department