You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Geelong Yao <ge...@gmail.com> on 2013/12/03 08:14:19 UTC
Perfect configuration setting
Hi Everyone
I am now testing the best performance of my cluster
Can anyone give me some *formulas or suggestion* for all setting values in
MR program?
eg.
io.sort.mb
io.sort.spill.percent
mapred.local.dir
mapred.child.java.opts etc.
Any details would be great
BRs
Geelong
--
>From Good To Great
Re: Perfect configuration setting
Posted by Azuryy Yu <az...@gmail.com>.
Sandy, I can access it now. but I do failed many times before I reply your
mail. Thanks.
On Wed, Dec 4, 2013 at 9:49 AM, Sandy Ryza <sa...@cloudera.com> wrote:
> Weird. I just tried exiting the Cloudera VPN and opening it from a
> browser I never use and was still able to access it.
>
>
> On Tue, Dec 3, 2013 at 5:46 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> Hi Sandy,
>> I think there are some permission limited on your shared URL. maybe only
>> Cloudera employees allowed.
>>
>>
>> On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com>wrote:
>>
>>> Hi Geelong,
>>>
>>> Check out Todd Lipcon's presentation on tuning MapReduce performance:
>>> http://www.slideshare.net/cloudera/mr-perf
>>>
>>> -Sandy
>>>
>>>
>>> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com>wrote:
>>>
>>>> Hi Everyone
>>>>
>>>> I am now testing the best performance of my cluster
>>>> Can anyone give me some *formulas or suggestion* for all setting
>>>> values in MR program?
>>>> eg.
>>>> io.sort.mb
>>>> io.sort.spill.percent
>>>> mapred.local.dir
>>>> mapred.child.java.opts etc.
>>>>
>>>> Any details would be great
>>>>
>>>> BRs
>>>> Geelong
>>>>
>>>> --
>>>> From Good To Great
>>>>
>>>
>>>
>>
>
Re: Perfect configuration setting
Posted by Azuryy Yu <az...@gmail.com>.
Sandy, I can access it now. but I do failed many times before I reply your
mail. Thanks.
On Wed, Dec 4, 2013 at 9:49 AM, Sandy Ryza <sa...@cloudera.com> wrote:
> Weird. I just tried exiting the Cloudera VPN and opening it from a
> browser I never use and was still able to access it.
>
>
> On Tue, Dec 3, 2013 at 5:46 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> Hi Sandy,
>> I think there are some permission limited on your shared URL. maybe only
>> Cloudera employees allowed.
>>
>>
>> On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com>wrote:
>>
>>> Hi Geelong,
>>>
>>> Check out Todd Lipcon's presentation on tuning MapReduce performance:
>>> http://www.slideshare.net/cloudera/mr-perf
>>>
>>> -Sandy
>>>
>>>
>>> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com>wrote:
>>>
>>>> Hi Everyone
>>>>
>>>> I am now testing the best performance of my cluster
>>>> Can anyone give me some *formulas or suggestion* for all setting
>>>> values in MR program?
>>>> eg.
>>>> io.sort.mb
>>>> io.sort.spill.percent
>>>> mapred.local.dir
>>>> mapred.child.java.opts etc.
>>>>
>>>> Any details would be great
>>>>
>>>> BRs
>>>> Geelong
>>>>
>>>> --
>>>> From Good To Great
>>>>
>>>
>>>
>>
>
Re: Perfect configuration setting
Posted by Azuryy Yu <az...@gmail.com>.
Sandy, I can access it now. but I do failed many times before I reply your
mail. Thanks.
On Wed, Dec 4, 2013 at 9:49 AM, Sandy Ryza <sa...@cloudera.com> wrote:
> Weird. I just tried exiting the Cloudera VPN and opening it from a
> browser I never use and was still able to access it.
>
>
> On Tue, Dec 3, 2013 at 5:46 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> Hi Sandy,
>> I think there are some permission limited on your shared URL. maybe only
>> Cloudera employees allowed.
>>
>>
>> On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com>wrote:
>>
>>> Hi Geelong,
>>>
>>> Check out Todd Lipcon's presentation on tuning MapReduce performance:
>>> http://www.slideshare.net/cloudera/mr-perf
>>>
>>> -Sandy
>>>
>>>
>>> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com>wrote:
>>>
>>>> Hi Everyone
>>>>
>>>> I am now testing the best performance of my cluster
>>>> Can anyone give me some *formulas or suggestion* for all setting
>>>> values in MR program?
>>>> eg.
>>>> io.sort.mb
>>>> io.sort.spill.percent
>>>> mapred.local.dir
>>>> mapred.child.java.opts etc.
>>>>
>>>> Any details would be great
>>>>
>>>> BRs
>>>> Geelong
>>>>
>>>> --
>>>> From Good To Great
>>>>
>>>
>>>
>>
>
Re: Perfect configuration setting
Posted by Azuryy Yu <az...@gmail.com>.
Sandy, I can access it now. but I do failed many times before I reply your
mail. Thanks.
On Wed, Dec 4, 2013 at 9:49 AM, Sandy Ryza <sa...@cloudera.com> wrote:
> Weird. I just tried exiting the Cloudera VPN and opening it from a
> browser I never use and was still able to access it.
>
>
> On Tue, Dec 3, 2013 at 5:46 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> Hi Sandy,
>> I think there are some permission limited on your shared URL. maybe only
>> Cloudera employees allowed.
>>
>>
>> On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com>wrote:
>>
>>> Hi Geelong,
>>>
>>> Check out Todd Lipcon's presentation on tuning MapReduce performance:
>>> http://www.slideshare.net/cloudera/mr-perf
>>>
>>> -Sandy
>>>
>>>
>>> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com>wrote:
>>>
>>>> Hi Everyone
>>>>
>>>> I am now testing the best performance of my cluster
>>>> Can anyone give me some *formulas or suggestion* for all setting
>>>> values in MR program?
>>>> eg.
>>>> io.sort.mb
>>>> io.sort.spill.percent
>>>> mapred.local.dir
>>>> mapred.child.java.opts etc.
>>>>
>>>> Any details would be great
>>>>
>>>> BRs
>>>> Geelong
>>>>
>>>> --
>>>> From Good To Great
>>>>
>>>
>>>
>>
>
Re: Perfect configuration setting
Posted by Sandy Ryza <sa...@cloudera.com>.
Weird. I just tried exiting the Cloudera VPN and opening it from a browser
I never use and was still able to access it.
On Tue, Dec 3, 2013 at 5:46 PM, Azuryy Yu <az...@gmail.com> wrote:
> Hi Sandy,
> I think there are some permission limited on your shared URL. maybe only
> Cloudera employees allowed.
>
>
> On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com>wrote:
>
>> Hi Geelong,
>>
>> Check out Todd Lipcon's presentation on tuning MapReduce performance:
>> http://www.slideshare.net/cloudera/mr-perf
>>
>> -Sandy
>>
>>
>> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com>wrote:
>>
>>> Hi Everyone
>>>
>>> I am now testing the best performance of my cluster
>>> Can anyone give me some *formulas or suggestion* for all setting values
>>> in MR program?
>>> eg.
>>> io.sort.mb
>>> io.sort.spill.percent
>>> mapred.local.dir
>>> mapred.child.java.opts etc.
>>>
>>> Any details would be great
>>>
>>> BRs
>>> Geelong
>>>
>>> --
>>> From Good To Great
>>>
>>
>>
>
Re: Perfect configuration setting
Posted by Sandy Ryza <sa...@cloudera.com>.
Weird. I just tried exiting the Cloudera VPN and opening it from a browser
I never use and was still able to access it.
On Tue, Dec 3, 2013 at 5:46 PM, Azuryy Yu <az...@gmail.com> wrote:
> Hi Sandy,
> I think there are some permission limited on your shared URL. maybe only
> Cloudera employees allowed.
>
>
> On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com>wrote:
>
>> Hi Geelong,
>>
>> Check out Todd Lipcon's presentation on tuning MapReduce performance:
>> http://www.slideshare.net/cloudera/mr-perf
>>
>> -Sandy
>>
>>
>> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com>wrote:
>>
>>> Hi Everyone
>>>
>>> I am now testing the best performance of my cluster
>>> Can anyone give me some *formulas or suggestion* for all setting values
>>> in MR program?
>>> eg.
>>> io.sort.mb
>>> io.sort.spill.percent
>>> mapred.local.dir
>>> mapred.child.java.opts etc.
>>>
>>> Any details would be great
>>>
>>> BRs
>>> Geelong
>>>
>>> --
>>> From Good To Great
>>>
>>
>>
>
Re: Perfect configuration setting
Posted by Sandy Ryza <sa...@cloudera.com>.
Weird. I just tried exiting the Cloudera VPN and opening it from a browser
I never use and was still able to access it.
On Tue, Dec 3, 2013 at 5:46 PM, Azuryy Yu <az...@gmail.com> wrote:
> Hi Sandy,
> I think there are some permission limited on your shared URL. maybe only
> Cloudera employees allowed.
>
>
> On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com>wrote:
>
>> Hi Geelong,
>>
>> Check out Todd Lipcon's presentation on tuning MapReduce performance:
>> http://www.slideshare.net/cloudera/mr-perf
>>
>> -Sandy
>>
>>
>> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com>wrote:
>>
>>> Hi Everyone
>>>
>>> I am now testing the best performance of my cluster
>>> Can anyone give me some *formulas or suggestion* for all setting values
>>> in MR program?
>>> eg.
>>> io.sort.mb
>>> io.sort.spill.percent
>>> mapred.local.dir
>>> mapred.child.java.opts etc.
>>>
>>> Any details would be great
>>>
>>> BRs
>>> Geelong
>>>
>>> --
>>> From Good To Great
>>>
>>
>>
>
Re: Perfect configuration setting
Posted by Sandy Ryza <sa...@cloudera.com>.
Weird. I just tried exiting the Cloudera VPN and opening it from a browser
I never use and was still able to access it.
On Tue, Dec 3, 2013 at 5:46 PM, Azuryy Yu <az...@gmail.com> wrote:
> Hi Sandy,
> I think there are some permission limited on your shared URL. maybe only
> Cloudera employees allowed.
>
>
> On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com>wrote:
>
>> Hi Geelong,
>>
>> Check out Todd Lipcon's presentation on tuning MapReduce performance:
>> http://www.slideshare.net/cloudera/mr-perf
>>
>> -Sandy
>>
>>
>> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com>wrote:
>>
>>> Hi Everyone
>>>
>>> I am now testing the best performance of my cluster
>>> Can anyone give me some *formulas or suggestion* for all setting values
>>> in MR program?
>>> eg.
>>> io.sort.mb
>>> io.sort.spill.percent
>>> mapred.local.dir
>>> mapred.child.java.opts etc.
>>>
>>> Any details would be great
>>>
>>> BRs
>>> Geelong
>>>
>>> --
>>> From Good To Great
>>>
>>
>>
>
Re: Perfect configuration setting
Posted by Azuryy Yu <az...@gmail.com>.
Hi Sandy,
I think there are some permission limited on your shared URL. maybe only
Cloudera employees allowed.
On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com> wrote:
> Hi Geelong,
>
> Check out Todd Lipcon's presentation on tuning MapReduce performance:
> http://www.slideshare.net/cloudera/mr-perf
>
> -Sandy
>
>
> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com> wrote:
>
>> Hi Everyone
>>
>> I am now testing the best performance of my cluster
>> Can anyone give me some *formulas or suggestion* for all setting values
>> in MR program?
>> eg.
>> io.sort.mb
>> io.sort.spill.percent
>> mapred.local.dir
>> mapred.child.java.opts etc.
>>
>> Any details would be great
>>
>> BRs
>> Geelong
>>
>> --
>> From Good To Great
>>
>
>
Re: Perfect configuration setting
Posted by Azuryy Yu <az...@gmail.com>.
Hi Sandy,
I think there are some permission limited on your shared URL. maybe only
Cloudera employees allowed.
On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com> wrote:
> Hi Geelong,
>
> Check out Todd Lipcon's presentation on tuning MapReduce performance:
> http://www.slideshare.net/cloudera/mr-perf
>
> -Sandy
>
>
> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com> wrote:
>
>> Hi Everyone
>>
>> I am now testing the best performance of my cluster
>> Can anyone give me some *formulas or suggestion* for all setting values
>> in MR program?
>> eg.
>> io.sort.mb
>> io.sort.spill.percent
>> mapred.local.dir
>> mapred.child.java.opts etc.
>>
>> Any details would be great
>>
>> BRs
>> Geelong
>>
>> --
>> From Good To Great
>>
>
>
Re: Perfect configuration setting
Posted by Azuryy Yu <az...@gmail.com>.
Hi Sandy,
I think there are some permission limited on your shared URL. maybe only
Cloudera employees allowed.
On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com> wrote:
> Hi Geelong,
>
> Check out Todd Lipcon's presentation on tuning MapReduce performance:
> http://www.slideshare.net/cloudera/mr-perf
>
> -Sandy
>
>
> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com> wrote:
>
>> Hi Everyone
>>
>> I am now testing the best performance of my cluster
>> Can anyone give me some *formulas or suggestion* for all setting values
>> in MR program?
>> eg.
>> io.sort.mb
>> io.sort.spill.percent
>> mapred.local.dir
>> mapred.child.java.opts etc.
>>
>> Any details would be great
>>
>> BRs
>> Geelong
>>
>> --
>> From Good To Great
>>
>
>
Re: Perfect configuration setting
Posted by Azuryy Yu <az...@gmail.com>.
Hi Sandy,
I think there are some permission limited on your shared URL. maybe only
Cloudera employees allowed.
On Tue, Dec 3, 2013 at 4:26 PM, Sandy Ryza <sa...@cloudera.com> wrote:
> Hi Geelong,
>
> Check out Todd Lipcon's presentation on tuning MapReduce performance:
> http://www.slideshare.net/cloudera/mr-perf
>
> -Sandy
>
>
> On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com> wrote:
>
>> Hi Everyone
>>
>> I am now testing the best performance of my cluster
>> Can anyone give me some *formulas or suggestion* for all setting values
>> in MR program?
>> eg.
>> io.sort.mb
>> io.sort.spill.percent
>> mapred.local.dir
>> mapred.child.java.opts etc.
>>
>> Any details would be great
>>
>> BRs
>> Geelong
>>
>> --
>> From Good To Great
>>
>
>
Re: Perfect configuration setting
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Geelong,
Check out Todd Lipcon's presentation on tuning MapReduce performance:
http://www.slideshare.net/cloudera/mr-perf
-Sandy
On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com> wrote:
> Hi Everyone
>
> I am now testing the best performance of my cluster
> Can anyone give me some *formulas or suggestion* for all setting values
> in MR program?
> eg.
> io.sort.mb
> io.sort.spill.percent
> mapred.local.dir
> mapred.child.java.opts etc.
>
> Any details would be great
>
> BRs
> Geelong
>
> --
> From Good To Great
>
Re: Perfect configuration setting
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Geelong,
Check out Todd Lipcon's presentation on tuning MapReduce performance:
http://www.slideshare.net/cloudera/mr-perf
-Sandy
On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com> wrote:
> Hi Everyone
>
> I am now testing the best performance of my cluster
> Can anyone give me some *formulas or suggestion* for all setting values
> in MR program?
> eg.
> io.sort.mb
> io.sort.spill.percent
> mapred.local.dir
> mapred.child.java.opts etc.
>
> Any details would be great
>
> BRs
> Geelong
>
> --
> From Good To Great
>
Re: Perfect configuration setting
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Geelong,
Check out Todd Lipcon's presentation on tuning MapReduce performance:
http://www.slideshare.net/cloudera/mr-perf
-Sandy
On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com> wrote:
> Hi Everyone
>
> I am now testing the best performance of my cluster
> Can anyone give me some *formulas or suggestion* for all setting values
> in MR program?
> eg.
> io.sort.mb
> io.sort.spill.percent
> mapred.local.dir
> mapred.child.java.opts etc.
>
> Any details would be great
>
> BRs
> Geelong
>
> --
> From Good To Great
>
Re: Perfect configuration setting
Posted by Sandy Ryza <sa...@cloudera.com>.
Hi Geelong,
Check out Todd Lipcon's presentation on tuning MapReduce performance:
http://www.slideshare.net/cloudera/mr-perf
-Sandy
On Mon, Dec 2, 2013 at 11:14 PM, Geelong Yao <ge...@gmail.com> wrote:
> Hi Everyone
>
> I am now testing the best performance of my cluster
> Can anyone give me some *formulas or suggestion* for all setting values
> in MR program?
> eg.
> io.sort.mb
> io.sort.spill.percent
> mapred.local.dir
> mapred.child.java.opts etc.
>
> Any details would be great
>
> BRs
> Geelong
>
> --
> From Good To Great
>