You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Patai Sangbutsarakum <si...@gmail.com> on 2013/04/02 19:59:25 UTC
Linux io scheduler
Hello Hadoopers,
Is anyone ever play linux io scheduler configuration in worker nodes?
default for centos is cfq, there are 3 more to choose from.
I wonder if we can play with those scheduler, probably we might get better
performance ?
hope this make sense.
Patai
Re: Linux io scheduler
Posted by Patai Sangbutsarakum <si...@gmail.com>.
Thanks Chris for response, I would love to try out. What would be the
metric(s) that we will be use to measure if we improve thing.?
On Tue, Apr 2, 2013 at 11:24 AM, Chris Embree <ce...@gmail.com> wrote:
> I assume your talking about the I/O scheduler. Based on normal advice,
> only change this if you have a "smart" device between the OS and the
> Drives. A SATA controller usually qualifies. I have our DataNodes to to
> NOOP to reduce the number of layers.
>
> As always your mileage may vary and you should do some testing on your own.
>
>
> On Tue, Apr 2, 2013 at 1:59 PM, Patai Sangbutsarakum <
> silvianhadoop@gmail.com> wrote:
>
>> Hello Hadoopers,
>>
>> Is anyone ever play linux io scheduler configuration in worker nodes?
>> default for centos is cfq, there are 3 more to choose from.
>> I wonder if we can play with those scheduler, probably we might get
>> better performance ?
>>
>> hope this make sense.
>> Patai
>>
>
>
Re: Linux io scheduler
Posted by Patai Sangbutsarakum <si...@gmail.com>.
Thanks Chris for response, I would love to try out. What would be the
metric(s) that we will be use to measure if we improve thing.?
On Tue, Apr 2, 2013 at 11:24 AM, Chris Embree <ce...@gmail.com> wrote:
> I assume your talking about the I/O scheduler. Based on normal advice,
> only change this if you have a "smart" device between the OS and the
> Drives. A SATA controller usually qualifies. I have our DataNodes to to
> NOOP to reduce the number of layers.
>
> As always your mileage may vary and you should do some testing on your own.
>
>
> On Tue, Apr 2, 2013 at 1:59 PM, Patai Sangbutsarakum <
> silvianhadoop@gmail.com> wrote:
>
>> Hello Hadoopers,
>>
>> Is anyone ever play linux io scheduler configuration in worker nodes?
>> default for centos is cfq, there are 3 more to choose from.
>> I wonder if we can play with those scheduler, probably we might get
>> better performance ?
>>
>> hope this make sense.
>> Patai
>>
>
>
Re: Linux io scheduler
Posted by Patai Sangbutsarakum <si...@gmail.com>.
Thanks Chris for response, I would love to try out. What would be the
metric(s) that we will be use to measure if we improve thing.?
On Tue, Apr 2, 2013 at 11:24 AM, Chris Embree <ce...@gmail.com> wrote:
> I assume your talking about the I/O scheduler. Based on normal advice,
> only change this if you have a "smart" device between the OS and the
> Drives. A SATA controller usually qualifies. I have our DataNodes to to
> NOOP to reduce the number of layers.
>
> As always your mileage may vary and you should do some testing on your own.
>
>
> On Tue, Apr 2, 2013 at 1:59 PM, Patai Sangbutsarakum <
> silvianhadoop@gmail.com> wrote:
>
>> Hello Hadoopers,
>>
>> Is anyone ever play linux io scheduler configuration in worker nodes?
>> default for centos is cfq, there are 3 more to choose from.
>> I wonder if we can play with those scheduler, probably we might get
>> better performance ?
>>
>> hope this make sense.
>> Patai
>>
>
>
Re: Linux io scheduler
Posted by Patai Sangbutsarakum <si...@gmail.com>.
Thanks Chris for response, I would love to try out. What would be the
metric(s) that we will be use to measure if we improve thing.?
On Tue, Apr 2, 2013 at 11:24 AM, Chris Embree <ce...@gmail.com> wrote:
> I assume your talking about the I/O scheduler. Based on normal advice,
> only change this if you have a "smart" device between the OS and the
> Drives. A SATA controller usually qualifies. I have our DataNodes to to
> NOOP to reduce the number of layers.
>
> As always your mileage may vary and you should do some testing on your own.
>
>
> On Tue, Apr 2, 2013 at 1:59 PM, Patai Sangbutsarakum <
> silvianhadoop@gmail.com> wrote:
>
>> Hello Hadoopers,
>>
>> Is anyone ever play linux io scheduler configuration in worker nodes?
>> default for centos is cfq, there are 3 more to choose from.
>> I wonder if we can play with those scheduler, probably we might get
>> better performance ?
>>
>> hope this make sense.
>> Patai
>>
>
>
Re: Linux io scheduler
Posted by Chris Embree <ce...@gmail.com>.
I assume your talking about the I/O scheduler. Based on normal advice,
only change this if you have a "smart" device between the OS and the
Drives. A SATA controller usually qualifies. I have our DataNodes to to
NOOP to reduce the number of layers.
As always your mileage may vary and you should do some testing on your own.
On Tue, Apr 2, 2013 at 1:59 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Hello Hadoopers,
>
> Is anyone ever play linux io scheduler configuration in worker nodes?
> default for centos is cfq, there are 3 more to choose from.
> I wonder if we can play with those scheduler, probably we might get better
> performance ?
>
> hope this make sense.
> Patai
>
Re: Linux io scheduler
Posted by Chris Embree <ce...@gmail.com>.
I assume your talking about the I/O scheduler. Based on normal advice,
only change this if you have a "smart" device between the OS and the
Drives. A SATA controller usually qualifies. I have our DataNodes to to
NOOP to reduce the number of layers.
As always your mileage may vary and you should do some testing on your own.
On Tue, Apr 2, 2013 at 1:59 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Hello Hadoopers,
>
> Is anyone ever play linux io scheduler configuration in worker nodes?
> default for centos is cfq, there are 3 more to choose from.
> I wonder if we can play with those scheduler, probably we might get better
> performance ?
>
> hope this make sense.
> Patai
>
Re: Linux io scheduler
Posted by Chris Embree <ce...@gmail.com>.
I assume your talking about the I/O scheduler. Based on normal advice,
only change this if you have a "smart" device between the OS and the
Drives. A SATA controller usually qualifies. I have our DataNodes to to
NOOP to reduce the number of layers.
As always your mileage may vary and you should do some testing on your own.
On Tue, Apr 2, 2013 at 1:59 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Hello Hadoopers,
>
> Is anyone ever play linux io scheduler configuration in worker nodes?
> default for centos is cfq, there are 3 more to choose from.
> I wonder if we can play with those scheduler, probably we might get better
> performance ?
>
> hope this make sense.
> Patai
>
Re: Linux io scheduler
Posted by Chris Embree <ce...@gmail.com>.
I assume your talking about the I/O scheduler. Based on normal advice,
only change this if you have a "smart" device between the OS and the
Drives. A SATA controller usually qualifies. I have our DataNodes to to
NOOP to reduce the number of layers.
As always your mileage may vary and you should do some testing on your own.
On Tue, Apr 2, 2013 at 1:59 PM, Patai Sangbutsarakum <
silvianhadoop@gmail.com> wrote:
> Hello Hadoopers,
>
> Is anyone ever play linux io scheduler configuration in worker nodes?
> default for centos is cfq, there are 3 more to choose from.
> I wonder if we can play with those scheduler, probably we might get better
> performance ?
>
> hope this make sense.
> Patai
>