You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by sunil kalva <sa...@gmail.com> on 2015/03/05 21:32:56 UTC
mapping between disk and partition
Hi
Can i map a specific partition to a different disk in a broker. And what is
the general recommendations for disk to partition mapping for which that
broker is leader. and also for replications that broker handles.
--
SunilKalva
Re: mapping between disk and partition
Posted by Todd Palino <tp...@gmail.com>.
This is one of the major issues that we have noted with using JBOD disk layouts, that there is no tool like partition reassignment to move partitions between disks.
Another is that the partition balance algorithm would need to be improved, allowing for better selection of a mount point than round robin allows. This could be handled with pluggable partition assignment schemes (see my comment on the KIP), but I think it also speaks to the need for compound reassignment schemes. For example, rack aware across brokers, and by mount point usage within the broker.
-Todd
> On Mar 7, 2015, at 11:18 AM, Jiangjie Qin <jq...@linkedin.com.INVALID> wrote:
>
> I don¹t think we can specify partition to disk mapping now. All the
> partition will resides in the same directory.
> Here is a wild idea but I haven¹t tried this.
> 1. Create the topic and make sure all the log files are created.
> 2. Move each partition log directory to the disk that you want them to
> resides.
> 3. Create a link in its original path pointing to the real file.
>
> Theoretically it should work.
>
> Jiangjie (Becket) Qin
>
>> On 3/7/15, 8:19 AM, "sunil kalva" <sa...@gmail.com> wrote:
>>
>> please be advice on this.
>>
>>> On Fri, Mar 6, 2015 at 2:02 AM, sunil kalva <sa...@gmail.com> wrote:
>>>
>>> Hi
>>>
>>> Can i map a specific partition to a different disk in a broker. And what
>>> is the general recommendations for disk to partition mapping for which
>>> that
>>> broker is leader. and also for replications that broker handles.
>>>
>>> --
>>> SunilKalva
>>
>>
>>
>> --
>> SunilKalva
>
Re: mapping between disk and partition
Posted by Jiangjie Qin <jq...@linkedin.com.INVALID>.
I don¹t think we can specify partition to disk mapping now. All the
partition will resides in the same directory.
Here is a wild idea but I haven¹t tried this.
1. Create the topic and make sure all the log files are created.
2. Move each partition log directory to the disk that you want them to
resides.
3. Create a link in its original path pointing to the real file.
Theoretically it should work.
Jiangjie (Becket) Qin
On 3/7/15, 8:19 AM, "sunil kalva" <sa...@gmail.com> wrote:
>please be advice on this.
>
>On Fri, Mar 6, 2015 at 2:02 AM, sunil kalva <sa...@gmail.com> wrote:
>
>> Hi
>>
>> Can i map a specific partition to a different disk in a broker. And what
>> is the general recommendations for disk to partition mapping for which
>>that
>> broker is leader. and also for replications that broker handles.
>>
>> --
>> SunilKalva
>>
>
>
>
>--
>SunilKalva
Re: mapping between disk and partition
Posted by sunil kalva <sa...@gmail.com>.
please be advice on this.
On Fri, Mar 6, 2015 at 2:02 AM, sunil kalva <sa...@gmail.com> wrote:
> Hi
>
> Can i map a specific partition to a different disk in a broker. And what
> is the general recommendations for disk to partition mapping for which that
> broker is leader. and also for replications that broker handles.
>
> --
> SunilKalva
>
--
SunilKalva