You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by dennis81 <dl...@gmail.com> on 2008/11/25 03:59:29 UTC
Block placement in HDFS
Hi everyone,
I was wondering whether it is possible to control the placement of the
blocks of a file in HDFS. Is it possible to instruct HDFS about which nodes
will hold the block replicas?
Thanks!
--
View this message in context: http://www.nabble.com/Block-placement-in-HDFS-tp20674353p20674353.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Re: Block placement in HDFS
Posted by Dhruba Borthakur <dh...@gmail.com>.
Hi Dennis,
There were some discussions on this topic earlier:
http://issues.apache.org/jira/browse/HADOOP-3799
Do you have any specific use-case for this feature?
thanks,
dhruba
On Mon, Nov 24, 2008 at 10:22 PM, Owen O'Malley <om...@apache.org> wrote:
>
> On Nov 24, 2008, at 8:44 PM, Mahadev Konar wrote:
>
> Hi Dennis,
>> I don't think that is possible to do.
>>
>
> No, it is not possible.
>
> The block placement is determined
>> by HDFS internally (which is local, rack local and off rack).
>>
>
> Actually, it was changed in 0.17 or so to be node-local, off-rack, and a
> second node off rack.
>
> -- Owen
>
Re: Block placement in HDFS
Posted by Pete Wyckoff <pw...@facebook.com>.
Fyi - Owen is referring to:
https://issues.apache.org/jira/browse/HADOOP-2559
On 11/24/08 10:22 PM, "Owen O'Malley" <om...@apache.org> wrote:
On Nov 24, 2008, at 8:44 PM, Mahadev Konar wrote:
> Hi Dennis,
> I don't think that is possible to do.
No, it is not possible.
> The block placement is determined
> by HDFS internally (which is local, rack local and off rack).
Actually, it was changed in 0.17 or so to be node-local, off-rack, and
a second node off rack.
-- Owen
Re: Block placement in HDFS
Posted by Owen O'Malley <om...@apache.org>.
On Nov 24, 2008, at 8:44 PM, Mahadev Konar wrote:
> Hi Dennis,
> I don't think that is possible to do.
No, it is not possible.
> The block placement is determined
> by HDFS internally (which is local, rack local and off rack).
Actually, it was changed in 0.17 or so to be node-local, off-rack, and
a second node off rack.
-- Owen
Re: Block placement in HDFS
Posted by Hyunsik Choi <hy...@korea.ac.kr>.
Hi All,
I try to divide some data into partitions explicitly (like regions of
Hbase). I wonder the following way to do is the best method.
For example, when we assume a block size 64MB, a file potion
corresponding to 0~63MB is allocated to first block?
I have three questions as follows:
Is the above method valid?
Is it the best method?
Is there alternative method?
Thank in advance.
--
Hyunsik Choi
Database & Information Systems Group
Dept. of Computer Science & Engineering, Korea University
On Mon, 2008-11-24 at 20:44 -0800, Mahadev Konar wrote:
> Hi Dennis,
> I don't think that is possible to do. The block placement is determined
> by HDFS internally (which is local, rack local and off rack).
>
>
> mahadev
>
>
> On 11/24/08 6:59 PM, "dennis81" <dl...@gmail.com> wrote:
>
> >
> > Hi everyone,
> >
> > I was wondering whether it is possible to control the placement of the
> > blocks of a file in HDFS. Is it possible to instruct HDFS about which nodes
> > will hold the block replicas?
> >
> > Thanks!
Re: Block placement in HDFS
Posted by Mahadev Konar <ma...@yahoo-inc.com>.
Hi Dennis,
I don't think that is possible to do. The block placement is determined
by HDFS internally (which is local, rack local and off rack).
mahadev
On 11/24/08 6:59 PM, "dennis81" <dl...@gmail.com> wrote:
>
> Hi everyone,
>
> I was wondering whether it is possible to control the placement of the
> blocks of a file in HDFS. Is it possible to instruct HDFS about which nodes
> will hold the block replicas?
>
> Thanks!