You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Fei Hu <hu...@gmail.com> on 2015/01/14 04:33:00 UTC
Data Placement Strategy for HDFS
Hi,
I want to customize the data placement strategy rather than using the default strategy in HDFS. Is there any way to control which datanode the replica is delivered to?
Thank you in advance.
Best regards,
Fei
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Thank you very much.
Fei
> On Jan 14, 2015, at 7:36 PM, Ted Yu <yu...@gmail.com> wrote:
>
> https://issues.apache.org/jira/browse/HDFS-7613 <https://issues.apache.org/jira/browse/HDFS-7613>
>
> Cheers
>
> On Wed, Jan 14, 2015 at 4:35 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> Thank you for your help.
>
> I searched HDFS-7613 by Google and the link https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22 <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>, but I could not find it.
>
> Could you email me the link? Thank you very much.
>
> Sincerely,
> Fei
>
>
>> On Jan 14, 2015, at 5:43 PM, Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
>>
>> Fei:
>> You can watch this issue:
>> HDFS-7613 Block placement policy for erasure coding groups
>>
>> the solution there would be helpful to us.
>>
>> Cheers
>>
>> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
>> Thank you for your quick response.
>>
>> After reading the materials you recommended, my conclusion is that Hadoop does not provide interface to customize the data placement policy. We need to add some codes to the source package of HDFS. Is that right?
>>
>> Thanks,
>> Fei
>>
>> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
>> See this thread: http://search-hadoop.com/m/lAh9i28K7 <http://search-hadoop.com/m/lAh9i28K7>
>>
>> See also HDFS-7228
>>
>> Cheers
>>
>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>>
>> I want to customize the data placement strategy rather than using the default strategy in HDFS. Is there any way to control which datanode the replica is delivered to?
>>
>> Thank you in advance.
>>
>> Best regards,
>> Fei
>>
>>
>
>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Thank you very much.
Fei
> On Jan 14, 2015, at 7:36 PM, Ted Yu <yu...@gmail.com> wrote:
>
> https://issues.apache.org/jira/browse/HDFS-7613 <https://issues.apache.org/jira/browse/HDFS-7613>
>
> Cheers
>
> On Wed, Jan 14, 2015 at 4:35 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> Thank you for your help.
>
> I searched HDFS-7613 by Google and the link https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22 <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>, but I could not find it.
>
> Could you email me the link? Thank you very much.
>
> Sincerely,
> Fei
>
>
>> On Jan 14, 2015, at 5:43 PM, Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
>>
>> Fei:
>> You can watch this issue:
>> HDFS-7613 Block placement policy for erasure coding groups
>>
>> the solution there would be helpful to us.
>>
>> Cheers
>>
>> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
>> Thank you for your quick response.
>>
>> After reading the materials you recommended, my conclusion is that Hadoop does not provide interface to customize the data placement policy. We need to add some codes to the source package of HDFS. Is that right?
>>
>> Thanks,
>> Fei
>>
>> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
>> See this thread: http://search-hadoop.com/m/lAh9i28K7 <http://search-hadoop.com/m/lAh9i28K7>
>>
>> See also HDFS-7228
>>
>> Cheers
>>
>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>>
>> I want to customize the data placement strategy rather than using the default strategy in HDFS. Is there any way to control which datanode the replica is delivered to?
>>
>> Thank you in advance.
>>
>> Best regards,
>> Fei
>>
>>
>
>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Thank you very much.
Fei
> On Jan 14, 2015, at 7:36 PM, Ted Yu <yu...@gmail.com> wrote:
>
> https://issues.apache.org/jira/browse/HDFS-7613 <https://issues.apache.org/jira/browse/HDFS-7613>
>
> Cheers
>
> On Wed, Jan 14, 2015 at 4:35 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> Thank you for your help.
>
> I searched HDFS-7613 by Google and the link https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22 <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>, but I could not find it.
>
> Could you email me the link? Thank you very much.
>
> Sincerely,
> Fei
>
>
>> On Jan 14, 2015, at 5:43 PM, Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
>>
>> Fei:
>> You can watch this issue:
>> HDFS-7613 Block placement policy for erasure coding groups
>>
>> the solution there would be helpful to us.
>>
>> Cheers
>>
>> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
>> Thank you for your quick response.
>>
>> After reading the materials you recommended, my conclusion is that Hadoop does not provide interface to customize the data placement policy. We need to add some codes to the source package of HDFS. Is that right?
>>
>> Thanks,
>> Fei
>>
>> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
>> See this thread: http://search-hadoop.com/m/lAh9i28K7 <http://search-hadoop.com/m/lAh9i28K7>
>>
>> See also HDFS-7228
>>
>> Cheers
>>
>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>>
>> I want to customize the data placement strategy rather than using the default strategy in HDFS. Is there any way to control which datanode the replica is delivered to?
>>
>> Thank you in advance.
>>
>> Best regards,
>> Fei
>>
>>
>
>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Thank you very much.
Fei
> On Jan 14, 2015, at 7:36 PM, Ted Yu <yu...@gmail.com> wrote:
>
> https://issues.apache.org/jira/browse/HDFS-7613 <https://issues.apache.org/jira/browse/HDFS-7613>
>
> Cheers
>
> On Wed, Jan 14, 2015 at 4:35 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> Thank you for your help.
>
> I searched HDFS-7613 by Google and the link https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22 <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>, but I could not find it.
>
> Could you email me the link? Thank you very much.
>
> Sincerely,
> Fei
>
>
>> On Jan 14, 2015, at 5:43 PM, Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
>>
>> Fei:
>> You can watch this issue:
>> HDFS-7613 Block placement policy for erasure coding groups
>>
>> the solution there would be helpful to us.
>>
>> Cheers
>>
>> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
>> Thank you for your quick response.
>>
>> After reading the materials you recommended, my conclusion is that Hadoop does not provide interface to customize the data placement policy. We need to add some codes to the source package of HDFS. Is that right?
>>
>> Thanks,
>> Fei
>>
>> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
>> See this thread: http://search-hadoop.com/m/lAh9i28K7 <http://search-hadoop.com/m/lAh9i28K7>
>>
>> See also HDFS-7228
>>
>> Cheers
>>
>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>>
>> I want to customize the data placement strategy rather than using the default strategy in HDFS. Is there any way to control which datanode the replica is delivered to?
>>
>> Thank you in advance.
>>
>> Best regards,
>> Fei
>>
>>
>
>
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
https://issues.apache.org/jira/browse/HDFS-7613
Cheers
On Wed, Jan 14, 2015 at 4:35 PM, Fei Hu <hu...@gmail.com> wrote:
> Hi,
>
> Thank you for your help.
>
> I searched HDFS-7613 by Google and the link
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22
> <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>,
> but I could not find it.
>
> Could you email me the link? Thank you very much.
>
> Sincerely,
> Fei
>
>
> On Jan 14, 2015, at 5:43 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Fei:
> You can watch this issue:
> HDFS-7613 Block placement policy for erasure coding groups
>
> the solution there would be helpful to us.
>
> Cheers
>
> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hu...@gmail.com> wrote:
>
>> Thank you for your quick response.
>>
>> After reading the materials you recommended, my conclusion is that Hadoop
>> does not provide interface to customize the data placement policy. We need
>> to add some codes to the source package of HDFS. Is that right?
>>
>> Thanks,
>> Fei
>>
>> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
>>
>>> See this thread: http://search-hadoop.com/m/lAh9i28K7
>>>
>>> See also HDFS-7228
>>>
>>> Cheers
>>>
>>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I want to customize the data placement strategy rather than using the
>>>> default strategy in HDFS. Is there any way to control which datanode the
>>>> replica is delivered to?
>>>>
>>>> Thank you in advance.
>>>>
>>>> Best regards,
>>>> Fei
>>>
>>>
>>>
>
>
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
https://issues.apache.org/jira/browse/HDFS-7613
Cheers
On Wed, Jan 14, 2015 at 4:35 PM, Fei Hu <hu...@gmail.com> wrote:
> Hi,
>
> Thank you for your help.
>
> I searched HDFS-7613 by Google and the link
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22
> <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>,
> but I could not find it.
>
> Could you email me the link? Thank you very much.
>
> Sincerely,
> Fei
>
>
> On Jan 14, 2015, at 5:43 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Fei:
> You can watch this issue:
> HDFS-7613 Block placement policy for erasure coding groups
>
> the solution there would be helpful to us.
>
> Cheers
>
> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hu...@gmail.com> wrote:
>
>> Thank you for your quick response.
>>
>> After reading the materials you recommended, my conclusion is that Hadoop
>> does not provide interface to customize the data placement policy. We need
>> to add some codes to the source package of HDFS. Is that right?
>>
>> Thanks,
>> Fei
>>
>> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
>>
>>> See this thread: http://search-hadoop.com/m/lAh9i28K7
>>>
>>> See also HDFS-7228
>>>
>>> Cheers
>>>
>>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I want to customize the data placement strategy rather than using the
>>>> default strategy in HDFS. Is there any way to control which datanode the
>>>> replica is delivered to?
>>>>
>>>> Thank you in advance.
>>>>
>>>> Best regards,
>>>> Fei
>>>
>>>
>>>
>
>
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
https://issues.apache.org/jira/browse/HDFS-7613
Cheers
On Wed, Jan 14, 2015 at 4:35 PM, Fei Hu <hu...@gmail.com> wrote:
> Hi,
>
> Thank you for your help.
>
> I searched HDFS-7613 by Google and the link
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22
> <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>,
> but I could not find it.
>
> Could you email me the link? Thank you very much.
>
> Sincerely,
> Fei
>
>
> On Jan 14, 2015, at 5:43 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Fei:
> You can watch this issue:
> HDFS-7613 Block placement policy for erasure coding groups
>
> the solution there would be helpful to us.
>
> Cheers
>
> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hu...@gmail.com> wrote:
>
>> Thank you for your quick response.
>>
>> After reading the materials you recommended, my conclusion is that Hadoop
>> does not provide interface to customize the data placement policy. We need
>> to add some codes to the source package of HDFS. Is that right?
>>
>> Thanks,
>> Fei
>>
>> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
>>
>>> See this thread: http://search-hadoop.com/m/lAh9i28K7
>>>
>>> See also HDFS-7228
>>>
>>> Cheers
>>>
>>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I want to customize the data placement strategy rather than using the
>>>> default strategy in HDFS. Is there any way to control which datanode the
>>>> replica is delivered to?
>>>>
>>>> Thank you in advance.
>>>>
>>>> Best regards,
>>>> Fei
>>>
>>>
>>>
>
>
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
https://issues.apache.org/jira/browse/HDFS-7613
Cheers
On Wed, Jan 14, 2015 at 4:35 PM, Fei Hu <hu...@gmail.com> wrote:
> Hi,
>
> Thank you for your help.
>
> I searched HDFS-7613 by Google and the link
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22
> <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>,
> but I could not find it.
>
> Could you email me the link? Thank you very much.
>
> Sincerely,
> Fei
>
>
> On Jan 14, 2015, at 5:43 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Fei:
> You can watch this issue:
> HDFS-7613 Block placement policy for erasure coding groups
>
> the solution there would be helpful to us.
>
> Cheers
>
> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hu...@gmail.com> wrote:
>
>> Thank you for your quick response.
>>
>> After reading the materials you recommended, my conclusion is that Hadoop
>> does not provide interface to customize the data placement policy. We need
>> to add some codes to the source package of HDFS. Is that right?
>>
>> Thanks,
>> Fei
>>
>> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
>>
>>> See this thread: http://search-hadoop.com/m/lAh9i28K7
>>>
>>> See also HDFS-7228
>>>
>>> Cheers
>>>
>>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I want to customize the data placement strategy rather than using the
>>>> default strategy in HDFS. Is there any way to control which datanode the
>>>> replica is delivered to?
>>>>
>>>> Thank you in advance.
>>>>
>>>> Best regards,
>>>> Fei
>>>
>>>
>>>
>
>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Hi,
Thank you for your help.
I searched HDFS-7613 by Google and the link https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22 <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>, but I could not find it.
Could you email me the link? Thank you very much.
Sincerely,
Fei
> On Jan 14, 2015, at 5:43 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Fei:
> You can watch this issue:
> HDFS-7613 Block placement policy for erasure coding groups
>
> the solution there would be helpful to us.
>
> Cheers
>
> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Thank you for your quick response.
>
> After reading the materials you recommended, my conclusion is that Hadoop does not provide interface to customize the data placement policy. We need to add some codes to the source package of HDFS. Is that right?
>
> Thanks,
> Fei
>
> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
> See this thread: http://search-hadoop.com/m/lAh9i28K7 <http://search-hadoop.com/m/lAh9i28K7>
>
> See also HDFS-7228
>
> Cheers
>
> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> I want to customize the data placement strategy rather than using the default strategy in HDFS. Is there any way to control which datanode the replica is delivered to?
>
> Thank you in advance.
>
> Best regards,
> Fei
>
>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Hi,
Thank you for your help.
I searched HDFS-7613 by Google and the link https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22 <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>, but I could not find it.
Could you email me the link? Thank you very much.
Sincerely,
Fei
> On Jan 14, 2015, at 5:43 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Fei:
> You can watch this issue:
> HDFS-7613 Block placement policy for erasure coding groups
>
> the solution there would be helpful to us.
>
> Cheers
>
> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Thank you for your quick response.
>
> After reading the materials you recommended, my conclusion is that Hadoop does not provide interface to customize the data placement policy. We need to add some codes to the source package of HDFS. Is that right?
>
> Thanks,
> Fei
>
> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
> See this thread: http://search-hadoop.com/m/lAh9i28K7 <http://search-hadoop.com/m/lAh9i28K7>
>
> See also HDFS-7228
>
> Cheers
>
> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> I want to customize the data placement strategy rather than using the default strategy in HDFS. Is there any way to control which datanode the replica is delivered to?
>
> Thank you in advance.
>
> Best regards,
> Fei
>
>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Hi,
Thank you for your help.
I searched HDFS-7613 by Google and the link https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22 <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>, but I could not find it.
Could you email me the link? Thank you very much.
Sincerely,
Fei
> On Jan 14, 2015, at 5:43 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Fei:
> You can watch this issue:
> HDFS-7613 Block placement policy for erasure coding groups
>
> the solution there would be helpful to us.
>
> Cheers
>
> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Thank you for your quick response.
>
> After reading the materials you recommended, my conclusion is that Hadoop does not provide interface to customize the data placement policy. We need to add some codes to the source package of HDFS. Is that right?
>
> Thanks,
> Fei
>
> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
> See this thread: http://search-hadoop.com/m/lAh9i28K7 <http://search-hadoop.com/m/lAh9i28K7>
>
> See also HDFS-7228
>
> Cheers
>
> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> I want to customize the data placement strategy rather than using the default strategy in HDFS. Is there any way to control which datanode the replica is delivered to?
>
> Thank you in advance.
>
> Best regards,
> Fei
>
>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Hi,
Thank you for your help.
I searched HDFS-7613 by Google and the link https://issues.apache.org/jira/issues/?jql=project%20%3D%20HDFS%20AND%20text%20~%20%227613%22 <https://issues.apache.org/jira/issues/?jql=project%20=%20HDFS%20AND%20text%20~%20%227613%22>, but I could not find it.
Could you email me the link? Thank you very much.
Sincerely,
Fei
> On Jan 14, 2015, at 5:43 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Fei:
> You can watch this issue:
> HDFS-7613 Block placement policy for erasure coding groups
>
> the solution there would be helpful to us.
>
> Cheers
>
> On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Thank you for your quick response.
>
> After reading the materials you recommended, my conclusion is that Hadoop does not provide interface to customize the data placement policy. We need to add some codes to the source package of HDFS. Is that right?
>
> Thanks,
> Fei
>
> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yuzhihong@gmail.com <ma...@gmail.com>> wrote:
> See this thread: http://search-hadoop.com/m/lAh9i28K7 <http://search-hadoop.com/m/lAh9i28K7>
>
> See also HDFS-7228
>
> Cheers
>
> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> I want to customize the data placement strategy rather than using the default strategy in HDFS. Is there any way to control which datanode the replica is delivered to?
>
> Thank you in advance.
>
> Best regards,
> Fei
>
>
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
Fei:
You can watch this issue:
HDFS-7613 Block placement policy for erasure coding groups
the solution there would be helpful to us.
Cheers
On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hu...@gmail.com> wrote:
> Thank you for your quick response.
>
> After reading the materials you recommended, my conclusion is that Hadoop
> does not provide interface to customize the data placement policy. We need
> to add some codes to the source package of HDFS. Is that right?
>
> Thanks,
> Fei
>
> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
>
>> See this thread: http://search-hadoop.com/m/lAh9i28K7
>>
>> See also HDFS-7228
>>
>> Cheers
>>
>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I want to customize the data placement strategy rather than using the
>>> default strategy in HDFS. Is there any way to control which datanode the
>>> replica is delivered to?
>>>
>>> Thank you in advance.
>>>
>>> Best regards,
>>> Fei
>>
>>
>>
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
Fei:
You can watch this issue:
HDFS-7613 Block placement policy for erasure coding groups
the solution there would be helpful to us.
Cheers
On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hu...@gmail.com> wrote:
> Thank you for your quick response.
>
> After reading the materials you recommended, my conclusion is that Hadoop
> does not provide interface to customize the data placement policy. We need
> to add some codes to the source package of HDFS. Is that right?
>
> Thanks,
> Fei
>
> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
>
>> See this thread: http://search-hadoop.com/m/lAh9i28K7
>>
>> See also HDFS-7228
>>
>> Cheers
>>
>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I want to customize the data placement strategy rather than using the
>>> default strategy in HDFS. Is there any way to control which datanode the
>>> replica is delivered to?
>>>
>>> Thank you in advance.
>>>
>>> Best regards,
>>> Fei
>>
>>
>>
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
Fei:
You can watch this issue:
HDFS-7613 Block placement policy for erasure coding groups
the solution there would be helpful to us.
Cheers
On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hu...@gmail.com> wrote:
> Thank you for your quick response.
>
> After reading the materials you recommended, my conclusion is that Hadoop
> does not provide interface to customize the data placement policy. We need
> to add some codes to the source package of HDFS. Is that right?
>
> Thanks,
> Fei
>
> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
>
>> See this thread: http://search-hadoop.com/m/lAh9i28K7
>>
>> See also HDFS-7228
>>
>> Cheers
>>
>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I want to customize the data placement strategy rather than using the
>>> default strategy in HDFS. Is there any way to control which datanode the
>>> replica is delivered to?
>>>
>>> Thank you in advance.
>>>
>>> Best regards,
>>> Fei
>>
>>
>>
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
Fei:
You can watch this issue:
HDFS-7613 Block placement policy for erasure coding groups
the solution there would be helpful to us.
Cheers
On Wed, Jan 14, 2015 at 11:04 AM, Fei Hu <hu...@gmail.com> wrote:
> Thank you for your quick response.
>
> After reading the materials you recommended, my conclusion is that Hadoop
> does not provide interface to customize the data placement policy. We need
> to add some codes to the source package of HDFS. Is that right?
>
> Thanks,
> Fei
>
> On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
>
>> See this thread: http://search-hadoop.com/m/lAh9i28K7
>>
>> See also HDFS-7228
>>
>> Cheers
>>
>> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I want to customize the data placement strategy rather than using the
>>> default strategy in HDFS. Is there any way to control which datanode the
>>> replica is delivered to?
>>>
>>> Thank you in advance.
>>>
>>> Best regards,
>>> Fei
>>
>>
>>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Thank you for your quick response.
After reading the materials you recommended, my conclusion is that Hadoop
does not provide interface to customize the data placement policy. We need
to add some codes to the source package of HDFS. Is that right?
Thanks,
Fei
On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
> See this thread: http://search-hadoop.com/m/lAh9i28K7
>
> See also HDFS-7228
>
> Cheers
>
> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>
>> Hi,
>>
>> I want to customize the data placement strategy rather than using the
>> default strategy in HDFS. Is there any way to control which datanode the
>> replica is delivered to?
>>
>> Thank you in advance.
>>
>> Best regards,
>> Fei
>
>
>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Thank you for your quick response.
After reading the materials you recommended, my conclusion is that Hadoop
does not provide interface to customize the data placement policy. We need
to add some codes to the source package of HDFS. Is that right?
Thanks,
Fei
On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
> See this thread: http://search-hadoop.com/m/lAh9i28K7
>
> See also HDFS-7228
>
> Cheers
>
> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>
>> Hi,
>>
>> I want to customize the data placement strategy rather than using the
>> default strategy in HDFS. Is there any way to control which datanode the
>> replica is delivered to?
>>
>> Thank you in advance.
>>
>> Best regards,
>> Fei
>
>
>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Thank you for your quick response.
After reading the materials you recommended, my conclusion is that Hadoop
does not provide interface to customize the data placement policy. We need
to add some codes to the source package of HDFS. Is that right?
Thanks,
Fei
On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
> See this thread: http://search-hadoop.com/m/lAh9i28K7
>
> See also HDFS-7228
>
> Cheers
>
> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>
>> Hi,
>>
>> I want to customize the data placement strategy rather than using the
>> default strategy in HDFS. Is there any way to control which datanode the
>> replica is delivered to?
>>
>> Thank you in advance.
>>
>> Best regards,
>> Fei
>
>
>
Re: Data Placement Strategy for HDFS
Posted by Fei Hu <hu...@gmail.com>.
Thank you for your quick response.
After reading the materials you recommended, my conclusion is that Hadoop
does not provide interface to customize the data placement policy. We need
to add some codes to the source package of HDFS. Is that right?
Thanks,
Fei
On Tue Jan 13 2015 at 10:42:27 PM Ted Yu <yu...@gmail.com> wrote:
> See this thread: http://search-hadoop.com/m/lAh9i28K7
>
> See also HDFS-7228
>
> Cheers
>
> On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
>
>> Hi,
>>
>> I want to customize the data placement strategy rather than using the
>> default strategy in HDFS. Is there any way to control which datanode the
>> replica is delivered to?
>>
>> Thank you in advance.
>>
>> Best regards,
>> Fei
>
>
>
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
See this thread: http://search-hadoop.com/m/lAh9i28K7
See also HDFS-7228
Cheers
On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
> Hi,
>
> I want to customize the data placement strategy rather than using the
> default strategy in HDFS. Is there any way to control which datanode the
> replica is delivered to?
>
> Thank you in advance.
>
> Best regards,
> Fei
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
See this thread: http://search-hadoop.com/m/lAh9i28K7
See also HDFS-7228
Cheers
On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
> Hi,
>
> I want to customize the data placement strategy rather than using the
> default strategy in HDFS. Is there any way to control which datanode the
> replica is delivered to?
>
> Thank you in advance.
>
> Best regards,
> Fei
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
See this thread: http://search-hadoop.com/m/lAh9i28K7
See also HDFS-7228
Cheers
On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
> Hi,
>
> I want to customize the data placement strategy rather than using the
> default strategy in HDFS. Is there any way to control which datanode the
> replica is delivered to?
>
> Thank you in advance.
>
> Best regards,
> Fei
Re: Data Placement Strategy for HDFS
Posted by Ted Yu <yu...@gmail.com>.
See this thread: http://search-hadoop.com/m/lAh9i28K7
See also HDFS-7228
Cheers
On Tue, Jan 13, 2015 at 7:33 PM, Fei Hu <hu...@gmail.com> wrote:
> Hi,
>
> I want to customize the data placement strategy rather than using the
> default strategy in HDFS. Is there any way to control which datanode the
> replica is delivered to?
>
> Thank you in advance.
>
> Best regards,
> Fei