You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Yehia Elshater <y....@gmail.com> on 2014/07/08 02:16:12 UTC
Copy hdfs block from one data node to another
Hi All,
How can copy a certain hdfs block (given the file name, start and end
bytes) from one node to another node ?
Thanks
Yehia
Re: Copy hdfs block from one data node to another
Posted by Yehia Elshater <y....@gmail.com>.
Thanks a lot. I will take a look on the balancer and decommissioning code.
On 10 July 2014 00:34, Arpit Agarwal <aa...@hortonworks.com> wrote:
> The balancer does something similar. It uses
> DataTransferProtocol.replaceBlock.
>
>
> On Wed, Jul 9, 2014 at 9:20 PM, sudhakara st <su...@gmail.com>
> wrote:
>
>> You can get info about all blocks stored in perticuler data node, i,e
>> block report. But you to handle, move in block level not in file or start
>> and end bytes level.
>>
>>
>> On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
>> wrote:
>>
>>> Haven't looked at the source but the thing you are trying to do sounds
>>> similar to what happens when you are decommissioning a datanode. I would
>>> hunt for that code.
>>> Cheers
>>> Chris
>>> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> Actually I need this functionality for my research, basically for fault
>>>> tolerance. I can calculate some failure probability for some data nodes
>>>> after certain unit of time. So I need to copy all the blocks reside on
>>>> these nodes to another nodes.
>>>>
>>>> Thanks
>>>> Yehia
>>>>
>>>>
>>>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>>>
>>>>> Can you outline why one would want to do that? The blocks are
>>>>> disposable so it is strange to manipulate them directly.
>>>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> How can copy a certain hdfs block (given the file name, start and end
>>>>>> bytes) from one node to another node ?
>>>>>>
>>>>>> Thanks
>>>>>> Yehia
>>>>>>
>>>>>
>>>>
>>
>>
>> --
>>
>> Regards,
>> ...sudhakara
>>
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Copy hdfs block from one data node to another
Posted by Yehia Elshater <y....@gmail.com>.
Thanks a lot. I will take a look on the balancer and decommissioning code.
On 10 July 2014 00:34, Arpit Agarwal <aa...@hortonworks.com> wrote:
> The balancer does something similar. It uses
> DataTransferProtocol.replaceBlock.
>
>
> On Wed, Jul 9, 2014 at 9:20 PM, sudhakara st <su...@gmail.com>
> wrote:
>
>> You can get info about all blocks stored in perticuler data node, i,e
>> block report. But you to handle, move in block level not in file or start
>> and end bytes level.
>>
>>
>> On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
>> wrote:
>>
>>> Haven't looked at the source but the thing you are trying to do sounds
>>> similar to what happens when you are decommissioning a datanode. I would
>>> hunt for that code.
>>> Cheers
>>> Chris
>>> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> Actually I need this functionality for my research, basically for fault
>>>> tolerance. I can calculate some failure probability for some data nodes
>>>> after certain unit of time. So I need to copy all the blocks reside on
>>>> these nodes to another nodes.
>>>>
>>>> Thanks
>>>> Yehia
>>>>
>>>>
>>>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>>>
>>>>> Can you outline why one would want to do that? The blocks are
>>>>> disposable so it is strange to manipulate them directly.
>>>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> How can copy a certain hdfs block (given the file name, start and end
>>>>>> bytes) from one node to another node ?
>>>>>>
>>>>>> Thanks
>>>>>> Yehia
>>>>>>
>>>>>
>>>>
>>
>>
>> --
>>
>> Regards,
>> ...sudhakara
>>
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Copy hdfs block from one data node to another
Posted by Yehia Elshater <y....@gmail.com>.
Thanks a lot. I will take a look on the balancer and decommissioning code.
On 10 July 2014 00:34, Arpit Agarwal <aa...@hortonworks.com> wrote:
> The balancer does something similar. It uses
> DataTransferProtocol.replaceBlock.
>
>
> On Wed, Jul 9, 2014 at 9:20 PM, sudhakara st <su...@gmail.com>
> wrote:
>
>> You can get info about all blocks stored in perticuler data node, i,e
>> block report. But you to handle, move in block level not in file or start
>> and end bytes level.
>>
>>
>> On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
>> wrote:
>>
>>> Haven't looked at the source but the thing you are trying to do sounds
>>> similar to what happens when you are decommissioning a datanode. I would
>>> hunt for that code.
>>> Cheers
>>> Chris
>>> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> Actually I need this functionality for my research, basically for fault
>>>> tolerance. I can calculate some failure probability for some data nodes
>>>> after certain unit of time. So I need to copy all the blocks reside on
>>>> these nodes to another nodes.
>>>>
>>>> Thanks
>>>> Yehia
>>>>
>>>>
>>>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>>>
>>>>> Can you outline why one would want to do that? The blocks are
>>>>> disposable so it is strange to manipulate them directly.
>>>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> How can copy a certain hdfs block (given the file name, start and end
>>>>>> bytes) from one node to another node ?
>>>>>>
>>>>>> Thanks
>>>>>> Yehia
>>>>>>
>>>>>
>>>>
>>
>>
>> --
>>
>> Regards,
>> ...sudhakara
>>
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Copy hdfs block from one data node to another
Posted by Yehia Elshater <y....@gmail.com>.
Thanks a lot. I will take a look on the balancer and decommissioning code.
On 10 July 2014 00:34, Arpit Agarwal <aa...@hortonworks.com> wrote:
> The balancer does something similar. It uses
> DataTransferProtocol.replaceBlock.
>
>
> On Wed, Jul 9, 2014 at 9:20 PM, sudhakara st <su...@gmail.com>
> wrote:
>
>> You can get info about all blocks stored in perticuler data node, i,e
>> block report. But you to handle, move in block level not in file or start
>> and end bytes level.
>>
>>
>> On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
>> wrote:
>>
>>> Haven't looked at the source but the thing you are trying to do sounds
>>> similar to what happens when you are decommissioning a datanode. I would
>>> hunt for that code.
>>> Cheers
>>> Chris
>>> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> Actually I need this functionality for my research, basically for fault
>>>> tolerance. I can calculate some failure probability for some data nodes
>>>> after certain unit of time. So I need to copy all the blocks reside on
>>>> these nodes to another nodes.
>>>>
>>>> Thanks
>>>> Yehia
>>>>
>>>>
>>>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>>>
>>>>> Can you outline why one would want to do that? The blocks are
>>>>> disposable so it is strange to manipulate them directly.
>>>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> How can copy a certain hdfs block (given the file name, start and end
>>>>>> bytes) from one node to another node ?
>>>>>>
>>>>>> Thanks
>>>>>> Yehia
>>>>>>
>>>>>
>>>>
>>
>>
>> --
>>
>> Regards,
>> ...sudhakara
>>
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Copy hdfs block from one data node to another
Posted by Arpit Agarwal <aa...@hortonworks.com>.
The balancer does something similar. It uses
DataTransferProtocol.replaceBlock.
On Wed, Jul 9, 2014 at 9:20 PM, sudhakara st <su...@gmail.com> wrote:
> You can get info about all blocks stored in perticuler data node, i,e
> block report. But you to handle, move in block level not in file or start
> and end bytes level.
>
>
> On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
> wrote:
>
>> Haven't looked at the source but the thing you are trying to do sounds
>> similar to what happens when you are decommissioning a datanode. I would
>> hunt for that code.
>> Cheers
>> Chris
>> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>
>>> Hi Chris,
>>>
>>> Actually I need this functionality for my research, basically for fault
>>> tolerance. I can calculate some failure probability for some data nodes
>>> after certain unit of time. So I need to copy all the blocks reside on
>>> these nodes to another nodes.
>>>
>>> Thanks
>>> Yehia
>>>
>>>
>>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>>
>>>> Can you outline why one would want to do that? The blocks are
>>>> disposable so it is strange to manipulate them directly.
>>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> How can copy a certain hdfs block (given the file name, start and end
>>>>> bytes) from one node to another node ?
>>>>>
>>>>> Thanks
>>>>> Yehia
>>>>>
>>>>
>>>
>
>
> --
>
> Regards,
> ...sudhakara
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Copy hdfs block from one data node to another
Posted by Arpit Agarwal <aa...@hortonworks.com>.
The balancer does something similar. It uses
DataTransferProtocol.replaceBlock.
On Wed, Jul 9, 2014 at 9:20 PM, sudhakara st <su...@gmail.com> wrote:
> You can get info about all blocks stored in perticuler data node, i,e
> block report. But you to handle, move in block level not in file or start
> and end bytes level.
>
>
> On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
> wrote:
>
>> Haven't looked at the source but the thing you are trying to do sounds
>> similar to what happens when you are decommissioning a datanode. I would
>> hunt for that code.
>> Cheers
>> Chris
>> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>
>>> Hi Chris,
>>>
>>> Actually I need this functionality for my research, basically for fault
>>> tolerance. I can calculate some failure probability for some data nodes
>>> after certain unit of time. So I need to copy all the blocks reside on
>>> these nodes to another nodes.
>>>
>>> Thanks
>>> Yehia
>>>
>>>
>>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>>
>>>> Can you outline why one would want to do that? The blocks are
>>>> disposable so it is strange to manipulate them directly.
>>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> How can copy a certain hdfs block (given the file name, start and end
>>>>> bytes) from one node to another node ?
>>>>>
>>>>> Thanks
>>>>> Yehia
>>>>>
>>>>
>>>
>
>
> --
>
> Regards,
> ...sudhakara
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Copy hdfs block from one data node to another
Posted by Arpit Agarwal <aa...@hortonworks.com>.
The balancer does something similar. It uses
DataTransferProtocol.replaceBlock.
On Wed, Jul 9, 2014 at 9:20 PM, sudhakara st <su...@gmail.com> wrote:
> You can get info about all blocks stored in perticuler data node, i,e
> block report. But you to handle, move in block level not in file or start
> and end bytes level.
>
>
> On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
> wrote:
>
>> Haven't looked at the source but the thing you are trying to do sounds
>> similar to what happens when you are decommissioning a datanode. I would
>> hunt for that code.
>> Cheers
>> Chris
>> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>
>>> Hi Chris,
>>>
>>> Actually I need this functionality for my research, basically for fault
>>> tolerance. I can calculate some failure probability for some data nodes
>>> after certain unit of time. So I need to copy all the blocks reside on
>>> these nodes to another nodes.
>>>
>>> Thanks
>>> Yehia
>>>
>>>
>>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>>
>>>> Can you outline why one would want to do that? The blocks are
>>>> disposable so it is strange to manipulate them directly.
>>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> How can copy a certain hdfs block (given the file name, start and end
>>>>> bytes) from one node to another node ?
>>>>>
>>>>> Thanks
>>>>> Yehia
>>>>>
>>>>
>>>
>
>
> --
>
> Regards,
> ...sudhakara
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Copy hdfs block from one data node to another
Posted by Arpit Agarwal <aa...@hortonworks.com>.
The balancer does something similar. It uses
DataTransferProtocol.replaceBlock.
On Wed, Jul 9, 2014 at 9:20 PM, sudhakara st <su...@gmail.com> wrote:
> You can get info about all blocks stored in perticuler data node, i,e
> block report. But you to handle, move in block level not in file or start
> and end bytes level.
>
>
> On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
> wrote:
>
>> Haven't looked at the source but the thing you are trying to do sounds
>> similar to what happens when you are decommissioning a datanode. I would
>> hunt for that code.
>> Cheers
>> Chris
>> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>
>>> Hi Chris,
>>>
>>> Actually I need this functionality for my research, basically for fault
>>> tolerance. I can calculate some failure probability for some data nodes
>>> after certain unit of time. So I need to copy all the blocks reside on
>>> these nodes to another nodes.
>>>
>>> Thanks
>>> Yehia
>>>
>>>
>>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>>
>>>> Can you outline why one would want to do that? The blocks are
>>>> disposable so it is strange to manipulate them directly.
>>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> How can copy a certain hdfs block (given the file name, start and end
>>>>> bytes) from one node to another node ?
>>>>>
>>>>> Thanks
>>>>> Yehia
>>>>>
>>>>
>>>
>
>
> --
>
> Regards,
> ...sudhakara
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Copy hdfs block from one data node to another
Posted by sudhakara st <su...@gmail.com>.
You can get info about all blocks stored in perticuler data node, i,e block
report. But you to handle, move in block level not in file or start and end
bytes level.
On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
wrote:
> Haven't looked at the source but the thing you are trying to do sounds
> similar to what happens when you are decommissioning a datanode. I would
> hunt for that code.
> Cheers
> Chris
> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>
>> Hi Chris,
>>
>> Actually I need this functionality for my research, basically for fault
>> tolerance. I can calculate some failure probability for some data nodes
>> after certain unit of time. So I need to copy all the blocks reside on
>> these nodes to another nodes.
>>
>> Thanks
>> Yehia
>>
>>
>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>
>>> Can you outline why one would want to do that? The blocks are disposable
>>> so it is strange to manipulate them directly.
>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> How can copy a certain hdfs block (given the file name, start and end
>>>> bytes) from one node to another node ?
>>>>
>>>> Thanks
>>>> Yehia
>>>>
>>>
>>
--
Regards,
...sudhakara
Re: Copy hdfs block from one data node to another
Posted by sudhakara st <su...@gmail.com>.
You can get info about all blocks stored in perticuler data node, i,e block
report. But you to handle, move in block level not in file or start and end
bytes level.
On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
wrote:
> Haven't looked at the source but the thing you are trying to do sounds
> similar to what happens when you are decommissioning a datanode. I would
> hunt for that code.
> Cheers
> Chris
> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>
>> Hi Chris,
>>
>> Actually I need this functionality for my research, basically for fault
>> tolerance. I can calculate some failure probability for some data nodes
>> after certain unit of time. So I need to copy all the blocks reside on
>> these nodes to another nodes.
>>
>> Thanks
>> Yehia
>>
>>
>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>
>>> Can you outline why one would want to do that? The blocks are disposable
>>> so it is strange to manipulate them directly.
>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> How can copy a certain hdfs block (given the file name, start and end
>>>> bytes) from one node to another node ?
>>>>
>>>> Thanks
>>>> Yehia
>>>>
>>>
>>
--
Regards,
...sudhakara
Re: Copy hdfs block from one data node to another
Posted by sudhakara st <su...@gmail.com>.
You can get info about all blocks stored in perticuler data node, i,e block
report. But you to handle, move in block level not in file or start and end
bytes level.
On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
wrote:
> Haven't looked at the source but the thing you are trying to do sounds
> similar to what happens when you are decommissioning a datanode. I would
> hunt for that code.
> Cheers
> Chris
> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>
>> Hi Chris,
>>
>> Actually I need this functionality for my research, basically for fault
>> tolerance. I can calculate some failure probability for some data nodes
>> after certain unit of time. So I need to copy all the blocks reside on
>> these nodes to another nodes.
>>
>> Thanks
>> Yehia
>>
>>
>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>
>>> Can you outline why one would want to do that? The blocks are disposable
>>> so it is strange to manipulate them directly.
>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> How can copy a certain hdfs block (given the file name, start and end
>>>> bytes) from one node to another node ?
>>>>
>>>> Thanks
>>>> Yehia
>>>>
>>>
>>
--
Regards,
...sudhakara
Re: Copy hdfs block from one data node to another
Posted by sudhakara st <su...@gmail.com>.
You can get info about all blocks stored in perticuler data node, i,e block
report. But you to handle, move in block level not in file or start and end
bytes level.
On Thu, Jul 10, 2014 at 2:49 AM, Chris Mawata <ch...@gmail.com>
wrote:
> Haven't looked at the source but the thing you are trying to do sounds
> similar to what happens when you are decommissioning a datanode. I would
> hunt for that code.
> Cheers
> Chris
> On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>
>> Hi Chris,
>>
>> Actually I need this functionality for my research, basically for fault
>> tolerance. I can calculate some failure probability for some data nodes
>> after certain unit of time. So I need to copy all the blocks reside on
>> these nodes to another nodes.
>>
>> Thanks
>> Yehia
>>
>>
>> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>>
>>> Can you outline why one would want to do that? The blocks are disposable
>>> so it is strange to manipulate them directly.
>>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> How can copy a certain hdfs block (given the file name, start and end
>>>> bytes) from one node to another node ?
>>>>
>>>> Thanks
>>>> Yehia
>>>>
>>>
>>
--
Regards,
...sudhakara
Re: Copy hdfs block from one data node to another
Posted by Chris Mawata <ch...@gmail.com>.
Haven't looked at the source but the thing you are trying to do sounds
similar to what happens when you are decommissioning a datanode. I would
hunt for that code.
Cheers
Chris
On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
> Hi Chris,
>
> Actually I need this functionality for my research, basically for fault
> tolerance. I can calculate some failure probability for some data nodes
> after certain unit of time. So I need to copy all the blocks reside on
> these nodes to another nodes.
>
> Thanks
> Yehia
>
>
> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>
>> Can you outline why one would want to do that? The blocks are disposable
>> so it is strange to manipulate them directly.
>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> How can copy a certain hdfs block (given the file name, start and end
>>> bytes) from one node to another node ?
>>>
>>> Thanks
>>> Yehia
>>>
>>
>
Re: Copy hdfs block from one data node to another
Posted by Chris Mawata <ch...@gmail.com>.
Haven't looked at the source but the thing you are trying to do sounds
similar to what happens when you are decommissioning a datanode. I would
hunt for that code.
Cheers
Chris
On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
> Hi Chris,
>
> Actually I need this functionality for my research, basically for fault
> tolerance. I can calculate some failure probability for some data nodes
> after certain unit of time. So I need to copy all the blocks reside on
> these nodes to another nodes.
>
> Thanks
> Yehia
>
>
> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>
>> Can you outline why one would want to do that? The blocks are disposable
>> so it is strange to manipulate them directly.
>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> How can copy a certain hdfs block (given the file name, start and end
>>> bytes) from one node to another node ?
>>>
>>> Thanks
>>> Yehia
>>>
>>
>
Re: Copy hdfs block from one data node to another
Posted by Chris Mawata <ch...@gmail.com>.
Haven't looked at the source but the thing you are trying to do sounds
similar to what happens when you are decommissioning a datanode. I would
hunt for that code.
Cheers
Chris
On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
> Hi Chris,
>
> Actually I need this functionality for my research, basically for fault
> tolerance. I can calculate some failure probability for some data nodes
> after certain unit of time. So I need to copy all the blocks reside on
> these nodes to another nodes.
>
> Thanks
> Yehia
>
>
> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>
>> Can you outline why one would want to do that? The blocks are disposable
>> so it is strange to manipulate them directly.
>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> How can copy a certain hdfs block (given the file name, start and end
>>> bytes) from one node to another node ?
>>>
>>> Thanks
>>> Yehia
>>>
>>
>
Re: Copy hdfs block from one data node to another
Posted by Chris Mawata <ch...@gmail.com>.
Haven't looked at the source but the thing you are trying to do sounds
similar to what happens when you are decommissioning a datanode. I would
hunt for that code.
Cheers
Chris
On Jul 9, 2014 3:41 PM, "Yehia Elshater" <y....@gmail.com> wrote:
> Hi Chris,
>
> Actually I need this functionality for my research, basically for fault
> tolerance. I can calculate some failure probability for some data nodes
> after certain unit of time. So I need to copy all the blocks reside on
> these nodes to another nodes.
>
> Thanks
> Yehia
>
>
> On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
>
>> Can you outline why one would want to do that? The blocks are disposable
>> so it is strange to manipulate them directly.
>> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> How can copy a certain hdfs block (given the file name, start and end
>>> bytes) from one node to another node ?
>>>
>>> Thanks
>>> Yehia
>>>
>>
>
Re: Copy hdfs block from one data node to another
Posted by Yehia Elshater <y....@gmail.com>.
Hi Chris,
Actually I need this functionality for my research, basically for fault
tolerance. I can calculate some failure probability for some data nodes
after certain unit of time. So I need to copy all the blocks reside on
these nodes to another nodes.
Thanks
Yehia
On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
> Can you outline why one would want to do that? The blocks are disposable
> so it is strange to manipulate them directly.
> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>
>> Hi All,
>>
>> How can copy a certain hdfs block (given the file name, start and end
>> bytes) from one node to another node ?
>>
>> Thanks
>> Yehia
>>
>
Re: Copy hdfs block from one data node to another
Posted by Yehia Elshater <y....@gmail.com>.
Hi Chris,
Actually I need this functionality for my research, basically for fault
tolerance. I can calculate some failure probability for some data nodes
after certain unit of time. So I need to copy all the blocks reside on
these nodes to another nodes.
Thanks
Yehia
On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
> Can you outline why one would want to do that? The blocks are disposable
> so it is strange to manipulate them directly.
> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>
>> Hi All,
>>
>> How can copy a certain hdfs block (given the file name, start and end
>> bytes) from one node to another node ?
>>
>> Thanks
>> Yehia
>>
>
Re: Copy hdfs block from one data node to another
Posted by Yehia Elshater <y....@gmail.com>.
Hi Chris,
Actually I need this functionality for my research, basically for fault
tolerance. I can calculate some failure probability for some data nodes
after certain unit of time. So I need to copy all the blocks reside on
these nodes to another nodes.
Thanks
Yehia
On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
> Can you outline why one would want to do that? The blocks are disposable
> so it is strange to manipulate them directly.
> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>
>> Hi All,
>>
>> How can copy a certain hdfs block (given the file name, start and end
>> bytes) from one node to another node ?
>>
>> Thanks
>> Yehia
>>
>
Re: Copy hdfs block from one data node to another
Posted by Yehia Elshater <y....@gmail.com>.
Hi Chris,
Actually I need this functionality for my research, basically for fault
tolerance. I can calculate some failure probability for some data nodes
after certain unit of time. So I need to copy all the blocks reside on
these nodes to another nodes.
Thanks
Yehia
On 7 July 2014 20:45, Chris Mawata <ch...@gmail.com> wrote:
> Can you outline why one would want to do that? The blocks are disposable
> so it is strange to manipulate them directly.
> On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
>
>> Hi All,
>>
>> How can copy a certain hdfs block (given the file name, start and end
>> bytes) from one node to another node ?
>>
>> Thanks
>> Yehia
>>
>
Re: Copy hdfs block from one data node to another
Posted by Chris Mawata <ch...@gmail.com>.
Can you outline why one would want to do that? The blocks are disposable so
it is strange to manipulate them directly.
On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
> Hi All,
>
> How can copy a certain hdfs block (given the file name, start and end
> bytes) from one node to another node ?
>
> Thanks
> Yehia
>
Re: Copy hdfs block from one data node to another
Posted by Chris Mawata <ch...@gmail.com>.
Can you outline why one would want to do that? The blocks are disposable so
it is strange to manipulate them directly.
On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
> Hi All,
>
> How can copy a certain hdfs block (given the file name, start and end
> bytes) from one node to another node ?
>
> Thanks
> Yehia
>
Re: Copy hdfs block from one data node to another
Posted by Chris Mawata <ch...@gmail.com>.
Can you outline why one would want to do that? The blocks are disposable so
it is strange to manipulate them directly.
On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
> Hi All,
>
> How can copy a certain hdfs block (given the file name, start and end
> bytes) from one node to another node ?
>
> Thanks
> Yehia
>
Re: Copy hdfs block from one data node to another
Posted by Chris Mawata <ch...@gmail.com>.
Can you outline why one would want to do that? The blocks are disposable so
it is strange to manipulate them directly.
On Jul 7, 2014 8:16 PM, "Yehia Elshater" <y....@gmail.com> wrote:
> Hi All,
>
> How can copy a certain hdfs block (given the file name, start and end
> bytes) from one node to another node ?
>
> Thanks
> Yehia
>