You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Atul Rajan <at...@gmail.com> on 2017/08/28 15:56:34 UTC

File copy from local to hdfs error

Hello Sir,

when i am copying data from local to hdfs i am getting below error :
can you please guide me how to proceed further with the error

copying could only be replicated to 0 nodes instead of multiplication (=1) There are 2 datanodes running and 2 nodes are excluded in this operation


Thanks and Regards
Atul Rajan

-Sent from my iPhone

Re: File copy from local to hdfs error

Posted by Atul Rajan <at...@gmail.com>.
Hello Istavan,

Thanks for the help it worked finally 
There was firewall issue solving that part made the hdfs work and take entry from local file system.

Thanks and Regards
Atul Rajan

-Sent from my iPhone

On 28-Aug-2017, at 11:20 PM, István Fajth <fa...@gmail.com> wrote:

Hi Atul,

as suggested before, set the blockmanager log level to debug, and check logs for reasons. You can either set the whole NameNode log to DEBUG level, and see for the messages logged by the BlockManager. Around the INFO level message in the NameNode log similar to the message you see now in the client as well, you will find the reason why BlockManager thinks that the two DN is not good to accept the block the client wish to allocate.

Cheers,
Istvan

> On Aug 28, 2017 19:24, "Atul Rajan" <at...@gmail.com> wrote:
> DataNodes were having issue earlier, i added the ports required in the iptables after that data logs are running but HDFS not able to distribute the file and make blocks. and any file copied on the cluster is throwing this error.
> 
>> On 28 August 2017 at 21:46, István Fajth <fa...@gmail.com> wrote:
>> Hi Atul,
>> 
>> you can check NameNode logs if the DataNodes were in service or there were issues with them. As well you can check for BlockManager's debug level logs for more exact reasons if you can reproduce the issue at will.
>> 
>> Istvan
>> 
>> On Aug 28, 2017 17:56, "Atul Rajan" <at...@gmail.com> wrote:
>> Hello Sir,
>> 
>> when i am copying data from local to hdfs i am getting below error :
>> can you please guide me how to proceed further with the error
>> 
>> copying could only be replicated to 0 nodes instead of multiplication (=1) There are 2 datanodes running and 2 nodes are excluded in this operation
>> 
>> 
>> Thanks and Regards
>> Atul Rajan
>> 
>> -Sent from my iPhone
>> 
> 
> 
> 
> -- 
> Best Regards
> Atul Rajan

Re: File copy from local to hdfs error

Posted by István Fajth <fa...@gmail.com>.
Hi Atul,

as suggested before, set the blockmanager log level to debug, and check
logs for reasons. You can either set the whole NameNode log to DEBUG level,
and see for the messages logged by the BlockManager. Around the INFO level
message in the NameNode log similar to the message you see now in the
client as well, you will find the reason why BlockManager thinks that the
two DN is not good to accept the block the client wish to allocate.

Cheers,
Istvan

On Aug 28, 2017 19:24, "Atul Rajan" <at...@gmail.com> wrote:

> DataNodes were having issue earlier, i added the ports required in the
> iptables after that data logs are running but HDFS not able to distribute
> the file and make blocks. and any file copied on the cluster is throwing
> this error.
>
> On 28 August 2017 at 21:46, István Fajth <fa...@gmail.com> wrote:
>
>> Hi Atul,
>>
>> you can check NameNode logs if the DataNodes were in service or there
>> were issues with them. As well you can check for BlockManager's debug level
>> logs for more exact reasons if you can reproduce the issue at will.
>>
>> Istvan
>>
>> On Aug 28, 2017 17:56, "Atul Rajan" <at...@gmail.com> wrote:
>>
>> Hello Sir,
>>
>> when i am copying data from local to hdfs i am getting below error :
>> can you please guide me how to proceed further with the error
>>
>> copying could only be replicated to 0 nodes instead of multiplication
>> (=1) There are 2 datanodes running and 2 nodes are excluded in this
>> operation
>>
>>
>>
>> Thanks and Regards
>> Atul Rajan
>>
>> -Sent from my iPhone
>>
>>
>>
>
>
> --
> *Best Regards*
> *Atul Rajan*
>

Re: File copy from local to hdfs error

Posted by Atul Rajan <at...@gmail.com>.
DataNodes were having issue earlier, i added the ports required in the
iptables after that data logs are running but HDFS not able to distribute
the file and make blocks. and any file copied on the cluster is throwing
this error.

On 28 August 2017 at 21:46, István Fajth <fa...@gmail.com> wrote:

> Hi Atul,
>
> you can check NameNode logs if the DataNodes were in service or there were
> issues with them. As well you can check for BlockManager's debug level logs
> for more exact reasons if you can reproduce the issue at will.
>
> Istvan
>
> On Aug 28, 2017 17:56, "Atul Rajan" <at...@gmail.com> wrote:
>
> Hello Sir,
>
> when i am copying data from local to hdfs i am getting below error :
> can you please guide me how to proceed further with the error
>
> copying could only be replicated to 0 nodes instead of multiplication (=1)
> There are 2 datanodes running and 2 nodes are excluded in this operation
>
>
>
> Thanks and Regards
> Atul Rajan
>
> -Sent from my iPhone
>
>
>


-- 
*Best Regards*
*Atul Rajan*

Re: File copy from local to hdfs error

Posted by István Fajth <fa...@gmail.com>.
Hi Atul,

you can check NameNode logs if the DataNodes were in service or there were
issues with them. As well you can check for BlockManager's debug level logs
for more exact reasons if you can reproduce the issue at will.

Istvan

On Aug 28, 2017 17:56, "Atul Rajan" <at...@gmail.com> wrote:

Hello Sir,

when i am copying data from local to hdfs i am getting below error :
can you please guide me how to proceed further with the error

copying could only be replicated to 0 nodes instead of multiplication (=1)
There are 2 datanodes running and 2 nodes are excluded in this operation



Thanks and Regards
Atul Rajan

-Sent from my iPhone