You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Niranjan Subramanian <ni...@webaction.com> on 2015/11/01 10:22:07 UTC
Re: Exception while appending to an existing file in HDFS
Bumping this. Any idea? What's wrong here?
On Fri, Oct 30, 2015 at 9:50 PM, Niranjan Subramanian <
niranjan@webaction.com> wrote:
> Hi guys,
>
> I'm basically trying to append data to an already exisiting file in HDFS.
> This is the exception I get
>
> 03:49:54,456WARN org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run:628 DataStreamer Exception
> java.lang.NullPointerException
> at com.google.protobuf.AbstractMessageLite$Builder.checkForNullValues(AbstractMessageLite.java:336)
> at com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:323)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$UpdatePipelineRequestProto$Builder.addAllStorageIDs(ClientNamenodeProtocolProtos.java)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.updatePipeline(ClientNamenodeProtocolTranslatorPB.java:842)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1238)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:532)
>
> My replication factor is 1. I'm using 2.5.0 of Apache's Hadoop
> distribution. This is the code snippet which I'm using for creating a file
> if it doesn't exist or create in append mode if it exists
>
> String url = getHadoopUrl()+fileName;
> Path file = new Path(url);
> try {
> if(append) {
> if(hadoopFileSystem.exists(file))
> fsDataOutputStream = hadoopFileSystem.append(file);
> else
> fsDataOutputStream = hadoopFileSystem.create(file);
> }
> else
> fsDataOutputStream = hadoopFileSystem.create(file);
>
> Appending stack trace that I found in datanode's log
>
> 2015-10-30 16:19:54,435 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-1012136337-192.168.123.103-1411103100884:blk_1073742239_1421 src: /127.0.0.1:54160 dest: /127.0.0.1:50010
> 2015-10-30 16:19:54,435 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Appending to FinalizedReplica, blk_1073742239_1421, FINALIZED
> getNumBytes() = 812
> getBytesOnDisk() = 812
> getVisibleLength()= 812
> getVolume() = /Users/niranjan/hadoop/hdfs/datanode/current
> getBlockFile() = /Users/niranjan/hadoop/hdfs/datanode/current/BP- 1012136337-192.168.123.103-1411103100884/current/finalized/blk_1073742239
> unlinked =false2015-10-30 16:19:54,461 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Exception for BP-1012136337-192.168.123.103-1411103100884:blk_1073742239_1422
> java.io.IOException: Premature EOF from inputStream
> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:194)
> at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)
> at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)
> at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)
> at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:435)
> at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:693)
> at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:569)
> at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:115)
> at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:68)
> at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:221)
> at java.lang.Thread.run(Thread.java:745)
>
>
> It's not quite clear what is causing this exception. Also I'm quite
> confused after reading various sources whether appending is supported in
> HDFS or not. Let me know what I'm missing here
>
> Regards,
> Niranjan
>
Re: Exception while appending to an existing file in HDFS
Posted by ABHISHEK SINGH <23...@gmail.com>.
Hi Niranjan,
Glad to hear your issue was resolved. Actually, upgrading was a wise
decision.
Thanks,
Abhishek
On Monday 02 November 2015 03:59 PM, Niranjan Subramanian wrote:
> Hi Abhishek,
>
> Thanks for your pointer on letting me know there is an issue from
> 2.2.0 to 2.5.1, I just upgraded to 2.7.1 and without adding any of
> these properties, it is working fine.
>
> Regards,
> Niranjan
> On 02-Nov-2015, at 1:38 pm, Niranjan Subramanian
> <niranjan@webaction.com <ma...@webaction.com>> wrote:
>
>> Hey Abhishek,
>>
>> I tried both tweaks, still I ended up with same exception. Maybe I
>> should try upgrading to 2.6.0 and see?
>>
>> Regards,
>> Niranjan
>> On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23singhabhishek@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>>> ("dfs.datanode.socket.write.timeout", "20000");
>>
>
Re: Exception while appending to an existing file in HDFS
Posted by ABHISHEK SINGH <23...@gmail.com>.
Hi Niranjan,
Glad to hear your issue was resolved. Actually, upgrading was a wise
decision.
Thanks,
Abhishek
On Monday 02 November 2015 03:59 PM, Niranjan Subramanian wrote:
> Hi Abhishek,
>
> Thanks for your pointer on letting me know there is an issue from
> 2.2.0 to 2.5.1, I just upgraded to 2.7.1 and without adding any of
> these properties, it is working fine.
>
> Regards,
> Niranjan
> On 02-Nov-2015, at 1:38 pm, Niranjan Subramanian
> <niranjan@webaction.com <ma...@webaction.com>> wrote:
>
>> Hey Abhishek,
>>
>> I tried both tweaks, still I ended up with same exception. Maybe I
>> should try upgrading to 2.6.0 and see?
>>
>> Regards,
>> Niranjan
>> On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23singhabhishek@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>>> ("dfs.datanode.socket.write.timeout", "20000");
>>
>
Re: Exception while appending to an existing file in HDFS
Posted by ABHISHEK SINGH <23...@gmail.com>.
Hi Niranjan,
Glad to hear your issue was resolved. Actually, upgrading was a wise
decision.
Thanks,
Abhishek
On Monday 02 November 2015 03:59 PM, Niranjan Subramanian wrote:
> Hi Abhishek,
>
> Thanks for your pointer on letting me know there is an issue from
> 2.2.0 to 2.5.1, I just upgraded to 2.7.1 and without adding any of
> these properties, it is working fine.
>
> Regards,
> Niranjan
> On 02-Nov-2015, at 1:38 pm, Niranjan Subramanian
> <niranjan@webaction.com <ma...@webaction.com>> wrote:
>
>> Hey Abhishek,
>>
>> I tried both tweaks, still I ended up with same exception. Maybe I
>> should try upgrading to 2.6.0 and see?
>>
>> Regards,
>> Niranjan
>> On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23singhabhishek@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>>> ("dfs.datanode.socket.write.timeout", "20000");
>>
>
Re: Exception while appending to an existing file in HDFS
Posted by ABHISHEK SINGH <23...@gmail.com>.
Hi Niranjan,
Glad to hear your issue was resolved. Actually, upgrading was a wise
decision.
Thanks,
Abhishek
On Monday 02 November 2015 03:59 PM, Niranjan Subramanian wrote:
> Hi Abhishek,
>
> Thanks for your pointer on letting me know there is an issue from
> 2.2.0 to 2.5.1, I just upgraded to 2.7.1 and without adding any of
> these properties, it is working fine.
>
> Regards,
> Niranjan
> On 02-Nov-2015, at 1:38 pm, Niranjan Subramanian
> <niranjan@webaction.com <ma...@webaction.com>> wrote:
>
>> Hey Abhishek,
>>
>> I tried both tweaks, still I ended up with same exception. Maybe I
>> should try upgrading to 2.6.0 and see?
>>
>> Regards,
>> Niranjan
>> On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23singhabhishek@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>>> ("dfs.datanode.socket.write.timeout", "20000");
>>
>
Re: Exception while appending to an existing file in HDFS
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hi Abhishek,
Thanks for your pointer on letting me know there is an issue from 2.2.0 to 2.5.1, I just upgraded to 2.7.1 and without adding any of these properties, it is working fine.
Regards,
Niranjan
On 02-Nov-2015, at 1:38 pm, Niranjan Subramanian <ni...@webaction.com> wrote:
> Hey Abhishek,
>
> I tried both tweaks, still I ended up with same exception. Maybe I should try upgrading to 2.6.0 and see?
>
> Regards,
> Niranjan
> On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23...@gmail.com> wrote:
>
>> ("dfs.datanode.socket.write.timeout", "20000");
>
Re: Exception while appending to an existing file in HDFS
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hi Abhishek,
Thanks for your pointer on letting me know there is an issue from 2.2.0 to 2.5.1, I just upgraded to 2.7.1 and without adding any of these properties, it is working fine.
Regards,
Niranjan
On 02-Nov-2015, at 1:38 pm, Niranjan Subramanian <ni...@webaction.com> wrote:
> Hey Abhishek,
>
> I tried both tweaks, still I ended up with same exception. Maybe I should try upgrading to 2.6.0 and see?
>
> Regards,
> Niranjan
> On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23...@gmail.com> wrote:
>
>> ("dfs.datanode.socket.write.timeout", "20000");
>
Re: Exception while appending to an existing file in HDFS
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hi Abhishek,
Thanks for your pointer on letting me know there is an issue from 2.2.0 to 2.5.1, I just upgraded to 2.7.1 and without adding any of these properties, it is working fine.
Regards,
Niranjan
On 02-Nov-2015, at 1:38 pm, Niranjan Subramanian <ni...@webaction.com> wrote:
> Hey Abhishek,
>
> I tried both tweaks, still I ended up with same exception. Maybe I should try upgrading to 2.6.0 and see?
>
> Regards,
> Niranjan
> On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23...@gmail.com> wrote:
>
>> ("dfs.datanode.socket.write.timeout", "20000");
>
Re: Exception while appending to an existing file in HDFS
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hi Abhishek,
Thanks for your pointer on letting me know there is an issue from 2.2.0 to 2.5.1, I just upgraded to 2.7.1 and without adding any of these properties, it is working fine.
Regards,
Niranjan
On 02-Nov-2015, at 1:38 pm, Niranjan Subramanian <ni...@webaction.com> wrote:
> Hey Abhishek,
>
> I tried both tweaks, still I ended up with same exception. Maybe I should try upgrading to 2.6.0 and see?
>
> Regards,
> Niranjan
> On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23...@gmail.com> wrote:
>
>> ("dfs.datanode.socket.write.timeout", "20000");
>
Re: Exception while appending to an existing file in HDFS
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hey Abhishek,
I tried both tweaks, still I ended up with same exception. Maybe I should try upgrading to 2.6.0 and see?
Regards,
Niranjan
On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23...@gmail.com> wrote:
> ("dfs.datanode.socket.write.timeout", "20000");
Re: Exception while appending to an existing file in HDFS
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hey Abhishek,
I tried both tweaks, still I ended up with same exception. Maybe I should try upgrading to 2.6.0 and see?
Regards,
Niranjan
On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23...@gmail.com> wrote:
> ("dfs.datanode.socket.write.timeout", "20000");
Re: Exception while appending to an existing file in HDFS
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hey Abhishek,
I tried both tweaks, still I ended up with same exception. Maybe I should try upgrading to 2.6.0 and see?
Regards,
Niranjan
On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23...@gmail.com> wrote:
> ("dfs.datanode.socket.write.timeout", "20000");
Re: Exception while appending to an existing file in HDFS
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hey Abhishek,
I tried both tweaks, still I ended up with same exception. Maybe I should try upgrading to 2.6.0 and see?
Regards,
Niranjan
On 01-Nov-2015, at 3:49 pm, ABHISHEK SINGH <23...@gmail.com> wrote:
> ("dfs.datanode.socket.write.timeout", "20000");
Re: Exception while appending to an existing file in HDFS
Posted by ABHISHEK SINGH <23...@gmail.com>.
Hi Niranjan,
Try these tweaks in hdfs-site.xml
1) <property>
<name>dfs.datanode.max.transfer.threads</name>
<value>8192</value>
<description>
Specifies the maximum number of threads to use for transferring
data
in and out of the DN.
</description>
</property>
<property>
<name>dfs.socket.timeout</name>
<value>20000</value>
</property>
<property>
<name>dfs.datanode.socket.write.timeout</name>
<value>20000</value>
</property>
2) In the code part, try this implementatin if above fails.
Configuration testConfig = new Configuration();
testConfig.set("fs.defaultFS", "hdfs://localhost:8020");
// deprecated key
// testConfig.set("dfs.socket.timeout", "20000");
testConfig.set("dfs.client.socket-timeout", "20000");
testConfig.set("dfs.datanode.socket.write.timeout", "20000");
This problem is witnessed in Hadoop version 2.2.0 to 2.5.1. Probable
solution could be to either use a lower idleTimeout or set higher
socket timeouts in hadoop config.
This is an issue with configuration.
Looking forward to hear from you.
Thanks,
Abhishek Singh
On Sunday 01 November 2015 02:52 PM, Niranjan Subramanian wrote:
> Bumping this. Any idea? What's wrong here?
>
> On Fri, Oct 30, 2015 at 9:50 PM, Niranjan Subramanian
> <niranjan@webaction.com <ma...@webaction.com>> wrote:
>
> Hi guys,
>
> I'm basically trying to append data to an already exisiting file
> in HDFS. This is the exception I get
>
> |03:49:54,456WARNorg.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run:628DataStreamerExceptionjava.lang.NullPointerExceptionat
> com.google.protobuf.AbstractMessageLite$Builder.checkForNullValues(AbstractMessageLite.java:336)at
> com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:323)at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$UpdatePipelineRequestProto$Builder.addAllStorageIDs(ClientNamenodeProtocolProtos.java)at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.updatePipeline(ClientNamenodeProtocolTranslatorPB.java:842)at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1238)at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:532)|
>
> My replication factor is 1. I'm using 2.5.0 of Apache's Hadoop
> distribution. This is the code snippet which I'm using for
> creating a file if it doesn't exist or create in append mode if it
> exists
>
> |Stringurl =getHadoopUrl()+fileName;Pathfile
> =newPath(url);try{if(append){if(hadoopFileSystem.exists(file))fsDataOutputStream
> =hadoopFileSystem.append(file);elsefsDataOutputStream
> =hadoopFileSystem.create(file);}elsefsDataOutputStream
> =hadoopFileSystem.create(file);|
>
> Appending stack trace that I found in datanode's log
>
> |2015-10-3016:19:54,435INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode:ReceivingBP-1012136337-192.168.123.103-1411103100884:blk_1073742239_1421
> src:/127.0.0.1:54160dest:/127.0.0.1:500102015-10-3016:19:54,435INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl:Appendingto
> FinalizedReplica,blk_1073742239_1421,FINALIZED
> getNumBytes()=812getBytesOnDisk()=812getVisibleLength()=812getVolume()=/Users/niranjan/hadoop/hdfs/datanode/current
> getBlockFile()=/Users/niranjan/hadoop/hdfs/datanode/current/BP-1012136337-192.168.123.103-1411103100884/current/finalized/blk_1073742239
> unlinked =false2015-10-3016:19:54,461INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode:ExceptionforBP-1012136337-192.168.123.103-1411103100884:blk_1073742239_1422
> java.io.IOException:PrematureEOF from inputStream at
> org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:194)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:435)at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:693)at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:569)at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:115)at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:68)at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:221)at
> java.lang.Thread.run(Thread.java:745)|
>
>
> It's not quite clear what is causing this exception. Also I'm
> quite confused after reading various sources whether appending is
> supported in HDFS or not. Let me know what I'm missing here
>
> Regards,
> Niranjan
>
>
Re: Exception while appending to an existing file in HDFS
Posted by ABHISHEK SINGH <23...@gmail.com>.
Hi Niranjan,
Try these tweaks in hdfs-site.xml
1) <property>
<name>dfs.datanode.max.transfer.threads</name>
<value>8192</value>
<description>
Specifies the maximum number of threads to use for transferring
data
in and out of the DN.
</description>
</property>
<property>
<name>dfs.socket.timeout</name>
<value>20000</value>
</property>
<property>
<name>dfs.datanode.socket.write.timeout</name>
<value>20000</value>
</property>
2) In the code part, try this implementatin if above fails.
Configuration testConfig = new Configuration();
testConfig.set("fs.defaultFS", "hdfs://localhost:8020");
// deprecated key
// testConfig.set("dfs.socket.timeout", "20000");
testConfig.set("dfs.client.socket-timeout", "20000");
testConfig.set("dfs.datanode.socket.write.timeout", "20000");
This problem is witnessed in Hadoop version 2.2.0 to 2.5.1. Probable
solution could be to either use a lower idleTimeout or set higher
socket timeouts in hadoop config.
This is an issue with configuration.
Looking forward to hear from you.
Thanks,
Abhishek Singh
On Sunday 01 November 2015 02:52 PM, Niranjan Subramanian wrote:
> Bumping this. Any idea? What's wrong here?
>
> On Fri, Oct 30, 2015 at 9:50 PM, Niranjan Subramanian
> <niranjan@webaction.com <ma...@webaction.com>> wrote:
>
> Hi guys,
>
> I'm basically trying to append data to an already exisiting file
> in HDFS. This is the exception I get
>
> |03:49:54,456WARNorg.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run:628DataStreamerExceptionjava.lang.NullPointerExceptionat
> com.google.protobuf.AbstractMessageLite$Builder.checkForNullValues(AbstractMessageLite.java:336)at
> com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:323)at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$UpdatePipelineRequestProto$Builder.addAllStorageIDs(ClientNamenodeProtocolProtos.java)at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.updatePipeline(ClientNamenodeProtocolTranslatorPB.java:842)at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1238)at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:532)|
>
> My replication factor is 1. I'm using 2.5.0 of Apache's Hadoop
> distribution. This is the code snippet which I'm using for
> creating a file if it doesn't exist or create in append mode if it
> exists
>
> |Stringurl =getHadoopUrl()+fileName;Pathfile
> =newPath(url);try{if(append){if(hadoopFileSystem.exists(file))fsDataOutputStream
> =hadoopFileSystem.append(file);elsefsDataOutputStream
> =hadoopFileSystem.create(file);}elsefsDataOutputStream
> =hadoopFileSystem.create(file);|
>
> Appending stack trace that I found in datanode's log
>
> |2015-10-3016:19:54,435INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode:ReceivingBP-1012136337-192.168.123.103-1411103100884:blk_1073742239_1421
> src:/127.0.0.1:54160dest:/127.0.0.1:500102015-10-3016:19:54,435INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl:Appendingto
> FinalizedReplica,blk_1073742239_1421,FINALIZED
> getNumBytes()=812getBytesOnDisk()=812getVisibleLength()=812getVolume()=/Users/niranjan/hadoop/hdfs/datanode/current
> getBlockFile()=/Users/niranjan/hadoop/hdfs/datanode/current/BP-1012136337-192.168.123.103-1411103100884/current/finalized/blk_1073742239
> unlinked =false2015-10-3016:19:54,461INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode:ExceptionforBP-1012136337-192.168.123.103-1411103100884:blk_1073742239_1422
> java.io.IOException:PrematureEOF from inputStream at
> org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:194)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:435)at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:693)at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:569)at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:115)at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:68)at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:221)at
> java.lang.Thread.run(Thread.java:745)|
>
>
> It's not quite clear what is causing this exception. Also I'm
> quite confused after reading various sources whether appending is
> supported in HDFS or not. Let me know what I'm missing here
>
> Regards,
> Niranjan
>
>
Re: Exception while appending to an existing file in HDFS
Posted by ABHISHEK SINGH <23...@gmail.com>.
Hi Niranjan,
Try these tweaks in hdfs-site.xml
1) <property>
<name>dfs.datanode.max.transfer.threads</name>
<value>8192</value>
<description>
Specifies the maximum number of threads to use for transferring
data
in and out of the DN.
</description>
</property>
<property>
<name>dfs.socket.timeout</name>
<value>20000</value>
</property>
<property>
<name>dfs.datanode.socket.write.timeout</name>
<value>20000</value>
</property>
2) In the code part, try this implementatin if above fails.
Configuration testConfig = new Configuration();
testConfig.set("fs.defaultFS", "hdfs://localhost:8020");
// deprecated key
// testConfig.set("dfs.socket.timeout", "20000");
testConfig.set("dfs.client.socket-timeout", "20000");
testConfig.set("dfs.datanode.socket.write.timeout", "20000");
This problem is witnessed in Hadoop version 2.2.0 to 2.5.1. Probable
solution could be to either use a lower idleTimeout or set higher
socket timeouts in hadoop config.
This is an issue with configuration.
Looking forward to hear from you.
Thanks,
Abhishek Singh
On Sunday 01 November 2015 02:52 PM, Niranjan Subramanian wrote:
> Bumping this. Any idea? What's wrong here?
>
> On Fri, Oct 30, 2015 at 9:50 PM, Niranjan Subramanian
> <niranjan@webaction.com <ma...@webaction.com>> wrote:
>
> Hi guys,
>
> I'm basically trying to append data to an already exisiting file
> in HDFS. This is the exception I get
>
> |03:49:54,456WARNorg.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run:628DataStreamerExceptionjava.lang.NullPointerExceptionat
> com.google.protobuf.AbstractMessageLite$Builder.checkForNullValues(AbstractMessageLite.java:336)at
> com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:323)at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$UpdatePipelineRequestProto$Builder.addAllStorageIDs(ClientNamenodeProtocolProtos.java)at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.updatePipeline(ClientNamenodeProtocolTranslatorPB.java:842)at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1238)at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:532)|
>
> My replication factor is 1. I'm using 2.5.0 of Apache's Hadoop
> distribution. This is the code snippet which I'm using for
> creating a file if it doesn't exist or create in append mode if it
> exists
>
> |Stringurl =getHadoopUrl()+fileName;Pathfile
> =newPath(url);try{if(append){if(hadoopFileSystem.exists(file))fsDataOutputStream
> =hadoopFileSystem.append(file);elsefsDataOutputStream
> =hadoopFileSystem.create(file);}elsefsDataOutputStream
> =hadoopFileSystem.create(file);|
>
> Appending stack trace that I found in datanode's log
>
> |2015-10-3016:19:54,435INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode:ReceivingBP-1012136337-192.168.123.103-1411103100884:blk_1073742239_1421
> src:/127.0.0.1:54160dest:/127.0.0.1:500102015-10-3016:19:54,435INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl:Appendingto
> FinalizedReplica,blk_1073742239_1421,FINALIZED
> getNumBytes()=812getBytesOnDisk()=812getVisibleLength()=812getVolume()=/Users/niranjan/hadoop/hdfs/datanode/current
> getBlockFile()=/Users/niranjan/hadoop/hdfs/datanode/current/BP-1012136337-192.168.123.103-1411103100884/current/finalized/blk_1073742239
> unlinked =false2015-10-3016:19:54,461INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode:ExceptionforBP-1012136337-192.168.123.103-1411103100884:blk_1073742239_1422
> java.io.IOException:PrematureEOF from inputStream at
> org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:194)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:435)at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:693)at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:569)at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:115)at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:68)at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:221)at
> java.lang.Thread.run(Thread.java:745)|
>
>
> It's not quite clear what is causing this exception. Also I'm
> quite confused after reading various sources whether appending is
> supported in HDFS or not. Let me know what I'm missing here
>
> Regards,
> Niranjan
>
>
Re: Exception while appending to an existing file in HDFS
Posted by ABHISHEK SINGH <23...@gmail.com>.
Hi Niranjan,
Try these tweaks in hdfs-site.xml
1) <property>
<name>dfs.datanode.max.transfer.threads</name>
<value>8192</value>
<description>
Specifies the maximum number of threads to use for transferring
data
in and out of the DN.
</description>
</property>
<property>
<name>dfs.socket.timeout</name>
<value>20000</value>
</property>
<property>
<name>dfs.datanode.socket.write.timeout</name>
<value>20000</value>
</property>
2) In the code part, try this implementatin if above fails.
Configuration testConfig = new Configuration();
testConfig.set("fs.defaultFS", "hdfs://localhost:8020");
// deprecated key
// testConfig.set("dfs.socket.timeout", "20000");
testConfig.set("dfs.client.socket-timeout", "20000");
testConfig.set("dfs.datanode.socket.write.timeout", "20000");
This problem is witnessed in Hadoop version 2.2.0 to 2.5.1. Probable
solution could be to either use a lower idleTimeout or set higher
socket timeouts in hadoop config.
This is an issue with configuration.
Looking forward to hear from you.
Thanks,
Abhishek Singh
On Sunday 01 November 2015 02:52 PM, Niranjan Subramanian wrote:
> Bumping this. Any idea? What's wrong here?
>
> On Fri, Oct 30, 2015 at 9:50 PM, Niranjan Subramanian
> <niranjan@webaction.com <ma...@webaction.com>> wrote:
>
> Hi guys,
>
> I'm basically trying to append data to an already exisiting file
> in HDFS. This is the exception I get
>
> |03:49:54,456WARNorg.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run:628DataStreamerExceptionjava.lang.NullPointerExceptionat
> com.google.protobuf.AbstractMessageLite$Builder.checkForNullValues(AbstractMessageLite.java:336)at
> com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:323)at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$UpdatePipelineRequestProto$Builder.addAllStorageIDs(ClientNamenodeProtocolProtos.java)at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.updatePipeline(ClientNamenodeProtocolTranslatorPB.java:842)at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1238)at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:532)|
>
> My replication factor is 1. I'm using 2.5.0 of Apache's Hadoop
> distribution. This is the code snippet which I'm using for
> creating a file if it doesn't exist or create in append mode if it
> exists
>
> |Stringurl =getHadoopUrl()+fileName;Pathfile
> =newPath(url);try{if(append){if(hadoopFileSystem.exists(file))fsDataOutputStream
> =hadoopFileSystem.append(file);elsefsDataOutputStream
> =hadoopFileSystem.create(file);}elsefsDataOutputStream
> =hadoopFileSystem.create(file);|
>
> Appending stack trace that I found in datanode's log
>
> |2015-10-3016:19:54,435INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode:ReceivingBP-1012136337-192.168.123.103-1411103100884:blk_1073742239_1421
> src:/127.0.0.1:54160dest:/127.0.0.1:500102015-10-3016:19:54,435INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl:Appendingto
> FinalizedReplica,blk_1073742239_1421,FINALIZED
> getNumBytes()=812getBytesOnDisk()=812getVisibleLength()=812getVolume()=/Users/niranjan/hadoop/hdfs/datanode/current
> getBlockFile()=/Users/niranjan/hadoop/hdfs/datanode/current/BP-1012136337-192.168.123.103-1411103100884/current/finalized/blk_1073742239
> unlinked =false2015-10-3016:19:54,461INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode:ExceptionforBP-1012136337-192.168.123.103-1411103100884:blk_1073742239_1422
> java.io.IOException:PrematureEOF from inputStream at
> org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:194)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)at
> org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:435)at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:693)at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:569)at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:115)at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:68)at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:221)at
> java.lang.Thread.run(Thread.java:745)|
>
>
> It's not quite clear what is causing this exception. Also I'm
> quite confused after reading various sources whether appending is
> supported in HDFS or not. Let me know what I'm missing here
>
> Regards,
> Niranjan
>
>