You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2007/04/03 01:38:32 UTC
[jira] Commented: (HADOOP-1093) NNBench generates millions of
NotReplicatedYetException in Namenode log
[ https://issues.apache.org/jira/browse/HADOOP-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12486174 ]
Hairong Kuang commented on HADOOP-1093:
---------------------------------------
For the change 2, I think it would be nice to check if the previous block has reached the minimum # of replicas or not but not for all the previous blocks. In this way we are able to slow down client writing when dfs is not able to handle the amount of data in time.
> NNBench generates millions of NotReplicatedYetException in Namenode log
> -----------------------------------------------------------------------
>
> Key: HADOOP-1093
> URL: https://issues.apache.org/jira/browse/HADOOP-1093
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Nigel Daley
> Assigned To: dhruba borthakur
> Fix For: 0.13.0
>
> Attachments: nyr2.patch
>
>
> Running NNBench on latest trunk (0.12.1 candidate) on a few hundred nodes yielded 2.3 million of these exceptions in the NN log:
> 2007-03-08 09:23:03,053 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8020 call error:
> org.apache.hadoop.dfs.NotReplicatedYetException: Not replicated yet
> at org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:803)
> at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:309)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:336)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:559)
> I run NNBench to create files with block size set to 1 and replication set to 1. NNBench then writes 1 byte to the file. Minimum replication for the cluster is the default, ie 1. If it encounters an exception while trying to do either the create or write operations, it loops and tries again. Multiply this by 1000 files per node and a few hundred nodes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.