You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "dhruba borthakur (JIRA)" <ji...@apache.org> on 2007/03/20 18:06:32 UTC
[jira] Created: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
-------------------------------------------------------------------------------------------
Key: HADOOP-1136
URL: https://issues.apache.org/jira/browse/HADOOP-1136
Project: Hadoop
Issue Type: Bug
Reporter: dhruba borthakur
Assigned To: Hairong Kuang
I was running a random writer followed by a sort when I saw this condition:
Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
at java.lang.Thread.run(Thread.java:619)
processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1136:
----------------------------------
Attachment: neededReplicationAdd1.patch
Resubmit the patch.
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch, neededReplicationAdd1.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1136:
----------------------------------
Attachment: neededReplicationAdd.patch
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-1136:
------------------------------
Status: Open (was: Patch Available)
Please resubmit the patch now HADOOP-702 has been committed. Thanks.
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-1136:
---------------------------------
Status: Open (was: Patch Available)
Will resubmit after HADOOP-702 is committed.
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1136:
-------------------------------------
Component/s: dfs
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1136:
----------------------------------
Status: Patch Available (was: Open)
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch, neededReplicationAdd1.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-1136:
------------------------------
Status: Open (was: Patch Available)
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12482916 ]
Hairong Kuang commented on HADOOP-1136:
---------------------------------------
I checked the code. If a block is overreplicated, UnderReplicatedBlocks.add handles the situation well. ArrayIndexOutOfBoundsException is impossible to be thrown. It seems that the only possibility that causes the exception is that the time-out block has zero copy.
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-1136:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I've just committed this. Thanks Hairong!
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch, neededReplicationAdd1.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12486547 ]
Hadoop QA commented on HADOOP-1136:
-----------------------------------
+1, because http://issues.apache.org/jira/secure/attachment/12353893/neededReplicationAdd.patch applied and successfully tested against trunk revision http://svn.apache.org/repos/asf/lucene/hadoop/trunk/525290. Results are at http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-1136:
---------------------------------
Status: Patch Available (was: Open)
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1136:
----------------------------------
Attachment: neededReplicationAdd.patch
Dhruba, could you please check if this patch gets rid of the exception? Thanks.
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1136:
----------------------------------
Status: Patch Available (was: Open)
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1136:
----------------------------------
Attachment: (was: neededReplicationAdd.patch)
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1136:
----------------------------------
Fix Version/s: 0.13.0
Affects Version/s: 0.12.2
Status: Patch Available (was: Open)
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12482960 ]
dhruba borthakur commented on HADOOP-1136:
------------------------------------------
I cannot reproduce this problem on demand. So I am not able to test this conclusively. But your code change looks good.
We need this fix.
+1.
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Attachments: neededReplicationAdd.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1136) exception in
UnderReplicatedBlocks:add when ther are more replicas of a block than
required
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12488051 ]
Hadoop QA commented on HADOOP-1136:
-----------------------------------
Integrated in Hadoop-Nightly #54 (See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/54/)
> exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-1136
> URL: https://issues.apache.org/jira/browse/HADOOP-1136
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.2
> Reporter: dhruba borthakur
> Assigned To: Hairong Kuang
> Fix For: 0.13.0
>
> Attachments: neededReplicationAdd.patch, neededReplicationAdd1.patch
>
>
> I was running a random writer followed by a sort when I saw this condition:
> Exception in thread "org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor@187814" java.lang.ArrayIndexOutOfBoundsException: 3
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:447)
> at org.apache.hadoop.dfs.FSNamesystem$UnderReplicatedBlocks.add(FSNamesystem.java:464)
> at org.apache.hadoop.dfs.FSNamesystem.processPendingReplications(FSNamesystem.java:1891)
> at org.apache.hadoop.dfs.FSNamesystem$ReplicationMonitor.run(FSNamesystem.java:1795)
> at java.lang.Thread.run(Thread.java:619)
> processPendingReplications is invoking neededReplications.add(). The add method then invokes add(block, 3, 4). This add method calls getPriority and that returns 3. Now, the call to priorityQueues[3].add() throws an OutOfBoundsException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.