You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Boris Shkolnik (JIRA)" <ji...@apache.org> on 2009/01/09 01:12:59 UTC
[jira] Created: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
IndexOutOfBoundsException in FSEditLog
--------------------------------------
Key: HADOOP-4999
URL: https://issues.apache.org/jira/browse/HADOOP-4999
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Reporter: Boris Shkolnik
when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
int numEditStreams = editStreams.size();
for (int idx = 0; idx < numEditStreams(); idx++) {
...
processIOError(idx);
...
}
but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662466#action_12662466 ]
Boris Shkolnik commented on HADOOP-4999:
----------------------------------------
yeah, you're right.
Let me fix it.
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated HADOOP-4999:
-----------------------------------
Attachment: HADOOP-4999.patch
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated HADOOP-4999:
-----------------------------------
Status: Open (was: Patch Available)
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated HADOOP-4999:
-----------------------------------
Status: Patch Available (was: Open)
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-4999)
IndexOutOfBoundsException in FSEditLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12667906#action_12667906 ]
rangadi edited comment on HADOOP-4999 at 1/27/09 5:22 PM:
---------------------------------------------------------------
I just committed this to 0.20 and trunk. Thanks Boris!
was (Author: rangadi):
I just committed this to 0.20 and trunk.
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.20.0
>
> Attachments: HADOOP-4999-1.patch, HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673955#action_12673955 ]
Hudson commented on HADOOP-4999:
--------------------------------
Integrated in Hadoop-trunk #756 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/756/])
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.20.0
>
> Attachments: HADOOP-4999-1.patch, HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated HADOOP-4999:
-----------------------------------
Attachment: (was: HADOOP-4999.patch)
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-4999:
----------------------------------
Status: Open (was: Patch Available)
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4999:
---------------------------------
Release Note: A failure to write to FsEditsLog results in IndexOutOfBounds exception.
Hadoop Flags: [Reviewed]
I just committed this to 0.20 and trunk.
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.20.0
>
> Attachments: HADOOP-4999-1.patch, HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated HADOOP-4999:
-----------------------------------
Status: Patch Available (was: Open)
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-4999:
------------------------------------
Release Note: (was: A failure to write to FsEditsLog results in IndexOutOfBounds exception.)
No release note for "just a bug."
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.20.0
>
> Attachments: HADOOP-4999-1.patch, HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12666781#action_12666781 ]
Raghu Angadi commented on HADOOP-4999:
--------------------------------------
+1. looks good.
nit : the comment goes way beyond 80 columns.
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated HADOOP-4999:
-----------------------------------
Attachment: HADOOP-4999.patch
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated HADOOP-4999:
-----------------------------------
Fix Version/s: 0.20.0
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.20.0
>
> Attachments: HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated HADOOP-4999:
-----------------------------------
Attachment: HADOOP-4999-1.patch
fixed the comments
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.20.0
>
> Attachments: HADOOP-4999-1.patch, HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik reassigned HADOOP-4999:
--------------------------------------
Assignee: Boris Shkolnik
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662460#action_12662460 ]
Raghu Angadi commented on HADOOP-4999:
--------------------------------------
The error condition does occur, wouldn't the new loop skip a stream? We need to decrement idx, if a stream is removed.
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated HADOOP-4999:
-----------------------------------
Status: Patch Available (was: Open)
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12667897#action_12667897 ]
Boris Shkolnik commented on HADOOP-4999:
----------------------------------------
ran test-patch
ANT_HOME=/home/hadoopqa/tools/ant/apache-ant-1.7.1 ant -Dpatch.file=../patches/HADOOP-4999-1.patch -Dfindbugs.home=/home/ndaley/tools/findbugs/latest -Dforrest.home=/home/ndaley/tools/forrest/latest -Djava5.home=/usr/releng/tools/java/jdk1.5.0_06 -Dscratch.dir=../scratch_dir/ test-patch
results:
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec]
[exec]
No test is included, because it is a failure case and I tested it manually.
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.20.0
>
> Attachments: HADOOP-4999-1.patch, HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4999) IndexOutOfBoundsException in
FSEditLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4999:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> IndexOutOfBoundsException in FSEditLog
> --------------------------------------
>
> Key: HADOOP-4999
> URL: https://issues.apache.org/jira/browse/HADOOP-4999
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Fix For: 0.20.0
>
> Attachments: HADOOP-4999-1.patch, HADOOP-4999.patch
>
>
> when we go over a collection of editStreams in FSEditLog::logEdit we pre-calculate number of iterations for the "for loop":
> int numEditStreams = editStreams.size();
> for (int idx = 0; idx < numEditStreams(); idx++) {
> ...
> processIOError(idx);
> ...
> }
> but there is a possibility of an IOError that will call processIOError(idx) which will remove an editStream from editStreams inside the loop, and that will cause IndexOutOfBoundsException when end of collection is reached.
> proposed fix: recalculate size of the collection on every iteration (it is very cheap, cause it just returns an integer).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.