You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Aaron Kimball (JIRA)" <ji...@apache.org> on 2008/09/10 02:39:44 UTC
[jira] Created: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
ScriptBasedMapping has infinite loop with invalid jobconf parameter
-------------------------------------------------------------------
Key: HADOOP-4141
URL: https://issues.apache.org/jira/browse/HADOOP-4141
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.18.0
Reporter: Aaron Kimball
Priority: Minor
Attachments: scriptMap.patch
A few bugs in ScriptBasedTopology:
1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball reopened HADOOP-4141:
-----------------------------------
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.1
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: HADOOP-4141.patch.3, scriptMap.patch, scriptMap.patch.2, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4141) ScriptBasedMapping has infinite
loop with invalid jobconf parameter
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629710#action_12629710 ]
Hadoop QA commented on HADOOP-4141:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12389792/TestScriptBasedMapping.java
against trunk revision 693705.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3226/console
This message is automatically generated.
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.0
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: scriptMap.patch, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-4141:
----------------------------------
Attachment: scriptMap.patch
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.0
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: scriptMap.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4141) ScriptBasedMapping has infinite
loop with invalid jobconf parameter
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12643770#action_12643770 ]
Aaron Kimball commented on HADOOP-4141:
---------------------------------------
Sorry this took so long. Been swamped; here you are.
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.1
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: HADOOP-4141.patch.3, scriptMap.patch, scriptMap.patch.2, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball resolved HADOOP-4141.
-----------------------------------
Resolution: Fixed
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.0
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: HADOOP-4141.patch.3, scriptMap.patch, scriptMap.patch.2, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-4141:
----------------------------------
Affects Version/s: (was: 0.18.0)
0.18.1
Status: Patch Available (was: Reopened)
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.1
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: HADOOP-4141.patch.3, scriptMap.patch, scriptMap.patch.2, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-4141:
----------------------------------
Attachment: scriptMap.patch.2
Added the Test file to the patch
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.0
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: scriptMap.patch, scriptMap.patch.2, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-4141:
----------------------------------
Status: Open (was: Patch Available)
Aaron - minor nit: could you please make the newly introduced 'static' variables 'package-private' rather than public? Thanks!
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.0
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: scriptMap.patch, scriptMap.patch.2, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-4141:
------------------------------
Resolution: Fixed
Fix Version/s: 0.20.0
Assignee: Aaron Kimball
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I've just committed this. Thanks Aaron!
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.1
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Priority: Minor
> Fix For: 0.20.0
>
> Attachments: HADOOP-4141.patch.3, scriptMap.patch, scriptMap.patch.2, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-4141:
----------------------------------
Status: Patch Available (was: Open)
Here's the patch for the fix.
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.0
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: scriptMap.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4141) ScriptBasedMapping has infinite
loop with invalid jobconf parameter
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12643801#action_12643801 ]
Hadoop QA commented on HADOOP-4141:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12393038/HADOOP-4141.patch.3
against trunk revision 709040.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3510/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3510/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3510/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3510/console
This message is automatically generated.
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.1
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: HADOOP-4141.patch.3, scriptMap.patch, scriptMap.patch.2, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4141) ScriptBasedMapping has infinite
loop with invalid jobconf parameter
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630071#action_12630071 ]
Aaron Kimball commented on HADOOP-4141:
---------------------------------------
As a related comment:
The patch I've submitted will log a warning-level message when the resolver is actually executed, and return an error value (null).
There's currently no great way to report these issues at configure()-time. You can throw an unchecked exception if you'd like, but should we maybe think about adding a checked MisconfigurationException so that if configure() tries to load a value that isn't valid, we have a structured way of propagating these results to the client?
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.0
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: scriptMap.patch, scriptMap.patch.2, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-4141:
----------------------------------
Attachment: HADOOP-4141.patch.3
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.0
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: HADOOP-4141.patch.3, scriptMap.patch, scriptMap.patch.2, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4141) ScriptBasedMapping has infinite loop
with invalid jobconf parameter
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-4141:
----------------------------------
Attachment: TestScriptBasedMapping.java
Here's a test case for this ('svn diff' doesn't include this new file so it's posted separately)
> ScriptBasedMapping has infinite loop with invalid jobconf parameter
> -------------------------------------------------------------------
>
> Key: HADOOP-4141
> URL: https://issues.apache.org/jira/browse/HADOOP-4141
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.0
> Reporter: Aaron Kimball
> Priority: Minor
> Attachments: scriptMap.patch, TestScriptBasedMapping.java
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> A few bugs in ScriptBasedTopology:
> 1) topology.script.number.args is not validated; if this is <= 0, the runResolveCommand() method will run its outer loop an infinite number of times.
> 2) CachedDNSToSwitchMapping assumes that if a List is returned, it has the same cardinality as the number of names. ScriptBasedMapping should return 'null' instead of a mis-sized list on error
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.