You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Junping Du (JIRA)" <ji...@apache.org> on 2012/04/23 12:37:35 UTC
[jira] [Created] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Junping Du created HADOOP-8304:
----------------------------------
Summary: DNSToSwitchMapping should add interface to resolve individual host besides a list of host
Key: HADOOP-8304
URL: https://issues.apache.org/jira/browse/HADOOP-8304
Project: Hadoop Common
Issue Type: Improvement
Components: io
Affects Versions: 1.0.0, 2.0.0
Reporter: Junping Du
Assignee: Junping Du
Fix For: 2.0.0
DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8304:
-------------------------------
Status: Patch Available (was: Open)
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262366#comment-13262366 ]
Junping Du commented on HADOOP-8304:
------------------------------------
I update patches in HDFS-3324 and MAPREDUCE-4198 as previous patch is wrongly including common project code (caused by switching between local branch). I should be more careful next time. Sorry for the trouble.
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260306#comment-13260306 ]
Hadoop QA commented on HADOOP-8304:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523938/HADOOP-8304.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified test files.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/883//console
This message is automatically generated.
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262374#comment-13262374 ]
Hadoop QA commented on HADOOP-8304:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12524393/HADOOP-8304-V2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 5 new or modified test files.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/890//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/890//console
This message is automatically generated.
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263145#comment-13263145 ]
Eli Collins commented on HADOOP-8304:
-------------------------------------
Good point, forgot that people override DNSToSwitchMapping. Technically DNSToSwitchMapping is evolving (and should stay that way), but yea no sense in breaking people if we don't have to.
Junping, how about introducing the fix in a compatible way with the List interface?
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270586#comment-13270586 ]
Eli Collins commented on HADOOP-8304:
-------------------------------------
bq. Do you see any scenario to resolve a list of host? (not counting the unit test)
DatanodeManager does that today with a list obtained from the hosts files.
bq. I don't understand the question of last comment there as I just want to fix the interface here
You mentioned earlier "identify a potential bug that a hostname start with number may not been resolved properly" - doesn't that issue still need to be addressed?
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270607#comment-13270607 ]
Junping Du commented on HADOOP-8304:
------------------------------------
That's for caching only. If you think list interface here is good, I agree that it may be better to leave it there which will not take complexity of incompatibility.
About the bug that wrongly resolve the hostname starting with number, yes. I will go ahead to file a jira and it could be easy to fix it.
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8304:
-------------------------------
Attachment: HADOOP-8304-V2.patch
Update patch to include hadoop-common project only, and put rest code into other (HDFS/MAPREDUCE) jira issue
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8304:
-------------------------------
Attachment: HADOOP-8304.patch
The patch to add resolve(String host) interface to DNSToSwitchMapping, and some related unit tests. Also, identify a potential bug that a hostname start with number may not been resolved properly.
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268933#comment-13268933 ]
Junping Du commented on HADOOP-8304:
------------------------------------
That's good comments on compatibility of this change. I saw evolving tag there but not consider people extends DNSToSwitchMapping as well (just thought ScriptBased, TableMapping and cached are good enough).
I won't say original interface cause some performance headache as the time of resolving rack info can be overwhelmed comparing with the whole flow (replica placement or task scheduling). However, it is more easy to use for major consumers of original interface which are expecting to resolve individual host. Do you see any scenario to resolve a list of host? (not counting the unit test)
Eli, I don't understand the question of last comment there as I just want to fix the interface here. :)
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262352#comment-13262352 ]
Junping Du commented on HADOOP-8304:
------------------------------------
Hello Eli,
I already updated the code to include common part only. The rest code (HDFS/MAPREDUCE) are going to MAPREDUCE-4198 and HDFS-3324. All are marked patch available for review.
Cheers,
Junping
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262326#comment-13262326 ]
Junping Du commented on HADOOP-8304:
------------------------------------
Hi Eli,
Thanks for suggestion. Will do that soon.
Best,
Junping
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8304:
-------------------------------
Attachment: HADOOP-8304-V2.patch
Grant license to ASF.
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262281#comment-13262281 ]
Eli Collins commented on HADOOP-8304:
-------------------------------------
Sounds reasonable to me Junping. test-patch doesn't run on cross-project patches. For this patch just introduce the new interface in common, and then file a separate HDFS and MR jira to use the new API.
Thanks,
Eli
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271359#comment-13271359 ]
Junping Du commented on HADOOP-8304:
------------------------------------
Hi Eli,
The jira is filed as https://issues.apache.org/jira/browse/HADOOP-8372. I uploaded a patch to fix this, but that will cause some side-effect.
Thanks,
Junping
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8304:
--------------------------------
Resolution: Not A Problem
Status: Resolved (was: Patch Available)
Cool, closing this one as won't fix. Thanks for filing a separate jira, please link it in here.
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262358#comment-13262358 ]
Hadoop QA commented on HADOOP-8304:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12524388/HADOOP-8304-V2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 5 new or modified test files.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.fs.viewfs.TestViewFsTrash
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/889//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/889//console
This message is automatically generated.
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8304) DNSToSwitchMapping should add
interface to resolve individual host besides a list of host
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262825#comment-13262825 ]
Tom White commented on HADOOP-8304:
-----------------------------------
This adds a method to an interface which is an incompatible change, and will break implementers. Is the overhead of wrapping a string in a list causing performance problems?
> DNSToSwitchMapping should add interface to resolve individual host besides a list of host
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-8304
> URL: https://issues.apache.org/jira/browse/HADOOP-8304
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Fix For: 2.0.0
>
> Attachments: HADOOP-8304-V2.patch, HADOOP-8304-V2.patch, HADOOP-8304.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DNSToSwitchMapping now has only one API to resolve a host list: public List<String> resolve(List<String> names). But the two major caller: RackResolver.resolve() and DatanodeManager.resolveNetworkLocation() are taking single host name but have to wrapper it to an single entry ArrayList. This is not necessary especially the host has been cached before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira