You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Kan Zhang (JIRA)" <ji...@apache.org> on 2009/02/28 03:25:12 UTC
[jira] Created: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Proxying for multiple HDFS clusters of different versions
---------------------------------------------------------
Key: HADOOP-5363
URL: https://issues.apache.org/jira/browse/HADOOP-5363
Project: Hadoop Core
Issue Type: New Feature
Components: contrib/hdfsproxy
Reporter: Kan Zhang
Assignee: zhiyong zhang
A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689145#action_12689145 ]
Hadoop QA commented on HADOOP-5363:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12403566/HADOOP-5363.patch
against trunk revision 758232.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/136/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/136/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/136/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/136/console
This message is automatically generated.
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "gary murry (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
gary murry updated HADOOP-5363:
-------------------------------
Attachment: ProxyTestPlan.html
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Fix For: 0.21.0
>
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch, ProxyTestPlan.html
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5363:
----------------------------------
Attachment: HADOOP-5363.patch
---++ General Design Idea.
* 1. Use forward servlet to forward requests to the right hadoop version servlet.
* 2. Package each version's servlet into a separate war file.
* 3. Communication between forwarding war and hadoop-specific version war only pass build-in type to avoid class loading issue.
* 4. Use a configuration file (hdfsproxy-site.xml) to identify the war file path. Format example (suppose www.apache1.com and www.apache2.com point to the same ip address through some dns configuration)
<property>
<name>www.apache1.com</name>
<value>/v18</value>
<description>one hostname corresponds to one web application archive
</description>
</property>
<property>
<name>www.apache2.com</name>
<value>/v21</value>
<description>one hostname corresponds to one web application archive
</description>
</property>
* 5. each war file package one proxy, which corresponds to one source cluster. even same hadoop version can use different war file to point to different source cluster. i.e. the war file are source-cluster oriented, but not hadoop version oriented.
* 6. tomcat need to set crossContext="true" in context.xml config file to allow cross war forwarding.
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695447#action_12695447 ]
Hudson commented on HADOOP-5363:
--------------------------------
Integrated in Hadoop-trunk #796 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/796/])
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Fix For: 0.21.0
>
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-5363:
----------------------------------
Status: Open (was: Patch Available)
Unfortunately, HADOOP-5390 conflicts with this patch. Would you mind regenerating it?
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12688593#action_12688593 ]
Chris Douglas commented on HADOOP-5363:
---------------------------------------
This looks good. Just a few small notes:
* Since pickOneAddress was added in support of hdfsproxy (HADOOP-4575), if it's no longer needed then it should be removed from HftpFileSystem
* Instead of pulling the configuration out of the context in doGet, it can be saved in HttpServlet::init in ProxyForwardServlet.
* Unless {{doForward}} has special meaning, it should probably not be confused with {{doGet}}, {{doHead}}, etc. that are part of HttpServlet. Either putting the body of that method in {{doGet}} or renaming it would make it more readable. {{doGet}} should have an @Override annotation
* Please include javadoc for public methods, where reasonable.
* I'm confused by {{getServletPath}} on the servlets; isn't this usually on the HttpServletRequest, set by the server to the matching part of the URI? Unless it has special meaning: rather than overloading {{getServletPath}} to return a fixed String, then changing behavior of {{doForward}} based on whether it matches "/path", it might make more sense to put the logic in the "else" block in a method, then override it in the ProxyFileForward servlet.
* ProxyUtil::main should print usage when it has too few arguments rather than throwing an exception for {{-get}}.
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas resolved HADOOP-5363.
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.21.0
Hadoop Flags: [Reviewed]
I committed this. Thanks, Zhiyong
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Fix For: 0.21.0
>
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-5363:
----------------------------------
Status: Patch Available (was: Open)
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12688587#action_12688587 ]
Hadoop QA commented on HADOOP-5363:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12402178/HADOOP-5363.patch
against trunk revision 757625.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 19 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/125/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/125/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/125/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/125/console
This message is automatically generated.
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5363:
----------------------------------
Attachment: HADOOP-5363.patch
Make a new ProxyFileForward.java as a subclass of ProxyForwardServlet to handle /file requests.
Thanks Chris
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689308#action_12689308 ]
Chris Douglas commented on HADOOP-5363:
---------------------------------------
+1 Looks good
{noformat}
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 18 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
{noformat}
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689222#action_12689222 ]
Chris Douglas commented on HADOOP-5363:
---------------------------------------
For the forwarding servlets, I was thinking that a single servlet could handle the requests for anything but "/file", which gets handled by a different servlet registered for those requests. The servlet handling "/file" can be a subclass of the normal forwarding servlet, overriding a function that contains the "else" clause in the base class, and the behavior for "/file" in the subclass servlet. So in ProxyForwardServlet, there's a call out of {{forwardRequest}} to some function {{buildForwardPath}} (or whatever), that in ProxyForwardServlet contains the equivalent to the "else":
{noformat}
+ if (request.getPathInfo() != null) {
+ path += request.getPathInfo();
+ }
+ if (request.getQueryString() != null) {
+ path += "?" + request.getQueryString();
+ }
{noformat}
but in the subclass registered for "/file" requests, {{buildForwardPath}} contains the equivalent to:
{noformat}
+ // use streamFile for file access
+ path = "/streamFile";
+ path += "?filename=" + request.getPathInfo();
+ UnixUserGroupInformation ugi = (UnixUserGroupInformation)request.getAttribute("authorized.ugi");
+ if (ugi != null) {
+ path += "&ugi=" + ugi.toString();
+ }
{noformat}
So then the behavior is bound to the servlets registered to these paths, rather than making the behavior of this servlet vary depending on the paths to which it is bound.
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-5363:
----------------------------------
Status: Open (was: Patch Available)
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689234#action_12689234 ]
zhiyong zhang commented on HADOOP-5363:
---------------------------------------
That's a good suggestion.
Further, to make it more clear and readable in structure, I would think to make ProxyForwardServlet only handle forwarding cases. Then make a new servlet named ProxyFileServlet.java, which is parallel to ProxyListPathsServlet, ProxyFileDataServlet, and ProxyStreamFile. ProxyFileServlet will handle /file requests and internally it will make a single redirect request to ProxyStreamFile. By doing that, we move the support to "/file" requests to each proxy version (18, 20, 21, etc) instead of the forwarding part. Then the forwarding functionality will be independent of each proxy versions and can be deployed in a standalone fashion.
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5363:
----------------------------------
Attachment: HADOOP-5363.patch
Forgot to remove the commented out lines. updated that with the new attachment.
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "zhiyong zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhiyong zhang updated HADOOP-5363:
----------------------------------
Attachment: HADOOP-5363.patch
* Since pickOneAddress was added in support of hdfsproxy (HADOOP-4575), if it's no longer needed then it should be removed from HftpFileSystem
*** Yes.
* Instead of pulling the configuration out of the context in doGet, it can be saved in HttpServlet::init in ProxyForwardServlet.
* yes should be more efficient
* Unless {{doForward}} has special meaning, it should probably not be confused with {{doGet}}, {{doHead}}, etc. that are part of HttpServlet. Either putting the body of that method in {{doGet}} or renaming it would make it more readable. {{doGet}} should have an @Override annotation
* yes. changed name now.
* Please include javadoc for public methods, where reasonable.
* yes.
* I'm confused by {{getServletPath}} on the servlets; isn't this usually on the HttpServletRequest, set by the server to the matching part of the URI? Unless it has special meaning: rather than overloading {{getServletPath}} to return a fixed String, then changing behavior of {{doForward}} based on whether it matches "/path", it might make more sense to put the logic in the "else" block in a method, then override it in the ProxyFileForward servlet.
* very good suggestion, now changed the structure, let all forwarding point to one forwardservlet. the code looks much thinner.
* ProxyUtil::main should print usage when it has too few arguments rather than throwing an exception for {{-get}}.
* yes.
Thanks.
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch, HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5363) Proxying for multiple HDFS clusters
of different versions
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-5363:
----------------------------------
Status: Patch Available (was: Open)
> Proxying for multiple HDFS clusters of different versions
> ---------------------------------------------------------
>
> Key: HADOOP-5363
> URL: https://issues.apache.org/jira/browse/HADOOP-5363
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/hdfsproxy
> Reporter: Kan Zhang
> Assignee: zhiyong zhang
> Attachments: HADOOP-5363.patch
>
>
> A single hdfsproxy server should be able to proxy for multiple HDFS clusters, whose Hadoop versions may be different from each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.