You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by "Larry McCay (Jira)" <ji...@apache.org> on 2019/11/27 02:06:00 UTC
[jira] [Updated] (KNOX-1414) DefaultHaDispatch is used by WEBHDFS
instead of WebHdfsHaDispatch
[ https://issues.apache.org/jira/browse/KNOX-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Larry McCay updated KNOX-1414:
------------------------------
Fix Version/s: (was: 1.4.0)
> DefaultHaDispatch is used by WEBHDFS instead of WebHdfsHaDispatch
> -----------------------------------------------------------------
>
> Key: KNOX-1414
> URL: https://issues.apache.org/jira/browse/KNOX-1414
> Project: Apache Knox
> Issue Type: Bug
> Affects Versions: 0.12.0
> Reporter: Kristopher Kane
> Priority: Minor
>
> Here is the series of failover attempts with DefaultHaDispatch (vs the WebHDFS one) when 403 is hit:
>
>
> {noformat}
> Caused by: java.io.IOException: java.io.IOException: Service connectivity error.
> at org.apache.hadoop.gateway.ha.dispatch.DefaultHaDispatch.failoverRequest(DefaultHaDispatch.java:125)
> at org.apache.hadoop.gateway.ha.dispatch.DefaultHaDispatch.executeRequest(DefaultHaDispatch.java:94)
> at org.apache.hadoop.gateway.ha.dispatch.DefaultHaDispatch.failoverRequest(DefaultHaDispatch.java:119)
> at org.apache.hadoop.gateway.ha.dispatch.DefaultHaDispatch.executeRequest(DefaultHaDispatch.java:94)
> at org.apache.hadoop.gateway.ha.dispatch.DefaultHaDispatch.failoverRequest(DefaultHaDispatch.java:119)
> at org.apache.hadoop.gateway.ha.dispatch.DefaultHaDispatch.executeRequest(DefaultHaDispatch.java:94)
> at org.apache.hadoop.gateway.ha.dispatch.DefaultHaDispatch.failoverRequest(DefaultHaDispatch.java:119)
> at org.apache.hadoop.gateway.ha.dispatch.DefaultHaDispatch.executeRequest(DefaultHaDispatch.java:94)
> {noformat}
> But `data/services/webhdfs/2.4.0->service.xml` has:
>
> <dispatch classname="org.apache.hadoop.gateway.hdfs.dispatch.HdfsHttpClientDispatch" ha-classname="org.apache.hadoop.gateway.hdfs.dispatch.WebHdfsHaDispatch"/>
>
> Which should use WebHdfsHaDispatch.
>
> Both HA dispatch classes conduct HA failover/retry but WebHdfsHaDispatch is only slightly more context aware. Both could appear to work in 2/3rds of the cases (1/3rd being when the NN is in Safemode which is not handled specifically by the default dispatch) and you would not know WebHdfsHaDispatch was not being used unless you looked at gateway.log
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)