You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Lian Jiang <ji...@gmail.com> on 2018/07/09 22:16:42 UTC
how to handle hdfs UI redirect in knox
Hi,
I am following
https://community.hortonworks.com/articles/81713/configure-knox-to-access-hdfs-ui.html
to expose hdfs ui via knox.
I have tested that all below curl commands worked:
on namenode:
curl http://localhost:50070/
curl http://localhost:50070/dfshealth.html
on a random machine:
curl -vvv -k -u guest:"{PASSWORD}" https://{DOMAIN}/gateway/ui/hdfs
However, in web browser, https://{DOMAIN}/gateway/ui/hdfs is redirected to
https://{DOMAIN}/gateway/ui/dfshealth.html which is not available. I
observed that
/var/lib/knox/data-2.6.4.0-91/services/hdfsui/2.7.0/rewrite.xml already has:
<rule dir="IN" name="HDFSUI/hdfs/inbound/namenode/dfs"
pattern="*://*:*/**/hdfs/dfshealth.html">
<rewrite template="{$serviceUrl[HDFSUI]}/dfshealth.html"/>
</rule>
what else do I need to do make hdfs ui work via knox? Appreciate any clue.
Re: how to handle hdfs UI redirect in knox
Posted by Sandeep Moré <mo...@gmail.com>.
What error are you getting ?
Also what is the Hive version ?
On Tue, Jul 10, 2018 at 3:36 PM Lian Jiang <ji...@gmail.com> wrote:
> Thanks for the details.
>
> Way1 did not work (as usual, redirect to dfshealth.html which is not
> found) and way 2 throw 500 server error. I have trouble to find related
> log. Any idea regarding debugging?
>
> On Tue, Jul 10, 2018 at 11:08 AM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
>> Sorry, I wasn't clear.
>> You can do that (download an replace service and rewrite xmls) and keep
>> your existing service definition intact i.e.
>> <service>
>> <role>HDFSUI</role>
>> <url>http://{DOMAIN}:50070</url>
>> </service>
>>
>> or
>>
>>
>> 1. You can create a new new folder
>> /var/lib/knox/data-2.6.4.0-91/services/hdfsui/3.0.0/
>> 2. Copy the new service.xml and rewrite.xml files there (into 3.0.0)
>> 3. Now update your topology with the service definition as follows
>>
>> <service>
>> <role>HDFSUI</role>
>> <version>3.0.0</version>
>> <url>http://{DOMAIN}:50070</url>
>> </service>
>>
>> Best,
>> Sandeep
>>
>> On Tue, Jul 10, 2018 at 1:46 PM Lian Jiang <ji...@gmail.com> wrote:
>>
>>> Thanks Sandeep.
>>>
>>> I am trying to understand what you said. Should I do below:
>>>
>>> 1. download service.xml and rewrite.xml from link [1].
>>> 2. replace the ones under
>>> /var/lib/knox/data-2.6.4.0-91/services/hdfsui/2.7.0/
>>> 3. in the topology file,
>>> change:
>>> <service>
>>> <role>HDFSUI</role>
>>> <url>http://{DOMAIN}:50070</url>
>>> </service>
>>> to:
>>> <service>
>>> <role>HDFSUI</role>
>>> <url>http://{DOMAIN}:50070/*2.7.0/*</url>
>>> </service>
>>>
>>> Thanks for clarification.
>>>
>>> On Mon, Jul 9, 2018 at 3:35 PM, Sandeep Moré <mo...@gmail.com>
>>> wrote:
>>>
>>>> There were a bunch of fixes that went into Knox 1.1.0 (currently in the
>>>> process of releasing) that fixes a bunch of issues with HDFSUI.
>>>> We also added new service defs [1] that has those fixes.
>>>>
>>>> If you are using older version of knox, try adding these to your hdfsui
>>>> service folder (services/hdfsui/). Because of a bug [2] Knox does not
>>>> always pick up the latest service defs. so you will have to specify HDFSUI
>>>> version in your topology (3.0.0 in this case).
>>>>
>>>> Let me know if you run into issues.
>>>>
>>>> Best,
>>>> Sandeep
>>>>
>>>>
>>>> [1]
>>>> https://github.com/apache/knox/tree/master/gateway-service-definitions/src/main/resources/services/hdfsui/3.0.0
>>>> [2] https://issues.apache.org/jira/browse/KNOX-1349
>>>>
>>>>
>>>> On Mon, Jul 9, 2018 at 6:16 PM Lian Jiang <ji...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am following
>>>>> https://community.hortonworks.com/articles/81713/configure-knox-to-access-hdfs-ui.html
>>>>> to expose hdfs ui via knox.
>>>>>
>>>>> I have tested that all below curl commands worked:
>>>>>
>>>>> on namenode:
>>>>> curl http://localhost:50070/
>>>>> curl http://localhost:50070/dfshealth.html
>>>>>
>>>>> on a random machine:
>>>>> curl -vvv -k -u guest:"{PASSWORD}" https://{DOMAIN}/gateway/ui/hdfs
>>>>>
>>>>> However, in web browser, https://{DOMAIN}/gateway/ui/hdfs is
>>>>> redirected to https://{DOMAIN}/gateway/ui/dfshealth.html which is not
>>>>> available. I observed that
>>>>> /var/lib/knox/data-2.6.4.0-91/services/hdfsui/2.7.0/rewrite.xml already has:
>>>>>
>>>>> <rule dir="IN" name="HDFSUI/hdfs/inbound/namenode/dfs"
>>>>> pattern="*://*:*/**/hdfs/dfshealth.html">
>>>>> <rewrite template="{$serviceUrl[HDFSUI]}/dfshealth.html"/>
>>>>> </rule>
>>>>>
>>>>> what else do I need to do make hdfs ui work via knox? Appreciate any
>>>>> clue.
>>>>>
>>>>
>>>
>
Re: how to handle hdfs UI redirect in knox
Posted by Lian Jiang <ji...@gmail.com>.
Thanks for the details.
Way1 did not work (as usual, redirect to dfshealth.html which is not found)
and way 2 throw 500 server error. I have trouble to find related log. Any
idea regarding debugging?
On Tue, Jul 10, 2018 at 11:08 AM, Sandeep Moré <mo...@gmail.com>
wrote:
> Sorry, I wasn't clear.
> You can do that (download an replace service and rewrite xmls) and keep
> your existing service definition intact i.e.
> <service>
> <role>HDFSUI</role>
> <url>http://{DOMAIN}:50070</url>
> </service>
>
> or
>
>
> 1. You can create a new new folder /var/lib/knox/data-2.6.4.0-
> 91/services/hdfsui/3.0.0/
> 2. Copy the new service.xml and rewrite.xml files there (into 3.0.0)
> 3. Now update your topology with the service definition as follows
>
> <service>
> <role>HDFSUI</role>
> <version>3.0.0</version>
> <url>http://{DOMAIN}:50070</url>
> </service>
>
> Best,
> Sandeep
>
> On Tue, Jul 10, 2018 at 1:46 PM Lian Jiang <ji...@gmail.com> wrote:
>
>> Thanks Sandeep.
>>
>> I am trying to understand what you said. Should I do below:
>>
>> 1. download service.xml and rewrite.xml from link [1].
>> 2. replace the ones under /var/lib/knox/data-2.6.4.0-91/
>> services/hdfsui/2.7.0/
>> 3. in the topology file,
>> change:
>> <service>
>> <role>HDFSUI</role>
>> <url>http://{DOMAIN}:50070</url>
>> </service>
>> to:
>> <service>
>> <role>HDFSUI</role>
>> <url>http://{DOMAIN}:50070/*2.7.0/*</url>
>> </service>
>>
>> Thanks for clarification.
>>
>> On Mon, Jul 9, 2018 at 3:35 PM, Sandeep Moré <mo...@gmail.com>
>> wrote:
>>
>>> There were a bunch of fixes that went into Knox 1.1.0 (currently in the
>>> process of releasing) that fixes a bunch of issues with HDFSUI.
>>> We also added new service defs [1] that has those fixes.
>>>
>>> If you are using older version of knox, try adding these to your hdfsui
>>> service folder (services/hdfsui/). Because of a bug [2] Knox does not
>>> always pick up the latest service defs. so you will have to specify HDFSUI
>>> version in your topology (3.0.0 in this case).
>>>
>>> Let me know if you run into issues.
>>>
>>> Best,
>>> Sandeep
>>>
>>>
>>> [1] https://github.com/apache/knox/tree/master/gateway-
>>> service-definitions/src/main/resources/services/hdfsui/3.0.0
>>> [2] https://issues.apache.org/jira/browse/KNOX-1349
>>>
>>>
>>> On Mon, Jul 9, 2018 at 6:16 PM Lian Jiang <ji...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am following https://community.hortonworks.
>>>> com/articles/81713/configure-knox-to-access-hdfs-ui.html to expose
>>>> hdfs ui via knox.
>>>>
>>>> I have tested that all below curl commands worked:
>>>>
>>>> on namenode:
>>>> curl http://localhost:50070/
>>>> curl http://localhost:50070/dfshealth.html
>>>>
>>>> on a random machine:
>>>> curl -vvv -k -u guest:"{PASSWORD}" https://{DOMAIN}/gateway/ui/hdfs
>>>>
>>>> However, in web browser, https://{DOMAIN}/gateway/ui/hdfs is
>>>> redirected to https://{DOMAIN}/gateway/ui/dfshealth.html which is not
>>>> available. I observed that /var/lib/knox/data-2.6.4.0-91/
>>>> services/hdfsui/2.7.0/rewrite.xml already has:
>>>>
>>>> <rule dir="IN" name="HDFSUI/hdfs/inbound/namenode/dfs"
>>>> pattern="*://*:*/**/hdfs/dfshealth.html">
>>>> <rewrite template="{$serviceUrl[HDFSUI]}/dfshealth.html"/>
>>>> </rule>
>>>>
>>>> what else do I need to do make hdfs ui work via knox? Appreciate any
>>>> clue.
>>>>
>>>
>>
Re: how to handle hdfs UI redirect in knox
Posted by Sandeep Moré <mo...@gmail.com>.
Sorry, I wasn't clear.
You can do that (download an replace service and rewrite xmls) and keep
your existing service definition intact i.e.
<service>
<role>HDFSUI</role>
<url>http://{DOMAIN}:50070</url>
</service>
or
1. You can create a new new folder
/var/lib/knox/data-2.6.4.0-91/services/hdfsui/3.0.0/
2. Copy the new service.xml and rewrite.xml files there (into 3.0.0)
3. Now update your topology with the service definition as follows
<service>
<role>HDFSUI</role>
<version>3.0.0</version>
<url>http://{DOMAIN}:50070</url>
</service>
Best,
Sandeep
On Tue, Jul 10, 2018 at 1:46 PM Lian Jiang <ji...@gmail.com> wrote:
> Thanks Sandeep.
>
> I am trying to understand what you said. Should I do below:
>
> 1. download service.xml and rewrite.xml from link [1].
> 2. replace the ones under
> /var/lib/knox/data-2.6.4.0-91/services/hdfsui/2.7.0/
> 3. in the topology file,
> change:
> <service>
> <role>HDFSUI</role>
> <url>http://{DOMAIN}:50070</url>
> </service>
> to:
> <service>
> <role>HDFSUI</role>
> <url>http://{DOMAIN}:50070/*2.7.0/*</url>
> </service>
>
> Thanks for clarification.
>
> On Mon, Jul 9, 2018 at 3:35 PM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
>> There were a bunch of fixes that went into Knox 1.1.0 (currently in the
>> process of releasing) that fixes a bunch of issues with HDFSUI.
>> We also added new service defs [1] that has those fixes.
>>
>> If you are using older version of knox, try adding these to your hdfsui
>> service folder (services/hdfsui/). Because of a bug [2] Knox does not
>> always pick up the latest service defs. so you will have to specify HDFSUI
>> version in your topology (3.0.0 in this case).
>>
>> Let me know if you run into issues.
>>
>> Best,
>> Sandeep
>>
>>
>> [1]
>> https://github.com/apache/knox/tree/master/gateway-service-definitions/src/main/resources/services/hdfsui/3.0.0
>> [2] https://issues.apache.org/jira/browse/KNOX-1349
>>
>>
>> On Mon, Jul 9, 2018 at 6:16 PM Lian Jiang <ji...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am following
>>> https://community.hortonworks.com/articles/81713/configure-knox-to-access-hdfs-ui.html
>>> to expose hdfs ui via knox.
>>>
>>> I have tested that all below curl commands worked:
>>>
>>> on namenode:
>>> curl http://localhost:50070/
>>> curl http://localhost:50070/dfshealth.html
>>>
>>> on a random machine:
>>> curl -vvv -k -u guest:"{PASSWORD}" https://{DOMAIN}/gateway/ui/hdfs
>>>
>>> However, in web browser, https://{DOMAIN}/gateway/ui/hdfs is redirected
>>> to https://{DOMAIN}/gateway/ui/dfshealth.html which is not available. I
>>> observed that
>>> /var/lib/knox/data-2.6.4.0-91/services/hdfsui/2.7.0/rewrite.xml already has:
>>>
>>> <rule dir="IN" name="HDFSUI/hdfs/inbound/namenode/dfs"
>>> pattern="*://*:*/**/hdfs/dfshealth.html">
>>> <rewrite template="{$serviceUrl[HDFSUI]}/dfshealth.html"/>
>>> </rule>
>>>
>>> what else do I need to do make hdfs ui work via knox? Appreciate any
>>> clue.
>>>
>>
>
Re: how to handle hdfs UI redirect in knox
Posted by Lian Jiang <ji...@gmail.com>.
Thanks Sandeep.
I am trying to understand what you said. Should I do below:
1. download service.xml and rewrite.xml from link [1].
2. replace the ones under /var/lib/knox/data-2.6.4.0-91/
services/hdfsui/2.7.0/
3. in the topology file,
change:
<service>
<role>HDFSUI</role>
<url>http://{DOMAIN}:50070</url>
</service>
to:
<service>
<role>HDFSUI</role>
<url>http://{DOMAIN}:50070/*2.7.0/*</url>
</service>
Thanks for clarification.
On Mon, Jul 9, 2018 at 3:35 PM, Sandeep Moré <mo...@gmail.com> wrote:
> There were a bunch of fixes that went into Knox 1.1.0 (currently in the
> process of releasing) that fixes a bunch of issues with HDFSUI.
> We also added new service defs [1] that has those fixes.
>
> If you are using older version of knox, try adding these to your hdfsui
> service folder (services/hdfsui/). Because of a bug [2] Knox does not
> always pick up the latest service defs. so you will have to specify HDFSUI
> version in your topology (3.0.0 in this case).
>
> Let me know if you run into issues.
>
> Best,
> Sandeep
>
>
> [1] https://github.com/apache/knox/tree/master/gateway-
> service-definitions/src/main/resources/services/hdfsui/3.0.0
> [2] https://issues.apache.org/jira/browse/KNOX-1349
>
>
> On Mon, Jul 9, 2018 at 6:16 PM Lian Jiang <ji...@gmail.com> wrote:
>
>> Hi,
>>
>> I am following https://community.hortonworks.
>> com/articles/81713/configure-knox-to-access-hdfs-ui.html to expose hdfs
>> ui via knox.
>>
>> I have tested that all below curl commands worked:
>>
>> on namenode:
>> curl http://localhost:50070/
>> curl http://localhost:50070/dfshealth.html
>>
>> on a random machine:
>> curl -vvv -k -u guest:"{PASSWORD}" https://{DOMAIN}/gateway/ui/hdfs
>>
>> However, in web browser, https://{DOMAIN}/gateway/ui/hdfs is redirected
>> to https://{DOMAIN}/gateway/ui/dfshealth.html which is not available. I
>> observed that /var/lib/knox/data-2.6.4.0-91/
>> services/hdfsui/2.7.0/rewrite.xml already has:
>>
>> <rule dir="IN" name="HDFSUI/hdfs/inbound/namenode/dfs"
>> pattern="*://*:*/**/hdfs/dfshealth.html">
>> <rewrite template="{$serviceUrl[HDFSUI]}/dfshealth.html"/>
>> </rule>
>>
>> what else do I need to do make hdfs ui work via knox? Appreciate any clue.
>>
>
Re: how to handle hdfs UI redirect in knox
Posted by Sandeep Moré <mo...@gmail.com>.
There were a bunch of fixes that went into Knox 1.1.0 (currently in the
process of releasing) that fixes a bunch of issues with HDFSUI.
We also added new service defs [1] that has those fixes.
If you are using older version of knox, try adding these to your hdfsui
service folder (services/hdfsui/). Because of a bug [2] Knox does not
always pick up the latest service defs. so you will have to specify HDFSUI
version in your topology (3.0.0 in this case).
Let me know if you run into issues.
Best,
Sandeep
[1]
https://github.com/apache/knox/tree/master/gateway-service-definitions/src/main/resources/services/hdfsui/3.0.0
[2] https://issues.apache.org/jira/browse/KNOX-1349
On Mon, Jul 9, 2018 at 6:16 PM Lian Jiang <ji...@gmail.com> wrote:
> Hi,
>
> I am following
> https://community.hortonworks.com/articles/81713/configure-knox-to-access-hdfs-ui.html
> to expose hdfs ui via knox.
>
> I have tested that all below curl commands worked:
>
> on namenode:
> curl http://localhost:50070/
> curl http://localhost:50070/dfshealth.html
>
> on a random machine:
> curl -vvv -k -u guest:"{PASSWORD}" https://{DOMAIN}/gateway/ui/hdfs
>
> However, in web browser, https://{DOMAIN}/gateway/ui/hdfs is redirected
> to https://{DOMAIN}/gateway/ui/dfshealth.html which is not available. I
> observed that
> /var/lib/knox/data-2.6.4.0-91/services/hdfsui/2.7.0/rewrite.xml already has:
>
> <rule dir="IN" name="HDFSUI/hdfs/inbound/namenode/dfs"
> pattern="*://*:*/**/hdfs/dfshealth.html">
> <rewrite template="{$serviceUrl[HDFSUI]}/dfshealth.html"/>
> </rule>
>
> what else do I need to do make hdfs ui work via knox? Appreciate any clue.
>