Re: Job history ui rewrite rule in yarnui

[Guang Yang <ko...@uber.com>]

Hi guys,

About the job urls, I found three more non-functional links. They are links
for jobmanager.(err/log/out) in the page of node container logs, like
http://hadoopworker001:8042/node/containerlogs/container_xxx/user_xxx.

Their original urls are like this <a href="
/node/containerlogs/container_xxx/user_xxx/jobmanager.err/?start=-4096
<http://hadoopworker1677-sjc1.prod.uber.internal:8042/node/containerlogs/container_e535_1521074600645_578887_01_000001/kedar/jobmanager.err/?start=-4096>
">jobmanager.err : Total file length is 0 bytes.</a>. As there is no node
host and port info in this uri, so when doing the rewrite, it can't be
written to the right url. Any suggestion about how to fix this issue?

Another interesting finding is, actually we have the node host and port
info in the web page url which contains these links, which is like "
https://knox/gateway/sandbox/yarn/nodemanager/node/containerlogs/container_xxx/user_xxx?scheme=http&host=hadoopworker001-sjc1.prod.uber.internal&port=8042".
So that means if there is a way we can get the parameter from the root url
when rewrite some other urls, we can probably fix that problem.

Do you have some suggestion? Appreciated.

Best,
Guang

On Mon, Mar 5, 2018 at 5:07 PM, Guang Yang <ko...@uber.com> wrote:

> Any suggestion, guys?
>
> On Fri, Mar 2, 2018 at 1:28 PM, Guang Yang <ko...@uber.com> wrote:
>
>> Hi Sandeep,
>>
>> Thanks for looking into it.
>>
>> Yes, I tried that before, the value can be replaced correctly, but the
>> webpage is just stuck there, no redirect happens.
>>
>> The result in html is <meta http-equiv="refresh"
>> content="https://host:port/something">, which however, should be <meta
>> http-equiv="refresh" content="*1; url=*https://host:port/something">. It
>> seems the html meta refresh format requires *'1; url='*, where the 1
>> means stop for 1 seconds before refresh.
>>
>> Thanks,
>> Guang
>>
>> On Fri, Mar 2, 2018 at 6:49 AM, Sandeep Moré <mo...@gmail.com>
>> wrote:
>>
>>> Hello Guang,
>>>
>>> This does look like a bug, after some digging it appears it was as a
>>> result of KNOX-973.
>>>
>>> Have you tried using
>>>
>>> <rewrite template="{$frontend[url]}/jobstoryui/jobstory/jobhistory/lo
>>> gs/{**}"/>
>>>
>>> I am curious to see what you get.
>>>
>>>
>>> Best,
>>> Sandeep
>>>
>>>
>>>
>>>
>>> On Thu, Mar 1, 2018 at 4:30 PM, Guang Yang <ko...@uber.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm currently working on the Map Reduce Job History UI rewrite rules,
>>>> and found several potential bugs here.
>>>>
>>>> <rewrite template="1; url=/gateway/jobstoryui/jobsto
>>>> ry/jobhistory/logs/{**}"/>
>>>> For this rewrite template
>>>> <https://github.com/apache/knox/blob/master/gateway-service-definitions/src/main/resources/services/yarnui/2.7.0/rewrite.xml#L104>,
>>>> let's not say what the `jobstory` is here for now. I think the target
>>>> url should start with something like {$frontend[url]}, just like other OUT
>>>> rules, because the previous one doesn't specify the deployment/environment
>>>> after word `gateway`.
>>>>
>>>> But after I change it to <rewrite template="1; url={$frontend[url]}/
>>>> jobstoryui/jobstory/jobhistory/logs/{**}"/>, the variable
>>>> {$frontend[url]} is not replaced with the right value, it's just literal
>>>> `{$frontend[url]}` in the target url. And I found that only when the
>>>> variable following the double quotes, it can be replaced, otherwise it just
>>>> stays there as literal text.
>>>>
>>>> My question is, anyone knows how to fix this bug? Or how to get {$frontend[url]}
>>>> replaced with right value even it's not at the beginning of the template?
>>>>
>>>> Btw, I think the right template should be <rewrite template="1;
>>>> url={$frontend[url]}/jobhistory/joblogs/{**}"/>.
>>>>
>>>> Appreciate for any help.
>>>>
>>>> Thanks,
>>>> Guang
>>>>
>>>>
>>>>
>>>
>>
>

Re: Job history ui rewrite rule in yarnui

[Sandeep Moré <mo...@gmail.com>]

Hello Guang,

For the first url, you cannot rewrite it because it does not have the
host:port, so this is different than the host:port your service is using ?
sorry, I don't remember exactly this works.

For the second case, yes I have see that used, but do not remember how we
route it, I think it get's routed to a different service, you can check the
rewrite.xml for details.

Best,
Sandeep


On Thu, Apr 19, 2018 at 7:12 PM, Guang Yang <ko...@uber.com> wrote:

> Hi guys,
>
> About the job urls, I found three more non-functional links. They are
> links for jobmanager.(err/log/out) in the page of node container logs, like
> http://hadoopworker001:8042/node/containerlogs/container_xxx/user_xxx.
>
> Their original urls are like this <a href="/node/containerlogs/
> container_xxx/user_xxx/jobmanager.err/?start=-4096
> <http://hadoopworker1677-sjc1.prod.uber.internal:8042/node/containerlogs/container_e535_1521074600645_578887_01_000001/kedar/jobmanager.err/?start=-4096>
> ">jobmanager.err : Total file length is 0 bytes.</a>. As there is no node
> host and port info in this uri, so when doing the rewrite, it can't be
> written to the right url. Any suggestion about how to fix this issue?
>
> Another interesting finding is, actually we have the node host and port
> info in the web page url which contains these links, which is like "
> https://knox/gateway/sandbox/yarn/nodemanager/node/
> containerlogs/container_xxx/user_xxx?scheme=http&host=
> hadoopworker001-sjc1.prod.uber.internal&port=8042". So that means if
> there is a way we can get the parameter from the root url when rewrite some
> other urls, we can probably fix that problem.
>
> Do you have some suggestion? Appreciated.
>
> Best,
> Guang
>
> On Mon, Mar 5, 2018 at 5:07 PM, Guang Yang <ko...@uber.com> wrote:
>
>> Any suggestion, guys?
>>
>> On Fri, Mar 2, 2018 at 1:28 PM, Guang Yang <ko...@uber.com> wrote:
>>
>>> Hi Sandeep,
>>>
>>> Thanks for looking into it.
>>>
>>> Yes, I tried that before, the value can be replaced correctly, but the
>>> webpage is just stuck there, no redirect happens.
>>>
>>> The result in html is <meta http-equiv="refresh"
>>> content="https://host:port/something">, which however, should be <meta
>>> http-equiv="refresh" content="*1; url=*https://host:port/something">.
>>> It seems the html meta refresh format requires *'1; url='*, where the 1
>>> means stop for 1 seconds before refresh.
>>>
>>> Thanks,
>>> Guang
>>>
>>> On Fri, Mar 2, 2018 at 6:49 AM, Sandeep Moré <mo...@gmail.com>
>>> wrote:
>>>
>>>> Hello Guang,
>>>>
>>>> This does look like a bug, after some digging it appears it was as a
>>>> result of KNOX-973.
>>>>
>>>> Have you tried using
>>>>
>>>> <rewrite template="{$frontend[url]}/jobstoryui/jobstory/jobhistory/lo
>>>> gs/{**}"/>
>>>>
>>>> I am curious to see what you get.
>>>>
>>>>
>>>> Best,
>>>> Sandeep
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Mar 1, 2018 at 4:30 PM, Guang Yang <ko...@uber.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm currently working on the Map Reduce Job History UI rewrite rules,
>>>>> and found several potential bugs here.
>>>>>
>>>>> <rewrite template="1; url=/gateway/jobstoryui/jobsto
>>>>> ry/jobhistory/logs/{**}"/>
>>>>> For this rewrite template
>>>>> <https://github.com/apache/knox/blob/master/gateway-service-definitions/src/main/resources/services/yarnui/2.7.0/rewrite.xml#L104>,
>>>>> let's not say what the `jobstory` is here for now. I think the target
>>>>> url should start with something like {$frontend[url]}, just like other OUT
>>>>> rules, because the previous one doesn't specify the deployment/environment
>>>>> after word `gateway`.
>>>>>
>>>>> But after I change it to <rewrite template="1; url={$frontend[url]}/
>>>>> jobstoryui/jobstory/jobhistory/logs/{**}"/>, the variable
>>>>> {$frontend[url]} is not replaced with the right value, it's just literal
>>>>> `{$frontend[url]}` in the target url. And I found that only when the
>>>>> variable following the double quotes, it can be replaced, otherwise it just
>>>>> stays there as literal text.
>>>>>
>>>>> My question is, anyone knows how to fix this bug? Or how to get {$frontend[url]}
>>>>> replaced with right value even it's not at the beginning of the template?
>>>>>
>>>>> Btw, I think the right template should be <rewrite template="1;
>>>>> url={$frontend[url]}/jobhistory/joblogs/{**}"/>.
>>>>>
>>>>> Appreciate for any help.
>>>>>
>>>>> Thanks,
>>>>> Guang
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>