[jira] [Commented] (KNOX-1334) Knox Service Defs for UIs in AWS EMR Deployments

["Mert Hocanin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KNOX-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730501#comment-16730501 ] 

Mert Hocanin commented on KNOX-1334:
------------------------------------

Hello all,

I apologize for the delay for my response. I will be much more responsive going forward. To answer questions from Larry:
{quote}There doesn't seem to be enough details in the second statement to justify the conclusion in the third. Can you indicate why there shouldn't be meaningful differences?
{quote}
=> I apologize for not being able to convey my thoughts clearly. I was trying say that from my experience, usually, minor version changes are backwards compatible and I assumed that this was true. However, it looks like many people are now using Knox for Hadoop 3.x as per documentation so my assumption is probably no longer correct. Can you please advise? I could probably start testing on both Hadoop 2.7.x as well as Hadoop 3.x, however, getting Hadoop 3.x up and running will take additional time as I would need to set it up manually.
{quote}You have a catch-all rewrite rule for matching things that you do not want to rewrite - we need to make sure that this doesn't match patterns from other services as well. It looks to me like it may. Need to test this colocated in a topology with other services (UIs and APIs) to be sure:
{quote}
=> I do not have access to many of the Knox supported applications so I wont be able to test. I personally do not remember adding this line (the commit is a combination of multiple contributions). I will remove the catch all because I agree with you, its probably best to add this and provide rules explicitly for anything that we do not have a rule for.
{quote}Based on your change of the dispatch package name below it seems that you are not using 1.0.0 or master (which will soon be 1.1.0) but are on at the latest 0.14.0 release. so, can you provide what version you are currently using. I wouldn't imagine that we would land this in anything older than 1.1.0 and TBH, I think we need to push it out to 1.2.0 which we can make a quick release cycle for a handful of like things.
{quote}
=> I took a combination of changes from others because that helped me get from a starting place. I will revert this particular change back to use org.apache.knox.gateway.dispatch.DefaultDispatch as that looks like a changed that I pulled. I was working on top of the master branch after version 1.0.0 was released. Since two minor versions were released since then, I will update the patch after a customer of ours and us conduct some more testing.
{quote}You mentioned that Knox supports Hadoop 2.7 - I assume this is based on the version of the service definitions that you are changing but you may also be referring to our pom dependency on hadoop-common (depending on what version you are using).
{quote}
=> Correct, I made this assumption from the service definition version for Job History UI and Yarn UI. But as you mentioned, looking at the dependency on Hadoop within the pom file, looks like its building against Hadoop 3.1.x.
{quote}Depending on the extent of your changes to the UIs that you are proxying here, a new version may not be the right way to go since newer version numbers become the default for the service and you have to specify older version in the topology for them to get picked up instead.
{quote}
=> I look at your guidance on how we can incorporate our changes within the project. Any advice/guidance you can provide, we would greatly appreciate! 

Thanks

> Knox Service Defs for UIs in AWS EMR Deployments
> ------------------------------------------------
>
>                 Key: KNOX-1334
>                 URL: https://issues.apache.org/jira/browse/KNOX-1334
>             Project: Apache Knox
>          Issue Type: Bug
>            Reporter: PRAVEEN K RAVIKUMAR
>            Priority: Major
>             Fix For: 1.3.0
>
>         Attachments: KNOX-1343.patch
>
>
> Hi
> I have requirement to send all the request coming to proxy to be forwarded to EMR master node.
> Example:
> https://<any dns-name>:<PORT>/ --> Proxy --> EMR master.
> The DNS name could be anything, because there are many slave nodes. I would like the proxy to "redirect" dynamically "https://<any dns-name>:<PORT>/" to EMR master host. Could you please let me know if a reverse proxy/ Apache Knox can do that. I am trying to mimic option2 "SSH Tunnel to the Master Node Using Dynamic Port Forwarding"
> https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-ssh-tunnel.html
> Sample URLS:
> YARN ResourceManager https://master-public-dns-name:8088/ YARN NodeManager https://slave-public-dns-name:8042/ Spark HistoryServer https://master-public-dns-name:18080/
> [https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-ssh-tunnel.html]
>  
> If so could you please share me the steps to configure Knox on EMR cluster- would be very helpful?
> Praveen.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)