You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Trevor Grant <tr...@gmail.com> on 2016/08/26 17:52:27 UTC
Flink WebUI on YARN behind firewall
I decided it made the most sense to open up a new thread.
I am running Flink on a cluster behind a firewall. Things seem to be
working fine, but when I access the YARN web-ui and click on the flink
application-ui, i get the jobmanager ui, but it is broken.
It is a broken link to a flink image and
[image: Apache Flink Dashboard]Apache Flink Dashboard
- Overview
- Running Jobs
- Completed Jobs
- Task Managers
- Job Manager
- Submit new Job
but none of the links work, no images no pretty formatting. does anyone
have a quick idea why this would be? I found
http://mail-archives.apache.org/mod_mbox/flink-user/201511.mbox/%3CCAGr9p8B_aKhbjLqVQQUZeO_eSz6P=eWKP+Kg1sQ65SB0npsX2Q@mail.gmail.com%3E
but that seems to be more related to running jobs than accessing the UI
any help would be appreciated
tg
Trevor Grant
Data Scientist
https://github.com/rawkintrevo
http://stackexchange.com/users/3002022/rawkintrevo
http://trevorgrant.org
*"Fortunate is he, who is able to know the causes of things." -Virgil*
Re: Flink WebUI on YARN behind firewall
Posted by Trevor Grant <tr...@gmail.com>.
Ahh, sorry I misunderstood.
Your comment provided insight for me though.
To anyone else who is having issues, maybe the following will help them. I
was trying to deploy Flink on an IBM BigInsights Cloud cluster (disclaimer,
I work for IBM, not trying to promote a company, but they do give me free
cluster time so I tend to play there).
At anyrate- flink_example.py is a script that will do it nice and clean for
anyone on BigInsights Cloud, and may be useful for other similar cloud YARN
deployments so I'll just leave it here...
https://github.com/rawkintrevo/bluemix-extra-services
The solution in short-
1) install flink-
2) run yarn-session.sh
3) parse the stdout from above command (see python script, easy to follow
or email me if not)
4) open the ApplicationUI via the Yarn UI, go to the #/submit tab
5) in the submit tab, IF you open with a web browser (bc it relies on
javascript) it may give you a hostname:port <- save that port
6) combine the port you just got , with an IP address you got when parsing
in step 3
7) open ssh tunneling to that IP:port
Ugly, but it does work.
best
tg
Trevor Grant
Data Scientist
https://github.com/rawkintrevo
http://stackexchange.com/users/3002022/rawkintrevo
http://trevorgrant.org
*"Fortunate is he, who is able to know the causes of things." -Virgil*
On Fri, Aug 26, 2016 at 5:54 PM, Vijay Srinivasaraghavan <
vijikarthi@yahoo.com> wrote:
> Hi Trevor,
>
> Sorry, I think I did not explain it clearly. I am not working on the issue
> that you have mentioned but I have noticed similar problem for another JIRA
> that I am currenty working (Flink Web UI basic authentication). The
> redirection from Yarn UI to Flink does not propogate the authentication
> headers. I was planning to investigate from Yarn UI as to what exactly is
> happening. Please go-ahead and create a JIRA anyway and someone will pick
> it up.
>
> Regards
> Vijay
>
>
> On Friday, August 26, 2016 1:34 PM, Trevor Grant <tr...@gmail.com>
> wrote:
>
>
> That's awesome Vijay, thanks for the work- will definitely help (esp as
> Flink gets picked up for more cloud cluster deployments)!
>
> Do you have the JIRA so I can watch it?
>
> Trevor Grant
> Data Scientist
> https://github.com/rawkintrevo
> http://stackexchange.com/users/3002022/rawkintrevo
> http://trevorgrant.org
>
> *"Fortunate is he, who is able to know the causes of things." -Virgil*
>
>
> On Fri, Aug 26, 2016 at 1:14 PM, Vijay Srinivasaraghavan <
> vijikarthi@yahoo.com> wrote:
>
> Hi Trevor,
>
> I am seeing similar issue for a JIRA that I am working now. I am yet to
> trace the Yarn Web UI code to find out how the "tracking URL" is being
> handled. To ublock, you could use the tracking URL (Flink UI URL) directly
> to access Flink Web UI to by-pass Yarn UI redirection. You can find the
> tracking URI in the Job Manager log file from Yarn container.
>
> Regards
> Vijay
>
>
> On Friday, August 26, 2016 10:52 AM, Trevor Grant <
> trevor.d.grant@gmail.com> wrote:
>
>
> I decided it made the most sense to open up a new thread.
>
> I am running Flink on a cluster behind a firewall. Things seem to be
> working fine, but when I access the YARN web-ui and click on the flink
> application-ui, i get the jobmanager ui, but it is broken.
>
> It is a broken link to a flink image and
> [image: Apache Flink Dashboard]Apache Flink Dashboard
>
> - Overview
> - Running Jobs
> - Completed Jobs
> - Task Managers
> - Job Manager
> - Submit new Job
>
> but none of the links work, no images no pretty formatting. does anyone
> have a quick idea why this would be? I found http://mail-archives.
> apache.org/mod_mbox/flink- user/201511.mbox/%3CCAGr9p8B_
> aKhbjLqVQQUZeO_eSz6P=eWKP+ Kg1sQ65SB0npsX2Q@mail.gmail. com%3E
> <http://mail-archives.apache.org/mod_mbox/flink-user/201511.mbox/%3CCAGr9p8B_aKhbjLqVQQUZeO_eSz6P=eWKP+Kg1sQ65SB0npsX2Q@mail.gmail.com%3E>
>
> but that seems to be more related to running jobs than accessing the UI
>
> any help would be appreciated
>
> tg
>
>
> Trevor Grant
> Data Scientist
> https://github.com/rawkintrevo
> http://stackexchange.com/ users/3002022/rawkintrevo
> <http://stackexchange.com/users/3002022/rawkintrevo>
> http://trevorgrant.org
>
> *"Fortunate is he, who is able to know the causes of things." -Virgil*
>
>
>
>
>
>
>
Re: Flink WebUI on YARN behind firewall
Posted by Vijay Srinivasaraghavan <vi...@yahoo.com>.
Hi Trevor,
I am seeing similar issue for a JIRA that I am working now. I am yet to trace the Yarn Web UI code to find out how the "tracking URL" is being handled. To ublock, you could use the tracking URL (Flink UI URL) directly to access Flink Web UI to by-pass Yarn UI redirection. You can find the tracking URI in the Job Manager log file from Yarn container.
RegardsVijay
On Friday, August 26, 2016 10:52 AM, Trevor Grant <tr...@gmail.com> wrote:
I decided it made the most sense to open up a new thread.
I am running Flink on a cluster behind a firewall. Things seem to be working fine, but when I access the YARN web-ui and click on the flink application-ui, i get the jobmanager ui, but it is broken.
It is a broken link to a flink image and Apache Flink Dashboard
- Overview
- Running Jobs
- Completed Jobs
- Task Managers
- Job Manager
- Submit new Job
but none of the links work, no images no pretty formatting. does anyone have a quick idea why this would be? I found http://mail-archives.apache.org/mod_mbox/flink-user/201511.mbox/%3CCAGr9p8B_aKhbjLqVQQUZeO_eSz6P=eWKP+Kg1sQ65SB0npsX2Q@mail.gmail.com%3E
but that seems to be more related to running jobs than accessing the UI
any help would be appreciated
tg
Trevor GrantData Scientisthttps://github.com/rawkintrevo
http://stackexchange.com/users/3002022/rawkintrevo
http://trevorgrant.org
"Fortunate is he, who is able to know the causes of things." -Virgil
Re: Flink WebUI on YARN behind firewall
Posted by Shannon Carey <sc...@expedia.com>.
I've noticed this too. The UI incorrectly tries to load its resources (CSS, JS) from root. Try adding a trailing slash to the URL.
-Shannon
From: Trevor Grant <tr...@gmail.com>>
Date: Friday, August 26, 2016 at 12:52 PM
To: <us...@flink.apache.org>>
Subject: Flink WebUI on YARN behind firewall
I decided it made the most sense to open up a new thread.
I am running Flink on a cluster behind a firewall. Things seem to be working fine, but when I access the YARN web-ui and click on the flink application-ui, i get the jobmanager ui, but it is broken.
It is a broken link to a flink image and
[Apache Flink Dashboard]Apache Flink Dashboard
* Overview
* Running Jobs
* Completed Jobs
* Task Managers
* Job Manager
* Submit new Job
but none of the links work, no images no pretty formatting. does anyone have a quick idea why this would be? I found http://mail-archives.apache.org/mod_mbox/flink-user/201511.mbox/%3CCAGr9p8B_aKhbjLqVQQUZeO_eSz6P=eWKP+Kg1sQ65SB0npsX2Q@mail.gmail.com%3E
but that seems to be more related to running jobs than accessing the UI
any help would be appreciated
tg
Trevor Grant
Data Scientist
https://github.com/rawkintrevo
http://stackexchange.com/users/3002022/rawkintrevo
http://trevorgrant.org
"Fortunate is he, who is able to know the causes of things." -Virgil