You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Fabian Wollert <fa...@zalando.de> on 2018/10/29 12:06:42 UTC
[DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Hi everyone,
in this email thread
<http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html>
and the tickets FLINK-10705
<https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
<https://issues.apache.org/jira/browse/FLINK-10706> the discussion came up
whether to change the underlying architecture of Flink's Web Dashboard from
Angular1 to something else. This email thread should be solely to discuss
the pro's and con's of this, and what could be the target architecture.
My choice would be React. Personally I agree with Till's comments in the
ticket, Angular 1 being basically outdated and is not having a large
following anymore. From my experience the choice between Angular 2-7 or
React is subjective, you can get things done with both. I personally only
have experience with React, so I personally would be faster to develop
with this one. I currently have not planned to learn Angular as well (being
a more backend focused developer in general) so if the decision would be to
go with Angular, i would be unfortunately out of this rework of the Flink
Dashboard most certainly.
Additionally i would like to get rid of bower, since its officially
deprecated <https://bower.io/blog/2017/how-to-migrate-away-from-bower/>. my
idea would be to just use a create-react-app package with npm and webpack
under the hood. no need for additional lib's here imho. But again: thats
mostly what i've been working with recently, so thats a subjective point. I
could imagine getting used to yarn in the future as well.
Cheers
Fabian
--
*Fabian WollertZalando SE*
E-Mail: fabian@zalando.de
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Yadong Xie <vt...@gmail.com>.
Thanks to Fabian for starting this discussion!
I liked the discussion. Here are some opinions I have:
[1] Till mentioned that "I think our web UI technology stack is quite dusty
by now and it would be beneficial to think about its technological future."
I totally agree with that we should upgrade the web UI technology stack. I
have been working on Flink Web UI for quite a while. We recently
reconstructed it to the latest version of Angular (7.0) in Alibaba. and we
would love to contribute the entire work to the Flink community. I will
show some snapshots and provide more details later in this discussion
thread.
[2] Fabian mentioned that "From my experience the choice between Angular
2-7 or React is subjective, you can get things done with both."
I totally agree with this. But to my experience, upgrade Angular from 1 to
2-7 is relatively easier and less work than completely rewrite the entire
web UI via React. And since we have already completed angular 1 to angular
7 upgrading, I hope we can keep the web UI to angular.
Best
Yadong
On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <tr...@apache.org> wrote:
> I agree with Chesnay that the web UI is already fairly isolated in Flink's
> code base (see flink-runtime-web module). Moreover, imo it is very
> important that Flink comes with a nice web UI which works out of the box
> because it lowers the entrance hurdle massively. What one could argue is to
> make it easier to plug in your own web UI if you want to develop it. But I
> would see this as an orthogonal issue.
>
> I would actually argue against Robert's proposal to make a grand rewrite if
> this is by any means possible. Experience shows that these kind of efforts
> are usually set up to fail or take super long. If possible, then I would
> rather prefer to convert one component at a time to React. But I'm not an
> expert here and cannot tell whether this is possible or not.
>
> What we would need is active help from the community to review these
> changes. I think that is the crucial piece to make such an effort work or
> fail.
>
> Cheers,
> Till
>
> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> > The WebUI goes entirely through the REST API.
> >
> > I don't see how moving it out would make it easier to develop/contribute.
> >
> > The UI is already encapsulated in it's own module (flink-runtime-web)
> > and changes can be made to it independently.
> > Historically the issue was simply that contributions to the UI were not
> > reviewed in a timely manner / not at all, which mind you
> > applies to virtually all components. This also won't change by moving it
> > out of Flink.
> >
> > Having it in Flink also guarantees that there is an easily-discoverable
> > version of the UI that is compatible with your Flink distribution.
> >
> > On 31.10.2018 09:53, dyana.rose wrote:
> > > Re: who's using the web ui
> > >
> > > Though many mature solutions, with a fair amount of time/resources
> > available are likely running their own front ends, for teams like mine
> > which are smaller and aren't focused solely on working with Flink, having
> > the web ui available removes a large barrier to getting up and running
> > quickly.
> > >
> > > But, with the web UI being in core Flink, it does make it a bit more of
> > a chore to contribute.
> > >
> > > I'm unaware of how necessary it is for the UI to deploy with Flink? Is
> > it making any calls that aren't through the REST API (and if so, can
> those
> > endpoints be added to the REST API)?
> > >
> > > In general I'd support moving it to its own package, making it easier
> to
> > develop with your standard UI dev tools. This also allows the web UI to
> be
> > developed and released independently to core Flink.
> > >
> > > Dyana
> > >
> > > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de> wrote:
> > >> Hi Till, I basically agree with all your points. i would stress the
> > >> "dustiness" of the current architecture: the package manager used
> > (bower)
> > >> is deprecated since a long time, the chance for the builds of the
> flink
> > web
> > >> dashboard not working anymore is increasing every day.
> > >>
> > >> About the knowledge in the community: Two days is not a lot of time,
> but
> > >> interest in this topic seems to be minor anyways. Is someone using the
> > >> Flink Web Dashboard at all, or is everyone running their own custom
> > >> solutions? Because if there is no interest in using the Web UI AND no
> > one
> > >> interested in developing, there would be no need to package this as
> > part of
> > >> the official Flink package, but rather develop an independent solution
> > (I'm
> > >> not voting for this right now, just putting it out), if at all. The
> > >> official package could then just ship with the API, which other
> > solutions
> > >> can build upon. This solution could be from an agile point of view
> also
> > the
> > >> best (enhanced testing, independent and more effective dev workflow,
> > etc.),
> > >> but is bad for the usage of the Flink UI, because people need to
> install
> > >> two things individually (Flink and the web dashboard).
> > >>
> > >> How did the choice for Angular1 happen back then? Who was writing the
> > >> Dashboard in the first place?
> > >>
> > >> Cheers
> > >>
> > >> --
> > >>
> > >>
> > >> *Fabian WollertZalando SE*
> > >>
> > >> E-Mail: fabian@zalando.de
> > >>
> > >>
> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> > >> trohrmann@apache.org>:
> > >>
> > >>> Thanks for starting this discussion Fabian! I think our web UI
> > technology
> > >>> stack is quite dusty by now and it would be beneficial to think about
> > its
> > >>> technological future.
> > >>>
> > >>> On the one hand, our current web UI works more or less reliable and
> > >>> changing the underlying technology has the risk of breaking things.
> > >>> Moreover, there might be the risk that the newly chosen technology
> > will be
> > >>> deprecated at some point in time as well.
> > >>>
> > >>> On the other hand, we don't have much Angular 1 knowledge in the
> > community
> > >>> and extending the web UI is, thus, quite hard at the moment. Maybe by
> > using
> > >>> some newer web technologies we might be able to attract more people
> > with a
> > >>> web technology background to join the community.
> > >>>
> > >>> The lack of people working on the web UI is for me the biggest
> problem
> > I
> > >>> would like to address. If there is interest in the web UI, then I'm
> > quite
> > >>> sure that we will be able to even migrate to some other technology in
> > the
> > >>> future. The next important issue for me is to do the change
> > incrementally
> > >>> if possible. Ideally we never break the web UI in the process of
> > migrating
> > >>> to a new technology. I'm not an expert here so it might or might not
> be
> > >>> possible. But if it is, then we should design the implementation
> steps
> > in
> > >>> such a way.
> > >>>
> > >>> Cheers,
> > >>> Till
> > >>>
> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de>
> > wrote:
> > >>>
> > >>>> Hi everyone,
> > >>>>
> > >>>> in this email thread
> > >>>> <
> > >>>>
> > >>>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > >>>> and the tickets FLINK-10705
> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the discussion
> > came
> > >>> up
> > >>>> whether to change the underlying architecture of Flink's Web
> Dashboard
> > >>> from
> > >>>> Angular1 to something else. This email thread should be solely to
> > discuss
> > >>>> the pro's and con's of this, and what could be the target
> > architecture.
> > >>>>
> > >>>> My choice would be React. Personally I agree with Till's comments in
> > the
> > >>>> ticket, Angular 1 being basically outdated and is not having a large
> > >>>> following anymore. From my experience the choice between Angular 2-7
> > or
> > >>>> React is subjective, you can get things done with both. I personally
> > only
> > >>>> have experience with React, so I personally would be faster to
> > develop
> > >>>> with this one. I currently have not planned to learn Angular as well
> > >>> (being
> > >>>> a more backend focused developer in general) so if the decision
> would
> > be
> > >>> to
> > >>>> go with Angular, i would be unfortunately out of this rework of the
> > Flink
> > >>>> Dashboard most certainly.
> > >>>>
> > >>>> Additionally i would like to get rid of bower, since its officially
> > >>>> deprecated <
> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> > >>>> my
> > >>>> idea would be to just use a create-react-app package with npm and
> > webpack
> > >>>> under the hood. no need for additional lib's here imho. But again:
> > thats
> > >>>> mostly what i've been working with recently, so thats a subjective
> > >>> point. I
> > >>>> could imagine getting used to yarn in the future as well.
> > >>>>
> > >>>> Cheers
> > >>>> Fabian
> > >>>>
> > >>>> --
> > >>>>
> > >>>>
> > >>>> *Fabian WollertZalando SE*
> > >>>>
> > >>>> E-Mail: fabian@zalando.de
> > >>>>
> >
> >
>
--
Yadong Xie
Ministry of Education Key Lab for Intelligent Networks and Network Security
Dept. of Computer Science and Technology
School of electronic information and engineering
Xi'an Jiaotong University
Xi'an, 710049, P.R. China
Email: vthinkxie@gmail.com
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Robert Metzger <rm...@apache.org>.
Hey all,
Yadong has now opened a pull request with the Angular 7-based web frontend:
https://github.com/apache/flink/pull/8016
The PR contains the complete dashboard, and is ready to check out, build
and run locally.
I believe it would be good to get some more feedback on the PR, from people
with different systems, browsers and experiences.
The PR removes the old UI.
I propose to merge the PR soon to get more testing exposure in the
1.9-SNAPSHOT version.
Regards,
Robert
On Tue, Nov 6, 2018 at 12:45 PM Shaoxuan Wang <ws...@gmail.com> wrote:
> Fabian,
> Thanks for pointing out the Jira. Sure, we will reuse it to start the
> contribution.
>
> Regards,
> Shaoxuan
>
> On Tue, Nov 6, 2018 at 7:28 PM Fabian Wollert <fa...@zalando.de> wrote:
>
> > i updated this JIRA already, feel free to reuse this:
> > https://issues.apache.org/jira/browse/FLINK-10706
> >
> > --
> >
> >
> > *Fabian WollertZalando SE*
> >
> > E-Mail: fabian@zalando.de
> >
> >
> > Am Di., 6. Nov. 2018 um 12:10 Uhr schrieb Shaoxuan Wang <
> > wshaoxuan@gmail.com
> > >:
> >
> > > Till,
> > > Yes, it is a good idea to have a feature flag to switch the web UI
> before
> > > we completely deprecate the old one.
> > >
> > > Yadong,
> > > It seems that everyone likes the new web UI. Can you please open a
> master
> > > Jira and start to merge the code to Flink master. What do you think?
> > >
> > > Regards,
> > > Shaoxuan
> > >
> > > On Mon, Nov 5, 2018 at 7:04 PM Till Rohrmann <tr...@apache.org>
> > wrote:
> > >
> > > > Thanks a lot for sharing the code with the community Yadong!
> > > >
> > > > It looks really cool and I also want to give it a try to see how easy
> > it
> > > is
> > > > to start Flink with it.
> > > >
> > > > If it is already implemented and working, we could also think about
> > > adding
> > > > it to Flink and add a feature flag to switch between the old and new
> > web
> > > > UI. We could think about enabling it by default to give it more user
> > > > exposure. After being confident and users having no complaints, we
> > could
> > > > think about deprecating the old web UI and eventually to drop it. Of
> > > > course, initially we should give it a thorough review.
> > > >
> > > > Cheers,
> > > > Till
> > > >
> > > > On Mon, Nov 5, 2018 at 8:40 AM Fabian Wollert <fa...@zalando.de>
> > wrote:
> > > >
> > > > > Hi Yadong, this is awesome, thx for the code! I will try it out on
> > our
> > > > > infrastructure and will post my feedback here, latest next week.
> > > > >
> > > > > I will also check if my ideas for FLINK-10707
> > > > > <https://issues.apache.org/jira/browse/FLINK-10707> are doable
> with
> > > your
> > > > > code since this was what pushed this discussion initially.
> > > > >
> > > > > Cheers
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > > *Fabian WollertZalando SE*
> > > > >
> > > > > E-Mail: fabian@zalando.de
> > > > >
> > > > >
> > > > > Am Mo., 5. Nov. 2018 um 04:42 Uhr schrieb Yadong Xie <
> > > > vthinkxie@gmail.com
> > > > > >:
> > > > >
> > > > > > Hi Fabian, Till, and Robert
> > > > > >
> > > > > > Thank you for your attention to this matter, I just push our
> codes
> > to
> > > > > > github: https://github.com/vthinkxie/flink-runtime-web.
> > > > > >
> > > > > > You can start the project by following the guidelines
> > > > > > <
> > > https://github.com/vthinkxie/flink-runtime-web#development--debugging
> > > > >
> > > > > > (just
> > > > > > run `npm install && npm run proxy`), just feel free to give any
> > > > comments
> > > > > :)
> > > > > >
> > > > > > If I missed anything please let me know. Look forward to your
> > > feedback
> > > > > and
> > > > > > suggestions soon.
> > > > > >
> > > > > > Best regards
> > > > > > Yadong
> > > > > >
> > > > > >
> > > > > > On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fabian@zalando.de
> >
> > > > wrote:
> > > > > >
> > > > > > > Hi Yadong, this looks awesome. is there any chance you can
> > already
> > > > > share
> > > > > > > the code of the new web UI, so we can take a look at what you
> > guys
> > > > > build
> > > > > > > there? I think that would speed up the discussion. If there is
> > > > already
> > > > > a
> > > > > > > fully fledged new Version with everything updated out there,
> and
> > > its
> > > > > even
> > > > > > > battle tested in production already, that sounds like the way
> to
> > go
> > > > for
> > > > > > me.
> > > > > > >
> > > > > > > i started aside from this discussion here (to strengthen and
> > learn
> > > > some
> > > > > > > new React stuff) my attempt on the React version already, for
> > > whoever
> > > > > is
> > > > > > > curious, you can check it out here:
> > > > > > > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the
> > base
> > > > url
> > > > > in
> > > > > > > actions.js
> > > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8
> > > > > > >,
> > > > > > > npm install and then npm start) ... i just started to convert
> the
> > > > first
> > > > > > > simple pages, so dont expect the whole UI yet, its just a start
> > ...
> > > > > > >
> > > > > > > Cheers
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > >
> > > > > > > *Fabian WollertZalando SE*
> > > > > > >
> > > > > > > E-Mail: fabian@zalando.de
> > > > > > >
> > > > > > >
> > > > > > > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <
> > > > zjffdu@gmail.com
> > > > > >:
> > > > > > >
> > > > > > >> +1 for the proposal of Yadong. I'd like to add one more
> > comment:
> > > > > > >>
> > > > > > >> We should provide more development guidance when developing
> the
> > > next
> > > > > > >> dashboard via angular 7. As Till mentioned before, one big
> > concern
> > > > is
> > > > > > lack
> > > > > > >> of frontend guys in flink community, I believe more
> development
> > > > > guidance
> > > > > > >> will attract more frontend guys to contribute to Flink web ui,
> > and
> > > > > also
> > > > > > be
> > > > > > >> helpful for backend guys who want to add something to
> dashboard.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> Shaoxuan Wang <ws...@gmail.com>于2018年11月2日周五 上午10:06写道：
> > > > > > >>
> > > > > > >> > +1 to keep the web UI stack to Angular. This will make the
> > stack
> > > > > > >> upgrading
> > > > > > >> > much easier and won't break the on-going efforts already
> made
> > > > > towards
> > > > > > >> the
> > > > > > >> > improvements of Flink Web UI.
> > > > > > >> >
> > > > > > >> > Till, Robert, and Fabian,
> > > > > > >> > Please take a look at the upgrading plan from Angular 1.x to
> > 7.0
> > > > > > >> proposed
> > > > > > >> > by Yadong. Any feedback is appreciated.
> > > > > > >> >
> > > > > > >> > Regards,
> > > > > > >> > Shaoxuan
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <
> > vthinkxie@gmail.com>
> > > > > > wrote:
> > > > > > >> >
> > > > > > >> > > For your interest, here are some snapshots of Flink web UI
> > > (with
> > > > > > >> angular
> > > > > > >> > > 7.0) we recently reconstructed.
> > > > > > >> > > [image: snapshot.001.jpeg]
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
> > > > > > >> > >
> > > > > > >> > > We choose Angular 7.0 due to the following reasons:
> > > > > > >> > >
> > > > > > >> > > 1. It is easy to migrate the current codebase to
> Angular
> > > 7.0
> > > > > > >> following
> > > > > > >> > > the official guide.
> > > > > > >> > > 2. It is easy for current Web UI developers who are
> > > > experienced
> > > > > > >> with
> > > > > > >> > > Angular 1.x to get familiar with 7.0 reading the
> official
> > > > doc.
> > > > > > >> > > 3. Migrating other than rewriting is helpful to ensure
> > its
> > > > > > >> functional
> > > > > > >> > > integrity.
> > > > > > >> > > 4. There are over 1 million Angular developers now (2.0
> > and
> > > > > > above,
> > > > > > >> > > still increasing rapidly), which means there are
> > > potentially
> > > > > more
> > > > > > >> > > developers willing to join the community.
> > > > > > >> > > 5. No bower or complex node / webpack configuration
> > needed
> > > > > (only
> > > > > > >> > > angular cli).
> > > > > > >> > >
> > > > > > >> > > We split Angular 1.x and 7.0 versions into two folders
> based
> > > on
> > > > > the
> > > > > > >> > > following consideration:
> > > > > > >> > >
> > > > > > >> > > 1. 1.x version could be the default page because there
> is
> > > no
> > > > > > >> > > breakchange for both users and developers.
> > > > > > >> > > 2. Make the upgrade progressive. Users can switch
> between
> > > two
> > > > > > >> versions
> > > > > > >> > > by clicking the button on the top right corner.
> > > > > > >> > > 3. Reduce maintenance costs and keep the performance of
> > web
> > > > > > without
> > > > > > >> > > bootstrap both 1.x and 7.0 versions in the same page.
> > > > > > >> > > 4. Won't interrupt the current development process. The
> > > code
> > > > > can
> > > > > > >> still
> > > > > > >> > > be submitted to 1.x version folder, we could help to
> sync
> > > all
> > > > > > >> > features and
> > > > > > >> > > bug fix to 7.0 version if needed.
> > > > > > >> > > 5. When the 7.0 version is ready and stable, it is easy
> > to
> > > > > change
> > > > > > >> > > default page to it, and remove the old version.
> > > > > > >> > >
> > > > > > >> > > [image: test.jpg]
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
> > > > > > >> > >
> > > > > > >> > > Here is the brief list of what we have done:
> > > > > > >> > >
> > > > > > >> > > 1. Create a new folder named next-web-dashboard in
> > > > > > >> flink-runtime-web.
> > > > > > >> > > 2. Migrate all Angular 1.x controllers, services,
> config
> > > etc
> > > > > one
> > > > > > by
> > > > > > >> > > one into 7.0 version with ngMigration.
> > > > > > >> > > 3. Modify frontend build scripts in order to package
> two
> > > > > versions
> > > > > > >> > > together.
> > > > > > >> > > 4. All works listed above are limited in the
> > > > flink-runtime-web
> > > > > > >> folder.
> > > > > > >> > > One folder was added and no file or folder was deleted.
> > > > > > >> > >
> > > > > > >> > > The new version of the Web UI is running stable for
> several
> > > > months
> > > > > > in
> > > > > > >> > > Alibaba. We are happy to contribute it to the Flink
> > community
> > > if
> > > > > > >> everyone
> > > > > > >> > > like the work.
> > > > > > >> > >
> > > > > > >> > > Best
> > > > > > >> > > Yadong
> > > > > > >> > >
> > > > > > >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <
> > > > > trohrmann@apache.org
> > > > > > >
> > > > > > >> > > wrote:
> > > > > > >> > >
> > > > > > >> > >> I agree with Chesnay that the web UI is already fairly
> > > isolated
> > > > > in
> > > > > > >> > Flink's
> > > > > > >> > >> code base (see flink-runtime-web module). Moreover, imo
> it
> > is
> > > > > very
> > > > > > >> > >> important that Flink comes with a nice web UI which works
> > out
> > > > of
> > > > > > the
> > > > > > >> box
> > > > > > >> > >> because it lowers the entrance hurdle massively. What one
> > > could
> > > > > > >> argue is
> > > > > > >> > >> to
> > > > > > >> > >> make it easier to plug in your own web UI if you want to
> > > > develop
> > > > > > it.
> > > > > > >> > But I
> > > > > > >> > >> would see this as an orthogonal issue.
> > > > > > >> > >>
> > > > > > >> > >> I would actually argue against Robert's proposal to make
> a
> > > > grand
> > > > > > >> rewrite
> > > > > > >> > >> if
> > > > > > >> > >> this is by any means possible. Experience shows that
> these
> > > kind
> > > > > of
> > > > > > >> > efforts
> > > > > > >> > >> are usually set up to fail or take super long. If
> possible,
> > > > then
> > > > > I
> > > > > > >> would
> > > > > > >> > >> rather prefer to convert one component at a time to
> React.
> > > But
> > > > > I'm
> > > > > > >> not
> > > > > > >> > an
> > > > > > >> > >> expert here and cannot tell whether this is possible or
> > not.
> > > > > > >> > >>
> > > > > > >> > >> What we would need is active help from the community to
> > > review
> > > > > > these
> > > > > > >> > >> changes. I think that is the crucial piece to make such
> an
> > > > effort
> > > > > > >> work
> > > > > > >> > or
> > > > > > >> > >> fail.
> > > > > > >> > >>
> > > > > > >> > >> Cheers,
> > > > > > >> > >> Till
> > > > > > >> > >>
> > > > > > >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <
> > > > > > >> chesnay@apache.org>
> > > > > > >> > >> wrote:
> > > > > > >> > >>
> > > > > > >> > >> > The WebUI goes entirely through the REST API.
> > > > > > >> > >> >
> > > > > > >> > >> > I don't see how moving it out would make it easier to
> > > > > > >> > >> develop/contribute.
> > > > > > >> > >> >
> > > > > > >> > >> > The UI is already encapsulated in it's own module
> > > > > > >> (flink-runtime-web)
> > > > > > >> > >> > and changes can be made to it independently.
> > > > > > >> > >> > Historically the issue was simply that contributions to
> > the
> > > > UI
> > > > > > were
> > > > > > >> > not
> > > > > > >> > >> > reviewed in a timely manner / not at all, which mind
> you
> > > > > > >> > >> > applies to virtually all components. This also won't
> > change
> > > > by
> > > > > > >> moving
> > > > > > >> > it
> > > > > > >> > >> > out of Flink.
> > > > > > >> > >> >
> > > > > > >> > >> > Having it in Flink also guarantees that there is an
> > > > > > >> > easily-discoverable
> > > > > > >> > >> > version of the UI that is compatible with your Flink
> > > > > > distribution.
> > > > > > >> > >> >
> > > > > > >> > >> > On 31.10.2018 09:53, dyana.rose wrote:
> > > > > > >> > >> > > Re: who's using the web ui
> > > > > > >> > >> > >
> > > > > > >> > >> > > Though many mature solutions, with a fair amount of
> > > > > > >> time/resources
> > > > > > >> > >> > available are likely running their own front ends, for
> > > teams
> > > > > like
> > > > > > >> mine
> > > > > > >> > >> > which are smaller and aren't focused solely on working
> > with
> > > > > > Flink,
> > > > > > >> > >> having
> > > > > > >> > >> > the web ui available removes a large barrier to getting
> > up
> > > > and
> > > > > > >> running
> > > > > > >> > >> > quickly.
> > > > > > >> > >> > >
> > > > > > >> > >> > > But, with the web UI being in core Flink, it does
> make
> > > it a
> > > > > bit
> > > > > > >> more
> > > > > > >> > >> of
> > > > > > >> > >> > a chore to contribute.
> > > > > > >> > >> > >
> > > > > > >> > >> > > I'm unaware of how necessary it is for the UI to
> deploy
> > > > with
> > > > > > >> Flink?
> > > > > > >> > Is
> > > > > > >> > >> > it making any calls that aren't through the REST API
> (and
> > > if
> > > > > so,
> > > > > > >> can
> > > > > > >> > >> those
> > > > > > >> > >> > endpoints be added to the REST API)?
> > > > > > >> > >> > >
> > > > > > >> > >> > > In general I'd support moving it to its own package,
> > > making
> > > > > it
> > > > > > >> > easier
> > > > > > >> > >> to
> > > > > > >> > >> > develop with your standard UI dev tools. This also
> allows
> > > the
> > > > > web
> > > > > > >> UI
> > > > > > >> > to
> > > > > > >> > >> be
> > > > > > >> > >> > developed and released independently to core Flink.
> > > > > > >> > >> > >
> > > > > > >> > >> > > Dyana
> > > > > > >> > >> > >
> > > > > > >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <
> > > fabian@zalando.de>
> > > > > > >> wrote:
> > > > > > >> > >> > >> Hi Till, I basically agree with all your points. i
> > would
> > > > > > stress
> > > > > > >> the
> > > > > > >> > >> > >> "dustiness" of the current architecture: the package
> > > > manager
> > > > > > >> used
> > > > > > >> > >> > (bower)
> > > > > > >> > >> > >> is deprecated since a long time, the chance for the
> > > builds
> > > > > of
> > > > > > >> the
> > > > > > >> > >> flink
> > > > > > >> > >> > web
> > > > > > >> > >> > >> dashboard not working anymore is increasing every
> day.
> > > > > > >> > >> > >>
> > > > > > >> > >> > >> About the knowledge in the community: Two days is
> not
> > a
> > > > lot
> > > > > of
> > > > > > >> > time,
> > > > > > >> > >> but
> > > > > > >> > >> > >> interest in this topic seems to be minor anyways. Is
> > > > someone
> > > > > > >> using
> > > > > > >> > >> the
> > > > > > >> > >> > >> Flink Web Dashboard at all, or is everyone running
> > their
> > > > own
> > > > > > >> custom
> > > > > > >> > >> > >> solutions? Because if there is no interest in using
> > the
> > > > Web
> > > > > UI
> > > > > > >> AND
> > > > > > >> > no
> > > > > > >> > >> > one
> > > > > > >> > >> > >> interested in developing, there would be no need to
> > > > package
> > > > > > >> this as
> > > > > > >> > >> > part of
> > > > > > >> > >> > >> the official Flink package, but rather develop an
> > > > > independent
> > > > > > >> > >> solution
> > > > > > >> > >> > (I'm
> > > > > > >> > >> > >> not voting for this right now, just putting it out),
> > if
> > > at
> > > > > > all.
> > > > > > >> The
> > > > > > >> > >> > >> official package could then just ship with the API,
> > > which
> > > > > > other
> > > > > > >> > >> > solutions
> > > > > > >> > >> > >> can build upon. This solution could be from an agile
> > > point
> > > > > of
> > > > > > >> view
> > > > > > >> > >> also
> > > > > > >> > >> > the
> > > > > > >> > >> > >> best (enhanced testing, independent and more
> effective
> > > dev
> > > > > > >> > workflow,
> > > > > > >> > >> > etc.),
> > > > > > >> > >> > >> but is bad for the usage of the Flink UI, because
> > people
> > > > > need
> > > > > > to
> > > > > > >> > >> install
> > > > > > >> > >> > >> two things individually (Flink and the web
> dashboard).
> > > > > > >> > >> > >>
> > > > > > >> > >> > >> How did the choice for Angular1 happen back then?
> Who
> > > was
> > > > > > >> writing
> > > > > > >> > the
> > > > > > >> > >> > >> Dashboard in the first place?
> > > > > > >> > >> > >>
> > > > > > >> > >> > >> Cheers
> > > > > > >> > >> > >>
> > > > > > >> > >> > >> --
> > > > > > >> > >> > >>
> > > > > > >> > >> > >>
> > > > > > >> > >> > >> *Fabian WollertZalando SE*
> > > > > > >> > >> > >>
> > > > > > >> > >> > >> E-Mail: fabian@zalando.de
> > > > > > >> > >> > >>
> > > > > > >> > >> > >>
> > > > > > >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till
> > > Rohrmann <
> > > > > > >> > >> > >> trohrmann@apache.org>:
> > > > > > >> > >> > >>
> > > > > > >> > >> > >>> Thanks for starting this discussion Fabian! I think
> > our
> > > > web
> > > > > > UI
> > > > > > >> > >> > technology
> > > > > > >> > >> > >>> stack is quite dusty by now and it would be
> > beneficial
> > > to
> > > > > > think
> > > > > > >> > >> about
> > > > > > >> > >> > its
> > > > > > >> > >> > >>> technological future.
> > > > > > >> > >> > >>>
> > > > > > >> > >> > >>> On the one hand, our current web UI works more or
> > less
> > > > > > reliable
> > > > > > >> > and
> > > > > > >> > >> > >>> changing the underlying technology has the risk of
> > > > breaking
> > > > > > >> > things.
> > > > > > >> > >> > >>> Moreover, there might be the risk that the newly
> > chosen
> > > > > > >> technology
> > > > > > >> > >> > will be
> > > > > > >> > >> > >>> deprecated at some point in time as well.
> > > > > > >> > >> > >>>
> > > > > > >> > >> > >>> On the other hand, we don't have much Angular 1
> > > knowledge
> > > > > in
> > > > > > >> the
> > > > > > >> > >> > community
> > > > > > >> > >> > >>> and extending the web UI is, thus, quite hard at
> the
> > > > > moment.
> > > > > > >> Maybe
> > > > > > >> > >> by
> > > > > > >> > >> > using
> > > > > > >> > >> > >>> some newer web technologies we might be able to
> > attract
> > > > > more
> > > > > > >> > people
> > > > > > >> > >> > with a
> > > > > > >> > >> > >>> web technology background to join the community.
> > > > > > >> > >> > >>>
> > > > > > >> > >> > >>> The lack of people working on the web UI is for me
> > the
> > > > > > biggest
> > > > > > >> > >> problem
> > > > > > >> > >> > I
> > > > > > >> > >> > >>> would like to address. If there is interest in the
> > web
> > > > UI,
> > > > > > then
> > > > > > >> > I'm
> > > > > > >> > >> > quite
> > > > > > >> > >> > >>> sure that we will be able to even migrate to some
> > other
> > > > > > >> technology
> > > > > > >> > >> in
> > > > > > >> > >> > the
> > > > > > >> > >> > >>> future. The next important issue for me is to do
> the
> > > > change
> > > > > > >> > >> > incrementally
> > > > > > >> > >> > >>> if possible. Ideally we never break the web UI in
> the
> > > > > process
> > > > > > >> of
> > > > > > >> > >> > migrating
> > > > > > >> > >> > >>> to a new technology. I'm not an expert here so it
> > might
> > > > or
> > > > > > >> might
> > > > > > >> > >> not be
> > > > > > >> > >> > >>> possible. But if it is, then we should design the
> > > > > > >> implementation
> > > > > > >> > >> steps
> > > > > > >> > >> > in
> > > > > > >> > >> > >>> such a way.
> > > > > > >> > >> > >>>
> > > > > > >> > >> > >>> Cheers,
> > > > > > >> > >> > >>> Till
> > > > > > >> > >> > >>>
> > > > > > >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <
> > > > > > >> fabian@zalando.de
> > > > > > >> > >
> > > > > > >> > >> > wrote:
> > > > > > >> > >> > >>>
> > > > > > >> > >> > >>>> Hi everyone,
> > > > > > >> > >> > >>>>
> > > > > > >> > >> > >>>> in this email thread
> > > > > > >> > >> > >>>> <
> > > > > > >> > >> > >>>>
> > > > > > >> > >> > >>>
> > > > > > >> > >> >
> > > > > > >> > >>
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > > > > > >> > >> > >>>> and the tickets FLINK-10705
> > > > > > >> > >> > >>>> <
> https://issues.apache.org/jira/browse/FLINK-10705>
> > > and
> > > > > > >> > >> FLINK-10706
> > > > > > >> > >> > >>>> <
> https://issues.apache.org/jira/browse/FLINK-10706>
> > > the
> > > > > > >> > discussion
> > > > > > >> > >> > came
> > > > > > >> > >> > >>> up
> > > > > > >> > >> > >>>> whether to change the underlying architecture of
> > > Flink's
> > > > > Web
> > > > > > >> > >> Dashboard
> > > > > > >> > >> > >>> from
> > > > > > >> > >> > >>>> Angular1 to something else. This email thread
> should
> > > be
> > > > > > >> solely to
> > > > > > >> > >> > discuss
> > > > > > >> > >> > >>>> the pro's and con's of this, and what could be the
> > > > target
> > > > > > >> > >> > architecture.
> > > > > > >> > >> > >>>>
> > > > > > >> > >> > >>>> My choice would be React. Personally I agree with
> > > Till's
> > > > > > >> comments
> > > > > > >> > >> in
> > > > > > >> > >> > the
> > > > > > >> > >> > >>>> ticket, Angular 1 being basically outdated and is
> > not
> > > > > > having a
> > > > > > >> > >> large
> > > > > > >> > >> > >>>> following anymore. From my experience the choice
> > > between
> > > > > > >> Angular
> > > > > > >> > >> 2-7
> > > > > > >> > >> > or
> > > > > > >> > >> > >>>> React is subjective, you can get things done with
> > > both.
> > > > I
> > > > > > >> > >> personally
> > > > > > >> > >> > only
> > > > > > >> > >> > >>>> have experience with React, so I personally would
> > be
> > > > > faster
> > > > > > >> to
> > > > > > >> > >> > develop
> > > > > > >> > >> > >>>> with this one. I currently have not planned to
> learn
> > > > > Angular
> > > > > > >> as
> > > > > > >> > >> well
> > > > > > >> > >> > >>> (being
> > > > > > >> > >> > >>>> a more backend focused developer in general) so if
> > the
> > > > > > >> decision
> > > > > > >> > >> would
> > > > > > >> > >> > be
> > > > > > >> > >> > >>> to
> > > > > > >> > >> > >>>> go with Angular, i would be unfortunately out of
> > this
> > > > > rework
> > > > > > >> of
> > > > > > >> > the
> > > > > > >> > >> > Flink
> > > > > > >> > >> > >>>> Dashboard most certainly.
> > > > > > >> > >> > >>>>
> > > > > > >> > >> > >>>> Additionally i would like to get rid of bower,
> since
> > > its
> > > > > > >> > officially
> > > > > > >> > >> > >>>> deprecated <
> > > > > > >> > >> >
> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/
> > > >.
> > > > > > >> > >> > >>>> my
> > > > > > >> > >> > >>>> idea would be to just use a create-react-app
> package
> > > > with
> > > > > > npm
> > > > > > >> and
> > > > > > >> > >> > webpack
> > > > > > >> > >> > >>>> under the hood. no need for additional lib's here
> > > imho.
> > > > > But
> > > > > > >> > again:
> > > > > > >> > >> > thats
> > > > > > >> > >> > >>>> mostly what i've been working with recently, so
> > thats
> > > a
> > > > > > >> > subjective
> > > > > > >> > >> > >>> point. I
> > > > > > >> > >> > >>>> could imagine getting used to yarn in the future
> as
> > > > well.
> > > > > > >> > >> > >>>>
> > > > > > >> > >> > >>>> Cheers
> > > > > > >> > >> > >>>> Fabian
> > > > > > >> > >> > >>>>
> > > > > > >> > >> > >>>> --
> > > > > > >> > >> > >>>>
> > > > > > >> > >> > >>>>
> > > > > > >> > >> > >>>> *Fabian WollertZalando SE*
> > > > > > >> > >> > >>>>
> > > > > > >> > >> > >>>> E-Mail: fabian@zalando.de
> > > > > > >> > >> > >>>>
> > > > > > >> > >> >
> > > > > > >> > >> >
> > > > > > >> > >>
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > --
> > > > > > >> > > Yadong Xie
> > > > > > >> > > Ministry of Education Key Lab for Intelligent Networks and
> > > > Network
> > > > > > >> > Security
> > > > > > >> > > Dept. of Computer Science and Technology
> > > > > > >> > > School of electronic information and engineering
> > > > > > >> > > Xi'an Jiaotong University
> > > > > > >> > > Xi'an, 710049, P.R. China
> > > > > > >> > > Email: vthinkxie@gmail.com
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Shaoxuan Wang <ws...@gmail.com>.
Fabian,
Thanks for pointing out the Jira. Sure, we will reuse it to start the
contribution.
Regards,
Shaoxuan
On Tue, Nov 6, 2018 at 7:28 PM Fabian Wollert <fa...@zalando.de> wrote:
> i updated this JIRA already, feel free to reuse this:
> https://issues.apache.org/jira/browse/FLINK-10706
>
> --
>
>
> *Fabian WollertZalando SE*
>
> E-Mail: fabian@zalando.de
>
>
> Am Di., 6. Nov. 2018 um 12:10 Uhr schrieb Shaoxuan Wang <
> wshaoxuan@gmail.com
> >:
>
> > Till,
> > Yes, it is a good idea to have a feature flag to switch the web UI before
> > we completely deprecate the old one.
> >
> > Yadong,
> > It seems that everyone likes the new web UI. Can you please open a master
> > Jira and start to merge the code to Flink master. What do you think?
> >
> > Regards,
> > Shaoxuan
> >
> > On Mon, Nov 5, 2018 at 7:04 PM Till Rohrmann <tr...@apache.org>
> wrote:
> >
> > > Thanks a lot for sharing the code with the community Yadong!
> > >
> > > It looks really cool and I also want to give it a try to see how easy
> it
> > is
> > > to start Flink with it.
> > >
> > > If it is already implemented and working, we could also think about
> > adding
> > > it to Flink and add a feature flag to switch between the old and new
> web
> > > UI. We could think about enabling it by default to give it more user
> > > exposure. After being confident and users having no complaints, we
> could
> > > think about deprecating the old web UI and eventually to drop it. Of
> > > course, initially we should give it a thorough review.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Mon, Nov 5, 2018 at 8:40 AM Fabian Wollert <fa...@zalando.de>
> wrote:
> > >
> > > > Hi Yadong, this is awesome, thx for the code! I will try it out on
> our
> > > > infrastructure and will post my feedback here, latest next week.
> > > >
> > > > I will also check if my ideas for FLINK-10707
> > > > <https://issues.apache.org/jira/browse/FLINK-10707> are doable with
> > your
> > > > code since this was what pushed this discussion initially.
> > > >
> > > > Cheers
> > > >
> > > > --
> > > >
> > > >
> > > > *Fabian WollertZalando SE*
> > > >
> > > > E-Mail: fabian@zalando.de
> > > >
> > > >
> > > > Am Mo., 5. Nov. 2018 um 04:42 Uhr schrieb Yadong Xie <
> > > vthinkxie@gmail.com
> > > > >:
> > > >
> > > > > Hi Fabian, Till, and Robert
> > > > >
> > > > > Thank you for your attention to this matter, I just push our codes
> to
> > > > > github: https://github.com/vthinkxie/flink-runtime-web.
> > > > >
> > > > > You can start the project by following the guidelines
> > > > > <
> > https://github.com/vthinkxie/flink-runtime-web#development--debugging
> > > >
> > > > > (just
> > > > > run `npm install && npm run proxy`), just feel free to give any
> > > comments
> > > > :)
> > > > >
> > > > > If I missed anything please let me know. Look forward to your
> > feedback
> > > > and
> > > > > suggestions soon.
> > > > >
> > > > > Best regards
> > > > > Yadong
> > > > >
> > > > >
> > > > > On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fa...@zalando.de>
> > > wrote:
> > > > >
> > > > > > Hi Yadong, this looks awesome. is there any chance you can
> already
> > > > share
> > > > > > the code of the new web UI, so we can take a look at what you
> guys
> > > > build
> > > > > > there? I think that would speed up the discussion. If there is
> > > already
> > > > a
> > > > > > fully fledged new Version with everything updated out there, and
> > its
> > > > even
> > > > > > battle tested in production already, that sounds like the way to
> go
> > > for
> > > > > me.
> > > > > >
> > > > > > i started aside from this discussion here (to strengthen and
> learn
> > > some
> > > > > > new React stuff) my attempt on the React version already, for
> > whoever
> > > > is
> > > > > > curious, you can check it out here:
> > > > > > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the
> base
> > > url
> > > > in
> > > > > > actions.js
> > > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8
> > > > > >,
> > > > > > npm install and then npm start) ... i just started to convert the
> > > first
> > > > > > simple pages, so dont expect the whole UI yet, its just a start
> ...
> > > > > >
> > > > > > Cheers
> > > > > >
> > > > > > --
> > > > > >
> > > > > >
> > > > > > *Fabian WollertZalando SE*
> > > > > >
> > > > > > E-Mail: fabian@zalando.de
> > > > > >
> > > > > >
> > > > > > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <
> > > zjffdu@gmail.com
> > > > >:
> > > > > >
> > > > > >> +1 for the proposal of Yadong. I'd like to add one more
> comment:
> > > > > >>
> > > > > >> We should provide more development guidance when developing the
> > next
> > > > > >> dashboard via angular 7. As Till mentioned before, one big
> concern
> > > is
> > > > > lack
> > > > > >> of frontend guys in flink community, I believe more development
> > > > guidance
> > > > > >> will attract more frontend guys to contribute to Flink web ui,
> and
> > > > also
> > > > > be
> > > > > >> helpful for backend guys who want to add something to dashboard.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> Shaoxuan Wang <ws...@gmail.com>于2018年11月2日周五 上午10:06写道：
> > > > > >>
> > > > > >> > +1 to keep the web UI stack to Angular. This will make the
> stack
> > > > > >> upgrading
> > > > > >> > much easier and won't break the on-going efforts already made
> > > > towards
> > > > > >> the
> > > > > >> > improvements of Flink Web UI.
> > > > > >> >
> > > > > >> > Till, Robert, and Fabian,
> > > > > >> > Please take a look at the upgrading plan from Angular 1.x to
> 7.0
> > > > > >> proposed
> > > > > >> > by Yadong. Any feedback is appreciated.
> > > > > >> >
> > > > > >> > Regards,
> > > > > >> > Shaoxuan
> > > > > >> >
> > > > > >> >
> > > > > >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <
> vthinkxie@gmail.com>
> > > > > wrote:
> > > > > >> >
> > > > > >> > > For your interest, here are some snapshots of Flink web UI
> > (with
> > > > > >> angular
> > > > > >> > > 7.0) we recently reconstructed.
> > > > > >> > > [image: snapshot.001.jpeg]
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
> > > > > >> > >
> > > > > >> > > We choose Angular 7.0 due to the following reasons:
> > > > > >> > >
> > > > > >> > > 1. It is easy to migrate the current codebase to Angular
> > 7.0
> > > > > >> following
> > > > > >> > > the official guide.
> > > > > >> > > 2. It is easy for current Web UI developers who are
> > > experienced
> > > > > >> with
> > > > > >> > > Angular 1.x to get familiar with 7.0 reading the official
> > > doc.
> > > > > >> > > 3. Migrating other than rewriting is helpful to ensure
> its
> > > > > >> functional
> > > > > >> > > integrity.
> > > > > >> > > 4. There are over 1 million Angular developers now (2.0
> and
> > > > > above,
> > > > > >> > > still increasing rapidly), which means there are
> > potentially
> > > > more
> > > > > >> > > developers willing to join the community.
> > > > > >> > > 5. No bower or complex node / webpack configuration
> needed
> > > > (only
> > > > > >> > > angular cli).
> > > > > >> > >
> > > > > >> > > We split Angular 1.x and 7.0 versions into two folders based
> > on
> > > > the
> > > > > >> > > following consideration:
> > > > > >> > >
> > > > > >> > > 1. 1.x version could be the default page because there is
> > no
> > > > > >> > > breakchange for both users and developers.
> > > > > >> > > 2. Make the upgrade progressive. Users can switch between
> > two
> > > > > >> versions
> > > > > >> > > by clicking the button on the top right corner.
> > > > > >> > > 3. Reduce maintenance costs and keep the performance of
> web
> > > > > without
> > > > > >> > > bootstrap both 1.x and 7.0 versions in the same page.
> > > > > >> > > 4. Won't interrupt the current development process. The
> > code
> > > > can
> > > > > >> still
> > > > > >> > > be submitted to 1.x version folder, we could help to sync
> > all
> > > > > >> > features and
> > > > > >> > > bug fix to 7.0 version if needed.
> > > > > >> > > 5. When the 7.0 version is ready and stable, it is easy
> to
> > > > change
> > > > > >> > > default page to it, and remove the old version.
> > > > > >> > >
> > > > > >> > > [image: test.jpg]
> > > > > >> > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
> > > > > >> > >
> > > > > >> > > Here is the brief list of what we have done:
> > > > > >> > >
> > > > > >> > > 1. Create a new folder named next-web-dashboard in
> > > > > >> flink-runtime-web.
> > > > > >> > > 2. Migrate all Angular 1.x controllers, services, config
> > etc
> > > > one
> > > > > by
> > > > > >> > > one into 7.0 version with ngMigration.
> > > > > >> > > 3. Modify frontend build scripts in order to package two
> > > > versions
> > > > > >> > > together.
> > > > > >> > > 4. All works listed above are limited in the
> > > flink-runtime-web
> > > > > >> folder.
> > > > > >> > > One folder was added and no file or folder was deleted.
> > > > > >> > >
> > > > > >> > > The new version of the Web UI is running stable for several
> > > months
> > > > > in
> > > > > >> > > Alibaba. We are happy to contribute it to the Flink
> community
> > if
> > > > > >> everyone
> > > > > >> > > like the work.
> > > > > >> > >
> > > > > >> > > Best
> > > > > >> > > Yadong
> > > > > >> > >
> > > > > >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <
> > > > trohrmann@apache.org
> > > > > >
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > >> I agree with Chesnay that the web UI is already fairly
> > isolated
> > > > in
> > > > > >> > Flink's
> > > > > >> > >> code base (see flink-runtime-web module). Moreover, imo it
> is
> > > > very
> > > > > >> > >> important that Flink comes with a nice web UI which works
> out
> > > of
> > > > > the
> > > > > >> box
> > > > > >> > >> because it lowers the entrance hurdle massively. What one
> > could
> > > > > >> argue is
> > > > > >> > >> to
> > > > > >> > >> make it easier to plug in your own web UI if you want to
> > > develop
> > > > > it.
> > > > > >> > But I
> > > > > >> > >> would see this as an orthogonal issue.
> > > > > >> > >>
> > > > > >> > >> I would actually argue against Robert's proposal to make a
> > > grand
> > > > > >> rewrite
> > > > > >> > >> if
> > > > > >> > >> this is by any means possible. Experience shows that these
> > kind
> > > > of
> > > > > >> > efforts
> > > > > >> > >> are usually set up to fail or take super long. If possible,
> > > then
> > > > I
> > > > > >> would
> > > > > >> > >> rather prefer to convert one component at a time to React.
> > But
> > > > I'm
> > > > > >> not
> > > > > >> > an
> > > > > >> > >> expert here and cannot tell whether this is possible or
> not.
> > > > > >> > >>
> > > > > >> > >> What we would need is active help from the community to
> > review
> > > > > these
> > > > > >> > >> changes. I think that is the crucial piece to make such an
> > > effort
> > > > > >> work
> > > > > >> > or
> > > > > >> > >> fail.
> > > > > >> > >>
> > > > > >> > >> Cheers,
> > > > > >> > >> Till
> > > > > >> > >>
> > > > > >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <
> > > > > >> chesnay@apache.org>
> > > > > >> > >> wrote:
> > > > > >> > >>
> > > > > >> > >> > The WebUI goes entirely through the REST API.
> > > > > >> > >> >
> > > > > >> > >> > I don't see how moving it out would make it easier to
> > > > > >> > >> develop/contribute.
> > > > > >> > >> >
> > > > > >> > >> > The UI is already encapsulated in it's own module
> > > > > >> (flink-runtime-web)
> > > > > >> > >> > and changes can be made to it independently.
> > > > > >> > >> > Historically the issue was simply that contributions to
> the
> > > UI
> > > > > were
> > > > > >> > not
> > > > > >> > >> > reviewed in a timely manner / not at all, which mind you
> > > > > >> > >> > applies to virtually all components. This also won't
> change
> > > by
> > > > > >> moving
> > > > > >> > it
> > > > > >> > >> > out of Flink.
> > > > > >> > >> >
> > > > > >> > >> > Having it in Flink also guarantees that there is an
> > > > > >> > easily-discoverable
> > > > > >> > >> > version of the UI that is compatible with your Flink
> > > > > distribution.
> > > > > >> > >> >
> > > > > >> > >> > On 31.10.2018 09:53, dyana.rose wrote:
> > > > > >> > >> > > Re: who's using the web ui
> > > > > >> > >> > >
> > > > > >> > >> > > Though many mature solutions, with a fair amount of
> > > > > >> time/resources
> > > > > >> > >> > available are likely running their own front ends, for
> > teams
> > > > like
> > > > > >> mine
> > > > > >> > >> > which are smaller and aren't focused solely on working
> with
> > > > > Flink,
> > > > > >> > >> having
> > > > > >> > >> > the web ui available removes a large barrier to getting
> up
> > > and
> > > > > >> running
> > > > > >> > >> > quickly.
> > > > > >> > >> > >
> > > > > >> > >> > > But, with the web UI being in core Flink, it does make
> > it a
> > > > bit
> > > > > >> more
> > > > > >> > >> of
> > > > > >> > >> > a chore to contribute.
> > > > > >> > >> > >
> > > > > >> > >> > > I'm unaware of how necessary it is for the UI to deploy
> > > with
> > > > > >> Flink?
> > > > > >> > Is
> > > > > >> > >> > it making any calls that aren't through the REST API (and
> > if
> > > > so,
> > > > > >> can
> > > > > >> > >> those
> > > > > >> > >> > endpoints be added to the REST API)?
> > > > > >> > >> > >
> > > > > >> > >> > > In general I'd support moving it to its own package,
> > making
> > > > it
> > > > > >> > easier
> > > > > >> > >> to
> > > > > >> > >> > develop with your standard UI dev tools. This also allows
> > the
> > > > web
> > > > > >> UI
> > > > > >> > to
> > > > > >> > >> be
> > > > > >> > >> > developed and released independently to core Flink.
> > > > > >> > >> > >
> > > > > >> > >> > > Dyana
> > > > > >> > >> > >
> > > > > >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <
> > fabian@zalando.de>
> > > > > >> wrote:
> > > > > >> > >> > >> Hi Till, I basically agree with all your points. i
> would
> > > > > stress
> > > > > >> the
> > > > > >> > >> > >> "dustiness" of the current architecture: the package
> > > manager
> > > > > >> used
> > > > > >> > >> > (bower)
> > > > > >> > >> > >> is deprecated since a long time, the chance for the
> > builds
> > > > of
> > > > > >> the
> > > > > >> > >> flink
> > > > > >> > >> > web
> > > > > >> > >> > >> dashboard not working anymore is increasing every day.
> > > > > >> > >> > >>
> > > > > >> > >> > >> About the knowledge in the community: Two days is not
> a
> > > lot
> > > > of
> > > > > >> > time,
> > > > > >> > >> but
> > > > > >> > >> > >> interest in this topic seems to be minor anyways. Is
> > > someone
> > > > > >> using
> > > > > >> > >> the
> > > > > >> > >> > >> Flink Web Dashboard at all, or is everyone running
> their
> > > own
> > > > > >> custom
> > > > > >> > >> > >> solutions? Because if there is no interest in using
> the
> > > Web
> > > > UI
> > > > > >> AND
> > > > > >> > no
> > > > > >> > >> > one
> > > > > >> > >> > >> interested in developing, there would be no need to
> > > package
> > > > > >> this as
> > > > > >> > >> > part of
> > > > > >> > >> > >> the official Flink package, but rather develop an
> > > > independent
> > > > > >> > >> solution
> > > > > >> > >> > (I'm
> > > > > >> > >> > >> not voting for this right now, just putting it out),
> if
> > at
> > > > > all.
> > > > > >> The
> > > > > >> > >> > >> official package could then just ship with the API,
> > which
> > > > > other
> > > > > >> > >> > solutions
> > > > > >> > >> > >> can build upon. This solution could be from an agile
> > point
> > > > of
> > > > > >> view
> > > > > >> > >> also
> > > > > >> > >> > the
> > > > > >> > >> > >> best (enhanced testing, independent and more effective
> > dev
> > > > > >> > workflow,
> > > > > >> > >> > etc.),
> > > > > >> > >> > >> but is bad for the usage of the Flink UI, because
> people
> > > > need
> > > > > to
> > > > > >> > >> install
> > > > > >> > >> > >> two things individually (Flink and the web dashboard).
> > > > > >> > >> > >>
> > > > > >> > >> > >> How did the choice for Angular1 happen back then? Who
> > was
> > > > > >> writing
> > > > > >> > the
> > > > > >> > >> > >> Dashboard in the first place?
> > > > > >> > >> > >>
> > > > > >> > >> > >> Cheers
> > > > > >> > >> > >>
> > > > > >> > >> > >> --
> > > > > >> > >> > >>
> > > > > >> > >> > >>
> > > > > >> > >> > >> *Fabian WollertZalando SE*
> > > > > >> > >> > >>
> > > > > >> > >> > >> E-Mail: fabian@zalando.de
> > > > > >> > >> > >>
> > > > > >> > >> > >>
> > > > > >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till
> > Rohrmann <
> > > > > >> > >> > >> trohrmann@apache.org>:
> > > > > >> > >> > >>
> > > > > >> > >> > >>> Thanks for starting this discussion Fabian! I think
> our
> > > web
> > > > > UI
> > > > > >> > >> > technology
> > > > > >> > >> > >>> stack is quite dusty by now and it would be
> beneficial
> > to
> > > > > think
> > > > > >> > >> about
> > > > > >> > >> > its
> > > > > >> > >> > >>> technological future.
> > > > > >> > >> > >>>
> > > > > >> > >> > >>> On the one hand, our current web UI works more or
> less
> > > > > reliable
> > > > > >> > and
> > > > > >> > >> > >>> changing the underlying technology has the risk of
> > > breaking
> > > > > >> > things.
> > > > > >> > >> > >>> Moreover, there might be the risk that the newly
> chosen
> > > > > >> technology
> > > > > >> > >> > will be
> > > > > >> > >> > >>> deprecated at some point in time as well.
> > > > > >> > >> > >>>
> > > > > >> > >> > >>> On the other hand, we don't have much Angular 1
> > knowledge
> > > > in
> > > > > >> the
> > > > > >> > >> > community
> > > > > >> > >> > >>> and extending the web UI is, thus, quite hard at the
> > > > moment.
> > > > > >> Maybe
> > > > > >> > >> by
> > > > > >> > >> > using
> > > > > >> > >> > >>> some newer web technologies we might be able to
> attract
> > > > more
> > > > > >> > people
> > > > > >> > >> > with a
> > > > > >> > >> > >>> web technology background to join the community.
> > > > > >> > >> > >>>
> > > > > >> > >> > >>> The lack of people working on the web UI is for me
> the
> > > > > biggest
> > > > > >> > >> problem
> > > > > >> > >> > I
> > > > > >> > >> > >>> would like to address. If there is interest in the
> web
> > > UI,
> > > > > then
> > > > > >> > I'm
> > > > > >> > >> > quite
> > > > > >> > >> > >>> sure that we will be able to even migrate to some
> other
> > > > > >> technology
> > > > > >> > >> in
> > > > > >> > >> > the
> > > > > >> > >> > >>> future. The next important issue for me is to do the
> > > change
> > > > > >> > >> > incrementally
> > > > > >> > >> > >>> if possible. Ideally we never break the web UI in the
> > > > process
> > > > > >> of
> > > > > >> > >> > migrating
> > > > > >> > >> > >>> to a new technology. I'm not an expert here so it
> might
> > > or
> > > > > >> might
> > > > > >> > >> not be
> > > > > >> > >> > >>> possible. But if it is, then we should design the
> > > > > >> implementation
> > > > > >> > >> steps
> > > > > >> > >> > in
> > > > > >> > >> > >>> such a way.
> > > > > >> > >> > >>>
> > > > > >> > >> > >>> Cheers,
> > > > > >> > >> > >>> Till
> > > > > >> > >> > >>>
> > > > > >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <
> > > > > >> fabian@zalando.de
> > > > > >> > >
> > > > > >> > >> > wrote:
> > > > > >> > >> > >>>
> > > > > >> > >> > >>>> Hi everyone,
> > > > > >> > >> > >>>>
> > > > > >> > >> > >>>> in this email thread
> > > > > >> > >> > >>>> <
> > > > > >> > >> > >>>>
> > > > > >> > >> > >>>
> > > > > >> > >> >
> > > > > >> > >>
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > > > > >> > >> > >>>> and the tickets FLINK-10705
> > > > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705>
> > and
> > > > > >> > >> FLINK-10706
> > > > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706>
> > the
> > > > > >> > discussion
> > > > > >> > >> > came
> > > > > >> > >> > >>> up
> > > > > >> > >> > >>>> whether to change the underlying architecture of
> > Flink's
> > > > Web
> > > > > >> > >> Dashboard
> > > > > >> > >> > >>> from
> > > > > >> > >> > >>>> Angular1 to something else. This email thread should
> > be
> > > > > >> solely to
> > > > > >> > >> > discuss
> > > > > >> > >> > >>>> the pro's and con's of this, and what could be the
> > > target
> > > > > >> > >> > architecture.
> > > > > >> > >> > >>>>
> > > > > >> > >> > >>>> My choice would be React. Personally I agree with
> > Till's
> > > > > >> comments
> > > > > >> > >> in
> > > > > >> > >> > the
> > > > > >> > >> > >>>> ticket, Angular 1 being basically outdated and is
> not
> > > > > having a
> > > > > >> > >> large
> > > > > >> > >> > >>>> following anymore. From my experience the choice
> > between
> > > > > >> Angular
> > > > > >> > >> 2-7
> > > > > >> > >> > or
> > > > > >> > >> > >>>> React is subjective, you can get things done with
> > both.
> > > I
> > > > > >> > >> personally
> > > > > >> > >> > only
> > > > > >> > >> > >>>> have experience with React, so I personally would
> be
> > > > faster
> > > > > >> to
> > > > > >> > >> > develop
> > > > > >> > >> > >>>> with this one. I currently have not planned to learn
> > > > Angular
> > > > > >> as
> > > > > >> > >> well
> > > > > >> > >> > >>> (being
> > > > > >> > >> > >>>> a more backend focused developer in general) so if
> the
> > > > > >> decision
> > > > > >> > >> would
> > > > > >> > >> > be
> > > > > >> > >> > >>> to
> > > > > >> > >> > >>>> go with Angular, i would be unfortunately out of
> this
> > > > rework
> > > > > >> of
> > > > > >> > the
> > > > > >> > >> > Flink
> > > > > >> > >> > >>>> Dashboard most certainly.
> > > > > >> > >> > >>>>
> > > > > >> > >> > >>>> Additionally i would like to get rid of bower, since
> > its
> > > > > >> > officially
> > > > > >> > >> > >>>> deprecated <
> > > > > >> > >> >
> https://bower.io/blog/2017/how-to-migrate-away-from-bower/
> > >.
> > > > > >> > >> > >>>> my
> > > > > >> > >> > >>>> idea would be to just use a create-react-app package
> > > with
> > > > > npm
> > > > > >> and
> > > > > >> > >> > webpack
> > > > > >> > >> > >>>> under the hood. no need for additional lib's here
> > imho.
> > > > But
> > > > > >> > again:
> > > > > >> > >> > thats
> > > > > >> > >> > >>>> mostly what i've been working with recently, so
> thats
> > a
> > > > > >> > subjective
> > > > > >> > >> > >>> point. I
> > > > > >> > >> > >>>> could imagine getting used to yarn in the future as
> > > well.
> > > > > >> > >> > >>>>
> > > > > >> > >> > >>>> Cheers
> > > > > >> > >> > >>>> Fabian
> > > > > >> > >> > >>>>
> > > > > >> > >> > >>>> --
> > > > > >> > >> > >>>>
> > > > > >> > >> > >>>>
> > > > > >> > >> > >>>> *Fabian WollertZalando SE*
> > > > > >> > >> > >>>>
> > > > > >> > >> > >>>> E-Mail: fabian@zalando.de
> > > > > >> > >> > >>>>
> > > > > >> > >> >
> > > > > >> > >> >
> > > > > >> > >>
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > --
> > > > > >> > > Yadong Xie
> > > > > >> > > Ministry of Education Key Lab for Intelligent Networks and
> > > Network
> > > > > >> > Security
> > > > > >> > > Dept. of Computer Science and Technology
> > > > > >> > > School of electronic information and engineering
> > > > > >> > > Xi'an Jiaotong University
> > > > > >> > > Xi'an, 710049, P.R. China
> > > > > >> > > Email: vthinkxie@gmail.com
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Fabian Wollert <fa...@zalando.de>.
i updated this JIRA already, feel free to reuse this:
https://issues.apache.org/jira/browse/FLINK-10706
--
*Fabian WollertZalando SE*
E-Mail: fabian@zalando.de
Am Di., 6. Nov. 2018 um 12:10 Uhr schrieb Shaoxuan Wang <wshaoxuan@gmail.com
>:
> Till,
> Yes, it is a good idea to have a feature flag to switch the web UI before
> we completely deprecate the old one.
>
> Yadong,
> It seems that everyone likes the new web UI. Can you please open a master
> Jira and start to merge the code to Flink master. What do you think?
>
> Regards,
> Shaoxuan
>
> On Mon, Nov 5, 2018 at 7:04 PM Till Rohrmann <tr...@apache.org> wrote:
>
> > Thanks a lot for sharing the code with the community Yadong!
> >
> > It looks really cool and I also want to give it a try to see how easy it
> is
> > to start Flink with it.
> >
> > If it is already implemented and working, we could also think about
> adding
> > it to Flink and add a feature flag to switch between the old and new web
> > UI. We could think about enabling it by default to give it more user
> > exposure. After being confident and users having no complaints, we could
> > think about deprecating the old web UI and eventually to drop it. Of
> > course, initially we should give it a thorough review.
> >
> > Cheers,
> > Till
> >
> > On Mon, Nov 5, 2018 at 8:40 AM Fabian Wollert <fa...@zalando.de> wrote:
> >
> > > Hi Yadong, this is awesome, thx for the code! I will try it out on our
> > > infrastructure and will post my feedback here, latest next week.
> > >
> > > I will also check if my ideas for FLINK-10707
> > > <https://issues.apache.org/jira/browse/FLINK-10707> are doable with
> your
> > > code since this was what pushed this discussion initially.
> > >
> > > Cheers
> > >
> > > --
> > >
> > >
> > > *Fabian WollertZalando SE*
> > >
> > > E-Mail: fabian@zalando.de
> > >
> > >
> > > Am Mo., 5. Nov. 2018 um 04:42 Uhr schrieb Yadong Xie <
> > vthinkxie@gmail.com
> > > >:
> > >
> > > > Hi Fabian, Till, and Robert
> > > >
> > > > Thank you for your attention to this matter, I just push our codes to
> > > > github: https://github.com/vthinkxie/flink-runtime-web.
> > > >
> > > > You can start the project by following the guidelines
> > > > <
> https://github.com/vthinkxie/flink-runtime-web#development--debugging
> > >
> > > > (just
> > > > run `npm install && npm run proxy`), just feel free to give any
> > comments
> > > :)
> > > >
> > > > If I missed anything please let me know. Look forward to your
> feedback
> > > and
> > > > suggestions soon.
> > > >
> > > > Best regards
> > > > Yadong
> > > >
> > > >
> > > > On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fa...@zalando.de>
> > wrote:
> > > >
> > > > > Hi Yadong, this looks awesome. is there any chance you can already
> > > share
> > > > > the code of the new web UI, so we can take a look at what you guys
> > > build
> > > > > there? I think that would speed up the discussion. If there is
> > already
> > > a
> > > > > fully fledged new Version with everything updated out there, and
> its
> > > even
> > > > > battle tested in production already, that sounds like the way to go
> > for
> > > > me.
> > > > >
> > > > > i started aside from this discussion here (to strengthen and learn
> > some
> > > > > new React stuff) my attempt on the React version already, for
> whoever
> > > is
> > > > > curious, you can check it out here:
> > > > > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base
> > url
> > > in
> > > > > actions.js
> > > > > <
> > > >
> > >
> >
> https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8
> > > > >,
> > > > > npm install and then npm start) ... i just started to convert the
> > first
> > > > > simple pages, so dont expect the whole UI yet, its just a start ...
> > > > >
> > > > > Cheers
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > > *Fabian WollertZalando SE*
> > > > >
> > > > > E-Mail: fabian@zalando.de
> > > > >
> > > > >
> > > > > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <
> > zjffdu@gmail.com
> > > >:
> > > > >
> > > > >> +1 for the proposal of Yadong. I'd like to add one more comment:
> > > > >>
> > > > >> We should provide more development guidance when developing the
> next
> > > > >> dashboard via angular 7. As Till mentioned before, one big concern
> > is
> > > > lack
> > > > >> of frontend guys in flink community, I believe more development
> > > guidance
> > > > >> will attract more frontend guys to contribute to Flink web ui, and
> > > also
> > > > be
> > > > >> helpful for backend guys who want to add something to dashboard.
> > > > >>
> > > > >>
> > > > >>
> > > > >> Shaoxuan Wang <ws...@gmail.com>于2018年11月2日周五 上午10:06写道：
> > > > >>
> > > > >> > +1 to keep the web UI stack to Angular. This will make the stack
> > > > >> upgrading
> > > > >> > much easier and won't break the on-going efforts already made
> > > towards
> > > > >> the
> > > > >> > improvements of Flink Web UI.
> > > > >> >
> > > > >> > Till, Robert, and Fabian,
> > > > >> > Please take a look at the upgrading plan from Angular 1.x to 7.0
> > > > >> proposed
> > > > >> > by Yadong. Any feedback is appreciated.
> > > > >> >
> > > > >> > Regards,
> > > > >> > Shaoxuan
> > > > >> >
> > > > >> >
> > > > >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vt...@gmail.com>
> > > > wrote:
> > > > >> >
> > > > >> > > For your interest, here are some snapshots of Flink web UI
> (with
> > > > >> angular
> > > > >> > > 7.0) we recently reconstructed.
> > > > >> > > [image: snapshot.001.jpeg]
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
> > > > >> > >
> > > > >> > > We choose Angular 7.0 due to the following reasons:
> > > > >> > >
> > > > >> > > 1. It is easy to migrate the current codebase to Angular
> 7.0
> > > > >> following
> > > > >> > > the official guide.
> > > > >> > > 2. It is easy for current Web UI developers who are
> > experienced
> > > > >> with
> > > > >> > > Angular 1.x to get familiar with 7.0 reading the official
> > doc.
> > > > >> > > 3. Migrating other than rewriting is helpful to ensure its
> > > > >> functional
> > > > >> > > integrity.
> > > > >> > > 4. There are over 1 million Angular developers now (2.0 and
> > > > above,
> > > > >> > > still increasing rapidly), which means there are
> potentially
> > > more
> > > > >> > > developers willing to join the community.
> > > > >> > > 5. No bower or complex node / webpack configuration needed
> > > (only
> > > > >> > > angular cli).
> > > > >> > >
> > > > >> > > We split Angular 1.x and 7.0 versions into two folders based
> on
> > > the
> > > > >> > > following consideration:
> > > > >> > >
> > > > >> > > 1. 1.x version could be the default page because there is
> no
> > > > >> > > breakchange for both users and developers.
> > > > >> > > 2. Make the upgrade progressive. Users can switch between
> two
> > > > >> versions
> > > > >> > > by clicking the button on the top right corner.
> > > > >> > > 3. Reduce maintenance costs and keep the performance of web
> > > > without
> > > > >> > > bootstrap both 1.x and 7.0 versions in the same page.
> > > > >> > > 4. Won't interrupt the current development process. The
> code
> > > can
> > > > >> still
> > > > >> > > be submitted to 1.x version folder, we could help to sync
> all
> > > > >> > features and
> > > > >> > > bug fix to 7.0 version if needed.
> > > > >> > > 5. When the 7.0 version is ready and stable, it is easy to
> > > change
> > > > >> > > default page to it, and remove the old version.
> > > > >> > >
> > > > >> > > [image: test.jpg]
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
> > > > >> > >
> > > > >> > > Here is the brief list of what we have done:
> > > > >> > >
> > > > >> > > 1. Create a new folder named next-web-dashboard in
> > > > >> flink-runtime-web.
> > > > >> > > 2. Migrate all Angular 1.x controllers, services, config
> etc
> > > one
> > > > by
> > > > >> > > one into 7.0 version with ngMigration.
> > > > >> > > 3. Modify frontend build scripts in order to package two
> > > versions
> > > > >> > > together.
> > > > >> > > 4. All works listed above are limited in the
> > flink-runtime-web
> > > > >> folder.
> > > > >> > > One folder was added and no file or folder was deleted.
> > > > >> > >
> > > > >> > > The new version of the Web UI is running stable for several
> > months
> > > > in
> > > > >> > > Alibaba. We are happy to contribute it to the Flink community
> if
> > > > >> everyone
> > > > >> > > like the work.
> > > > >> > >
> > > > >> > > Best
> > > > >> > > Yadong
> > > > >> > >
> > > > >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <
> > > trohrmann@apache.org
> > > > >
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > >> I agree with Chesnay that the web UI is already fairly
> isolated
> > > in
> > > > >> > Flink's
> > > > >> > >> code base (see flink-runtime-web module). Moreover, imo it is
> > > very
> > > > >> > >> important that Flink comes with a nice web UI which works out
> > of
> > > > the
> > > > >> box
> > > > >> > >> because it lowers the entrance hurdle massively. What one
> could
> > > > >> argue is
> > > > >> > >> to
> > > > >> > >> make it easier to plug in your own web UI if you want to
> > develop
> > > > it.
> > > > >> > But I
> > > > >> > >> would see this as an orthogonal issue.
> > > > >> > >>
> > > > >> > >> I would actually argue against Robert's proposal to make a
> > grand
> > > > >> rewrite
> > > > >> > >> if
> > > > >> > >> this is by any means possible. Experience shows that these
> kind
> > > of
> > > > >> > efforts
> > > > >> > >> are usually set up to fail or take super long. If possible,
> > then
> > > I
> > > > >> would
> > > > >> > >> rather prefer to convert one component at a time to React.
> But
> > > I'm
> > > > >> not
> > > > >> > an
> > > > >> > >> expert here and cannot tell whether this is possible or not.
> > > > >> > >>
> > > > >> > >> What we would need is active help from the community to
> review
> > > > these
> > > > >> > >> changes. I think that is the crucial piece to make such an
> > effort
> > > > >> work
> > > > >> > or
> > > > >> > >> fail.
> > > > >> > >>
> > > > >> > >> Cheers,
> > > > >> > >> Till
> > > > >> > >>
> > > > >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <
> > > > >> chesnay@apache.org>
> > > > >> > >> wrote:
> > > > >> > >>
> > > > >> > >> > The WebUI goes entirely through the REST API.
> > > > >> > >> >
> > > > >> > >> > I don't see how moving it out would make it easier to
> > > > >> > >> develop/contribute.
> > > > >> > >> >
> > > > >> > >> > The UI is already encapsulated in it's own module
> > > > >> (flink-runtime-web)
> > > > >> > >> > and changes can be made to it independently.
> > > > >> > >> > Historically the issue was simply that contributions to the
> > UI
> > > > were
> > > > >> > not
> > > > >> > >> > reviewed in a timely manner / not at all, which mind you
> > > > >> > >> > applies to virtually all components. This also won't change
> > by
> > > > >> moving
> > > > >> > it
> > > > >> > >> > out of Flink.
> > > > >> > >> >
> > > > >> > >> > Having it in Flink also guarantees that there is an
> > > > >> > easily-discoverable
> > > > >> > >> > version of the UI that is compatible with your Flink
> > > > distribution.
> > > > >> > >> >
> > > > >> > >> > On 31.10.2018 09:53, dyana.rose wrote:
> > > > >> > >> > > Re: who's using the web ui
> > > > >> > >> > >
> > > > >> > >> > > Though many mature solutions, with a fair amount of
> > > > >> time/resources
> > > > >> > >> > available are likely running their own front ends, for
> teams
> > > like
> > > > >> mine
> > > > >> > >> > which are smaller and aren't focused solely on working with
> > > > Flink,
> > > > >> > >> having
> > > > >> > >> > the web ui available removes a large barrier to getting up
> > and
> > > > >> running
> > > > >> > >> > quickly.
> > > > >> > >> > >
> > > > >> > >> > > But, with the web UI being in core Flink, it does make
> it a
> > > bit
> > > > >> more
> > > > >> > >> of
> > > > >> > >> > a chore to contribute.
> > > > >> > >> > >
> > > > >> > >> > > I'm unaware of how necessary it is for the UI to deploy
> > with
> > > > >> Flink?
> > > > >> > Is
> > > > >> > >> > it making any calls that aren't through the REST API (and
> if
> > > so,
> > > > >> can
> > > > >> > >> those
> > > > >> > >> > endpoints be added to the REST API)?
> > > > >> > >> > >
> > > > >> > >> > > In general I'd support moving it to its own package,
> making
> > > it
> > > > >> > easier
> > > > >> > >> to
> > > > >> > >> > develop with your standard UI dev tools. This also allows
> the
> > > web
> > > > >> UI
> > > > >> > to
> > > > >> > >> be
> > > > >> > >> > developed and released independently to core Flink.
> > > > >> > >> > >
> > > > >> > >> > > Dyana
> > > > >> > >> > >
> > > > >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <
> fabian@zalando.de>
> > > > >> wrote:
> > > > >> > >> > >> Hi Till, I basically agree with all your points. i would
> > > > stress
> > > > >> the
> > > > >> > >> > >> "dustiness" of the current architecture: the package
> > manager
> > > > >> used
> > > > >> > >> > (bower)
> > > > >> > >> > >> is deprecated since a long time, the chance for the
> builds
> > > of
> > > > >> the
> > > > >> > >> flink
> > > > >> > >> > web
> > > > >> > >> > >> dashboard not working anymore is increasing every day.
> > > > >> > >> > >>
> > > > >> > >> > >> About the knowledge in the community: Two days is not a
> > lot
> > > of
> > > > >> > time,
> > > > >> > >> but
> > > > >> > >> > >> interest in this topic seems to be minor anyways. Is
> > someone
> > > > >> using
> > > > >> > >> the
> > > > >> > >> > >> Flink Web Dashboard at all, or is everyone running their
> > own
> > > > >> custom
> > > > >> > >> > >> solutions? Because if there is no interest in using the
> > Web
> > > UI
> > > > >> AND
> > > > >> > no
> > > > >> > >> > one
> > > > >> > >> > >> interested in developing, there would be no need to
> > package
> > > > >> this as
> > > > >> > >> > part of
> > > > >> > >> > >> the official Flink package, but rather develop an
> > > independent
> > > > >> > >> solution
> > > > >> > >> > (I'm
> > > > >> > >> > >> not voting for this right now, just putting it out), if
> at
> > > > all.
> > > > >> The
> > > > >> > >> > >> official package could then just ship with the API,
> which
> > > > other
> > > > >> > >> > solutions
> > > > >> > >> > >> can build upon. This solution could be from an agile
> point
> > > of
> > > > >> view
> > > > >> > >> also
> > > > >> > >> > the
> > > > >> > >> > >> best (enhanced testing, independent and more effective
> dev
> > > > >> > workflow,
> > > > >> > >> > etc.),
> > > > >> > >> > >> but is bad for the usage of the Flink UI, because people
> > > need
> > > > to
> > > > >> > >> install
> > > > >> > >> > >> two things individually (Flink and the web dashboard).
> > > > >> > >> > >>
> > > > >> > >> > >> How did the choice for Angular1 happen back then? Who
> was
> > > > >> writing
> > > > >> > the
> > > > >> > >> > >> Dashboard in the first place?
> > > > >> > >> > >>
> > > > >> > >> > >> Cheers
> > > > >> > >> > >>
> > > > >> > >> > >> --
> > > > >> > >> > >>
> > > > >> > >> > >>
> > > > >> > >> > >> *Fabian WollertZalando SE*
> > > > >> > >> > >>
> > > > >> > >> > >> E-Mail: fabian@zalando.de
> > > > >> > >> > >>
> > > > >> > >> > >>
> > > > >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till
> Rohrmann <
> > > > >> > >> > >> trohrmann@apache.org>:
> > > > >> > >> > >>
> > > > >> > >> > >>> Thanks for starting this discussion Fabian! I think our
> > web
> > > > UI
> > > > >> > >> > technology
> > > > >> > >> > >>> stack is quite dusty by now and it would be beneficial
> to
> > > > think
> > > > >> > >> about
> > > > >> > >> > its
> > > > >> > >> > >>> technological future.
> > > > >> > >> > >>>
> > > > >> > >> > >>> On the one hand, our current web UI works more or less
> > > > reliable
> > > > >> > and
> > > > >> > >> > >>> changing the underlying technology has the risk of
> > breaking
> > > > >> > things.
> > > > >> > >> > >>> Moreover, there might be the risk that the newly chosen
> > > > >> technology
> > > > >> > >> > will be
> > > > >> > >> > >>> deprecated at some point in time as well.
> > > > >> > >> > >>>
> > > > >> > >> > >>> On the other hand, we don't have much Angular 1
> knowledge
> > > in
> > > > >> the
> > > > >> > >> > community
> > > > >> > >> > >>> and extending the web UI is, thus, quite hard at the
> > > moment.
> > > > >> Maybe
> > > > >> > >> by
> > > > >> > >> > using
> > > > >> > >> > >>> some newer web technologies we might be able to attract
> > > more
> > > > >> > people
> > > > >> > >> > with a
> > > > >> > >> > >>> web technology background to join the community.
> > > > >> > >> > >>>
> > > > >> > >> > >>> The lack of people working on the web UI is for me the
> > > > biggest
> > > > >> > >> problem
> > > > >> > >> > I
> > > > >> > >> > >>> would like to address. If there is interest in the web
> > UI,
> > > > then
> > > > >> > I'm
> > > > >> > >> > quite
> > > > >> > >> > >>> sure that we will be able to even migrate to some other
> > > > >> technology
> > > > >> > >> in
> > > > >> > >> > the
> > > > >> > >> > >>> future. The next important issue for me is to do the
> > change
> > > > >> > >> > incrementally
> > > > >> > >> > >>> if possible. Ideally we never break the web UI in the
> > > process
> > > > >> of
> > > > >> > >> > migrating
> > > > >> > >> > >>> to a new technology. I'm not an expert here so it might
> > or
> > > > >> might
> > > > >> > >> not be
> > > > >> > >> > >>> possible. But if it is, then we should design the
> > > > >> implementation
> > > > >> > >> steps
> > > > >> > >> > in
> > > > >> > >> > >>> such a way.
> > > > >> > >> > >>>
> > > > >> > >> > >>> Cheers,
> > > > >> > >> > >>> Till
> > > > >> > >> > >>>
> > > > >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <
> > > > >> fabian@zalando.de
> > > > >> > >
> > > > >> > >> > wrote:
> > > > >> > >> > >>>
> > > > >> > >> > >>>> Hi everyone,
> > > > >> > >> > >>>>
> > > > >> > >> > >>>> in this email thread
> > > > >> > >> > >>>> <
> > > > >> > >> > >>>>
> > > > >> > >> > >>>
> > > > >> > >> >
> > > > >> > >>
> > > > >> >
> > > > >>
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > > > >> > >> > >>>> and the tickets FLINK-10705
> > > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705>
> and
> > > > >> > >> FLINK-10706
> > > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706>
> the
> > > > >> > discussion
> > > > >> > >> > came
> > > > >> > >> > >>> up
> > > > >> > >> > >>>> whether to change the underlying architecture of
> Flink's
> > > Web
> > > > >> > >> Dashboard
> > > > >> > >> > >>> from
> > > > >> > >> > >>>> Angular1 to something else. This email thread should
> be
> > > > >> solely to
> > > > >> > >> > discuss
> > > > >> > >> > >>>> the pro's and con's of this, and what could be the
> > target
> > > > >> > >> > architecture.
> > > > >> > >> > >>>>
> > > > >> > >> > >>>> My choice would be React. Personally I agree with
> Till's
> > > > >> comments
> > > > >> > >> in
> > > > >> > >> > the
> > > > >> > >> > >>>> ticket, Angular 1 being basically outdated and is not
> > > > having a
> > > > >> > >> large
> > > > >> > >> > >>>> following anymore. From my experience the choice
> between
> > > > >> Angular
> > > > >> > >> 2-7
> > > > >> > >> > or
> > > > >> > >> > >>>> React is subjective, you can get things done with
> both.
> > I
> > > > >> > >> personally
> > > > >> > >> > only
> > > > >> > >> > >>>> have experience with React, so I personally would be
> > > faster
> > > > >> to
> > > > >> > >> > develop
> > > > >> > >> > >>>> with this one. I currently have not planned to learn
> > > Angular
> > > > >> as
> > > > >> > >> well
> > > > >> > >> > >>> (being
> > > > >> > >> > >>>> a more backend focused developer in general) so if the
> > > > >> decision
> > > > >> > >> would
> > > > >> > >> > be
> > > > >> > >> > >>> to
> > > > >> > >> > >>>> go with Angular, i would be unfortunately out of this
> > > rework
> > > > >> of
> > > > >> > the
> > > > >> > >> > Flink
> > > > >> > >> > >>>> Dashboard most certainly.
> > > > >> > >> > >>>>
> > > > >> > >> > >>>> Additionally i would like to get rid of bower, since
> its
> > > > >> > officially
> > > > >> > >> > >>>> deprecated <
> > > > >> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/
> >.
> > > > >> > >> > >>>> my
> > > > >> > >> > >>>> idea would be to just use a create-react-app package
> > with
> > > > npm
> > > > >> and
> > > > >> > >> > webpack
> > > > >> > >> > >>>> under the hood. no need for additional lib's here
> imho.
> > > But
> > > > >> > again:
> > > > >> > >> > thats
> > > > >> > >> > >>>> mostly what i've been working with recently, so thats
> a
> > > > >> > subjective
> > > > >> > >> > >>> point. I
> > > > >> > >> > >>>> could imagine getting used to yarn in the future as
> > well.
> > > > >> > >> > >>>>
> > > > >> > >> > >>>> Cheers
> > > > >> > >> > >>>> Fabian
> > > > >> > >> > >>>>
> > > > >> > >> > >>>> --
> > > > >> > >> > >>>>
> > > > >> > >> > >>>>
> > > > >> > >> > >>>> *Fabian WollertZalando SE*
> > > > >> > >> > >>>>
> > > > >> > >> > >>>> E-Mail: fabian@zalando.de
> > > > >> > >> > >>>>
> > > > >> > >> >
> > > > >> > >> >
> > > > >> > >>
> > > > >> > >
> > > > >> > >
> > > > >> > > --
> > > > >> > > Yadong Xie
> > > > >> > > Ministry of Education Key Lab for Intelligent Networks and
> > Network
> > > > >> > Security
> > > > >> > > Dept. of Computer Science and Technology
> > > > >> > > School of electronic information and engineering
> > > > >> > > Xi'an Jiaotong University
> > > > >> > > Xi'an, 710049, P.R. China
> > > > >> > > Email: vthinkxie@gmail.com
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Shaoxuan Wang <ws...@gmail.com>.
Till,
Yes, it is a good idea to have a feature flag to switch the web UI before
we completely deprecate the old one.
Yadong,
It seems that everyone likes the new web UI. Can you please open a master
Jira and start to merge the code to Flink master. What do you think?
Regards,
Shaoxuan
On Mon, Nov 5, 2018 at 7:04 PM Till Rohrmann <tr...@apache.org> wrote:
> Thanks a lot for sharing the code with the community Yadong!
>
> It looks really cool and I also want to give it a try to see how easy it is
> to start Flink with it.
>
> If it is already implemented and working, we could also think about adding
> it to Flink and add a feature flag to switch between the old and new web
> UI. We could think about enabling it by default to give it more user
> exposure. After being confident and users having no complaints, we could
> think about deprecating the old web UI and eventually to drop it. Of
> course, initially we should give it a thorough review.
>
> Cheers,
> Till
>
> On Mon, Nov 5, 2018 at 8:40 AM Fabian Wollert <fa...@zalando.de> wrote:
>
> > Hi Yadong, this is awesome, thx for the code! I will try it out on our
> > infrastructure and will post my feedback here, latest next week.
> >
> > I will also check if my ideas for FLINK-10707
> > <https://issues.apache.org/jira/browse/FLINK-10707> are doable with your
> > code since this was what pushed this discussion initially.
> >
> > Cheers
> >
> > --
> >
> >
> > *Fabian WollertZalando SE*
> >
> > E-Mail: fabian@zalando.de
> >
> >
> > Am Mo., 5. Nov. 2018 um 04:42 Uhr schrieb Yadong Xie <
> vthinkxie@gmail.com
> > >:
> >
> > > Hi Fabian, Till, and Robert
> > >
> > > Thank you for your attention to this matter, I just push our codes to
> > > github: https://github.com/vthinkxie/flink-runtime-web.
> > >
> > > You can start the project by following the guidelines
> > > <https://github.com/vthinkxie/flink-runtime-web#development--debugging
> >
> > > (just
> > > run `npm install && npm run proxy`), just feel free to give any
> comments
> > :)
> > >
> > > If I missed anything please let me know. Look forward to your feedback
> > and
> > > suggestions soon.
> > >
> > > Best regards
> > > Yadong
> > >
> > >
> > > On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fa...@zalando.de>
> wrote:
> > >
> > > > Hi Yadong, this looks awesome. is there any chance you can already
> > share
> > > > the code of the new web UI, so we can take a look at what you guys
> > build
> > > > there? I think that would speed up the discussion. If there is
> already
> > a
> > > > fully fledged new Version with everything updated out there, and its
> > even
> > > > battle tested in production already, that sounds like the way to go
> for
> > > me.
> > > >
> > > > i started aside from this discussion here (to strengthen and learn
> some
> > > > new React stuff) my attempt on the React version already, for whoever
> > is
> > > > curious, you can check it out here:
> > > > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base
> url
> > in
> > > > actions.js
> > > > <
> > >
> >
> https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8
> > > >,
> > > > npm install and then npm start) ... i just started to convert the
> first
> > > > simple pages, so dont expect the whole UI yet, its just a start ...
> > > >
> > > > Cheers
> > > >
> > > > --
> > > >
> > > >
> > > > *Fabian WollertZalando SE*
> > > >
> > > > E-Mail: fabian@zalando.de
> > > >
> > > >
> > > > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <
> zjffdu@gmail.com
> > >:
> > > >
> > > >> +1 for the proposal of Yadong. I'd like to add one more comment:
> > > >>
> > > >> We should provide more development guidance when developing the next
> > > >> dashboard via angular 7. As Till mentioned before, one big concern
> is
> > > lack
> > > >> of frontend guys in flink community, I believe more development
> > guidance
> > > >> will attract more frontend guys to contribute to Flink web ui, and
> > also
> > > be
> > > >> helpful for backend guys who want to add something to dashboard.
> > > >>
> > > >>
> > > >>
> > > >> Shaoxuan Wang <ws...@gmail.com>于2018年11月2日周五 上午10:06写道：
> > > >>
> > > >> > +1 to keep the web UI stack to Angular. This will make the stack
> > > >> upgrading
> > > >> > much easier and won't break the on-going efforts already made
> > towards
> > > >> the
> > > >> > improvements of Flink Web UI.
> > > >> >
> > > >> > Till, Robert, and Fabian,
> > > >> > Please take a look at the upgrading plan from Angular 1.x to 7.0
> > > >> proposed
> > > >> > by Yadong. Any feedback is appreciated.
> > > >> >
> > > >> > Regards,
> > > >> > Shaoxuan
> > > >> >
> > > >> >
> > > >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vt...@gmail.com>
> > > wrote:
> > > >> >
> > > >> > > For your interest, here are some snapshots of Flink web UI (with
> > > >> angular
> > > >> > > 7.0) we recently reconstructed.
> > > >> > > [image: snapshot.001.jpeg]
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
> > > >> > >
> > > >> > > We choose Angular 7.0 due to the following reasons:
> > > >> > >
> > > >> > > 1. It is easy to migrate the current codebase to Angular 7.0
> > > >> following
> > > >> > > the official guide.
> > > >> > > 2. It is easy for current Web UI developers who are
> experienced
> > > >> with
> > > >> > > Angular 1.x to get familiar with 7.0 reading the official
> doc.
> > > >> > > 3. Migrating other than rewriting is helpful to ensure its
> > > >> functional
> > > >> > > integrity.
> > > >> > > 4. There are over 1 million Angular developers now (2.0 and
> > > above,
> > > >> > > still increasing rapidly), which means there are potentially
> > more
> > > >> > > developers willing to join the community.
> > > >> > > 5. No bower or complex node / webpack configuration needed
> > (only
> > > >> > > angular cli).
> > > >> > >
> > > >> > > We split Angular 1.x and 7.0 versions into two folders based on
> > the
> > > >> > > following consideration:
> > > >> > >
> > > >> > > 1. 1.x version could be the default page because there is no
> > > >> > > breakchange for both users and developers.
> > > >> > > 2. Make the upgrade progressive. Users can switch between two
> > > >> versions
> > > >> > > by clicking the button on the top right corner.
> > > >> > > 3. Reduce maintenance costs and keep the performance of web
> > > without
> > > >> > > bootstrap both 1.x and 7.0 versions in the same page.
> > > >> > > 4. Won't interrupt the current development process. The code
> > can
> > > >> still
> > > >> > > be submitted to 1.x version folder, we could help to sync all
> > > >> > features and
> > > >> > > bug fix to 7.0 version if needed.
> > > >> > > 5. When the 7.0 version is ready and stable, it is easy to
> > change
> > > >> > > default page to it, and remove the old version.
> > > >> > >
> > > >> > > [image: test.jpg]
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
> > > >> > >
> > > >> > > Here is the brief list of what we have done:
> > > >> > >
> > > >> > > 1. Create a new folder named next-web-dashboard in
> > > >> flink-runtime-web.
> > > >> > > 2. Migrate all Angular 1.x controllers, services, config etc
> > one
> > > by
> > > >> > > one into 7.0 version with ngMigration.
> > > >> > > 3. Modify frontend build scripts in order to package two
> > versions
> > > >> > > together.
> > > >> > > 4. All works listed above are limited in the
> flink-runtime-web
> > > >> folder.
> > > >> > > One folder was added and no file or folder was deleted.
> > > >> > >
> > > >> > > The new version of the Web UI is running stable for several
> months
> > > in
> > > >> > > Alibaba. We are happy to contribute it to the Flink community if
> > > >> everyone
> > > >> > > like the work.
> > > >> > >
> > > >> > > Best
> > > >> > > Yadong
> > > >> > >
> > > >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <
> > trohrmann@apache.org
> > > >
> > > >> > > wrote:
> > > >> > >
> > > >> > >> I agree with Chesnay that the web UI is already fairly isolated
> > in
> > > >> > Flink's
> > > >> > >> code base (see flink-runtime-web module). Moreover, imo it is
> > very
> > > >> > >> important that Flink comes with a nice web UI which works out
> of
> > > the
> > > >> box
> > > >> > >> because it lowers the entrance hurdle massively. What one could
> > > >> argue is
> > > >> > >> to
> > > >> > >> make it easier to plug in your own web UI if you want to
> develop
> > > it.
> > > >> > But I
> > > >> > >> would see this as an orthogonal issue.
> > > >> > >>
> > > >> > >> I would actually argue against Robert's proposal to make a
> grand
> > > >> rewrite
> > > >> > >> if
> > > >> > >> this is by any means possible. Experience shows that these kind
> > of
> > > >> > efforts
> > > >> > >> are usually set up to fail or take super long. If possible,
> then
> > I
> > > >> would
> > > >> > >> rather prefer to convert one component at a time to React. But
> > I'm
> > > >> not
> > > >> > an
> > > >> > >> expert here and cannot tell whether this is possible or not.
> > > >> > >>
> > > >> > >> What we would need is active help from the community to review
> > > these
> > > >> > >> changes. I think that is the crucial piece to make such an
> effort
> > > >> work
> > > >> > or
> > > >> > >> fail.
> > > >> > >>
> > > >> > >> Cheers,
> > > >> > >> Till
> > > >> > >>
> > > >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <
> > > >> chesnay@apache.org>
> > > >> > >> wrote:
> > > >> > >>
> > > >> > >> > The WebUI goes entirely through the REST API.
> > > >> > >> >
> > > >> > >> > I don't see how moving it out would make it easier to
> > > >> > >> develop/contribute.
> > > >> > >> >
> > > >> > >> > The UI is already encapsulated in it's own module
> > > >> (flink-runtime-web)
> > > >> > >> > and changes can be made to it independently.
> > > >> > >> > Historically the issue was simply that contributions to the
> UI
> > > were
> > > >> > not
> > > >> > >> > reviewed in a timely manner / not at all, which mind you
> > > >> > >> > applies to virtually all components. This also won't change
> by
> > > >> moving
> > > >> > it
> > > >> > >> > out of Flink.
> > > >> > >> >
> > > >> > >> > Having it in Flink also guarantees that there is an
> > > >> > easily-discoverable
> > > >> > >> > version of the UI that is compatible with your Flink
> > > distribution.
> > > >> > >> >
> > > >> > >> > On 31.10.2018 09:53, dyana.rose wrote:
> > > >> > >> > > Re: who's using the web ui
> > > >> > >> > >
> > > >> > >> > > Though many mature solutions, with a fair amount of
> > > >> time/resources
> > > >> > >> > available are likely running their own front ends, for teams
> > like
> > > >> mine
> > > >> > >> > which are smaller and aren't focused solely on working with
> > > Flink,
> > > >> > >> having
> > > >> > >> > the web ui available removes a large barrier to getting up
> and
> > > >> running
> > > >> > >> > quickly.
> > > >> > >> > >
> > > >> > >> > > But, with the web UI being in core Flink, it does make it a
> > bit
> > > >> more
> > > >> > >> of
> > > >> > >> > a chore to contribute.
> > > >> > >> > >
> > > >> > >> > > I'm unaware of how necessary it is for the UI to deploy
> with
> > > >> Flink?
> > > >> > Is
> > > >> > >> > it making any calls that aren't through the REST API (and if
> > so,
> > > >> can
> > > >> > >> those
> > > >> > >> > endpoints be added to the REST API)?
> > > >> > >> > >
> > > >> > >> > > In general I'd support moving it to its own package, making
> > it
> > > >> > easier
> > > >> > >> to
> > > >> > >> > develop with your standard UI dev tools. This also allows the
> > web
> > > >> UI
> > > >> > to
> > > >> > >> be
> > > >> > >> > developed and released independently to core Flink.
> > > >> > >> > >
> > > >> > >> > > Dyana
> > > >> > >> > >
> > > >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de>
> > > >> wrote:
> > > >> > >> > >> Hi Till, I basically agree with all your points. i would
> > > stress
> > > >> the
> > > >> > >> > >> "dustiness" of the current architecture: the package
> manager
> > > >> used
> > > >> > >> > (bower)
> > > >> > >> > >> is deprecated since a long time, the chance for the builds
> > of
> > > >> the
> > > >> > >> flink
> > > >> > >> > web
> > > >> > >> > >> dashboard not working anymore is increasing every day.
> > > >> > >> > >>
> > > >> > >> > >> About the knowledge in the community: Two days is not a
> lot
> > of
> > > >> > time,
> > > >> > >> but
> > > >> > >> > >> interest in this topic seems to be minor anyways. Is
> someone
> > > >> using
> > > >> > >> the
> > > >> > >> > >> Flink Web Dashboard at all, or is everyone running their
> own
> > > >> custom
> > > >> > >> > >> solutions? Because if there is no interest in using the
> Web
> > UI
> > > >> AND
> > > >> > no
> > > >> > >> > one
> > > >> > >> > >> interested in developing, there would be no need to
> package
> > > >> this as
> > > >> > >> > part of
> > > >> > >> > >> the official Flink package, but rather develop an
> > independent
> > > >> > >> solution
> > > >> > >> > (I'm
> > > >> > >> > >> not voting for this right now, just putting it out), if at
> > > all.
> > > >> The
> > > >> > >> > >> official package could then just ship with the API, which
> > > other
> > > >> > >> > solutions
> > > >> > >> > >> can build upon. This solution could be from an agile point
> > of
> > > >> view
> > > >> > >> also
> > > >> > >> > the
> > > >> > >> > >> best (enhanced testing, independent and more effective dev
> > > >> > workflow,
> > > >> > >> > etc.),
> > > >> > >> > >> but is bad for the usage of the Flink UI, because people
> > need
> > > to
> > > >> > >> install
> > > >> > >> > >> two things individually (Flink and the web dashboard).
> > > >> > >> > >>
> > > >> > >> > >> How did the choice for Angular1 happen back then? Who was
> > > >> writing
> > > >> > the
> > > >> > >> > >> Dashboard in the first place?
> > > >> > >> > >>
> > > >> > >> > >> Cheers
> > > >> > >> > >>
> > > >> > >> > >> --
> > > >> > >> > >>
> > > >> > >> > >>
> > > >> > >> > >> *Fabian WollertZalando SE*
> > > >> > >> > >>
> > > >> > >> > >> E-Mail: fabian@zalando.de
> > > >> > >> > >>
> > > >> > >> > >>
> > > >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> > > >> > >> > >> trohrmann@apache.org>:
> > > >> > >> > >>
> > > >> > >> > >>> Thanks for starting this discussion Fabian! I think our
> web
> > > UI
> > > >> > >> > technology
> > > >> > >> > >>> stack is quite dusty by now and it would be beneficial to
> > > think
> > > >> > >> about
> > > >> > >> > its
> > > >> > >> > >>> technological future.
> > > >> > >> > >>>
> > > >> > >> > >>> On the one hand, our current web UI works more or less
> > > reliable
> > > >> > and
> > > >> > >> > >>> changing the underlying technology has the risk of
> breaking
> > > >> > things.
> > > >> > >> > >>> Moreover, there might be the risk that the newly chosen
> > > >> technology
> > > >> > >> > will be
> > > >> > >> > >>> deprecated at some point in time as well.
> > > >> > >> > >>>
> > > >> > >> > >>> On the other hand, we don't have much Angular 1 knowledge
> > in
> > > >> the
> > > >> > >> > community
> > > >> > >> > >>> and extending the web UI is, thus, quite hard at the
> > moment.
> > > >> Maybe
> > > >> > >> by
> > > >> > >> > using
> > > >> > >> > >>> some newer web technologies we might be able to attract
> > more
> > > >> > people
> > > >> > >> > with a
> > > >> > >> > >>> web technology background to join the community.
> > > >> > >> > >>>
> > > >> > >> > >>> The lack of people working on the web UI is for me the
> > > biggest
> > > >> > >> problem
> > > >> > >> > I
> > > >> > >> > >>> would like to address. If there is interest in the web
> UI,
> > > then
> > > >> > I'm
> > > >> > >> > quite
> > > >> > >> > >>> sure that we will be able to even migrate to some other
> > > >> technology
> > > >> > >> in
> > > >> > >> > the
> > > >> > >> > >>> future. The next important issue for me is to do the
> change
> > > >> > >> > incrementally
> > > >> > >> > >>> if possible. Ideally we never break the web UI in the
> > process
> > > >> of
> > > >> > >> > migrating
> > > >> > >> > >>> to a new technology. I'm not an expert here so it might
> or
> > > >> might
> > > >> > >> not be
> > > >> > >> > >>> possible. But if it is, then we should design the
> > > >> implementation
> > > >> > >> steps
> > > >> > >> > in
> > > >> > >> > >>> such a way.
> > > >> > >> > >>>
> > > >> > >> > >>> Cheers,
> > > >> > >> > >>> Till
> > > >> > >> > >>>
> > > >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <
> > > >> fabian@zalando.de
> > > >> > >
> > > >> > >> > wrote:
> > > >> > >> > >>>
> > > >> > >> > >>>> Hi everyone,
> > > >> > >> > >>>>
> > > >> > >> > >>>> in this email thread
> > > >> > >> > >>>> <
> > > >> > >> > >>>>
> > > >> > >> > >>>
> > > >> > >> >
> > > >> > >>
> > > >> >
> > > >>
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > > >> > >> > >>>> and the tickets FLINK-10705
> > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and
> > > >> > >> FLINK-10706
> > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the
> > > >> > discussion
> > > >> > >> > came
> > > >> > >> > >>> up
> > > >> > >> > >>>> whether to change the underlying architecture of Flink's
> > Web
> > > >> > >> Dashboard
> > > >> > >> > >>> from
> > > >> > >> > >>>> Angular1 to something else. This email thread should be
> > > >> solely to
> > > >> > >> > discuss
> > > >> > >> > >>>> the pro's and con's of this, and what could be the
> target
> > > >> > >> > architecture.
> > > >> > >> > >>>>
> > > >> > >> > >>>> My choice would be React. Personally I agree with Till's
> > > >> comments
> > > >> > >> in
> > > >> > >> > the
> > > >> > >> > >>>> ticket, Angular 1 being basically outdated and is not
> > > having a
> > > >> > >> large
> > > >> > >> > >>>> following anymore. From my experience the choice between
> > > >> Angular
> > > >> > >> 2-7
> > > >> > >> > or
> > > >> > >> > >>>> React is subjective, you can get things done with both.
> I
> > > >> > >> personally
> > > >> > >> > only
> > > >> > >> > >>>> have experience with React, so I personally would be
> > faster
> > > >> to
> > > >> > >> > develop
> > > >> > >> > >>>> with this one. I currently have not planned to learn
> > Angular
> > > >> as
> > > >> > >> well
> > > >> > >> > >>> (being
> > > >> > >> > >>>> a more backend focused developer in general) so if the
> > > >> decision
> > > >> > >> would
> > > >> > >> > be
> > > >> > >> > >>> to
> > > >> > >> > >>>> go with Angular, i would be unfortunately out of this
> > rework
> > > >> of
> > > >> > the
> > > >> > >> > Flink
> > > >> > >> > >>>> Dashboard most certainly.
> > > >> > >> > >>>>
> > > >> > >> > >>>> Additionally i would like to get rid of bower, since its
> > > >> > officially
> > > >> > >> > >>>> deprecated <
> > > >> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> > > >> > >> > >>>> my
> > > >> > >> > >>>> idea would be to just use a create-react-app package
> with
> > > npm
> > > >> and
> > > >> > >> > webpack
> > > >> > >> > >>>> under the hood. no need for additional lib's here imho.
> > But
> > > >> > again:
> > > >> > >> > thats
> > > >> > >> > >>>> mostly what i've been working with recently, so thats a
> > > >> > subjective
> > > >> > >> > >>> point. I
> > > >> > >> > >>>> could imagine getting used to yarn in the future as
> well.
> > > >> > >> > >>>>
> > > >> > >> > >>>> Cheers
> > > >> > >> > >>>> Fabian
> > > >> > >> > >>>>
> > > >> > >> > >>>> --
> > > >> > >> > >>>>
> > > >> > >> > >>>>
> > > >> > >> > >>>> *Fabian WollertZalando SE*
> > > >> > >> > >>>>
> > > >> > >> > >>>> E-Mail: fabian@zalando.de
> > > >> > >> > >>>>
> > > >> > >> >
> > > >> > >> >
> > > >> > >>
> > > >> > >
> > > >> > >
> > > >> > > --
> > > >> > > Yadong Xie
> > > >> > > Ministry of Education Key Lab for Intelligent Networks and
> Network
> > > >> > Security
> > > >> > > Dept. of Computer Science and Technology
> > > >> > > School of electronic information and engineering
> > > >> > > Xi'an Jiaotong University
> > > >> > > Xi'an, 710049, P.R. China
> > > >> > > Email: vthinkxie@gmail.com
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Till Rohrmann <tr...@apache.org>.
Thanks a lot for sharing the code with the community Yadong!
It looks really cool and I also want to give it a try to see how easy it is
to start Flink with it.
If it is already implemented and working, we could also think about adding
it to Flink and add a feature flag to switch between the old and new web
UI. We could think about enabling it by default to give it more user
exposure. After being confident and users having no complaints, we could
think about deprecating the old web UI and eventually to drop it. Of
course, initially we should give it a thorough review.
Cheers,
Till
On Mon, Nov 5, 2018 at 8:40 AM Fabian Wollert <fa...@zalando.de> wrote:
> Hi Yadong, this is awesome, thx for the code! I will try it out on our
> infrastructure and will post my feedback here, latest next week.
>
> I will also check if my ideas for FLINK-10707
> <https://issues.apache.org/jira/browse/FLINK-10707> are doable with your
> code since this was what pushed this discussion initially.
>
> Cheers
>
> --
>
>
> *Fabian WollertZalando SE*
>
> E-Mail: fabian@zalando.de
>
>
> Am Mo., 5. Nov. 2018 um 04:42 Uhr schrieb Yadong Xie <vthinkxie@gmail.com
> >:
>
> > Hi Fabian, Till, and Robert
> >
> > Thank you for your attention to this matter, I just push our codes to
> > github: https://github.com/vthinkxie/flink-runtime-web.
> >
> > You can start the project by following the guidelines
> > <https://github.com/vthinkxie/flink-runtime-web#development--debugging>
> > (just
> > run `npm install && npm run proxy`), just feel free to give any comments
> :)
> >
> > If I missed anything please let me know. Look forward to your feedback
> and
> > suggestions soon.
> >
> > Best regards
> > Yadong
> >
> >
> > On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fa...@zalando.de> wrote:
> >
> > > Hi Yadong, this looks awesome. is there any chance you can already
> share
> > > the code of the new web UI, so we can take a look at what you guys
> build
> > > there? I think that would speed up the discussion. If there is already
> a
> > > fully fledged new Version with everything updated out there, and its
> even
> > > battle tested in production already, that sounds like the way to go for
> > me.
> > >
> > > i started aside from this discussion here (to strengthen and learn some
> > > new React stuff) my attempt on the React version already, for whoever
> is
> > > curious, you can check it out here:
> > > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base url
> in
> > > actions.js
> > > <
> >
> https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8
> > >,
> > > npm install and then npm start) ... i just started to convert the first
> > > simple pages, so dont expect the whole UI yet, its just a start ...
> > >
> > > Cheers
> > >
> > > --
> > >
> > >
> > > *Fabian WollertZalando SE*
> > >
> > > E-Mail: fabian@zalando.de
> > >
> > >
> > > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <zjffdu@gmail.com
> >:
> > >
> > >> +1 for the proposal of Yadong. I'd like to add one more comment:
> > >>
> > >> We should provide more development guidance when developing the next
> > >> dashboard via angular 7. As Till mentioned before, one big concern is
> > lack
> > >> of frontend guys in flink community, I believe more development
> guidance
> > >> will attract more frontend guys to contribute to Flink web ui, and
> also
> > be
> > >> helpful for backend guys who want to add something to dashboard.
> > >>
> > >>
> > >>
> > >> Shaoxuan Wang <ws...@gmail.com>于2018年11月2日周五 上午10:06写道：
> > >>
> > >> > +1 to keep the web UI stack to Angular. This will make the stack
> > >> upgrading
> > >> > much easier and won't break the on-going efforts already made
> towards
> > >> the
> > >> > improvements of Flink Web UI.
> > >> >
> > >> > Till, Robert, and Fabian,
> > >> > Please take a look at the upgrading plan from Angular 1.x to 7.0
> > >> proposed
> > >> > by Yadong. Any feedback is appreciated.
> > >> >
> > >> > Regards,
> > >> > Shaoxuan
> > >> >
> > >> >
> > >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vt...@gmail.com>
> > wrote:
> > >> >
> > >> > > For your interest, here are some snapshots of Flink web UI (with
> > >> angular
> > >> > > 7.0) we recently reconstructed.
> > >> > > [image: snapshot.001.jpeg]
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
> > >> > >
> > >> > > We choose Angular 7.0 due to the following reasons:
> > >> > >
> > >> > > 1. It is easy to migrate the current codebase to Angular 7.0
> > >> following
> > >> > > the official guide.
> > >> > > 2. It is easy for current Web UI developers who are experienced
> > >> with
> > >> > > Angular 1.x to get familiar with 7.0 reading the official doc.
> > >> > > 3. Migrating other than rewriting is helpful to ensure its
> > >> functional
> > >> > > integrity.
> > >> > > 4. There are over 1 million Angular developers now (2.0 and
> > above,
> > >> > > still increasing rapidly), which means there are potentially
> more
> > >> > > developers willing to join the community.
> > >> > > 5. No bower or complex node / webpack configuration needed
> (only
> > >> > > angular cli).
> > >> > >
> > >> > > We split Angular 1.x and 7.0 versions into two folders based on
> the
> > >> > > following consideration:
> > >> > >
> > >> > > 1. 1.x version could be the default page because there is no
> > >> > > breakchange for both users and developers.
> > >> > > 2. Make the upgrade progressive. Users can switch between two
> > >> versions
> > >> > > by clicking the button on the top right corner.
> > >> > > 3. Reduce maintenance costs and keep the performance of web
> > without
> > >> > > bootstrap both 1.x and 7.0 versions in the same page.
> > >> > > 4. Won't interrupt the current development process. The code
> can
> > >> still
> > >> > > be submitted to 1.x version folder, we could help to sync all
> > >> > features and
> > >> > > bug fix to 7.0 version if needed.
> > >> > > 5. When the 7.0 version is ready and stable, it is easy to
> change
> > >> > > default page to it, and remove the old version.
> > >> > >
> > >> > > [image: test.jpg]
> > >> > >
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
> > >> > >
> > >> > > Here is the brief list of what we have done:
> > >> > >
> > >> > > 1. Create a new folder named next-web-dashboard in
> > >> flink-runtime-web.
> > >> > > 2. Migrate all Angular 1.x controllers, services, config etc
> one
> > by
> > >> > > one into 7.0 version with ngMigration.
> > >> > > 3. Modify frontend build scripts in order to package two
> versions
> > >> > > together.
> > >> > > 4. All works listed above are limited in the flink-runtime-web
> > >> folder.
> > >> > > One folder was added and no file or folder was deleted.
> > >> > >
> > >> > > The new version of the Web UI is running stable for several months
> > in
> > >> > > Alibaba. We are happy to contribute it to the Flink community if
> > >> everyone
> > >> > > like the work.
> > >> > >
> > >> > > Best
> > >> > > Yadong
> > >> > >
> > >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <
> trohrmann@apache.org
> > >
> > >> > > wrote:
> > >> > >
> > >> > >> I agree with Chesnay that the web UI is already fairly isolated
> in
> > >> > Flink's
> > >> > >> code base (see flink-runtime-web module). Moreover, imo it is
> very
> > >> > >> important that Flink comes with a nice web UI which works out of
> > the
> > >> box
> > >> > >> because it lowers the entrance hurdle massively. What one could
> > >> argue is
> > >> > >> to
> > >> > >> make it easier to plug in your own web UI if you want to develop
> > it.
> > >> > But I
> > >> > >> would see this as an orthogonal issue.
> > >> > >>
> > >> > >> I would actually argue against Robert's proposal to make a grand
> > >> rewrite
> > >> > >> if
> > >> > >> this is by any means possible. Experience shows that these kind
> of
> > >> > efforts
> > >> > >> are usually set up to fail or take super long. If possible, then
> I
> > >> would
> > >> > >> rather prefer to convert one component at a time to React. But
> I'm
> > >> not
> > >> > an
> > >> > >> expert here and cannot tell whether this is possible or not.
> > >> > >>
> > >> > >> What we would need is active help from the community to review
> > these
> > >> > >> changes. I think that is the crucial piece to make such an effort
> > >> work
> > >> > or
> > >> > >> fail.
> > >> > >>
> > >> > >> Cheers,
> > >> > >> Till
> > >> > >>
> > >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <
> > >> chesnay@apache.org>
> > >> > >> wrote:
> > >> > >>
> > >> > >> > The WebUI goes entirely through the REST API.
> > >> > >> >
> > >> > >> > I don't see how moving it out would make it easier to
> > >> > >> develop/contribute.
> > >> > >> >
> > >> > >> > The UI is already encapsulated in it's own module
> > >> (flink-runtime-web)
> > >> > >> > and changes can be made to it independently.
> > >> > >> > Historically the issue was simply that contributions to the UI
> > were
> > >> > not
> > >> > >> > reviewed in a timely manner / not at all, which mind you
> > >> > >> > applies to virtually all components. This also won't change by
> > >> moving
> > >> > it
> > >> > >> > out of Flink.
> > >> > >> >
> > >> > >> > Having it in Flink also guarantees that there is an
> > >> > easily-discoverable
> > >> > >> > version of the UI that is compatible with your Flink
> > distribution.
> > >> > >> >
> > >> > >> > On 31.10.2018 09:53, dyana.rose wrote:
> > >> > >> > > Re: who's using the web ui
> > >> > >> > >
> > >> > >> > > Though many mature solutions, with a fair amount of
> > >> time/resources
> > >> > >> > available are likely running their own front ends, for teams
> like
> > >> mine
> > >> > >> > which are smaller and aren't focused solely on working with
> > Flink,
> > >> > >> having
> > >> > >> > the web ui available removes a large barrier to getting up and
> > >> running
> > >> > >> > quickly.
> > >> > >> > >
> > >> > >> > > But, with the web UI being in core Flink, it does make it a
> bit
> > >> more
> > >> > >> of
> > >> > >> > a chore to contribute.
> > >> > >> > >
> > >> > >> > > I'm unaware of how necessary it is for the UI to deploy with
> > >> Flink?
> > >> > Is
> > >> > >> > it making any calls that aren't through the REST API (and if
> so,
> > >> can
> > >> > >> those
> > >> > >> > endpoints be added to the REST API)?
> > >> > >> > >
> > >> > >> > > In general I'd support moving it to its own package, making
> it
> > >> > easier
> > >> > >> to
> > >> > >> > develop with your standard UI dev tools. This also allows the
> web
> > >> UI
> > >> > to
> > >> > >> be
> > >> > >> > developed and released independently to core Flink.
> > >> > >> > >
> > >> > >> > > Dyana
> > >> > >> > >
> > >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de>
> > >> wrote:
> > >> > >> > >> Hi Till, I basically agree with all your points. i would
> > stress
> > >> the
> > >> > >> > >> "dustiness" of the current architecture: the package manager
> > >> used
> > >> > >> > (bower)
> > >> > >> > >> is deprecated since a long time, the chance for the builds
> of
> > >> the
> > >> > >> flink
> > >> > >> > web
> > >> > >> > >> dashboard not working anymore is increasing every day.
> > >> > >> > >>
> > >> > >> > >> About the knowledge in the community: Two days is not a lot
> of
> > >> > time,
> > >> > >> but
> > >> > >> > >> interest in this topic seems to be minor anyways. Is someone
> > >> using
> > >> > >> the
> > >> > >> > >> Flink Web Dashboard at all, or is everyone running their own
> > >> custom
> > >> > >> > >> solutions? Because if there is no interest in using the Web
> UI
> > >> AND
> > >> > no
> > >> > >> > one
> > >> > >> > >> interested in developing, there would be no need to package
> > >> this as
> > >> > >> > part of
> > >> > >> > >> the official Flink package, but rather develop an
> independent
> > >> > >> solution
> > >> > >> > (I'm
> > >> > >> > >> not voting for this right now, just putting it out), if at
> > all.
> > >> The
> > >> > >> > >> official package could then just ship with the API, which
> > other
> > >> > >> > solutions
> > >> > >> > >> can build upon. This solution could be from an agile point
> of
> > >> view
> > >> > >> also
> > >> > >> > the
> > >> > >> > >> best (enhanced testing, independent and more effective dev
> > >> > workflow,
> > >> > >> > etc.),
> > >> > >> > >> but is bad for the usage of the Flink UI, because people
> need
> > to
> > >> > >> install
> > >> > >> > >> two things individually (Flink and the web dashboard).
> > >> > >> > >>
> > >> > >> > >> How did the choice for Angular1 happen back then? Who was
> > >> writing
> > >> > the
> > >> > >> > >> Dashboard in the first place?
> > >> > >> > >>
> > >> > >> > >> Cheers
> > >> > >> > >>
> > >> > >> > >> --
> > >> > >> > >>
> > >> > >> > >>
> > >> > >> > >> *Fabian WollertZalando SE*
> > >> > >> > >>
> > >> > >> > >> E-Mail: fabian@zalando.de
> > >> > >> > >>
> > >> > >> > >>
> > >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> > >> > >> > >> trohrmann@apache.org>:
> > >> > >> > >>
> > >> > >> > >>> Thanks for starting this discussion Fabian! I think our web
> > UI
> > >> > >> > technology
> > >> > >> > >>> stack is quite dusty by now and it would be beneficial to
> > think
> > >> > >> about
> > >> > >> > its
> > >> > >> > >>> technological future.
> > >> > >> > >>>
> > >> > >> > >>> On the one hand, our current web UI works more or less
> > reliable
> > >> > and
> > >> > >> > >>> changing the underlying technology has the risk of breaking
> > >> > things.
> > >> > >> > >>> Moreover, there might be the risk that the newly chosen
> > >> technology
> > >> > >> > will be
> > >> > >> > >>> deprecated at some point in time as well.
> > >> > >> > >>>
> > >> > >> > >>> On the other hand, we don't have much Angular 1 knowledge
> in
> > >> the
> > >> > >> > community
> > >> > >> > >>> and extending the web UI is, thus, quite hard at the
> moment.
> > >> Maybe
> > >> > >> by
> > >> > >> > using
> > >> > >> > >>> some newer web technologies we might be able to attract
> more
> > >> > people
> > >> > >> > with a
> > >> > >> > >>> web technology background to join the community.
> > >> > >> > >>>
> > >> > >> > >>> The lack of people working on the web UI is for me the
> > biggest
> > >> > >> problem
> > >> > >> > I
> > >> > >> > >>> would like to address. If there is interest in the web UI,
> > then
> > >> > I'm
> > >> > >> > quite
> > >> > >> > >>> sure that we will be able to even migrate to some other
> > >> technology
> > >> > >> in
> > >> > >> > the
> > >> > >> > >>> future. The next important issue for me is to do the change
> > >> > >> > incrementally
> > >> > >> > >>> if possible. Ideally we never break the web UI in the
> process
> > >> of
> > >> > >> > migrating
> > >> > >> > >>> to a new technology. I'm not an expert here so it might or
> > >> might
> > >> > >> not be
> > >> > >> > >>> possible. But if it is, then we should design the
> > >> implementation
> > >> > >> steps
> > >> > >> > in
> > >> > >> > >>> such a way.
> > >> > >> > >>>
> > >> > >> > >>> Cheers,
> > >> > >> > >>> Till
> > >> > >> > >>>
> > >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <
> > >> fabian@zalando.de
> > >> > >
> > >> > >> > wrote:
> > >> > >> > >>>
> > >> > >> > >>>> Hi everyone,
> > >> > >> > >>>>
> > >> > >> > >>>> in this email thread
> > >> > >> > >>>> <
> > >> > >> > >>>>
> > >> > >> > >>>
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > >> > >> > >>>> and the tickets FLINK-10705
> > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and
> > >> > >> FLINK-10706
> > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the
> > >> > discussion
> > >> > >> > came
> > >> > >> > >>> up
> > >> > >> > >>>> whether to change the underlying architecture of Flink's
> Web
> > >> > >> Dashboard
> > >> > >> > >>> from
> > >> > >> > >>>> Angular1 to something else. This email thread should be
> > >> solely to
> > >> > >> > discuss
> > >> > >> > >>>> the pro's and con's of this, and what could be the target
> > >> > >> > architecture.
> > >> > >> > >>>>
> > >> > >> > >>>> My choice would be React. Personally I agree with Till's
> > >> comments
> > >> > >> in
> > >> > >> > the
> > >> > >> > >>>> ticket, Angular 1 being basically outdated and is not
> > having a
> > >> > >> large
> > >> > >> > >>>> following anymore. From my experience the choice between
> > >> Angular
> > >> > >> 2-7
> > >> > >> > or
> > >> > >> > >>>> React is subjective, you can get things done with both. I
> > >> > >> personally
> > >> > >> > only
> > >> > >> > >>>> have experience with React, so I personally would be
> faster
> > >> to
> > >> > >> > develop
> > >> > >> > >>>> with this one. I currently have not planned to learn
> Angular
> > >> as
> > >> > >> well
> > >> > >> > >>> (being
> > >> > >> > >>>> a more backend focused developer in general) so if the
> > >> decision
> > >> > >> would
> > >> > >> > be
> > >> > >> > >>> to
> > >> > >> > >>>> go with Angular, i would be unfortunately out of this
> rework
> > >> of
> > >> > the
> > >> > >> > Flink
> > >> > >> > >>>> Dashboard most certainly.
> > >> > >> > >>>>
> > >> > >> > >>>> Additionally i would like to get rid of bower, since its
> > >> > officially
> > >> > >> > >>>> deprecated <
> > >> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> > >> > >> > >>>> my
> > >> > >> > >>>> idea would be to just use a create-react-app package with
> > npm
> > >> and
> > >> > >> > webpack
> > >> > >> > >>>> under the hood. no need for additional lib's here imho.
> But
> > >> > again:
> > >> > >> > thats
> > >> > >> > >>>> mostly what i've been working with recently, so thats a
> > >> > subjective
> > >> > >> > >>> point. I
> > >> > >> > >>>> could imagine getting used to yarn in the future as well.
> > >> > >> > >>>>
> > >> > >> > >>>> Cheers
> > >> > >> > >>>> Fabian
> > >> > >> > >>>>
> > >> > >> > >>>> --
> > >> > >> > >>>>
> > >> > >> > >>>>
> > >> > >> > >>>> *Fabian WollertZalando SE*
> > >> > >> > >>>>
> > >> > >> > >>>> E-Mail: fabian@zalando.de
> > >> > >> > >>>>
> > >> > >> >
> > >> > >> >
> > >> > >>
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Yadong Xie
> > >> > > Ministry of Education Key Lab for Intelligent Networks and Network
> > >> > Security
> > >> > > Dept. of Computer Science and Technology
> > >> > > School of electronic information and engineering
> > >> > > Xi'an Jiaotong University
> > >> > > Xi'an, 710049, P.R. China
> > >> > > Email: vthinkxie@gmail.com
> > >> > >
> > >> >
> > >>
> > >
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Fabian Wollert <fa...@zalando.de>.
Hi Yadong, this is awesome, thx for the code! I will try it out on our
infrastructure and will post my feedback here, latest next week.
I will also check if my ideas for FLINK-10707
<https://issues.apache.org/jira/browse/FLINK-10707> are doable with your
code since this was what pushed this discussion initially.
Cheers
--
*Fabian WollertZalando SE*
E-Mail: fabian@zalando.de
Am Mo., 5. Nov. 2018 um 04:42 Uhr schrieb Yadong Xie <vt...@gmail.com>:
> Hi Fabian, Till, and Robert
>
> Thank you for your attention to this matter, I just push our codes to
> github: https://github.com/vthinkxie/flink-runtime-web.
>
> You can start the project by following the guidelines
> <https://github.com/vthinkxie/flink-runtime-web#development--debugging>
> (just
> run `npm install && npm run proxy`), just feel free to give any comments :)
>
> If I missed anything please let me know. Look forward to your feedback and
> suggestions soon.
>
> Best regards
> Yadong
>
>
> On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fa...@zalando.de> wrote:
>
> > Hi Yadong, this looks awesome. is there any chance you can already share
> > the code of the new web UI, so we can take a look at what you guys build
> > there? I think that would speed up the discussion. If there is already a
> > fully fledged new Version with everything updated out there, and its even
> > battle tested in production already, that sounds like the way to go for
> me.
> >
> > i started aside from this discussion here (to strengthen and learn some
> > new React stuff) my attempt on the React version already, for whoever is
> > curious, you can check it out here:
> > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base url in
> > actions.js
> > <
> https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8
> >,
> > npm install and then npm start) ... i just started to convert the first
> > simple pages, so dont expect the whole UI yet, its just a start ...
> >
> > Cheers
> >
> > --
> >
> >
> > *Fabian WollertZalando SE*
> >
> > E-Mail: fabian@zalando.de
> >
> >
> > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <zj...@gmail.com>:
> >
> >> +1 for the proposal of Yadong. I'd like to add one more comment:
> >>
> >> We should provide more development guidance when developing the next
> >> dashboard via angular 7. As Till mentioned before, one big concern is
> lack
> >> of frontend guys in flink community, I believe more development guidance
> >> will attract more frontend guys to contribute to Flink web ui, and also
> be
> >> helpful for backend guys who want to add something to dashboard.
> >>
> >>
> >>
> >> Shaoxuan Wang <ws...@gmail.com>于2018年11月2日周五 上午10:06写道：
> >>
> >> > +1 to keep the web UI stack to Angular. This will make the stack
> >> upgrading
> >> > much easier and won't break the on-going efforts already made towards
> >> the
> >> > improvements of Flink Web UI.
> >> >
> >> > Till, Robert, and Fabian,
> >> > Please take a look at the upgrading plan from Angular 1.x to 7.0
> >> proposed
> >> > by Yadong. Any feedback is appreciated.
> >> >
> >> > Regards,
> >> > Shaoxuan
> >> >
> >> >
> >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vt...@gmail.com>
> wrote:
> >> >
> >> > > For your interest, here are some snapshots of Flink web UI (with
> >> angular
> >> > > 7.0) we recently reconstructed.
> >> > > [image: snapshot.001.jpeg]
> >> > >
> >> > >
> >> >
> >>
> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
> >> > >
> >> > > We choose Angular 7.0 due to the following reasons:
> >> > >
> >> > > 1. It is easy to migrate the current codebase to Angular 7.0
> >> following
> >> > > the official guide.
> >> > > 2. It is easy for current Web UI developers who are experienced
> >> with
> >> > > Angular 1.x to get familiar with 7.0 reading the official doc.
> >> > > 3. Migrating other than rewriting is helpful to ensure its
> >> functional
> >> > > integrity.
> >> > > 4. There are over 1 million Angular developers now (2.0 and
> above,
> >> > > still increasing rapidly), which means there are potentially more
> >> > > developers willing to join the community.
> >> > > 5. No bower or complex node / webpack configuration needed (only
> >> > > angular cli).
> >> > >
> >> > > We split Angular 1.x and 7.0 versions into two folders based on the
> >> > > following consideration:
> >> > >
> >> > > 1. 1.x version could be the default page because there is no
> >> > > breakchange for both users and developers.
> >> > > 2. Make the upgrade progressive. Users can switch between two
> >> versions
> >> > > by clicking the button on the top right corner.
> >> > > 3. Reduce maintenance costs and keep the performance of web
> without
> >> > > bootstrap both 1.x and 7.0 versions in the same page.
> >> > > 4. Won't interrupt the current development process. The code can
> >> still
> >> > > be submitted to 1.x version folder, we could help to sync all
> >> > features and
> >> > > bug fix to 7.0 version if needed.
> >> > > 5. When the 7.0 version is ready and stable, it is easy to change
> >> > > default page to it, and remove the old version.
> >> > >
> >> > > [image: test.jpg]
> >> > >
> >> > >
> >> > >
> >> >
> >>
> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
> >> > >
> >> > > Here is the brief list of what we have done:
> >> > >
> >> > > 1. Create a new folder named next-web-dashboard in
> >> flink-runtime-web.
> >> > > 2. Migrate all Angular 1.x controllers, services, config etc one
> by
> >> > > one into 7.0 version with ngMigration.
> >> > > 3. Modify frontend build scripts in order to package two versions
> >> > > together.
> >> > > 4. All works listed above are limited in the flink-runtime-web
> >> folder.
> >> > > One folder was added and no file or folder was deleted.
> >> > >
> >> > > The new version of the Web UI is running stable for several months
> in
> >> > > Alibaba. We are happy to contribute it to the Flink community if
> >> everyone
> >> > > like the work.
> >> > >
> >> > > Best
> >> > > Yadong
> >> > >
> >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <trohrmann@apache.org
> >
> >> > > wrote:
> >> > >
> >> > >> I agree with Chesnay that the web UI is already fairly isolated in
> >> > Flink's
> >> > >> code base (see flink-runtime-web module). Moreover, imo it is very
> >> > >> important that Flink comes with a nice web UI which works out of
> the
> >> box
> >> > >> because it lowers the entrance hurdle massively. What one could
> >> argue is
> >> > >> to
> >> > >> make it easier to plug in your own web UI if you want to develop
> it.
> >> > But I
> >> > >> would see this as an orthogonal issue.
> >> > >>
> >> > >> I would actually argue against Robert's proposal to make a grand
> >> rewrite
> >> > >> if
> >> > >> this is by any means possible. Experience shows that these kind of
> >> > efforts
> >> > >> are usually set up to fail or take super long. If possible, then I
> >> would
> >> > >> rather prefer to convert one component at a time to React. But I'm
> >> not
> >> > an
> >> > >> expert here and cannot tell whether this is possible or not.
> >> > >>
> >> > >> What we would need is active help from the community to review
> these
> >> > >> changes. I think that is the crucial piece to make such an effort
> >> work
> >> > or
> >> > >> fail.
> >> > >>
> >> > >> Cheers,
> >> > >> Till
> >> > >>
> >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <
> >> chesnay@apache.org>
> >> > >> wrote:
> >> > >>
> >> > >> > The WebUI goes entirely through the REST API.
> >> > >> >
> >> > >> > I don't see how moving it out would make it easier to
> >> > >> develop/contribute.
> >> > >> >
> >> > >> > The UI is already encapsulated in it's own module
> >> (flink-runtime-web)
> >> > >> > and changes can be made to it independently.
> >> > >> > Historically the issue was simply that contributions to the UI
> were
> >> > not
> >> > >> > reviewed in a timely manner / not at all, which mind you
> >> > >> > applies to virtually all components. This also won't change by
> >> moving
> >> > it
> >> > >> > out of Flink.
> >> > >> >
> >> > >> > Having it in Flink also guarantees that there is an
> >> > easily-discoverable
> >> > >> > version of the UI that is compatible with your Flink
> distribution.
> >> > >> >
> >> > >> > On 31.10.2018 09:53, dyana.rose wrote:
> >> > >> > > Re: who's using the web ui
> >> > >> > >
> >> > >> > > Though many mature solutions, with a fair amount of
> >> time/resources
> >> > >> > available are likely running their own front ends, for teams like
> >> mine
> >> > >> > which are smaller and aren't focused solely on working with
> Flink,
> >> > >> having
> >> > >> > the web ui available removes a large barrier to getting up and
> >> running
> >> > >> > quickly.
> >> > >> > >
> >> > >> > > But, with the web UI being in core Flink, it does make it a bit
> >> more
> >> > >> of
> >> > >> > a chore to contribute.
> >> > >> > >
> >> > >> > > I'm unaware of how necessary it is for the UI to deploy with
> >> Flink?
> >> > Is
> >> > >> > it making any calls that aren't through the REST API (and if so,
> >> can
> >> > >> those
> >> > >> > endpoints be added to the REST API)?
> >> > >> > >
> >> > >> > > In general I'd support moving it to its own package, making it
> >> > easier
> >> > >> to
> >> > >> > develop with your standard UI dev tools. This also allows the web
> >> UI
> >> > to
> >> > >> be
> >> > >> > developed and released independently to core Flink.
> >> > >> > >
> >> > >> > > Dyana
> >> > >> > >
> >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de>
> >> wrote:
> >> > >> > >> Hi Till, I basically agree with all your points. i would
> stress
> >> the
> >> > >> > >> "dustiness" of the current architecture: the package manager
> >> used
> >> > >> > (bower)
> >> > >> > >> is deprecated since a long time, the chance for the builds of
> >> the
> >> > >> flink
> >> > >> > web
> >> > >> > >> dashboard not working anymore is increasing every day.
> >> > >> > >>
> >> > >> > >> About the knowledge in the community: Two days is not a lot of
> >> > time,
> >> > >> but
> >> > >> > >> interest in this topic seems to be minor anyways. Is someone
> >> using
> >> > >> the
> >> > >> > >> Flink Web Dashboard at all, or is everyone running their own
> >> custom
> >> > >> > >> solutions? Because if there is no interest in using the Web UI
> >> AND
> >> > no
> >> > >> > one
> >> > >> > >> interested in developing, there would be no need to package
> >> this as
> >> > >> > part of
> >> > >> > >> the official Flink package, but rather develop an independent
> >> > >> solution
> >> > >> > (I'm
> >> > >> > >> not voting for this right now, just putting it out), if at
> all.
> >> The
> >> > >> > >> official package could then just ship with the API, which
> other
> >> > >> > solutions
> >> > >> > >> can build upon. This solution could be from an agile point of
> >> view
> >> > >> also
> >> > >> > the
> >> > >> > >> best (enhanced testing, independent and more effective dev
> >> > workflow,
> >> > >> > etc.),
> >> > >> > >> but is bad for the usage of the Flink UI, because people need
> to
> >> > >> install
> >> > >> > >> two things individually (Flink and the web dashboard).
> >> > >> > >>
> >> > >> > >> How did the choice for Angular1 happen back then? Who was
> >> writing
> >> > the
> >> > >> > >> Dashboard in the first place?
> >> > >> > >>
> >> > >> > >> Cheers
> >> > >> > >>
> >> > >> > >> --
> >> > >> > >>
> >> > >> > >>
> >> > >> > >> *Fabian WollertZalando SE*
> >> > >> > >>
> >> > >> > >> E-Mail: fabian@zalando.de
> >> > >> > >>
> >> > >> > >>
> >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> >> > >> > >> trohrmann@apache.org>:
> >> > >> > >>
> >> > >> > >>> Thanks for starting this discussion Fabian! I think our web
> UI
> >> > >> > technology
> >> > >> > >>> stack is quite dusty by now and it would be beneficial to
> think
> >> > >> about
> >> > >> > its
> >> > >> > >>> technological future.
> >> > >> > >>>
> >> > >> > >>> On the one hand, our current web UI works more or less
> reliable
> >> > and
> >> > >> > >>> changing the underlying technology has the risk of breaking
> >> > things.
> >> > >> > >>> Moreover, there might be the risk that the newly chosen
> >> technology
> >> > >> > will be
> >> > >> > >>> deprecated at some point in time as well.
> >> > >> > >>>
> >> > >> > >>> On the other hand, we don't have much Angular 1 knowledge in
> >> the
> >> > >> > community
> >> > >> > >>> and extending the web UI is, thus, quite hard at the moment.
> >> Maybe
> >> > >> by
> >> > >> > using
> >> > >> > >>> some newer web technologies we might be able to attract more
> >> > people
> >> > >> > with a
> >> > >> > >>> web technology background to join the community.
> >> > >> > >>>
> >> > >> > >>> The lack of people working on the web UI is for me the
> biggest
> >> > >> problem
> >> > >> > I
> >> > >> > >>> would like to address. If there is interest in the web UI,
> then
> >> > I'm
> >> > >> > quite
> >> > >> > >>> sure that we will be able to even migrate to some other
> >> technology
> >> > >> in
> >> > >> > the
> >> > >> > >>> future. The next important issue for me is to do the change
> >> > >> > incrementally
> >> > >> > >>> if possible. Ideally we never break the web UI in the process
> >> of
> >> > >> > migrating
> >> > >> > >>> to a new technology. I'm not an expert here so it might or
> >> might
> >> > >> not be
> >> > >> > >>> possible. But if it is, then we should design the
> >> implementation
> >> > >> steps
> >> > >> > in
> >> > >> > >>> such a way.
> >> > >> > >>>
> >> > >> > >>> Cheers,
> >> > >> > >>> Till
> >> > >> > >>>
> >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <
> >> fabian@zalando.de
> >> > >
> >> > >> > wrote:
> >> > >> > >>>
> >> > >> > >>>> Hi everyone,
> >> > >> > >>>>
> >> > >> > >>>> in this email thread
> >> > >> > >>>> <
> >> > >> > >>>>
> >> > >> > >>>
> >> > >> >
> >> > >>
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> >> > >> > >>>> and the tickets FLINK-10705
> >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and
> >> > >> FLINK-10706
> >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the
> >> > discussion
> >> > >> > came
> >> > >> > >>> up
> >> > >> > >>>> whether to change the underlying architecture of Flink's Web
> >> > >> Dashboard
> >> > >> > >>> from
> >> > >> > >>>> Angular1 to something else. This email thread should be
> >> solely to
> >> > >> > discuss
> >> > >> > >>>> the pro's and con's of this, and what could be the target
> >> > >> > architecture.
> >> > >> > >>>>
> >> > >> > >>>> My choice would be React. Personally I agree with Till's
> >> comments
> >> > >> in
> >> > >> > the
> >> > >> > >>>> ticket, Angular 1 being basically outdated and is not
> having a
> >> > >> large
> >> > >> > >>>> following anymore. From my experience the choice between
> >> Angular
> >> > >> 2-7
> >> > >> > or
> >> > >> > >>>> React is subjective, you can get things done with both. I
> >> > >> personally
> >> > >> > only
> >> > >> > >>>> have experience with React, so I personally would be faster
> >> to
> >> > >> > develop
> >> > >> > >>>> with this one. I currently have not planned to learn Angular
> >> as
> >> > >> well
> >> > >> > >>> (being
> >> > >> > >>>> a more backend focused developer in general) so if the
> >> decision
> >> > >> would
> >> > >> > be
> >> > >> > >>> to
> >> > >> > >>>> go with Angular, i would be unfortunately out of this rework
> >> of
> >> > the
> >> > >> > Flink
> >> > >> > >>>> Dashboard most certainly.
> >> > >> > >>>>
> >> > >> > >>>> Additionally i would like to get rid of bower, since its
> >> > officially
> >> > >> > >>>> deprecated <
> >> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> >> > >> > >>>> my
> >> > >> > >>>> idea would be to just use a create-react-app package with
> npm
> >> and
> >> > >> > webpack
> >> > >> > >>>> under the hood. no need for additional lib's here imho. But
> >> > again:
> >> > >> > thats
> >> > >> > >>>> mostly what i've been working with recently, so thats a
> >> > subjective
> >> > >> > >>> point. I
> >> > >> > >>>> could imagine getting used to yarn in the future as well.
> >> > >> > >>>>
> >> > >> > >>>> Cheers
> >> > >> > >>>> Fabian
> >> > >> > >>>>
> >> > >> > >>>> --
> >> > >> > >>>>
> >> > >> > >>>>
> >> > >> > >>>> *Fabian WollertZalando SE*
> >> > >> > >>>>
> >> > >> > >>>> E-Mail: fabian@zalando.de
> >> > >> > >>>>
> >> > >> >
> >> > >> >
> >> > >>
> >> > >
> >> > >
> >> > > --
> >> > > Yadong Xie
> >> > > Ministry of Education Key Lab for Intelligent Networks and Network
> >> > Security
> >> > > Dept. of Computer Science and Technology
> >> > > School of electronic information and engineering
> >> > > Xi'an Jiaotong University
> >> > > Xi'an, 710049, P.R. China
> >> > > Email: vthinkxie@gmail.com
> >> > >
> >> >
> >>
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Yadong Xie <vt...@gmail.com>.
Hi Fabian, Till, and Robert
Thank you for your attention to this matter, I just push our codes to
github: https://github.com/vthinkxie/flink-runtime-web.
You can start the project by following the guidelines
<https://github.com/vthinkxie/flink-runtime-web#development--debugging> (just
run `npm install && npm run proxy`), just feel free to give any comments :)
If I missed anything please let me know. Look forward to your feedback and
suggestions soon.
Best regards
Yadong
On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fa...@zalando.de> wrote:
> Hi Yadong, this looks awesome. is there any chance you can already share
> the code of the new web UI, so we can take a look at what you guys build
> there? I think that would speed up the discussion. If there is already a
> fully fledged new Version with everything updated out there, and its even
> battle tested in production already, that sounds like the way to go for me.
>
> i started aside from this discussion here (to strengthen and learn some
> new React stuff) my attempt on the React version already, for whoever is
> curious, you can check it out here:
> https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base url in
> actions.js
> <https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8>,
> npm install and then npm start) ... i just started to convert the first
> simple pages, so dont expect the whole UI yet, its just a start ...
>
> Cheers
>
> --
>
>
> *Fabian WollertZalando SE*
>
> E-Mail: fabian@zalando.de
>
>
> Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <zj...@gmail.com>:
>
>> +1 for the proposal of Yadong. I'd like to add one more comment:
>>
>> We should provide more development guidance when developing the next
>> dashboard via angular 7. As Till mentioned before, one big concern is lack
>> of frontend guys in flink community, I believe more development guidance
>> will attract more frontend guys to contribute to Flink web ui, and also be
>> helpful for backend guys who want to add something to dashboard.
>>
>>
>>
>> Shaoxuan Wang <ws...@gmail.com>于2018年11月2日周五 上午10:06写道：
>>
>> > +1 to keep the web UI stack to Angular. This will make the stack
>> upgrading
>> > much easier and won't break the on-going efforts already made towards
>> the
>> > improvements of Flink Web UI.
>> >
>> > Till, Robert, and Fabian,
>> > Please take a look at the upgrading plan from Angular 1.x to 7.0
>> proposed
>> > by Yadong. Any feedback is appreciated.
>> >
>> > Regards,
>> > Shaoxuan
>> >
>> >
>> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vt...@gmail.com> wrote:
>> >
>> > > For your interest, here are some snapshots of Flink web UI (with
>> angular
>> > > 7.0) we recently reconstructed.
>> > > [image: snapshot.001.jpeg]
>> > >
>> > >
>> >
>> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
>> > >
>> > > We choose Angular 7.0 due to the following reasons:
>> > >
>> > > 1. It is easy to migrate the current codebase to Angular 7.0
>> following
>> > > the official guide.
>> > > 2. It is easy for current Web UI developers who are experienced
>> with
>> > > Angular 1.x to get familiar with 7.0 reading the official doc.
>> > > 3. Migrating other than rewriting is helpful to ensure its
>> functional
>> > > integrity.
>> > > 4. There are over 1 million Angular developers now (2.0 and above,
>> > > still increasing rapidly), which means there are potentially more
>> > > developers willing to join the community.
>> > > 5. No bower or complex node / webpack configuration needed (only
>> > > angular cli).
>> > >
>> > > We split Angular 1.x and 7.0 versions into two folders based on the
>> > > following consideration:
>> > >
>> > > 1. 1.x version could be the default page because there is no
>> > > breakchange for both users and developers.
>> > > 2. Make the upgrade progressive. Users can switch between two
>> versions
>> > > by clicking the button on the top right corner.
>> > > 3. Reduce maintenance costs and keep the performance of web without
>> > > bootstrap both 1.x and 7.0 versions in the same page.
>> > > 4. Won't interrupt the current development process. The code can
>> still
>> > > be submitted to 1.x version folder, we could help to sync all
>> > features and
>> > > bug fix to 7.0 version if needed.
>> > > 5. When the 7.0 version is ready and stable, it is easy to change
>> > > default page to it, and remove the old version.
>> > >
>> > > [image: test.jpg]
>> > >
>> > >
>> > >
>> >
>> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
>> > >
>> > > Here is the brief list of what we have done:
>> > >
>> > > 1. Create a new folder named next-web-dashboard in
>> flink-runtime-web.
>> > > 2. Migrate all Angular 1.x controllers, services, config etc one by
>> > > one into 7.0 version with ngMigration.
>> > > 3. Modify frontend build scripts in order to package two versions
>> > > together.
>> > > 4. All works listed above are limited in the flink-runtime-web
>> folder.
>> > > One folder was added and no file or folder was deleted.
>> > >
>> > > The new version of the Web UI is running stable for several months in
>> > > Alibaba. We are happy to contribute it to the Flink community if
>> everyone
>> > > like the work.
>> > >
>> > > Best
>> > > Yadong
>> > >
>> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <tr...@apache.org>
>> > > wrote:
>> > >
>> > >> I agree with Chesnay that the web UI is already fairly isolated in
>> > Flink's
>> > >> code base (see flink-runtime-web module). Moreover, imo it is very
>> > >> important that Flink comes with a nice web UI which works out of the
>> box
>> > >> because it lowers the entrance hurdle massively. What one could
>> argue is
>> > >> to
>> > >> make it easier to plug in your own web UI if you want to develop it.
>> > But I
>> > >> would see this as an orthogonal issue.
>> > >>
>> > >> I would actually argue against Robert's proposal to make a grand
>> rewrite
>> > >> if
>> > >> this is by any means possible. Experience shows that these kind of
>> > efforts
>> > >> are usually set up to fail or take super long. If possible, then I
>> would
>> > >> rather prefer to convert one component at a time to React. But I'm
>> not
>> > an
>> > >> expert here and cannot tell whether this is possible or not.
>> > >>
>> > >> What we would need is active help from the community to review these
>> > >> changes. I think that is the crucial piece to make such an effort
>> work
>> > or
>> > >> fail.
>> > >>
>> > >> Cheers,
>> > >> Till
>> > >>
>> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <
>> chesnay@apache.org>
>> > >> wrote:
>> > >>
>> > >> > The WebUI goes entirely through the REST API.
>> > >> >
>> > >> > I don't see how moving it out would make it easier to
>> > >> develop/contribute.
>> > >> >
>> > >> > The UI is already encapsulated in it's own module
>> (flink-runtime-web)
>> > >> > and changes can be made to it independently.
>> > >> > Historically the issue was simply that contributions to the UI were
>> > not
>> > >> > reviewed in a timely manner / not at all, which mind you
>> > >> > applies to virtually all components. This also won't change by
>> moving
>> > it
>> > >> > out of Flink.
>> > >> >
>> > >> > Having it in Flink also guarantees that there is an
>> > easily-discoverable
>> > >> > version of the UI that is compatible with your Flink distribution.
>> > >> >
>> > >> > On 31.10.2018 09:53, dyana.rose wrote:
>> > >> > > Re: who's using the web ui
>> > >> > >
>> > >> > > Though many mature solutions, with a fair amount of
>> time/resources
>> > >> > available are likely running their own front ends, for teams like
>> mine
>> > >> > which are smaller and aren't focused solely on working with Flink,
>> > >> having
>> > >> > the web ui available removes a large barrier to getting up and
>> running
>> > >> > quickly.
>> > >> > >
>> > >> > > But, with the web UI being in core Flink, it does make it a bit
>> more
>> > >> of
>> > >> > a chore to contribute.
>> > >> > >
>> > >> > > I'm unaware of how necessary it is for the UI to deploy with
>> Flink?
>> > Is
>> > >> > it making any calls that aren't through the REST API (and if so,
>> can
>> > >> those
>> > >> > endpoints be added to the REST API)?
>> > >> > >
>> > >> > > In general I'd support moving it to its own package, making it
>> > easier
>> > >> to
>> > >> > develop with your standard UI dev tools. This also allows the web
>> UI
>> > to
>> > >> be
>> > >> > developed and released independently to core Flink.
>> > >> > >
>> > >> > > Dyana
>> > >> > >
>> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de>
>> wrote:
>> > >> > >> Hi Till, I basically agree with all your points. i would stress
>> the
>> > >> > >> "dustiness" of the current architecture: the package manager
>> used
>> > >> > (bower)
>> > >> > >> is deprecated since a long time, the chance for the builds of
>> the
>> > >> flink
>> > >> > web
>> > >> > >> dashboard not working anymore is increasing every day.
>> > >> > >>
>> > >> > >> About the knowledge in the community: Two days is not a lot of
>> > time,
>> > >> but
>> > >> > >> interest in this topic seems to be minor anyways. Is someone
>> using
>> > >> the
>> > >> > >> Flink Web Dashboard at all, or is everyone running their own
>> custom
>> > >> > >> solutions? Because if there is no interest in using the Web UI
>> AND
>> > no
>> > >> > one
>> > >> > >> interested in developing, there would be no need to package
>> this as
>> > >> > part of
>> > >> > >> the official Flink package, but rather develop an independent
>> > >> solution
>> > >> > (I'm
>> > >> > >> not voting for this right now, just putting it out), if at all.
>> The
>> > >> > >> official package could then just ship with the API, which other
>> > >> > solutions
>> > >> > >> can build upon. This solution could be from an agile point of
>> view
>> > >> also
>> > >> > the
>> > >> > >> best (enhanced testing, independent and more effective dev
>> > workflow,
>> > >> > etc.),
>> > >> > >> but is bad for the usage of the Flink UI, because people need to
>> > >> install
>> > >> > >> two things individually (Flink and the web dashboard).
>> > >> > >>
>> > >> > >> How did the choice for Angular1 happen back then? Who was
>> writing
>> > the
>> > >> > >> Dashboard in the first place?
>> > >> > >>
>> > >> > >> Cheers
>> > >> > >>
>> > >> > >> --
>> > >> > >>
>> > >> > >>
>> > >> > >> *Fabian WollertZalando SE*
>> > >> > >>
>> > >> > >> E-Mail: fabian@zalando.de
>> > >> > >>
>> > >> > >>
>> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
>> > >> > >> trohrmann@apache.org>:
>> > >> > >>
>> > >> > >>> Thanks for starting this discussion Fabian! I think our web UI
>> > >> > technology
>> > >> > >>> stack is quite dusty by now and it would be beneficial to think
>> > >> about
>> > >> > its
>> > >> > >>> technological future.
>> > >> > >>>
>> > >> > >>> On the one hand, our current web UI works more or less reliable
>> > and
>> > >> > >>> changing the underlying technology has the risk of breaking
>> > things.
>> > >> > >>> Moreover, there might be the risk that the newly chosen
>> technology
>> > >> > will be
>> > >> > >>> deprecated at some point in time as well.
>> > >> > >>>
>> > >> > >>> On the other hand, we don't have much Angular 1 knowledge in
>> the
>> > >> > community
>> > >> > >>> and extending the web UI is, thus, quite hard at the moment.
>> Maybe
>> > >> by
>> > >> > using
>> > >> > >>> some newer web technologies we might be able to attract more
>> > people
>> > >> > with a
>> > >> > >>> web technology background to join the community.
>> > >> > >>>
>> > >> > >>> The lack of people working on the web UI is for me the biggest
>> > >> problem
>> > >> > I
>> > >> > >>> would like to address. If there is interest in the web UI, then
>> > I'm
>> > >> > quite
>> > >> > >>> sure that we will be able to even migrate to some other
>> technology
>> > >> in
>> > >> > the
>> > >> > >>> future. The next important issue for me is to do the change
>> > >> > incrementally
>> > >> > >>> if possible. Ideally we never break the web UI in the process
>> of
>> > >> > migrating
>> > >> > >>> to a new technology. I'm not an expert here so it might or
>> might
>> > >> not be
>> > >> > >>> possible. But if it is, then we should design the
>> implementation
>> > >> steps
>> > >> > in
>> > >> > >>> such a way.
>> > >> > >>>
>> > >> > >>> Cheers,
>> > >> > >>> Till
>> > >> > >>>
>> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <
>> fabian@zalando.de
>> > >
>> > >> > wrote:
>> > >> > >>>
>> > >> > >>>> Hi everyone,
>> > >> > >>>>
>> > >> > >>>> in this email thread
>> > >> > >>>> <
>> > >> > >>>>
>> > >> > >>>
>> > >> >
>> > >>
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
>> > >> > >>>> and the tickets FLINK-10705
>> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and
>> > >> FLINK-10706
>> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the
>> > discussion
>> > >> > came
>> > >> > >>> up
>> > >> > >>>> whether to change the underlying architecture of Flink's Web
>> > >> Dashboard
>> > >> > >>> from
>> > >> > >>>> Angular1 to something else. This email thread should be
>> solely to
>> > >> > discuss
>> > >> > >>>> the pro's and con's of this, and what could be the target
>> > >> > architecture.
>> > >> > >>>>
>> > >> > >>>> My choice would be React. Personally I agree with Till's
>> comments
>> > >> in
>> > >> > the
>> > >> > >>>> ticket, Angular 1 being basically outdated and is not having a
>> > >> large
>> > >> > >>>> following anymore. From my experience the choice between
>> Angular
>> > >> 2-7
>> > >> > or
>> > >> > >>>> React is subjective, you can get things done with both. I
>> > >> personally
>> > >> > only
>> > >> > >>>> have experience with React, so I personally would be faster
>> to
>> > >> > develop
>> > >> > >>>> with this one. I currently have not planned to learn Angular
>> as
>> > >> well
>> > >> > >>> (being
>> > >> > >>>> a more backend focused developer in general) so if the
>> decision
>> > >> would
>> > >> > be
>> > >> > >>> to
>> > >> > >>>> go with Angular, i would be unfortunately out of this rework
>> of
>> > the
>> > >> > Flink
>> > >> > >>>> Dashboard most certainly.
>> > >> > >>>>
>> > >> > >>>> Additionally i would like to get rid of bower, since its
>> > officially
>> > >> > >>>> deprecated <
>> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
>> > >> > >>>> my
>> > >> > >>>> idea would be to just use a create-react-app package with npm
>> and
>> > >> > webpack
>> > >> > >>>> under the hood. no need for additional lib's here imho. But
>> > again:
>> > >> > thats
>> > >> > >>>> mostly what i've been working with recently, so thats a
>> > subjective
>> > >> > >>> point. I
>> > >> > >>>> could imagine getting used to yarn in the future as well.
>> > >> > >>>>
>> > >> > >>>> Cheers
>> > >> > >>>> Fabian
>> > >> > >>>>
>> > >> > >>>> --
>> > >> > >>>>
>> > >> > >>>>
>> > >> > >>>> *Fabian WollertZalando SE*
>> > >> > >>>>
>> > >> > >>>> E-Mail: fabian@zalando.de
>> > >> > >>>>
>> > >> >
>> > >> >
>> > >>
>> > >
>> > >
>> > > --
>> > > Yadong Xie
>> > > Ministry of Education Key Lab for Intelligent Networks and Network
>> > Security
>> > > Dept. of Computer Science and Technology
>> > > School of electronic information and engineering
>> > > Xi'an Jiaotong University
>> > > Xi'an, 710049, P.R. China
>> > > Email: vthinkxie@gmail.com
>> > >
>> >
>>
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Fabian Wollert <fa...@zalando.de>.
Hi Yadong, this looks awesome. is there any chance you can already share
the code of the new web UI, so we can take a look at what you guys build
there? I think that would speed up the discussion. If there is already a
fully fledged new Version with everything updated out there, and its even
battle tested in production already, that sounds like the way to go for me.
i started aside from this discussion here (to strengthen and learn some new
React stuff) my attempt on the React version already, for whoever is
curious, you can check it out here:
https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base url in
actions.js
<https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8>,
npm install and then npm start) ... i just started to convert the first
simple pages, so dont expect the whole UI yet, its just a start ...
Cheers
--
*Fabian WollertZalando SE*
E-Mail: fabian@zalando.de
Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <zj...@gmail.com>:
> +1 for the proposal of Yadong. I'd like to add one more comment:
>
> We should provide more development guidance when developing the next
> dashboard via angular 7. As Till mentioned before, one big concern is lack
> of frontend guys in flink community, I believe more development guidance
> will attract more frontend guys to contribute to Flink web ui, and also be
> helpful for backend guys who want to add something to dashboard.
>
>
>
> Shaoxuan Wang <ws...@gmail.com>于2018年11月2日周五 上午10:06写道：
>
> > +1 to keep the web UI stack to Angular. This will make the stack
> upgrading
> > much easier and won't break the on-going efforts already made towards the
> > improvements of Flink Web UI.
> >
> > Till, Robert, and Fabian,
> > Please take a look at the upgrading plan from Angular 1.x to 7.0 proposed
> > by Yadong. Any feedback is appreciated.
> >
> > Regards,
> > Shaoxuan
> >
> >
> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vt...@gmail.com> wrote:
> >
> > > For your interest, here are some snapshots of Flink web UI (with
> angular
> > > 7.0) we recently reconstructed.
> > > [image: snapshot.001.jpeg]
> > >
> > >
> >
> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
> > >
> > > We choose Angular 7.0 due to the following reasons:
> > >
> > > 1. It is easy to migrate the current codebase to Angular 7.0
> following
> > > the official guide.
> > > 2. It is easy for current Web UI developers who are experienced with
> > > Angular 1.x to get familiar with 7.0 reading the official doc.
> > > 3. Migrating other than rewriting is helpful to ensure its
> functional
> > > integrity.
> > > 4. There are over 1 million Angular developers now (2.0 and above,
> > > still increasing rapidly), which means there are potentially more
> > > developers willing to join the community.
> > > 5. No bower or complex node / webpack configuration needed (only
> > > angular cli).
> > >
> > > We split Angular 1.x and 7.0 versions into two folders based on the
> > > following consideration:
> > >
> > > 1. 1.x version could be the default page because there is no
> > > breakchange for both users and developers.
> > > 2. Make the upgrade progressive. Users can switch between two
> versions
> > > by clicking the button on the top right corner.
> > > 3. Reduce maintenance costs and keep the performance of web without
> > > bootstrap both 1.x and 7.0 versions in the same page.
> > > 4. Won't interrupt the current development process. The code can
> still
> > > be submitted to 1.x version folder, we could help to sync all
> > features and
> > > bug fix to 7.0 version if needed.
> > > 5. When the 7.0 version is ready and stable, it is easy to change
> > > default page to it, and remove the old version.
> > >
> > > [image: test.jpg]
> > >
> > >
> > >
> >
> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
> > >
> > > Here is the brief list of what we have done:
> > >
> > > 1. Create a new folder named next-web-dashboard in
> flink-runtime-web.
> > > 2. Migrate all Angular 1.x controllers, services, config etc one by
> > > one into 7.0 version with ngMigration.
> > > 3. Modify frontend build scripts in order to package two versions
> > > together.
> > > 4. All works listed above are limited in the flink-runtime-web
> folder.
> > > One folder was added and no file or folder was deleted.
> > >
> > > The new version of the Web UI is running stable for several months in
> > > Alibaba. We are happy to contribute it to the Flink community if
> everyone
> > > like the work.
> > >
> > > Best
> > > Yadong
> > >
> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <tr...@apache.org>
> > > wrote:
> > >
> > >> I agree with Chesnay that the web UI is already fairly isolated in
> > Flink's
> > >> code base (see flink-runtime-web module). Moreover, imo it is very
> > >> important that Flink comes with a nice web UI which works out of the
> box
> > >> because it lowers the entrance hurdle massively. What one could argue
> is
> > >> to
> > >> make it easier to plug in your own web UI if you want to develop it.
> > But I
> > >> would see this as an orthogonal issue.
> > >>
> > >> I would actually argue against Robert's proposal to make a grand
> rewrite
> > >> if
> > >> this is by any means possible. Experience shows that these kind of
> > efforts
> > >> are usually set up to fail or take super long. If possible, then I
> would
> > >> rather prefer to convert one component at a time to React. But I'm not
> > an
> > >> expert here and cannot tell whether this is possible or not.
> > >>
> > >> What we would need is active help from the community to review these
> > >> changes. I think that is the crucial piece to make such an effort work
> > or
> > >> fail.
> > >>
> > >> Cheers,
> > >> Till
> > >>
> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <chesnay@apache.org
> >
> > >> wrote:
> > >>
> > >> > The WebUI goes entirely through the REST API.
> > >> >
> > >> > I don't see how moving it out would make it easier to
> > >> develop/contribute.
> > >> >
> > >> > The UI is already encapsulated in it's own module
> (flink-runtime-web)
> > >> > and changes can be made to it independently.
> > >> > Historically the issue was simply that contributions to the UI were
> > not
> > >> > reviewed in a timely manner / not at all, which mind you
> > >> > applies to virtually all components. This also won't change by
> moving
> > it
> > >> > out of Flink.
> > >> >
> > >> > Having it in Flink also guarantees that there is an
> > easily-discoverable
> > >> > version of the UI that is compatible with your Flink distribution.
> > >> >
> > >> > On 31.10.2018 09:53, dyana.rose wrote:
> > >> > > Re: who's using the web ui
> > >> > >
> > >> > > Though many mature solutions, with a fair amount of time/resources
> > >> > available are likely running their own front ends, for teams like
> mine
> > >> > which are smaller and aren't focused solely on working with Flink,
> > >> having
> > >> > the web ui available removes a large barrier to getting up and
> running
> > >> > quickly.
> > >> > >
> > >> > > But, with the web UI being in core Flink, it does make it a bit
> more
> > >> of
> > >> > a chore to contribute.
> > >> > >
> > >> > > I'm unaware of how necessary it is for the UI to deploy with
> Flink?
> > Is
> > >> > it making any calls that aren't through the REST API (and if so, can
> > >> those
> > >> > endpoints be added to the REST API)?
> > >> > >
> > >> > > In general I'd support moving it to its own package, making it
> > easier
> > >> to
> > >> > develop with your standard UI dev tools. This also allows the web UI
> > to
> > >> be
> > >> > developed and released independently to core Flink.
> > >> > >
> > >> > > Dyana
> > >> > >
> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de> wrote:
> > >> > >> Hi Till, I basically agree with all your points. i would stress
> the
> > >> > >> "dustiness" of the current architecture: the package manager used
> > >> > (bower)
> > >> > >> is deprecated since a long time, the chance for the builds of the
> > >> flink
> > >> > web
> > >> > >> dashboard not working anymore is increasing every day.
> > >> > >>
> > >> > >> About the knowledge in the community: Two days is not a lot of
> > time,
> > >> but
> > >> > >> interest in this topic seems to be minor anyways. Is someone
> using
> > >> the
> > >> > >> Flink Web Dashboard at all, or is everyone running their own
> custom
> > >> > >> solutions? Because if there is no interest in using the Web UI
> AND
> > no
> > >> > one
> > >> > >> interested in developing, there would be no need to package this
> as
> > >> > part of
> > >> > >> the official Flink package, but rather develop an independent
> > >> solution
> > >> > (I'm
> > >> > >> not voting for this right now, just putting it out), if at all.
> The
> > >> > >> official package could then just ship with the API, which other
> > >> > solutions
> > >> > >> can build upon. This solution could be from an agile point of
> view
> > >> also
> > >> > the
> > >> > >> best (enhanced testing, independent and more effective dev
> > workflow,
> > >> > etc.),
> > >> > >> but is bad for the usage of the Flink UI, because people need to
> > >> install
> > >> > >> two things individually (Flink and the web dashboard).
> > >> > >>
> > >> > >> How did the choice for Angular1 happen back then? Who was writing
> > the
> > >> > >> Dashboard in the first place?
> > >> > >>
> > >> > >> Cheers
> > >> > >>
> > >> > >> --
> > >> > >>
> > >> > >>
> > >> > >> *Fabian WollertZalando SE*
> > >> > >>
> > >> > >> E-Mail: fabian@zalando.de
> > >> > >>
> > >> > >>
> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> > >> > >> trohrmann@apache.org>:
> > >> > >>
> > >> > >>> Thanks for starting this discussion Fabian! I think our web UI
> > >> > technology
> > >> > >>> stack is quite dusty by now and it would be beneficial to think
> > >> about
> > >> > its
> > >> > >>> technological future.
> > >> > >>>
> > >> > >>> On the one hand, our current web UI works more or less reliable
> > and
> > >> > >>> changing the underlying technology has the risk of breaking
> > things.
> > >> > >>> Moreover, there might be the risk that the newly chosen
> technology
> > >> > will be
> > >> > >>> deprecated at some point in time as well.
> > >> > >>>
> > >> > >>> On the other hand, we don't have much Angular 1 knowledge in the
> > >> > community
> > >> > >>> and extending the web UI is, thus, quite hard at the moment.
> Maybe
> > >> by
> > >> > using
> > >> > >>> some newer web technologies we might be able to attract more
> > people
> > >> > with a
> > >> > >>> web technology background to join the community.
> > >> > >>>
> > >> > >>> The lack of people working on the web UI is for me the biggest
> > >> problem
> > >> > I
> > >> > >>> would like to address. If there is interest in the web UI, then
> > I'm
> > >> > quite
> > >> > >>> sure that we will be able to even migrate to some other
> technology
> > >> in
> > >> > the
> > >> > >>> future. The next important issue for me is to do the change
> > >> > incrementally
> > >> > >>> if possible. Ideally we never break the web UI in the process of
> > >> > migrating
> > >> > >>> to a new technology. I'm not an expert here so it might or might
> > >> not be
> > >> > >>> possible. But if it is, then we should design the implementation
> > >> steps
> > >> > in
> > >> > >>> such a way.
> > >> > >>>
> > >> > >>> Cheers,
> > >> > >>> Till
> > >> > >>>
> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <
> fabian@zalando.de
> > >
> > >> > wrote:
> > >> > >>>
> > >> > >>>> Hi everyone,
> > >> > >>>>
> > >> > >>>> in this email thread
> > >> > >>>> <
> > >> > >>>>
> > >> > >>>
> > >> >
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > >> > >>>> and the tickets FLINK-10705
> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and
> > >> FLINK-10706
> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the
> > discussion
> > >> > came
> > >> > >>> up
> > >> > >>>> whether to change the underlying architecture of Flink's Web
> > >> Dashboard
> > >> > >>> from
> > >> > >>>> Angular1 to something else. This email thread should be solely
> to
> > >> > discuss
> > >> > >>>> the pro's and con's of this, and what could be the target
> > >> > architecture.
> > >> > >>>>
> > >> > >>>> My choice would be React. Personally I agree with Till's
> comments
> > >> in
> > >> > the
> > >> > >>>> ticket, Angular 1 being basically outdated and is not having a
> > >> large
> > >> > >>>> following anymore. From my experience the choice between
> Angular
> > >> 2-7
> > >> > or
> > >> > >>>> React is subjective, you can get things done with both. I
> > >> personally
> > >> > only
> > >> > >>>> have experience with React, so I personally would be faster to
> > >> > develop
> > >> > >>>> with this one. I currently have not planned to learn Angular as
> > >> well
> > >> > >>> (being
> > >> > >>>> a more backend focused developer in general) so if the decision
> > >> would
> > >> > be
> > >> > >>> to
> > >> > >>>> go with Angular, i would be unfortunately out of this rework of
> > the
> > >> > Flink
> > >> > >>>> Dashboard most certainly.
> > >> > >>>>
> > >> > >>>> Additionally i would like to get rid of bower, since its
> > officially
> > >> > >>>> deprecated <
> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> > >> > >>>> my
> > >> > >>>> idea would be to just use a create-react-app package with npm
> and
> > >> > webpack
> > >> > >>>> under the hood. no need for additional lib's here imho. But
> > again:
> > >> > thats
> > >> > >>>> mostly what i've been working with recently, so thats a
> > subjective
> > >> > >>> point. I
> > >> > >>>> could imagine getting used to yarn in the future as well.
> > >> > >>>>
> > >> > >>>> Cheers
> > >> > >>>> Fabian
> > >> > >>>>
> > >> > >>>> --
> > >> > >>>>
> > >> > >>>>
> > >> > >>>> *Fabian WollertZalando SE*
> > >> > >>>>
> > >> > >>>> E-Mail: fabian@zalando.de
> > >> > >>>>
> > >> >
> > >> >
> > >>
> > >
> > >
> > > --
> > > Yadong Xie
> > > Ministry of Education Key Lab for Intelligent Networks and Network
> > Security
> > > Dept. of Computer Science and Technology
> > > School of electronic information and engineering
> > > Xi'an Jiaotong University
> > > Xi'an, 710049, P.R. China
> > > Email: vthinkxie@gmail.com
> > >
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Jeff Zhang <zj...@gmail.com>.
+1 for the proposal of Yadong. I'd like to add one more comment:
We should provide more development guidance when developing the next
dashboard via angular 7. As Till mentioned before, one big concern is lack
of frontend guys in flink community, I believe more development guidance
will attract more frontend guys to contribute to Flink web ui, and also be
helpful for backend guys who want to add something to dashboard.
Shaoxuan Wang <ws...@gmail.com>于2018年11月2日周五 上午10:06写道：
> +1 to keep the web UI stack to Angular. This will make the stack upgrading
> much easier and won't break the on-going efforts already made towards the
> improvements of Flink Web UI.
>
> Till, Robert, and Fabian,
> Please take a look at the upgrading plan from Angular 1.x to 7.0 proposed
> by Yadong. Any feedback is appreciated.
>
> Regards,
> Shaoxuan
>
>
> On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vt...@gmail.com> wrote:
>
> > For your interest, here are some snapshots of Flink web UI (with angular
> > 7.0) we recently reconstructed.
> > [image: snapshot.001.jpeg]
> >
> >
> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
> >
> > We choose Angular 7.0 due to the following reasons:
> >
> > 1. It is easy to migrate the current codebase to Angular 7.0 following
> > the official guide.
> > 2. It is easy for current Web UI developers who are experienced with
> > Angular 1.x to get familiar with 7.0 reading the official doc.
> > 3. Migrating other than rewriting is helpful to ensure its functional
> > integrity.
> > 4. There are over 1 million Angular developers now (2.0 and above,
> > still increasing rapidly), which means there are potentially more
> > developers willing to join the community.
> > 5. No bower or complex node / webpack configuration needed (only
> > angular cli).
> >
> > We split Angular 1.x and 7.0 versions into two folders based on the
> > following consideration:
> >
> > 1. 1.x version could be the default page because there is no
> > breakchange for both users and developers.
> > 2. Make the upgrade progressive. Users can switch between two versions
> > by clicking the button on the top right corner.
> > 3. Reduce maintenance costs and keep the performance of web without
> > bootstrap both 1.x and 7.0 versions in the same page.
> > 4. Won't interrupt the current development process. The code can still
> > be submitted to 1.x version folder, we could help to sync all
> features and
> > bug fix to 7.0 version if needed.
> > 5. When the 7.0 version is ready and stable, it is easy to change
> > default page to it, and remove the old version.
> >
> > [image: test.jpg]
> >
> >
> >
> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
> >
> > Here is the brief list of what we have done:
> >
> > 1. Create a new folder named next-web-dashboard in flink-runtime-web.
> > 2. Migrate all Angular 1.x controllers, services, config etc one by
> > one into 7.0 version with ngMigration.
> > 3. Modify frontend build scripts in order to package two versions
> > together.
> > 4. All works listed above are limited in the flink-runtime-web folder.
> > One folder was added and no file or folder was deleted.
> >
> > The new version of the Web UI is running stable for several months in
> > Alibaba. We are happy to contribute it to the Flink community if everyone
> > like the work.
> >
> > Best
> > Yadong
> >
> > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <tr...@apache.org>
> > wrote:
> >
> >> I agree with Chesnay that the web UI is already fairly isolated in
> Flink's
> >> code base (see flink-runtime-web module). Moreover, imo it is very
> >> important that Flink comes with a nice web UI which works out of the box
> >> because it lowers the entrance hurdle massively. What one could argue is
> >> to
> >> make it easier to plug in your own web UI if you want to develop it.
> But I
> >> would see this as an orthogonal issue.
> >>
> >> I would actually argue against Robert's proposal to make a grand rewrite
> >> if
> >> this is by any means possible. Experience shows that these kind of
> efforts
> >> are usually set up to fail or take super long. If possible, then I would
> >> rather prefer to convert one component at a time to React. But I'm not
> an
> >> expert here and cannot tell whether this is possible or not.
> >>
> >> What we would need is active help from the community to review these
> >> changes. I think that is the crucial piece to make such an effort work
> or
> >> fail.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <ch...@apache.org>
> >> wrote:
> >>
> >> > The WebUI goes entirely through the REST API.
> >> >
> >> > I don't see how moving it out would make it easier to
> >> develop/contribute.
> >> >
> >> > The UI is already encapsulated in it's own module (flink-runtime-web)
> >> > and changes can be made to it independently.
> >> > Historically the issue was simply that contributions to the UI were
> not
> >> > reviewed in a timely manner / not at all, which mind you
> >> > applies to virtually all components. This also won't change by moving
> it
> >> > out of Flink.
> >> >
> >> > Having it in Flink also guarantees that there is an
> easily-discoverable
> >> > version of the UI that is compatible with your Flink distribution.
> >> >
> >> > On 31.10.2018 09:53, dyana.rose wrote:
> >> > > Re: who's using the web ui
> >> > >
> >> > > Though many mature solutions, with a fair amount of time/resources
> >> > available are likely running their own front ends, for teams like mine
> >> > which are smaller and aren't focused solely on working with Flink,
> >> having
> >> > the web ui available removes a large barrier to getting up and running
> >> > quickly.
> >> > >
> >> > > But, with the web UI being in core Flink, it does make it a bit more
> >> of
> >> > a chore to contribute.
> >> > >
> >> > > I'm unaware of how necessary it is for the UI to deploy with Flink?
> Is
> >> > it making any calls that aren't through the REST API (and if so, can
> >> those
> >> > endpoints be added to the REST API)?
> >> > >
> >> > > In general I'd support moving it to its own package, making it
> easier
> >> to
> >> > develop with your standard UI dev tools. This also allows the web UI
> to
> >> be
> >> > developed and released independently to core Flink.
> >> > >
> >> > > Dyana
> >> > >
> >> > > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de> wrote:
> >> > >> Hi Till, I basically agree with all your points. i would stress the
> >> > >> "dustiness" of the current architecture: the package manager used
> >> > (bower)
> >> > >> is deprecated since a long time, the chance for the builds of the
> >> flink
> >> > web
> >> > >> dashboard not working anymore is increasing every day.
> >> > >>
> >> > >> About the knowledge in the community: Two days is not a lot of
> time,
> >> but
> >> > >> interest in this topic seems to be minor anyways. Is someone using
> >> the
> >> > >> Flink Web Dashboard at all, or is everyone running their own custom
> >> > >> solutions? Because if there is no interest in using the Web UI AND
> no
> >> > one
> >> > >> interested in developing, there would be no need to package this as
> >> > part of
> >> > >> the official Flink package, but rather develop an independent
> >> solution
> >> > (I'm
> >> > >> not voting for this right now, just putting it out), if at all. The
> >> > >> official package could then just ship with the API, which other
> >> > solutions
> >> > >> can build upon. This solution could be from an agile point of view
> >> also
> >> > the
> >> > >> best (enhanced testing, independent and more effective dev
> workflow,
> >> > etc.),
> >> > >> but is bad for the usage of the Flink UI, because people need to
> >> install
> >> > >> two things individually (Flink and the web dashboard).
> >> > >>
> >> > >> How did the choice for Angular1 happen back then? Who was writing
> the
> >> > >> Dashboard in the first place?
> >> > >>
> >> > >> Cheers
> >> > >>
> >> > >> --
> >> > >>
> >> > >>
> >> > >> *Fabian WollertZalando SE*
> >> > >>
> >> > >> E-Mail: fabian@zalando.de
> >> > >>
> >> > >>
> >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> >> > >> trohrmann@apache.org>:
> >> > >>
> >> > >>> Thanks for starting this discussion Fabian! I think our web UI
> >> > technology
> >> > >>> stack is quite dusty by now and it would be beneficial to think
> >> about
> >> > its
> >> > >>> technological future.
> >> > >>>
> >> > >>> On the one hand, our current web UI works more or less reliable
> and
> >> > >>> changing the underlying technology has the risk of breaking
> things.
> >> > >>> Moreover, there might be the risk that the newly chosen technology
> >> > will be
> >> > >>> deprecated at some point in time as well.
> >> > >>>
> >> > >>> On the other hand, we don't have much Angular 1 knowledge in the
> >> > community
> >> > >>> and extending the web UI is, thus, quite hard at the moment. Maybe
> >> by
> >> > using
> >> > >>> some newer web technologies we might be able to attract more
> people
> >> > with a
> >> > >>> web technology background to join the community.
> >> > >>>
> >> > >>> The lack of people working on the web UI is for me the biggest
> >> problem
> >> > I
> >> > >>> would like to address. If there is interest in the web UI, then
> I'm
> >> > quite
> >> > >>> sure that we will be able to even migrate to some other technology
> >> in
> >> > the
> >> > >>> future. The next important issue for me is to do the change
> >> > incrementally
> >> > >>> if possible. Ideally we never break the web UI in the process of
> >> > migrating
> >> > >>> to a new technology. I'm not an expert here so it might or might
> >> not be
> >> > >>> possible. But if it is, then we should design the implementation
> >> steps
> >> > in
> >> > >>> such a way.
> >> > >>>
> >> > >>> Cheers,
> >> > >>> Till
> >> > >>>
> >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fabian@zalando.de
> >
> >> > wrote:
> >> > >>>
> >> > >>>> Hi everyone,
> >> > >>>>
> >> > >>>> in this email thread
> >> > >>>> <
> >> > >>>>
> >> > >>>
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> >> > >>>> and the tickets FLINK-10705
> >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and
> >> FLINK-10706
> >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the
> discussion
> >> > came
> >> > >>> up
> >> > >>>> whether to change the underlying architecture of Flink's Web
> >> Dashboard
> >> > >>> from
> >> > >>>> Angular1 to something else. This email thread should be solely to
> >> > discuss
> >> > >>>> the pro's and con's of this, and what could be the target
> >> > architecture.
> >> > >>>>
> >> > >>>> My choice would be React. Personally I agree with Till's comments
> >> in
> >> > the
> >> > >>>> ticket, Angular 1 being basically outdated and is not having a
> >> large
> >> > >>>> following anymore. From my experience the choice between Angular
> >> 2-7
> >> > or
> >> > >>>> React is subjective, you can get things done with both. I
> >> personally
> >> > only
> >> > >>>> have experience with React, so I personally would be faster to
> >> > develop
> >> > >>>> with this one. I currently have not planned to learn Angular as
> >> well
> >> > >>> (being
> >> > >>>> a more backend focused developer in general) so if the decision
> >> would
> >> > be
> >> > >>> to
> >> > >>>> go with Angular, i would be unfortunately out of this rework of
> the
> >> > Flink
> >> > >>>> Dashboard most certainly.
> >> > >>>>
> >> > >>>> Additionally i would like to get rid of bower, since its
> officially
> >> > >>>> deprecated <
> >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> >> > >>>> my
> >> > >>>> idea would be to just use a create-react-app package with npm and
> >> > webpack
> >> > >>>> under the hood. no need for additional lib's here imho. But
> again:
> >> > thats
> >> > >>>> mostly what i've been working with recently, so thats a
> subjective
> >> > >>> point. I
> >> > >>>> could imagine getting used to yarn in the future as well.
> >> > >>>>
> >> > >>>> Cheers
> >> > >>>> Fabian
> >> > >>>>
> >> > >>>> --
> >> > >>>>
> >> > >>>>
> >> > >>>> *Fabian WollertZalando SE*
> >> > >>>>
> >> > >>>> E-Mail: fabian@zalando.de
> >> > >>>>
> >> >
> >> >
> >>
> >
> >
> > --
> > Yadong Xie
> > Ministry of Education Key Lab for Intelligent Networks and Network
> Security
> > Dept. of Computer Science and Technology
> > School of electronic information and engineering
> > Xi'an Jiaotong University
> > Xi'an, 710049, P.R. China
> > Email: vthinkxie@gmail.com
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Shaoxuan Wang <ws...@gmail.com>.
+1 to keep the web UI stack to Angular. This will make the stack upgrading
much easier and won't break the on-going efforts already made towards the
improvements of Flink Web UI.
Till, Robert, and Fabian,
Please take a look at the upgrading plan from Angular 1.x to 7.0 proposed
by Yadong. Any feedback is appreciated.
Regards,
Shaoxuan
On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vt...@gmail.com> wrote:
> For your interest, here are some snapshots of Flink web UI (with angular
> 7.0) we recently reconstructed.
> [image: snapshot.001.jpeg]
>
> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
>
> We choose Angular 7.0 due to the following reasons:
>
> 1. It is easy to migrate the current codebase to Angular 7.0 following
> the official guide.
> 2. It is easy for current Web UI developers who are experienced with
> Angular 1.x to get familiar with 7.0 reading the official doc.
> 3. Migrating other than rewriting is helpful to ensure its functional
> integrity.
> 4. There are over 1 million Angular developers now (2.0 and above,
> still increasing rapidly), which means there are potentially more
> developers willing to join the community.
> 5. No bower or complex node / webpack configuration needed (only
> angular cli).
>
> We split Angular 1.x and 7.0 versions into two folders based on the
> following consideration:
>
> 1. 1.x version could be the default page because there is no
> breakchange for both users and developers.
> 2. Make the upgrade progressive. Users can switch between two versions
> by clicking the button on the top right corner.
> 3. Reduce maintenance costs and keep the performance of web without
> bootstrap both 1.x and 7.0 versions in the same page.
> 4. Won't interrupt the current development process. The code can still
> be submitted to 1.x version folder, we could help to sync all features and
> bug fix to 7.0 version if needed.
> 5. When the 7.0 version is ready and stable, it is easy to change
> default page to it, and remove the old version.
>
> [image: test.jpg]
>
>
> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
>
> Here is the brief list of what we have done:
>
> 1. Create a new folder named next-web-dashboard in flink-runtime-web.
> 2. Migrate all Angular 1.x controllers, services, config etc one by
> one into 7.0 version with ngMigration.
> 3. Modify frontend build scripts in order to package two versions
> together.
> 4. All works listed above are limited in the flink-runtime-web folder.
> One folder was added and no file or folder was deleted.
>
> The new version of the Web UI is running stable for several months in
> Alibaba. We are happy to contribute it to the Flink community if everyone
> like the work.
>
> Best
> Yadong
>
> On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <tr...@apache.org>
> wrote:
>
>> I agree with Chesnay that the web UI is already fairly isolated in Flink's
>> code base (see flink-runtime-web module). Moreover, imo it is very
>> important that Flink comes with a nice web UI which works out of the box
>> because it lowers the entrance hurdle massively. What one could argue is
>> to
>> make it easier to plug in your own web UI if you want to develop it. But I
>> would see this as an orthogonal issue.
>>
>> I would actually argue against Robert's proposal to make a grand rewrite
>> if
>> this is by any means possible. Experience shows that these kind of efforts
>> are usually set up to fail or take super long. If possible, then I would
>> rather prefer to convert one component at a time to React. But I'm not an
>> expert here and cannot tell whether this is possible or not.
>>
>> What we would need is active help from the community to review these
>> changes. I think that is the crucial piece to make such an effort work or
>> fail.
>>
>> Cheers,
>> Till
>>
>> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <ch...@apache.org>
>> wrote:
>>
>> > The WebUI goes entirely through the REST API.
>> >
>> > I don't see how moving it out would make it easier to
>> develop/contribute.
>> >
>> > The UI is already encapsulated in it's own module (flink-runtime-web)
>> > and changes can be made to it independently.
>> > Historically the issue was simply that contributions to the UI were not
>> > reviewed in a timely manner / not at all, which mind you
>> > applies to virtually all components. This also won't change by moving it
>> > out of Flink.
>> >
>> > Having it in Flink also guarantees that there is an easily-discoverable
>> > version of the UI that is compatible with your Flink distribution.
>> >
>> > On 31.10.2018 09:53, dyana.rose wrote:
>> > > Re: who's using the web ui
>> > >
>> > > Though many mature solutions, with a fair amount of time/resources
>> > available are likely running their own front ends, for teams like mine
>> > which are smaller and aren't focused solely on working with Flink,
>> having
>> > the web ui available removes a large barrier to getting up and running
>> > quickly.
>> > >
>> > > But, with the web UI being in core Flink, it does make it a bit more
>> of
>> > a chore to contribute.
>> > >
>> > > I'm unaware of how necessary it is for the UI to deploy with Flink? Is
>> > it making any calls that aren't through the REST API (and if so, can
>> those
>> > endpoints be added to the REST API)?
>> > >
>> > > In general I'd support moving it to its own package, making it easier
>> to
>> > develop with your standard UI dev tools. This also allows the web UI to
>> be
>> > developed and released independently to core Flink.
>> > >
>> > > Dyana
>> > >
>> > > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de> wrote:
>> > >> Hi Till, I basically agree with all your points. i would stress the
>> > >> "dustiness" of the current architecture: the package manager used
>> > (bower)
>> > >> is deprecated since a long time, the chance for the builds of the
>> flink
>> > web
>> > >> dashboard not working anymore is increasing every day.
>> > >>
>> > >> About the knowledge in the community: Two days is not a lot of time,
>> but
>> > >> interest in this topic seems to be minor anyways. Is someone using
>> the
>> > >> Flink Web Dashboard at all, or is everyone running their own custom
>> > >> solutions? Because if there is no interest in using the Web UI AND no
>> > one
>> > >> interested in developing, there would be no need to package this as
>> > part of
>> > >> the official Flink package, but rather develop an independent
>> solution
>> > (I'm
>> > >> not voting for this right now, just putting it out), if at all. The
>> > >> official package could then just ship with the API, which other
>> > solutions
>> > >> can build upon. This solution could be from an agile point of view
>> also
>> > the
>> > >> best (enhanced testing, independent and more effective dev workflow,
>> > etc.),
>> > >> but is bad for the usage of the Flink UI, because people need to
>> install
>> > >> two things individually (Flink and the web dashboard).
>> > >>
>> > >> How did the choice for Angular1 happen back then? Who was writing the
>> > >> Dashboard in the first place?
>> > >>
>> > >> Cheers
>> > >>
>> > >> --
>> > >>
>> > >>
>> > >> *Fabian WollertZalando SE*
>> > >>
>> > >> E-Mail: fabian@zalando.de
>> > >>
>> > >>
>> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
>> > >> trohrmann@apache.org>:
>> > >>
>> > >>> Thanks for starting this discussion Fabian! I think our web UI
>> > technology
>> > >>> stack is quite dusty by now and it would be beneficial to think
>> about
>> > its
>> > >>> technological future.
>> > >>>
>> > >>> On the one hand, our current web UI works more or less reliable and
>> > >>> changing the underlying technology has the risk of breaking things.
>> > >>> Moreover, there might be the risk that the newly chosen technology
>> > will be
>> > >>> deprecated at some point in time as well.
>> > >>>
>> > >>> On the other hand, we don't have much Angular 1 knowledge in the
>> > community
>> > >>> and extending the web UI is, thus, quite hard at the moment. Maybe
>> by
>> > using
>> > >>> some newer web technologies we might be able to attract more people
>> > with a
>> > >>> web technology background to join the community.
>> > >>>
>> > >>> The lack of people working on the web UI is for me the biggest
>> problem
>> > I
>> > >>> would like to address. If there is interest in the web UI, then I'm
>> > quite
>> > >>> sure that we will be able to even migrate to some other technology
>> in
>> > the
>> > >>> future. The next important issue for me is to do the change
>> > incrementally
>> > >>> if possible. Ideally we never break the web UI in the process of
>> > migrating
>> > >>> to a new technology. I'm not an expert here so it might or might
>> not be
>> > >>> possible. But if it is, then we should design the implementation
>> steps
>> > in
>> > >>> such a way.
>> > >>>
>> > >>> Cheers,
>> > >>> Till
>> > >>>
>> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de>
>> > wrote:
>> > >>>
>> > >>>> Hi everyone,
>> > >>>>
>> > >>>> in this email thread
>> > >>>> <
>> > >>>>
>> > >>>
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
>> > >>>> and the tickets FLINK-10705
>> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and
>> FLINK-10706
>> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the discussion
>> > came
>> > >>> up
>> > >>>> whether to change the underlying architecture of Flink's Web
>> Dashboard
>> > >>> from
>> > >>>> Angular1 to something else. This email thread should be solely to
>> > discuss
>> > >>>> the pro's and con's of this, and what could be the target
>> > architecture.
>> > >>>>
>> > >>>> My choice would be React. Personally I agree with Till's comments
>> in
>> > the
>> > >>>> ticket, Angular 1 being basically outdated and is not having a
>> large
>> > >>>> following anymore. From my experience the choice between Angular
>> 2-7
>> > or
>> > >>>> React is subjective, you can get things done with both. I
>> personally
>> > only
>> > >>>> have experience with React, so I personally would be faster to
>> > develop
>> > >>>> with this one. I currently have not planned to learn Angular as
>> well
>> > >>> (being
>> > >>>> a more backend focused developer in general) so if the decision
>> would
>> > be
>> > >>> to
>> > >>>> go with Angular, i would be unfortunately out of this rework of the
>> > Flink
>> > >>>> Dashboard most certainly.
>> > >>>>
>> > >>>> Additionally i would like to get rid of bower, since its officially
>> > >>>> deprecated <
>> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
>> > >>>> my
>> > >>>> idea would be to just use a create-react-app package with npm and
>> > webpack
>> > >>>> under the hood. no need for additional lib's here imho. But again:
>> > thats
>> > >>>> mostly what i've been working with recently, so thats a subjective
>> > >>> point. I
>> > >>>> could imagine getting used to yarn in the future as well.
>> > >>>>
>> > >>>> Cheers
>> > >>>> Fabian
>> > >>>>
>> > >>>> --
>> > >>>>
>> > >>>>
>> > >>>> *Fabian WollertZalando SE*
>> > >>>>
>> > >>>> E-Mail: fabian@zalando.de
>> > >>>>
>> >
>> >
>>
>
>
> --
> Yadong Xie
> Ministry of Education Key Lab for Intelligent Networks and Network Security
> Dept. of Computer Science and Technology
> School of electronic information and engineering
> Xi'an Jiaotong University
> Xi'an, 710049, P.R. China
> Email: vthinkxie@gmail.com
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Yadong Xie <vt...@gmail.com>.
For your interest, here are some snapshots of Flink web UI (with angular
7.0) we recently reconstructed.
[image: snapshot.001.jpeg]
https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
We choose Angular 7.0 due to the following reasons:
1. It is easy to migrate the current codebase to Angular 7.0 following
the official guide.
2. It is easy for current Web UI developers who are experienced with
Angular 1.x to get familiar with 7.0 reading the official doc.
3. Migrating other than rewriting is helpful to ensure its functional
integrity.
4. There are over 1 million Angular developers now (2.0 and above, still
increasing rapidly), which means there are potentially more developers
willing to join the community.
5. No bower or complex node / webpack configuration needed (only angular
cli).
We split Angular 1.x and 7.0 versions into two folders based on the
following consideration:
1. 1.x version could be the default page because there is no breakchange
for both users and developers.
2. Make the upgrade progressive. Users can switch between two versions
by clicking the button on the top right corner.
3. Reduce maintenance costs and keep the performance of web without
bootstrap both 1.x and 7.0 versions in the same page.
4. Won't interrupt the current development process. The code can still
be submitted to 1.x version folder, we could help to sync all features and
bug fix to 7.0 version if needed.
5. When the 7.0 version is ready and stable, it is easy to change
default page to it, and remove the old version.
[image: test.jpg]
https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
Here is the brief list of what we have done:
1. Create a new folder named next-web-dashboard in flink-runtime-web.
2. Migrate all Angular 1.x controllers, services, config etc one by one
into 7.0 version with ngMigration.
3. Modify frontend build scripts in order to package two versions
together.
4. All works listed above are limited in the flink-runtime-web folder.
One folder was added and no file or folder was deleted.
The new version of the Web UI is running stable for several months in
Alibaba. We are happy to contribute it to the Flink community if everyone
like the work.
Best
Yadong
On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <tr...@apache.org> wrote:
> I agree with Chesnay that the web UI is already fairly isolated in Flink's
> code base (see flink-runtime-web module). Moreover, imo it is very
> important that Flink comes with a nice web UI which works out of the box
> because it lowers the entrance hurdle massively. What one could argue is to
> make it easier to plug in your own web UI if you want to develop it. But I
> would see this as an orthogonal issue.
>
> I would actually argue against Robert's proposal to make a grand rewrite if
> this is by any means possible. Experience shows that these kind of efforts
> are usually set up to fail or take super long. If possible, then I would
> rather prefer to convert one component at a time to React. But I'm not an
> expert here and cannot tell whether this is possible or not.
>
> What we would need is active help from the community to review these
> changes. I think that is the crucial piece to make such an effort work or
> fail.
>
> Cheers,
> Till
>
> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> > The WebUI goes entirely through the REST API.
> >
> > I don't see how moving it out would make it easier to develop/contribute.
> >
> > The UI is already encapsulated in it's own module (flink-runtime-web)
> > and changes can be made to it independently.
> > Historically the issue was simply that contributions to the UI were not
> > reviewed in a timely manner / not at all, which mind you
> > applies to virtually all components. This also won't change by moving it
> > out of Flink.
> >
> > Having it in Flink also guarantees that there is an easily-discoverable
> > version of the UI that is compatible with your Flink distribution.
> >
> > On 31.10.2018 09:53, dyana.rose wrote:
> > > Re: who's using the web ui
> > >
> > > Though many mature solutions, with a fair amount of time/resources
> > available are likely running their own front ends, for teams like mine
> > which are smaller and aren't focused solely on working with Flink, having
> > the web ui available removes a large barrier to getting up and running
> > quickly.
> > >
> > > But, with the web UI being in core Flink, it does make it a bit more of
> > a chore to contribute.
> > >
> > > I'm unaware of how necessary it is for the UI to deploy with Flink? Is
> > it making any calls that aren't through the REST API (and if so, can
> those
> > endpoints be added to the REST API)?
> > >
> > > In general I'd support moving it to its own package, making it easier
> to
> > develop with your standard UI dev tools. This also allows the web UI to
> be
> > developed and released independently to core Flink.
> > >
> > > Dyana
> > >
> > > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de> wrote:
> > >> Hi Till, I basically agree with all your points. i would stress the
> > >> "dustiness" of the current architecture: the package manager used
> > (bower)
> > >> is deprecated since a long time, the chance for the builds of the
> flink
> > web
> > >> dashboard not working anymore is increasing every day.
> > >>
> > >> About the knowledge in the community: Two days is not a lot of time,
> but
> > >> interest in this topic seems to be minor anyways. Is someone using the
> > >> Flink Web Dashboard at all, or is everyone running their own custom
> > >> solutions? Because if there is no interest in using the Web UI AND no
> > one
> > >> interested in developing, there would be no need to package this as
> > part of
> > >> the official Flink package, but rather develop an independent solution
> > (I'm
> > >> not voting for this right now, just putting it out), if at all. The
> > >> official package could then just ship with the API, which other
> > solutions
> > >> can build upon. This solution could be from an agile point of view
> also
> > the
> > >> best (enhanced testing, independent and more effective dev workflow,
> > etc.),
> > >> but is bad for the usage of the Flink UI, because people need to
> install
> > >> two things individually (Flink and the web dashboard).
> > >>
> > >> How did the choice for Angular1 happen back then? Who was writing the
> > >> Dashboard in the first place?
> > >>
> > >> Cheers
> > >>
> > >> --
> > >>
> > >>
> > >> *Fabian WollertZalando SE*
> > >>
> > >> E-Mail: fabian@zalando.de
> > >>
> > >>
> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> > >> trohrmann@apache.org>:
> > >>
> > >>> Thanks for starting this discussion Fabian! I think our web UI
> > technology
> > >>> stack is quite dusty by now and it would be beneficial to think about
> > its
> > >>> technological future.
> > >>>
> > >>> On the one hand, our current web UI works more or less reliable and
> > >>> changing the underlying technology has the risk of breaking things.
> > >>> Moreover, there might be the risk that the newly chosen technology
> > will be
> > >>> deprecated at some point in time as well.
> > >>>
> > >>> On the other hand, we don't have much Angular 1 knowledge in the
> > community
> > >>> and extending the web UI is, thus, quite hard at the moment. Maybe by
> > using
> > >>> some newer web technologies we might be able to attract more people
> > with a
> > >>> web technology background to join the community.
> > >>>
> > >>> The lack of people working on the web UI is for me the biggest
> problem
> > I
> > >>> would like to address. If there is interest in the web UI, then I'm
> > quite
> > >>> sure that we will be able to even migrate to some other technology in
> > the
> > >>> future. The next important issue for me is to do the change
> > incrementally
> > >>> if possible. Ideally we never break the web UI in the process of
> > migrating
> > >>> to a new technology. I'm not an expert here so it might or might not
> be
> > >>> possible. But if it is, then we should design the implementation
> steps
> > in
> > >>> such a way.
> > >>>
> > >>> Cheers,
> > >>> Till
> > >>>
> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de>
> > wrote:
> > >>>
> > >>>> Hi everyone,
> > >>>>
> > >>>> in this email thread
> > >>>> <
> > >>>>
> > >>>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > >>>> and the tickets FLINK-10705
> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the discussion
> > came
> > >>> up
> > >>>> whether to change the underlying architecture of Flink's Web
> Dashboard
> > >>> from
> > >>>> Angular1 to something else. This email thread should be solely to
> > discuss
> > >>>> the pro's and con's of this, and what could be the target
> > architecture.
> > >>>>
> > >>>> My choice would be React. Personally I agree with Till's comments in
> > the
> > >>>> ticket, Angular 1 being basically outdated and is not having a large
> > >>>> following anymore. From my experience the choice between Angular 2-7
> > or
> > >>>> React is subjective, you can get things done with both. I personally
> > only
> > >>>> have experience with React, so I personally would be faster to
> > develop
> > >>>> with this one. I currently have not planned to learn Angular as well
> > >>> (being
> > >>>> a more backend focused developer in general) so if the decision
> would
> > be
> > >>> to
> > >>>> go with Angular, i would be unfortunately out of this rework of the
> > Flink
> > >>>> Dashboard most certainly.
> > >>>>
> > >>>> Additionally i would like to get rid of bower, since its officially
> > >>>> deprecated <
> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> > >>>> my
> > >>>> idea would be to just use a create-react-app package with npm and
> > webpack
> > >>>> under the hood. no need for additional lib's here imho. But again:
> > thats
> > >>>> mostly what i've been working with recently, so thats a subjective
> > >>> point. I
> > >>>> could imagine getting used to yarn in the future as well.
> > >>>>
> > >>>> Cheers
> > >>>> Fabian
> > >>>>
> > >>>> --
> > >>>>
> > >>>>
> > >>>> *Fabian WollertZalando SE*
> > >>>>
> > >>>> E-Mail: fabian@zalando.de
> > >>>>
> >
> >
>
--
Yadong Xie
Ministry of Education Key Lab for Intelligent Networks and Network Security
Dept. of Computer Science and Technology
School of electronic information and engineering
Xi'an Jiaotong University
Xi'an, 710049, P.R. China
Email: vthinkxie@gmail.com
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Till Rohrmann <tr...@apache.org>.
I agree with Chesnay that the web UI is already fairly isolated in Flink's
code base (see flink-runtime-web module). Moreover, imo it is very
important that Flink comes with a nice web UI which works out of the box
because it lowers the entrance hurdle massively. What one could argue is to
make it easier to plug in your own web UI if you want to develop it. But I
would see this as an orthogonal issue.
I would actually argue against Robert's proposal to make a grand rewrite if
this is by any means possible. Experience shows that these kind of efforts
are usually set up to fail or take super long. If possible, then I would
rather prefer to convert one component at a time to React. But I'm not an
expert here and cannot tell whether this is possible or not.
What we would need is active help from the community to review these
changes. I think that is the crucial piece to make such an effort work or
fail.
Cheers,
Till
On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <ch...@apache.org>
wrote:
> The WebUI goes entirely through the REST API.
>
> I don't see how moving it out would make it easier to develop/contribute.
>
> The UI is already encapsulated in it's own module (flink-runtime-web)
> and changes can be made to it independently.
> Historically the issue was simply that contributions to the UI were not
> reviewed in a timely manner / not at all, which mind you
> applies to virtually all components. This also won't change by moving it
> out of Flink.
>
> Having it in Flink also guarantees that there is an easily-discoverable
> version of the UI that is compatible with your Flink distribution.
>
> On 31.10.2018 09:53, dyana.rose wrote:
> > Re: who's using the web ui
> >
> > Though many mature solutions, with a fair amount of time/resources
> available are likely running their own front ends, for teams like mine
> which are smaller and aren't focused solely on working with Flink, having
> the web ui available removes a large barrier to getting up and running
> quickly.
> >
> > But, with the web UI being in core Flink, it does make it a bit more of
> a chore to contribute.
> >
> > I'm unaware of how necessary it is for the UI to deploy with Flink? Is
> it making any calls that aren't through the REST API (and if so, can those
> endpoints be added to the REST API)?
> >
> > In general I'd support moving it to its own package, making it easier to
> develop with your standard UI dev tools. This also allows the web UI to be
> developed and released independently to core Flink.
> >
> > Dyana
> >
> > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de> wrote:
> >> Hi Till, I basically agree with all your points. i would stress the
> >> "dustiness" of the current architecture: the package manager used
> (bower)
> >> is deprecated since a long time, the chance for the builds of the flink
> web
> >> dashboard not working anymore is increasing every day.
> >>
> >> About the knowledge in the community: Two days is not a lot of time, but
> >> interest in this topic seems to be minor anyways. Is someone using the
> >> Flink Web Dashboard at all, or is everyone running their own custom
> >> solutions? Because if there is no interest in using the Web UI AND no
> one
> >> interested in developing, there would be no need to package this as
> part of
> >> the official Flink package, but rather develop an independent solution
> (I'm
> >> not voting for this right now, just putting it out), if at all. The
> >> official package could then just ship with the API, which other
> solutions
> >> can build upon. This solution could be from an agile point of view also
> the
> >> best (enhanced testing, independent and more effective dev workflow,
> etc.),
> >> but is bad for the usage of the Flink UI, because people need to install
> >> two things individually (Flink and the web dashboard).
> >>
> >> How did the choice for Angular1 happen back then? Who was writing the
> >> Dashboard in the first place?
> >>
> >> Cheers
> >>
> >> --
> >>
> >>
> >> *Fabian WollertZalando SE*
> >>
> >> E-Mail: fabian@zalando.de
> >>
> >>
> >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> >> trohrmann@apache.org>:
> >>
> >>> Thanks for starting this discussion Fabian! I think our web UI
> technology
> >>> stack is quite dusty by now and it would be beneficial to think about
> its
> >>> technological future.
> >>>
> >>> On the one hand, our current web UI works more or less reliable and
> >>> changing the underlying technology has the risk of breaking things.
> >>> Moreover, there might be the risk that the newly chosen technology
> will be
> >>> deprecated at some point in time as well.
> >>>
> >>> On the other hand, we don't have much Angular 1 knowledge in the
> community
> >>> and extending the web UI is, thus, quite hard at the moment. Maybe by
> using
> >>> some newer web technologies we might be able to attract more people
> with a
> >>> web technology background to join the community.
> >>>
> >>> The lack of people working on the web UI is for me the biggest problem
> I
> >>> would like to address. If there is interest in the web UI, then I'm
> quite
> >>> sure that we will be able to even migrate to some other technology in
> the
> >>> future. The next important issue for me is to do the change
> incrementally
> >>> if possible. Ideally we never break the web UI in the process of
> migrating
> >>> to a new technology. I'm not an expert here so it might or might not be
> >>> possible. But if it is, then we should design the implementation steps
> in
> >>> such a way.
> >>>
> >>> Cheers,
> >>> Till
> >>>
> >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de>
> wrote:
> >>>
> >>>> Hi everyone,
> >>>>
> >>>> in this email thread
> >>>> <
> >>>>
> >>>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> >>>> and the tickets FLINK-10705
> >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
> >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the discussion
> came
> >>> up
> >>>> whether to change the underlying architecture of Flink's Web Dashboard
> >>> from
> >>>> Angular1 to something else. This email thread should be solely to
> discuss
> >>>> the pro's and con's of this, and what could be the target
> architecture.
> >>>>
> >>>> My choice would be React. Personally I agree with Till's comments in
> the
> >>>> ticket, Angular 1 being basically outdated and is not having a large
> >>>> following anymore. From my experience the choice between Angular 2-7
> or
> >>>> React is subjective, you can get things done with both. I personally
> only
> >>>> have experience with React, so I personally would be faster to
> develop
> >>>> with this one. I currently have not planned to learn Angular as well
> >>> (being
> >>>> a more backend focused developer in general) so if the decision would
> be
> >>> to
> >>>> go with Angular, i would be unfortunately out of this rework of the
> Flink
> >>>> Dashboard most certainly.
> >>>>
> >>>> Additionally i would like to get rid of bower, since its officially
> >>>> deprecated <
> https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> >>>> my
> >>>> idea would be to just use a create-react-app package with npm and
> webpack
> >>>> under the hood. no need for additional lib's here imho. But again:
> thats
> >>>> mostly what i've been working with recently, so thats a subjective
> >>> point. I
> >>>> could imagine getting used to yarn in the future as well.
> >>>>
> >>>> Cheers
> >>>> Fabian
> >>>>
> >>>> --
> >>>>
> >>>>
> >>>> *Fabian WollertZalando SE*
> >>>>
> >>>> E-Mail: fabian@zalando.de
> >>>>
>
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web
Dashboard
Posted by Chesnay Schepler <ch...@apache.org>.
The WebUI goes entirely through the REST API.
I don't see how moving it out would make it easier to develop/contribute.
The UI is already encapsulated in it's own module (flink-runtime-web)
and changes can be made to it independently.
Historically the issue was simply that contributions to the UI were not
reviewed in a timely manner / not at all, which mind you
applies to virtually all components. This also won't change by moving it
out of Flink.
Having it in Flink also guarantees that there is an easily-discoverable
version of the UI that is compatible with your Flink distribution.
On 31.10.2018 09:53, dyana.rose wrote:
> Re: who's using the web ui
>
> Though many mature solutions, with a fair amount of time/resources available are likely running their own front ends, for teams like mine which are smaller and aren't focused solely on working with Flink, having the web ui available removes a large barrier to getting up and running quickly.
>
> But, with the web UI being in core Flink, it does make it a bit more of a chore to contribute.
>
> I'm unaware of how necessary it is for the UI to deploy with Flink? Is it making any calls that aren't through the REST API (and if so, can those endpoints be added to the REST API)?
>
> In general I'd support moving it to its own package, making it easier to develop with your standard UI dev tools. This also allows the web UI to be developed and released independently to core Flink.
>
> Dyana
>
> On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de> wrote:
>> Hi Till, I basically agree with all your points. i would stress the
>> "dustiness" of the current architecture: the package manager used (bower)
>> is deprecated since a long time, the chance for the builds of the flink web
>> dashboard not working anymore is increasing every day.
>>
>> About the knowledge in the community: Two days is not a lot of time, but
>> interest in this topic seems to be minor anyways. Is someone using the
>> Flink Web Dashboard at all, or is everyone running their own custom
>> solutions? Because if there is no interest in using the Web UI AND no one
>> interested in developing, there would be no need to package this as part of
>> the official Flink package, but rather develop an independent solution (I'm
>> not voting for this right now, just putting it out), if at all. The
>> official package could then just ship with the API, which other solutions
>> can build upon. This solution could be from an agile point of view also the
>> best (enhanced testing, independent and more effective dev workflow, etc.),
>> but is bad for the usage of the Flink UI, because people need to install
>> two things individually (Flink and the web dashboard).
>>
>> How did the choice for Angular1 happen back then? Who was writing the
>> Dashboard in the first place?
>>
>> Cheers
>>
>> --
>>
>>
>> *Fabian WollertZalando SE*
>>
>> E-Mail: fabian@zalando.de
>>
>>
>> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
>> trohrmann@apache.org>:
>>
>>> Thanks for starting this discussion Fabian! I think our web UI technology
>>> stack is quite dusty by now and it would be beneficial to think about its
>>> technological future.
>>>
>>> On the one hand, our current web UI works more or less reliable and
>>> changing the underlying technology has the risk of breaking things.
>>> Moreover, there might be the risk that the newly chosen technology will be
>>> deprecated at some point in time as well.
>>>
>>> On the other hand, we don't have much Angular 1 knowledge in the community
>>> and extending the web UI is, thus, quite hard at the moment. Maybe by using
>>> some newer web technologies we might be able to attract more people with a
>>> web technology background to join the community.
>>>
>>> The lack of people working on the web UI is for me the biggest problem I
>>> would like to address. If there is interest in the web UI, then I'm quite
>>> sure that we will be able to even migrate to some other technology in the
>>> future. The next important issue for me is to do the change incrementally
>>> if possible. Ideally we never break the web UI in the process of migrating
>>> to a new technology. I'm not an expert here so it might or might not be
>>> possible. But if it is, then we should design the implementation steps in
>>> such a way.
>>>
>>> Cheers,
>>> Till
>>>
>>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> in this email thread
>>>> <
>>>>
>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
>>>> and the tickets FLINK-10705
>>>> <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
>>>> <https://issues.apache.org/jira/browse/FLINK-10706> the discussion came
>>> up
>>>> whether to change the underlying architecture of Flink's Web Dashboard
>>> from
>>>> Angular1 to something else. This email thread should be solely to discuss
>>>> the pro's and con's of this, and what could be the target architecture.
>>>>
>>>> My choice would be React. Personally I agree with Till's comments in the
>>>> ticket, Angular 1 being basically outdated and is not having a large
>>>> following anymore. From my experience the choice between Angular 2-7 or
>>>> React is subjective, you can get things done with both. I personally only
>>>> have experience with React, so I personally would be faster to develop
>>>> with this one. I currently have not planned to learn Angular as well
>>> (being
>>>> a more backend focused developer in general) so if the decision would be
>>> to
>>>> go with Angular, i would be unfortunately out of this rework of the Flink
>>>> Dashboard most certainly.
>>>>
>>>> Additionally i would like to get rid of bower, since its officially
>>>> deprecated <https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
>>>> my
>>>> idea would be to just use a create-react-app package with npm and webpack
>>>> under the hood. no need for additional lib's here imho. But again: thats
>>>> mostly what i've been working with recently, so thats a subjective
>>> point. I
>>>> could imagine getting used to yarn in the future as well.
>>>>
>>>> Cheers
>>>> Fabian
>>>>
>>>> --
>>>>
>>>>
>>>> *Fabian WollertZalando SE*
>>>>
>>>> E-Mail: fabian@zalando.de
>>>>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by "dyana.rose" <dy...@salecycle.com>.
Re: who's using the web ui
Though many mature solutions, with a fair amount of time/resources available are likely running their own front ends, for teams like mine which are smaller and aren't focused solely on working with Flink, having the web ui available removes a large barrier to getting up and running quickly.
But, with the web UI being in core Flink, it does make it a bit more of a chore to contribute.
I'm unaware of how necessary it is for the UI to deploy with Flink? Is it making any calls that aren't through the REST API (and if so, can those endpoints be added to the REST API)?
In general I'd support moving it to its own package, making it easier to develop with your standard UI dev tools. This also allows the web UI to be developed and released independently to core Flink.
Dyana
On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de> wrote:
> Hi Till, I basically agree with all your points. i would stress the
> "dustiness" of the current architecture: the package manager used (bower)
> is deprecated since a long time, the chance for the builds of the flink web
> dashboard not working anymore is increasing every day.
>
> About the knowledge in the community: Two days is not a lot of time, but
> interest in this topic seems to be minor anyways. Is someone using the
> Flink Web Dashboard at all, or is everyone running their own custom
> solutions? Because if there is no interest in using the Web UI AND no one
> interested in developing, there would be no need to package this as part of
> the official Flink package, but rather develop an independent solution (I'm
> not voting for this right now, just putting it out), if at all. The
> official package could then just ship with the API, which other solutions
> can build upon. This solution could be from an agile point of view also the
> best (enhanced testing, independent and more effective dev workflow, etc.),
> but is bad for the usage of the Flink UI, because people need to install
> two things individually (Flink and the web dashboard).
>
> How did the choice for Angular1 happen back then? Who was writing the
> Dashboard in the first place?
>
> Cheers
>
> --
>
>
> *Fabian WollertZalando SE*
>
> E-Mail: fabian@zalando.de
>
>
> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> trohrmann@apache.org>:
>
> > Thanks for starting this discussion Fabian! I think our web UI technology
> > stack is quite dusty by now and it would be beneficial to think about its
> > technological future.
> >
> > On the one hand, our current web UI works more or less reliable and
> > changing the underlying technology has the risk of breaking things.
> > Moreover, there might be the risk that the newly chosen technology will be
> > deprecated at some point in time as well.
> >
> > On the other hand, we don't have much Angular 1 knowledge in the community
> > and extending the web UI is, thus, quite hard at the moment. Maybe by using
> > some newer web technologies we might be able to attract more people with a
> > web technology background to join the community.
> >
> > The lack of people working on the web UI is for me the biggest problem I
> > would like to address. If there is interest in the web UI, then I'm quite
> > sure that we will be able to even migrate to some other technology in the
> > future. The next important issue for me is to do the change incrementally
> > if possible. Ideally we never break the web UI in the process of migrating
> > to a new technology. I'm not an expert here so it might or might not be
> > possible. But if it is, then we should design the implementation steps in
> > such a way.
> >
> > Cheers,
> > Till
> >
> > On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de> wrote:
> >
> > > Hi everyone,
> > >
> > > in this email thread
> > > <
> > >
> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > > >
> > > and the tickets FLINK-10705
> > > <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
> > > <https://issues.apache.org/jira/browse/FLINK-10706> the discussion came
> > up
> > > whether to change the underlying architecture of Flink's Web Dashboard
> > from
> > > Angular1 to something else. This email thread should be solely to discuss
> > > the pro's and con's of this, and what could be the target architecture.
> > >
> > > My choice would be React. Personally I agree with Till's comments in the
> > > ticket, Angular 1 being basically outdated and is not having a large
> > > following anymore. From my experience the choice between Angular 2-7 or
> > > React is subjective, you can get things done with both. I personally only
> > > have experience with React, so I personally would be faster to develop
> > > with this one. I currently have not planned to learn Angular as well
> > (being
> > > a more backend focused developer in general) so if the decision would be
> > to
> > > go with Angular, i would be unfortunately out of this rework of the Flink
> > > Dashboard most certainly.
> > >
> > > Additionally i would like to get rid of bower, since its officially
> > > deprecated <https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> > > my
> > > idea would be to just use a create-react-app package with npm and webpack
> > > under the hood. no need for additional lib's here imho. But again: thats
> > > mostly what i've been working with recently, so thats a subjective
> > point. I
> > > could imagine getting used to yarn in the future as well.
> > >
> > > Cheers
> > > Fabian
> > >
> > > --
> > >
> > >
> > > *Fabian WollertZalando SE*
> > >
> > > E-Mail: fabian@zalando.de
> > >
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Robert Metzger <rm...@apache.org>.
Hey Fabian,
First of all, big +1 on the effort!
Two quick thoughts:
- I would suggest to build a new UI outside of the main Flink project, and
look at replacing the current one, once a new one has reached feature
parity.
- I would also suggest to go with React, as it seems the leading frontend
framework of the time.
On Wed, Oct 31, 2018 at 10:06 AM Flavio Pompermaier <po...@okkam.it>
wrote:
> I think that it is important to have a nice "official" (or at least free)
> Flink UI, we use it to see the detail of the jobs.
> It's very useful for people starting working with Flink and also for those
> that does not have the resources to write a custom UI.
> How are you going to monitor the status of a job otherwise??
>
> On Wed, Oct 31, 2018 at 8:48 AM Fabian Wollert <fa...@zalando.de> wrote:
>
> > Hi Till, I basically agree with all your points. i would stress the
> > "dustiness" of the current architecture: the package manager used (bower)
> > is deprecated since a long time, the chance for the builds of the flink
> web
> > dashboard not working anymore is increasing every day.
> >
> > About the knowledge in the community: Two days is not a lot of time, but
> > interest in this topic seems to be minor anyways. Is someone using the
> > Flink Web Dashboard at all, or is everyone running their own custom
> > solutions? Because if there is no interest in using the Web UI AND no one
> > interested in developing, there would be no need to package this as part
> of
> > the official Flink package, but rather develop an independent solution
> (I'm
> > not voting for this right now, just putting it out), if at all. The
> > official package could then just ship with the API, which other solutions
> > can build upon. This solution could be from an agile point of view also
> the
> > best (enhanced testing, independent and more effective dev workflow,
> etc.),
> > but is bad for the usage of the Flink UI, because people need to install
> > two things individually (Flink and the web dashboard).
> >
> > How did the choice for Angular1 happen back then? Who was writing the
> > Dashboard in the first place?
> >
> > Cheers
> >
> > --
> >
> >
> > *Fabian WollertZalando SE*
> >
> > E-Mail: fabian@zalando.de
> >
> >
> > Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> > trohrmann@apache.org>:
> >
> > > Thanks for starting this discussion Fabian! I think our web UI
> technology
> > > stack is quite dusty by now and it would be beneficial to think about
> its
> > > technological future.
> > >
> > > On the one hand, our current web UI works more or less reliable and
> > > changing the underlying technology has the risk of breaking things.
> > > Moreover, there might be the risk that the newly chosen technology will
> > be
> > > deprecated at some point in time as well.
> > >
> > > On the other hand, we don't have much Angular 1 knowledge in the
> > community
> > > and extending the web UI is, thus, quite hard at the moment. Maybe by
> > using
> > > some newer web technologies we might be able to attract more people
> with
> > a
> > > web technology background to join the community.
> > >
> > > The lack of people working on the web UI is for me the biggest problem
> I
> > > would like to address. If there is interest in the web UI, then I'm
> quite
> > > sure that we will be able to even migrate to some other technology in
> the
> > > future. The next important issue for me is to do the change
> incrementally
> > > if possible. Ideally we never break the web UI in the process of
> > migrating
> > > to a new technology. I'm not an expert here so it might or might not be
> > > possible. But if it is, then we should design the implementation steps
> in
> > > such a way.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de>
> > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > in this email thread
> > > > <
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > > > >
> > > > and the tickets FLINK-10705
> > > > <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
> > > > <https://issues.apache.org/jira/browse/FLINK-10706> the discussion
> > came
> > > up
> > > > whether to change the underlying architecture of Flink's Web
> Dashboard
> > > from
> > > > Angular1 to something else. This email thread should be solely to
> > discuss
> > > > the pro's and con's of this, and what could be the target
> architecture.
> > > >
> > > > My choice would be React. Personally I agree with Till's comments in
> > the
> > > > ticket, Angular 1 being basically outdated and is not having a large
> > > > following anymore. From my experience the choice between Angular 2-7
> or
> > > > React is subjective, you can get things done with both. I personally
> > only
> > > > have experience with React, so I personally would be faster to
> develop
> > > > with this one. I currently have not planned to learn Angular as well
> > > (being
> > > > a more backend focused developer in general) so if the decision would
> > be
> > > to
> > > > go with Angular, i would be unfortunately out of this rework of the
> > Flink
> > > > Dashboard most certainly.
> > > >
> > > > Additionally i would like to get rid of bower, since its officially
> > > > deprecated <
> https://bower.io/blog/2017/how-to-migrate-away-from-bower/
> > >.
> > > > my
> > > > idea would be to just use a create-react-app package with npm and
> > webpack
> > > > under the hood. no need for additional lib's here imho. But again:
> > thats
> > > > mostly what i've been working with recently, so thats a subjective
> > > point. I
> > > > could imagine getting used to yarn in the future as well.
> > > >
> > > > Cheers
> > > > Fabian
> > > >
> > > > --
> > > >
> > > >
> > > > *Fabian WollertZalando SE*
> > > >
> > > > E-Mail: fabian@zalando.de
> > > >
> > >
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Flavio Pompermaier <po...@okkam.it>.
I think that it is important to have a nice "official" (or at least free)
Flink UI, we use it to see the detail of the jobs.
It's very useful for people starting working with Flink and also for those
that does not have the resources to write a custom UI.
How are you going to monitor the status of a job otherwise??
On Wed, Oct 31, 2018 at 8:48 AM Fabian Wollert <fa...@zalando.de> wrote:
> Hi Till, I basically agree with all your points. i would stress the
> "dustiness" of the current architecture: the package manager used (bower)
> is deprecated since a long time, the chance for the builds of the flink web
> dashboard not working anymore is increasing every day.
>
> About the knowledge in the community: Two days is not a lot of time, but
> interest in this topic seems to be minor anyways. Is someone using the
> Flink Web Dashboard at all, or is everyone running their own custom
> solutions? Because if there is no interest in using the Web UI AND no one
> interested in developing, there would be no need to package this as part of
> the official Flink package, but rather develop an independent solution (I'm
> not voting for this right now, just putting it out), if at all. The
> official package could then just ship with the API, which other solutions
> can build upon. This solution could be from an agile point of view also the
> best (enhanced testing, independent and more effective dev workflow, etc.),
> but is bad for the usage of the Flink UI, because people need to install
> two things individually (Flink and the web dashboard).
>
> How did the choice for Angular1 happen back then? Who was writing the
> Dashboard in the first place?
>
> Cheers
>
> --
>
>
> *Fabian WollertZalando SE*
>
> E-Mail: fabian@zalando.de
>
>
> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> trohrmann@apache.org>:
>
> > Thanks for starting this discussion Fabian! I think our web UI technology
> > stack is quite dusty by now and it would be beneficial to think about its
> > technological future.
> >
> > On the one hand, our current web UI works more or less reliable and
> > changing the underlying technology has the risk of breaking things.
> > Moreover, there might be the risk that the newly chosen technology will
> be
> > deprecated at some point in time as well.
> >
> > On the other hand, we don't have much Angular 1 knowledge in the
> community
> > and extending the web UI is, thus, quite hard at the moment. Maybe by
> using
> > some newer web technologies we might be able to attract more people with
> a
> > web technology background to join the community.
> >
> > The lack of people working on the web UI is for me the biggest problem I
> > would like to address. If there is interest in the web UI, then I'm quite
> > sure that we will be able to even migrate to some other technology in the
> > future. The next important issue for me is to do the change incrementally
> > if possible. Ideally we never break the web UI in the process of
> migrating
> > to a new technology. I'm not an expert here so it might or might not be
> > possible. But if it is, then we should design the implementation steps in
> > such a way.
> >
> > Cheers,
> > Till
> >
> > On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de>
> wrote:
> >
> > > Hi everyone,
> > >
> > > in this email thread
> > > <
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > > >
> > > and the tickets FLINK-10705
> > > <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
> > > <https://issues.apache.org/jira/browse/FLINK-10706> the discussion
> came
> > up
> > > whether to change the underlying architecture of Flink's Web Dashboard
> > from
> > > Angular1 to something else. This email thread should be solely to
> discuss
> > > the pro's and con's of this, and what could be the target architecture.
> > >
> > > My choice would be React. Personally I agree with Till's comments in
> the
> > > ticket, Angular 1 being basically outdated and is not having a large
> > > following anymore. From my experience the choice between Angular 2-7 or
> > > React is subjective, you can get things done with both. I personally
> only
> > > have experience with React, so I personally would be faster to develop
> > > with this one. I currently have not planned to learn Angular as well
> > (being
> > > a more backend focused developer in general) so if the decision would
> be
> > to
> > > go with Angular, i would be unfortunately out of this rework of the
> Flink
> > > Dashboard most certainly.
> > >
> > > Additionally i would like to get rid of bower, since its officially
> > > deprecated <https://bower.io/blog/2017/how-to-migrate-away-from-bower/
> >.
> > > my
> > > idea would be to just use a create-react-app package with npm and
> webpack
> > > under the hood. no need for additional lib's here imho. But again:
> thats
> > > mostly what i've been working with recently, so thats a subjective
> > point. I
> > > could imagine getting used to yarn in the future as well.
> > >
> > > Cheers
> > > Fabian
> > >
> > > --
> > >
> > >
> > > *Fabian WollertZalando SE*
> > >
> > > E-Mail: fabian@zalando.de
> > >
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Fabian Wollert <fa...@zalando.de>.
Hi Till, I basically agree with all your points. i would stress the
"dustiness" of the current architecture: the package manager used (bower)
is deprecated since a long time, the chance for the builds of the flink web
dashboard not working anymore is increasing every day.
About the knowledge in the community: Two days is not a lot of time, but
interest in this topic seems to be minor anyways. Is someone using the
Flink Web Dashboard at all, or is everyone running their own custom
solutions? Because if there is no interest in using the Web UI AND no one
interested in developing, there would be no need to package this as part of
the official Flink package, but rather develop an independent solution (I'm
not voting for this right now, just putting it out), if at all. The
official package could then just ship with the API, which other solutions
can build upon. This solution could be from an agile point of view also the
best (enhanced testing, independent and more effective dev workflow, etc.),
but is bad for the usage of the Flink UI, because people need to install
two things individually (Flink and the web dashboard).
How did the choice for Angular1 happen back then? Who was writing the
Dashboard in the first place?
Cheers
--
*Fabian WollertZalando SE*
E-Mail: fabian@zalando.de
Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
trohrmann@apache.org>:
> Thanks for starting this discussion Fabian! I think our web UI technology
> stack is quite dusty by now and it would be beneficial to think about its
> technological future.
>
> On the one hand, our current web UI works more or less reliable and
> changing the underlying technology has the risk of breaking things.
> Moreover, there might be the risk that the newly chosen technology will be
> deprecated at some point in time as well.
>
> On the other hand, we don't have much Angular 1 knowledge in the community
> and extending the web UI is, thus, quite hard at the moment. Maybe by using
> some newer web technologies we might be able to attract more people with a
> web technology background to join the community.
>
> The lack of people working on the web UI is for me the biggest problem I
> would like to address. If there is interest in the web UI, then I'm quite
> sure that we will be able to even migrate to some other technology in the
> future. The next important issue for me is to do the change incrementally
> if possible. Ideally we never break the web UI in the process of migrating
> to a new technology. I'm not an expert here so it might or might not be
> possible. But if it is, then we should design the implementation steps in
> such a way.
>
> Cheers,
> Till
>
> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de> wrote:
>
> > Hi everyone,
> >
> > in this email thread
> > <
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > >
> > and the tickets FLINK-10705
> > <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
> > <https://issues.apache.org/jira/browse/FLINK-10706> the discussion came
> up
> > whether to change the underlying architecture of Flink's Web Dashboard
> from
> > Angular1 to something else. This email thread should be solely to discuss
> > the pro's and con's of this, and what could be the target architecture.
> >
> > My choice would be React. Personally I agree with Till's comments in the
> > ticket, Angular 1 being basically outdated and is not having a large
> > following anymore. From my experience the choice between Angular 2-7 or
> > React is subjective, you can get things done with both. I personally only
> > have experience with React, so I personally would be faster to develop
> > with this one. I currently have not planned to learn Angular as well
> (being
> > a more backend focused developer in general) so if the decision would be
> to
> > go with Angular, i would be unfortunately out of this rework of the Flink
> > Dashboard most certainly.
> >
> > Additionally i would like to get rid of bower, since its officially
> > deprecated <https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> > my
> > idea would be to just use a create-react-app package with npm and webpack
> > under the hood. no need for additional lib's here imho. But again: thats
> > mostly what i've been working with recently, so thats a subjective
> point. I
> > could imagine getting used to yarn in the future as well.
> >
> > Cheers
> > Fabian
> >
> > --
> >
> >
> > *Fabian WollertZalando SE*
> >
> > E-Mail: fabian@zalando.de
> >
>
Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard
Posted by Till Rohrmann <tr...@apache.org>.
Thanks for starting this discussion Fabian! I think our web UI technology
stack is quite dusty by now and it would be beneficial to think about its
technological future.
On the one hand, our current web UI works more or less reliable and
changing the underlying technology has the risk of breaking things.
Moreover, there might be the risk that the newly chosen technology will be
deprecated at some point in time as well.
On the other hand, we don't have much Angular 1 knowledge in the community
and extending the web UI is, thus, quite hard at the moment. Maybe by using
some newer web technologies we might be able to attract more people with a
web technology background to join the community.
The lack of people working on the web UI is for me the biggest problem I
would like to address. If there is interest in the web UI, then I'm quite
sure that we will be able to even migrate to some other technology in the
future. The next important issue for me is to do the change incrementally
if possible. Ideally we never break the web UI in the process of migrating
to a new technology. I'm not an expert here so it might or might not be
possible. But if it is, then we should design the implementation steps in
such a way.
Cheers,
Till
On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de> wrote:
> Hi everyone,
>
> in this email thread
> <
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> >
> and the tickets FLINK-10705
> <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
> <https://issues.apache.org/jira/browse/FLINK-10706> the discussion came up
> whether to change the underlying architecture of Flink's Web Dashboard from
> Angular1 to something else. This email thread should be solely to discuss
> the pro's and con's of this, and what could be the target architecture.
>
> My choice would be React. Personally I agree with Till's comments in the
> ticket, Angular 1 being basically outdated and is not having a large
> following anymore. From my experience the choice between Angular 2-7 or
> React is subjective, you can get things done with both. I personally only
> have experience with React, so I personally would be faster to develop
> with this one. I currently have not planned to learn Angular as well (being
> a more backend focused developer in general) so if the decision would be to
> go with Angular, i would be unfortunately out of this rework of the Flink
> Dashboard most certainly.
>
> Additionally i would like to get rid of bower, since its officially
> deprecated <https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> my
> idea would be to just use a create-react-app package with npm and webpack
> under the hood. no need for additional lib's here imho. But again: thats
> mostly what i've been working with recently, so thats a subjective point. I
> could imagine getting used to yarn in the future as well.
>
> Cheers
> Fabian
>
> --
>
>
> *Fabian WollertZalando SE*
>
> E-Mail: fabian@zalando.de
>