Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard

[Yadong Xie <vt...@gmail.com>]

Thanks to Fabian for starting this discussion!
I liked the discussion. Here are some opinions I have:
[1] Till mentioned that "I think our web UI technology stack is quite dusty
by now and it would be beneficial to think about its technological future."
I totally agree with that we should upgrade the web UI technology stack. I
have been working on Flink Web UI for quite a while. We recently
reconstructed it to the latest version of Angular (7.0) in Alibaba. and we
would love to contribute the entire work to the Flink community. I will
show some snapshots and provide more details later in this discussion
thread.
[2] Fabian mentioned that "From my experience the choice between Angular
2-7 or React is subjective, you can get things done with both."
I totally agree with this. But to my experience, upgrade Angular from 1 to
2-7 is relatively easier and less work than completely rewrite the entire
web UI via React. And since we have already completed angular 1 to angular
7 upgrading, I hope we can keep the web UI to angular.

Best
Yadong

On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <tr...@apache.org> wrote:

> I agree with Chesnay that the web UI is already fairly isolated in Flink's
> code base (see flink-runtime-web module). Moreover, imo it is very
> important that Flink comes with a nice web UI which works out of the box
> because it lowers the entrance hurdle massively. What one could argue is to
> make it easier to plug in your own web UI if you want to develop it. But I
> would see this as an orthogonal issue.
>
> I would actually argue against Robert's proposal to make a grand rewrite if
> this is by any means possible. Experience shows that these kind of efforts
> are usually set up to fail or take super long. If possible, then I would
> rather prefer to convert one component at a time to React. But I'm not an
> expert here and cannot tell whether this is possible or not.
>
> What we would need is active help from the community to review these
> changes. I think that is the crucial piece to make such an effort work or
> fail.
>
> Cheers,
> Till
>
> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> > The WebUI goes entirely through the REST API.
> >
> > I don't see how moving it out would make it easier to develop/contribute.
> >
> > The UI is already encapsulated in it's own module (flink-runtime-web)
> > and changes can be made to it independently.
> > Historically the issue was simply that contributions to the UI were not
> > reviewed in a timely manner /  not at all, which mind you
> > applies to virtually all components. This also won't change by moving it
> > out of Flink.
> >
> > Having it in Flink also guarantees that there is an easily-discoverable
> > version of the UI that is compatible with your Flink distribution.
> >
> > On 31.10.2018 09:53, dyana.rose wrote:
> > > Re: who's using the web ui
> > >
> > > Though many mature solutions, with a fair amount of time/resources
> > available are likely running their own front ends, for teams like mine
> > which are smaller and aren't focused solely on working with Flink, having
> > the web ui available removes a large barrier to getting up and running
> > quickly.
> > >
> > > But, with the web UI being in core Flink, it does make it a bit more of
> > a chore to contribute.
> > >
> > > I'm unaware of how necessary it is for the UI to deploy with Flink? Is
> > it making any calls that aren't through the REST API (and if so, can
> those
> > endpoints be added to the REST API)?
> > >
> > > In general I'd support moving it to its own package, making it easier
> to
> > develop with your standard UI dev tools. This also allows the web UI to
> be
> > developed and released independently to core Flink.
> > >
> > > Dyana
> > >
> > > On 2018/10/31 07:47:50, Fabian Wollert <fa...@zalando.de> wrote:
> > >> Hi Till, I basically agree with all your points. i would stress the
> > >> "dustiness" of the current architecture: the package manager used
> > (bower)
> > >> is deprecated since a long time, the chance for the builds of the
> flink
> > web
> > >> dashboard not working anymore is increasing every day.
> > >>
> > >> About the knowledge in the community: Two days is not a lot of time,
> but
> > >> interest in this topic seems to be minor anyways. Is someone using the
> > >> Flink Web Dashboard at all, or is everyone running their own custom
> > >> solutions? Because if there is no interest in using the Web UI AND no
> > one
> > >> interested in developing, there would be no need to package this as
> > part of
> > >> the official Flink package, but rather develop an independent solution
> > (I'm
> > >> not voting for this right now, just putting it out), if at all. The
> > >> official package could then just ship with the API, which other
> > solutions
> > >> can build upon. This solution could be from an agile point of view
> also
> > the
> > >> best (enhanced testing, independent and more effective dev workflow,
> > etc.),
> > >> but is bad for the usage of the Flink UI, because people need to
> install
> > >> two things individually (Flink and the web dashboard).
> > >>
> > >> How did the choice for Angular1 happen back then? Who was writing the
> > >> Dashboard in the first place?
> > >>
> > >> Cheers
> > >>
> > >> --
> > >>
> > >>
> > >> *Fabian WollertZalando SE*
> > >>
> > >> E-Mail: fabian@zalando.de
> > >>
> > >>
> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> > >> trohrmann@apache.org>:
> > >>
> > >>> Thanks for starting this discussion Fabian! I think our web UI
> > technology
> > >>> stack is quite dusty by now and it would be beneficial to think about
> > its
> > >>> technological future.
> > >>>
> > >>> On the one hand, our current web UI works more or less reliable and
> > >>> changing the underlying technology has the risk of breaking things.
> > >>> Moreover, there might be the risk that the newly chosen technology
> > will be
> > >>> deprecated at some point in time as well.
> > >>>
> > >>> On the other hand, we don't have much Angular 1 knowledge in the
> > community
> > >>> and extending the web UI is, thus, quite hard at the moment. Maybe by
> > using
> > >>> some newer web technologies we might be able to attract more people
> > with a
> > >>> web technology background to join the community.
> > >>>
> > >>> The lack of people working on the web UI is for me the biggest
> problem
> > I
> > >>> would like to address. If there is interest in the web UI, then I'm
> > quite
> > >>> sure that we will be able to even migrate to some other technology in
> > the
> > >>> future. The next important issue for me is to do the change
> > incrementally
> > >>> if possible. Ideally we never break the web UI in the process of
> > migrating
> > >>> to a new technology. I'm not an expert here so it might or might not
> be
> > >>> possible. But if it is, then we should design the implementation
> steps
> > in
> > >>> such a way.
> > >>>
> > >>> Cheers,
> > >>> Till
> > >>>
> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fa...@zalando.de>
> > wrote:
> > >>>
> > >>>> Hi everyone,
> > >>>>
> > >>>> in this email thread
> > >>>> <
> > >>>>
> > >>>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> > >>>> and the tickets FLINK-10705
> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the discussion
> > came
> > >>> up
> > >>>> whether to change the underlying architecture of Flink's Web
> Dashboard
> > >>> from
> > >>>> Angular1 to something else. This email thread should be solely to
> > discuss
> > >>>> the pro's and con's of this, and what could be the target
> > architecture.
> > >>>>
> > >>>> My choice would be React. Personally I agree with Till's comments in
> > the
> > >>>> ticket, Angular 1 being basically outdated and is not having a large
> > >>>> following anymore. From my experience the choice between Angular 2-7
> > or
> > >>>> React is subjective, you can get things done with both. I personally
> > only
> > >>>> have experience with React, so I  personally would be faster to
> > develop
> > >>>> with this one. I currently have not planned to learn Angular as well
> > >>> (being
> > >>>> a more backend focused developer in general) so if the decision
> would
> > be
> > >>> to
> > >>>> go with Angular, i would be unfortunately out of this rework of the
> > Flink
> > >>>> Dashboard most certainly.
> > >>>>
> > >>>> Additionally i would like to get rid of bower, since its officially
> > >>>> deprecated <
> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> > >>>> my
> > >>>> idea would be to just use a create-react-app package with npm and
> > webpack
> > >>>> under the hood. no need for additional lib's here imho. But again:
> > thats
> > >>>> mostly what i've been working with recently, so thats a subjective
> > >>> point. I
> > >>>> could imagine getting used to yarn in the future as well.
> > >>>>
> > >>>> Cheers
> > >>>> Fabian
> > >>>>
> > >>>> --
> > >>>>
> > >>>>
> > >>>> *Fabian WollertZalando SE*
> > >>>>
> > >>>> E-Mail: fabian@zalando.de
> > >>>>
> >
> >
>


-- 
Yadong Xie
Ministry of Education Key Lab for Intelligent Networks and Network Security
Dept. of Computer Science and Technology
School of electronic information and engineering
Xi'an Jiaotong University
Xi'an, 710049, P.R. China
Email: vthinkxie@gmail.com