You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Maciek Próchniak <mp...@touk.pl> on 2017/09/04 13:03:40 UTC
ANN: TouK Nussknacker - designing Flink processes with GUI
Hello,
we would like to announce availability of TouK Nussknacker - tool for
creating Flink processes with GUI.
It's our attempt to bring Flink a bit closer to analysts and business
people - by letting them design processes with MS Visio-like tool ;)
Of course not every Flink process can be created by drawing diagrams,
but we wanted to make it possible for analyst to design process that
does filtering, enrichment or simple aggregations.
Our idea is pretty simple:
- first, data model and additional services have to be developed and
packaged into jar (this is the jar that is used by Flink jobs) - this is
the part where coding skills are required
- users can create new processes by drawing diagrams - the nodes in
toolbox are based on the data model. This includes e.g. creating filters
with SPEL (Spring Expression Language) expressions.
- the process is deployed to Flink cluster as jar with model and diagram
serialized as JSON
We make it easier to work with our diagrams by integration with Grafana
(each process registers quite detailed metrics about it's behaviour),
letting users test their processes via GUI with Flink mini-cluster and
various other goodies (subprocesses, basic code completion, export to
PDF etc.)
So far we have one quite large (>30 processes, >100k/s total throughput
in all processes) production deployment and a few more in progress. Our
main use case so far is RTM (Real Time Marketing) and fraud detection in
telco, banking and similar industries.
We think that our tool can be of interest mainly for more traditional
enterprises, that do not have large development teams, but have
competent analysts in their BI departments.
Nussknacker is released under ASL 2.0 (however we use React heavily so
beware of patents clause...)
Of course, we'd like to hear your opinions.
The code is here: https://github.com/touk/nussknacker
The quickstart is here: https://touk.github.io/nussknacker/Quickstart.html
And our blog post is here:
https://touk.pl/blog/2017/09/04/touk-nussknacker-using-apache-flink-made-easier-for-analysts-and-business/
I'll also be talking about Nussknacker next week at Flink Forward -
https://berlin.flink-forward.org/kb_sessions/touk-nussknacker-creating-flink-jobs-with-gui/
- hope to see you there :)
thanks,
maciek próchniak
TouK
Re: ANN: TouK Nussknacker - designing Flink processes with GUI
Posted by Maciek Próchniak <mp...@touk.pl>.
Hi Flavio,
we have rather modest goals - so currently we don't plan to handle batch
(although in theory it could be done). We also don't even think about
visualizing and editing existing programs - in generality this could be
pretty difficult.
We only let users define pretty simple diagrams (in particular they have
to be trees, not DAGs) to be able to handle them properly (although we
do have mechanism to plug-in custom Flink code - it just have to be
coded separately as part of the model)
thanks,
maciek
On 04/09/2017 15:43, Flavio Pompermaier wrote:
> Thanks for sharing this nice tool maciek. Does it handle both batch
> and Streaming? Is it able to visualize also an existing Flink program?
>
> Best,
> Flavio
>
> On Mon, Sep 4, 2017 at 3:03 PM, Maciek Próchniak <mpr@touk.pl
> <ma...@touk.pl>> wrote:
>
> Hello,
>
> we would like to announce availability of TouK Nussknacker - tool
> for creating Flink processes with GUI.
>
>
> It's our attempt to bring Flink a bit closer to analysts and
> business people - by letting them design processes with MS
> Visio-like tool ;)
>
> Of course not every Flink process can be created by drawing
> diagrams, but we wanted to make it possible for analyst to design
> process that does filtering, enrichment or simple aggregations.
>
> Our idea is pretty simple:
>
> - first, data model and additional services have to be developed
> and packaged into jar (this is the jar that is used by Flink jobs)
> - this is the part where coding skills are required
>
> - users can create new processes by drawing diagrams - the nodes
> in toolbox are based on the data model. This includes e.g.
> creating filters with SPEL (Spring Expression Language) expressions.
>
> - the process is deployed to Flink cluster as jar with model and
> diagram serialized as JSON
>
> We make it easier to work with our diagrams by integration with
> Grafana (each process registers quite detailed metrics about it's
> behaviour), letting users test their processes via GUI with Flink
> mini-cluster and various other goodies (subprocesses, basic code
> completion, export to PDF etc.)
>
>
> So far we have one quite large (>30 processes, >100k/s total
> throughput in all processes) production deployment and a few more
> in progress. Our main use case so far is RTM (Real Time Marketing)
> and fraud detection in telco, banking and similar industries.
>
> We think that our tool can be of interest mainly for more
> traditional enterprises, that do not have large development teams,
> but have competent analysts in their BI departments.
>
>
> Nussknacker is released under ASL 2.0 (however we use React
> heavily so beware of patents clause...)
>
> Of course, we'd like to hear your opinions.
>
>
> The code is here: https://github.com/touk/nussknacker
> <https://github.com/touk/nussknacker>
>
> The quickstart is here:
> https://touk.github.io/nussknacker/Quickstart.html
> <https://touk.github.io/nussknacker/Quickstart.html>
>
> And our blog post is here:
> https://touk.pl/blog/2017/09/04/touk-nussknacker-using-apache-flink-made-easier-for-analysts-and-business/
> <https://touk.pl/blog/2017/09/04/touk-nussknacker-using-apache-flink-made-easier-for-analysts-and-business/>
>
> I'll also be talking about Nussknacker next week at Flink Forward
> -
> https://berlin.flink-forward.org/kb_sessions/touk-nussknacker-creating-flink-jobs-with-gui/
> <https://berlin.flink-forward.org/kb_sessions/touk-nussknacker-creating-flink-jobs-with-gui/>
> - hope to see you there :)
>
>
> thanks,
>
> maciek próchniak
>
> TouK
>
>
>
Re: ANN: TouK Nussknacker - designing Flink processes with GUI
Posted by Flavio Pompermaier <po...@okkam.it>.
Thanks for sharing this nice tool maciek. Does it handle both batch and
Streaming? Is it able to visualize also an existing Flink program?
Best,
Flavio
On Mon, Sep 4, 2017 at 3:03 PM, Maciek Próchniak <mp...@touk.pl> wrote:
> Hello,
>
> we would like to announce availability of TouK Nussknacker - tool for
> creating Flink processes with GUI.
>
>
> It's our attempt to bring Flink a bit closer to analysts and business
> people - by letting them design processes with MS Visio-like tool ;)
>
> Of course not every Flink process can be created by drawing diagrams, but
> we wanted to make it possible for analyst to design process that does
> filtering, enrichment or simple aggregations.
>
> Our idea is pretty simple:
>
> - first, data model and additional services have to be developed and
> packaged into jar (this is the jar that is used by Flink jobs) - this is
> the part where coding skills are required
>
> - users can create new processes by drawing diagrams - the nodes in
> toolbox are based on the data model. This includes e.g. creating filters
> with SPEL (Spring Expression Language) expressions.
>
> - the process is deployed to Flink cluster as jar with model and diagram
> serialized as JSON
>
> We make it easier to work with our diagrams by integration with Grafana
> (each process registers quite detailed metrics about it's behaviour),
> letting users test their processes via GUI with Flink mini-cluster and
> various other goodies (subprocesses, basic code completion, export to PDF
> etc.)
>
>
> So far we have one quite large (>30 processes, >100k/s total throughput in
> all processes) production deployment and a few more in progress. Our main
> use case so far is RTM (Real Time Marketing) and fraud detection in telco,
> banking and similar industries.
>
> We think that our tool can be of interest mainly for more traditional
> enterprises, that do not have large development teams, but have competent
> analysts in their BI departments.
>
>
> Nussknacker is released under ASL 2.0 (however we use React heavily so
> beware of patents clause...)
>
> Of course, we'd like to hear your opinions.
>
>
> The code is here: https://github.com/touk/nussknacker
>
> The quickstart is here: https://touk.github.io/nussknacker/Quickstart.html
>
> And our blog post is here: https://touk.pl/blog/2017/09/0
> 4/touk-nussknacker-using-apache-flink-made-easier-for-analys
> ts-and-business/
>
> I'll also be talking about Nussknacker next week at Flink Forward -
> https://berlin.flink-forward.org/kb_sessions/touk-nussknacke
> r-creating-flink-jobs-with-gui/ - hope to see you there :)
>
>
> thanks,
>
> maciek próchniak
>
> TouK
>
>