Highcharts > NVD3

[Maxime Beauchemin <ma...@gmail.com>]

We're moving to Apache, and the Highcharts license is not compatible with
ours so we're removing Highcharts in favor of NVD3, and attempt to keep the
charting application working where possible. *Note that some chart types
that won't be supported anymore (heatmaps, parallel coordinates) and some
may behave differently as they did in the past.*

Since the small charting application located under "Data Profiling ->
Chart" is peripheral to Airflow, this change is not meant to be fully
compatible. For much better charting and long term support, check out
Airbnb's Caravel.
https://github.com/airbnb/caravel

If you'd like to test the library replacement before it hits, please pull
that branch and try it on your charts and provide feedback on the PR here:
https://github.com/apache/incubator-airflow/pull/1469

As a more general topic, who uses the "Data Profiling" features and how? Is
this over the line of what Airflow should take on, or is it an area we
should keep pushing and growing? I'm curious to hear feedback from the
community on this topic!

Max

Re: Highcharts > NVD3

[Chris Riccomini <cr...@apache.org>]

Hey Maxime,

> Note that some chart types that won't be supported anymore (heatmaps,
parallel coordinates) and some may behave differently as they did in the
past

I'm being lazy--could you post some screenshots of how it'll look? :)

> As a more general topic, who uses the "Data Profiling" features and how?

We don't use it at all (intentionally).

> Is this over the line of what Airflow should take on, or is it an area we should
keep pushing and growing

We should either significantly improve it, or cut it. I'm leaning towards
the latter. At LinkedIn, we built something like this, and it was very
popular. We eventually merged it into Azkaban (
https://github.com/azkaban/azkaban-plugins/tree/master/plugins/reportal).
There is a very natural need to do scheduled report generation on a
periodic basis, and leveraging your scheduler (i.e. Airflow) to do the
periodic execution is really nice. However, most modern reporting is moving
more towards realtime (e.g. Druid + Caravel) or moving to the cloud
(Chart.io, bi.me, etc), and I think the scheduled reporting approach is
getting a bit outdated. For that reason, I'd say cut it. Sid might
disagree, though, since he's using it. ;)

Cheers,
Chris

On Thu, May 5, 2016 at 4:30 PM, Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:

> We're moving to Apache, and the Highcharts license is not compatible with
> ours so we're removing Highcharts in favor of NVD3, and attempt to keep the
> charting application working where possible. *Note that some chart types
> that won't be supported anymore (heatmaps, parallel coordinates) and some
> may behave differently as they did in the past.*
>
> Since the small charting application located under "Data Profiling ->
> Chart" is peripheral to Airflow, this change is not meant to be fully
> compatible. For much better charting and long term support, check out
> Airbnb's Caravel.
> https://github.com/airbnb/caravel
>
> If you'd like to test the library replacement before it hits, please pull
> that branch and try it on your charts and provide feedback on the PR here:
> https://github.com/apache/incubator-airflow/pull/1469
>
> As a more general topic, who uses the "Data Profiling" features and how? Is
> this over the line of what Airflow should take on, or is it an area we
> should keep pushing and growing? I'm curious to hear feedback from the
> community on this topic!
>
> Max
>