[DISCUSSION] TM1 Airflow provider

[Ákos Nagy <ak...@knowledgeseed.ch>]

Hi Airflow Devs,

Recently I posted in #new-contributors<https://app.slack.com/client/TCQ18L22Z/CCZRF2U5A> channel:

Hello folks!
I represent KnowledgeSeed GmbH<https://knowledgeseed.ch/>. We've been using Apache Airflow for a while to complete our stack with a robust ETL execution engine to move data in/out IBM TM1<https://www.ibm.com/topics/tm1> database that is our central component to support real-time analytics and forecasting. Please see this link<https://knowledgeseed.ch/blog/a_python-based_approach_to_data_loading_in_TM1.html> for more details about our use case.
To connect TM1 and Airflow, we use an external Airflow TM1 provider<https://github.com/KnowledgeSeed/airflow-tm1> that is basically a fork including slight enhancements of Cubewise<https://www.cubewise.com/>'s original Airflow provider. It's always been a pain to set up Airflow including the custom TM1 provider and the concept is usually challenged by client architect boards to ensure it passes security audits, etc. It would be a big leap forward for us if TM1 provider could become part of the standard Airflow distribution and would most probably open some doors in the business intelligence world to use Airflow combined with TM1 for some big companies including but not limited to Amazon, Deutche Bahn, Uber, etc.
Knowledgeseed is happy to volunteer on this journey! Let me know if you guys are up to a preliminary concept review.

 Thanks,
Akos



Best regards,

Akos Nagy

Knowledgeseed GmbH

System Integration Specialist

Mobile: +36 30 694 7641

www.knowledgeseed.ch<http://www.knowledgeseed.ch>

[ks_logo]<http://www.knowledgeseed.ch/>

Re: [DISCUSSION] TM1 Airflow provider

[Elad Kalif <el...@apache.org>]

Hi Akos,

Can you please elaborate about the reasoning?
You mentioned the pain of setting up a custom provider. How will this be
resolved by having it as an official Airflow provider?
I took a look at the stats of the 3rd party package
https://pypistats.org/packages/airflow-tm1 it has less than 90 downloads a
month - It doesn't seem to be a very popular integration.

Thanks,
Elad

On Tue, May 7, 2024 at 4:28 PM Ákos Nagy <ak...@knowledgeseed.ch> wrote:

> Hi Airflow Devs,
>
> Recently I posted in #new-contributors
> <https://app.slack.com/client/TCQ18L22Z/CCZRF2U5A> channel:
>
> Hello folks!
> I represent KnowledgeSeed GmbH <https://knowledgeseed.ch/>. We've been
> using Apache Airflow for a while to complete our stack with a robust ETL
> execution engine to move data in/out IBM TM1
> <https://www.ibm.com/topics/tm1> database that is our central component
> to support real-time analytics and forecasting. Please see this link
> <https://knowledgeseed.ch/blog/a_python-based_approach_to_data_loading_in_TM1.html> for
> more details about our use case.
> To connect TM1 and Airflow, we use an external Airflow TM1 provider
> <https://github.com/KnowledgeSeed/airflow-tm1> that is basically a fork
> including slight enhancements of Cubewise <https://www.cubewise.com/>'s
> original Airflow provider. It's always been a pain to set up Airflow
> including the custom TM1 provider and the concept is usually challenged by
> client architect boards to ensure it passes security audits, etc. It would
> be a big leap forward for us if TM1 provider could become part of the
> standard Airflow distribution and would most probably open some doors in
> the business intelligence world to use Airflow combined with TM1 for some
> big companies including but not limited to Amazon, Deutche Bahn, Uber, etc.
> Knowledgeseed is happy to volunteer on this journey! Let me know if you
> guys are up to a preliminary concept review.
>
>  Thanks,
> Akos
>
>
> Best regards,
>
> Akos Nagy
>
> Knowledgeseed GmbH
>
> System Integration Specialist
>
> Mobile: +36 30 694 7641
>
> www.knowledgeseed.ch
>
> [image: ks_logo] <http://www.knowledgeseed.ch/>
>
>