You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Blain David <da...@infrabel.be> on 2026/03/03 08:30:39 UTC
[DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Hi all,
At our company we recently implemented an IBM MQ integration for Airflow and I would like to gauge interest in contributing this as a new provider package.
Motivation
With the introduction of event-driven scheduling and the MessageQueueProvider abstraction in Airflow, it has become significantly easier to trigger DAGs from external message brokers (as described in Astronomer's guide on event-driven scheduling):
https://www.astronomer.io/docs/learn/airflow-event-driven-scheduling
Many enterprises still rely heavily on IBM MQ as their primary enterprise messaging backbone. However, at the moment there is no official Airflow provider supporting IBM MQ.
Our implementation enables:
* An IBMMQHook
* A MessageQueueProvider implementation for IBM MQ
* The ability to trigger DAGs from IBM MQ events
* Standard producer/consumer patterns from within tasks
This allows IBM MQ to function similarly to Kafka, SQS, etc., within the Airflow event-driven scheduling framework.
Technical Details
The implementation is built on top of the open-source IBM MQ Python wrapper:
* IBM MQ Python (ibmmq) library:
https://github.com/ibm-messaging/mq-mqi-python
IBM has recently released and documented their modern Python binding here:
https://community.ibm.com/community/user/blogs/dylan-goode/2025/10/16/new-python-binding-for-ibm-mq
The hook supports:
* Secure connections (TLS)
* Queue get/put operations
* Configurable polling behavior
* Transaction handling where applicable
The MessageQueueProvider implementation integrates with Airflow's event-driven scheduling so that DAGs can be triggered based on IBM MQ messages.
Why this might make sense:
* IBM MQ is still widely used in regulated industries (banking, insurance, government).
* Many enterprises using Airflow also run IBM MQ.
* This would allow IBM MQ to be a first-class citizen in Airflow's event-driven ecosystem.
* The dependency is officially maintained by IBM and open source.
I am willing to act as initial maintainer and code owner, of course this is purely a proposition.
I would appreciate feedback on:
* Whether there is interest in such a provider
* If yes, whether it should live under apache-airflow-providers-ibm
* And if we formalize this as an AIP or draft PR
Happy to share a draft implementation through a PR if there is interest.
Thanks!
David
Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Posted by Kaxil Naik <ka...@gmail.com>.
Aah, good the dependencies are completely isolated too:
https://pypi.org/project/ibmmq/ -- so yeah I am fine with that.
On Fri, 6 Mar 2026 at 22:44, Blain David <da...@infrabel.be> wrote:
> Indeed, I did the same as with the Microsoft providers, as those are the
> one I was familiar with.
>
> -----Original Message-----
> From: Jarek Potiuk <ja...@potiuk.com>
> Sent: 06 March 2026 21:49
> To: Elad Kalif <el...@apache.org>
> Cc: dev@airflow.apache.org; Vikram Koka <vi...@astronomer.io>
> Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook +
> MessageQueueProvider) to Airflow
>
> EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze
> niet vertrouwt, klik niet op een link of open geen bijlages. Bij twijfel,
> stuur deze e-mail als bijlage naar abuse@infrabel.be<mailto:
> abuse@infrabel.be>.
>
> Actually Vikram, Elad - you are right, I was initially under the
> impression that -mq is "our message queue" -> but since this is about "IBM
> MQ" as a separate product/entity in IBM you are completely right it should
> be a separate provider.
>
> On Fri, Mar 6, 2026 at 7:07 PM Elad Kalif <el...@apache.org> wrote:
>
> > If IBM services are independent from one another we should follow how
> > we do it on microsoft.
> >
> > While there is no apache-airflow-providers-microsoft provider.
> > We do have:
> > apache-airflow-providers-microsoft-mssql
> > apache-airflow-providers-microsoft-azure
> > and others... azure is not the same division as mssql.
> >
> > The google provider is a unique case. We have
> > apache-airflow-providers-google due to historical reasons (google ads
> > has nothing to do with google cloud) and splitting the provider is not
> > easy task
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> > ub.com%2Fapache%2Fairflow%2Fissues%2F15933&data=05%7C02%7Cdavid.blain%
> > 40infrabel.be%7C753b70a0859d495498a408de7bc1e34f%7Cb82bc314ab8e4d6fb18
> > 946f02e1f27f2%7C0%7C0%7C639084269845151921%7CUnknown%7CTWFpbGZsb3d8eyJ
> > FbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpb
> > CIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=I2QJFAYkyMpe8ytFByC3omVDvtz1drY
> > rnNL5NTf4SIw%3D&reserved=0 I would also argue that
> > apache-airflow-providers-amazon is not that good.
> > If someone would like to add integration of Amazon marketplaces APIs
> > (retail) that has nothing to do with AWS.
> >
> >
> > On Fri, Mar 6, 2026 at 5:47 PM Vikram Koka via dev
> > <de...@airflow.apache.org>
> > wrote:
> >
> >> I am not sure about that Kaxil and Jarek.
> >>
> >> I understand the prior model of Google and Amazon, but those had
> >> teams responsible for all the integrations with that service.
> >> In the new governance model, the support burden for an interested
> >> individual or individuals seems too high when the team is not a
> >> service provider.
> >>
> >> Why not let individuals or SIs (system integrators) take the
> >> "individual service supported" approach?
> >> This is similar to the Apache services model.
> >>
> >> Vikram
> >>
> >>
> >> On Wed, Mar 4, 2026 at 1:20 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> >>
> >> > +1
> >> >
> >> > On Wed, Mar 4, 2026 at 10:17 PM Kaxil Naik <ka...@gmail.com>
> wrote:
> >> >
> >> >> It should just be called apache-airflow-providers-ibm, in future
> >> >> folks might want to add more IBM hooks that are not just message
> queues.
> >> >>
> >> >> Similar to how Amazon & Google provider treats PubSub or Kinesis
> >> >>
> >> >> On Tue, 3 Mar 2026 at 16:57, Vikram Koka via dev <
> >> dev@airflow.apache.org>
> >> >> wrote:
> >> >>
> >> >> > Thanks David.
> >> >> > I agree with Jason, that a PR is sufficient for this, and an AIP
> >> >> > is
> >> not
> >> >> > required.
> >> >> >
> >> >> > Looking forward to this,
> >> >> > Vikram
> >> >> >
> >> >> > On Tue, Mar 3, 2026 at 7:28 AM Blain David
> >> >> > <da...@infrabel.be>
> >> >> > wrote:
> >> >> >
> >> >> > > Hello Jason,
> >> >> > >
> >> >> > > Thanks for you reply, I've created a draft PR for this provider:
> >> >> > >
> >> >> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2
> >> >> > > F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F62790&data=05%7C02%
> >> >> > > 7Cdavid.blain%40infrabel.be%7C753b70a0859d495498a408de7bc1e34f
> >> >> > > %7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C6390842698451855
> >> >> > > 21%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjA
> >> >> > > uMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%
> >> >> > > 7C%7C%7C&sdata=TcqQfxqzMsZ%2BJME%2BZz1F1J0imMpaU30A5pgvN%2BkhX
> >> >> > > KE%3D&reserved=0
> >> >> > >
> >> >> > > Kind regards,
> >> >> > > David
> >> >> > >
> >> >> > > -----Original Message-----
> >> >> > > From: Zhe-You(Jason) Liu <ja...@apache.org>
> >> >> > > Sent: 03 March 2026 10:37
> >> >> > > To: dev@airflow.apache.org
> >> >> > > Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider
> >> >> > > (Hook
> >> +
> >> >> > > MessageQueueProvider) to Airflow
> >> >> > >
> >> >> > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent
> >> >> > > en
> >> deze
> >> >> > > niet vertrouwt, klik niet op een link of open geen bijlages.
> >> >> > > Bij
> >> >> twijfel,
> >> >> > > stuur deze e-mail als bijlage naar abuse@infrabel.be<mailto:
> >> >> > > abuse@infrabel.be>.
> >> >> > >
> >> >> > > Hi David,
> >> >> > >
> >> >> > > Thank you for your interest in adding a new
> >> >> > > MessageQueueProvider to Airflow!
> >> >> > >
> >> >> > > I previously opened an issue about adding more providers that
> >> support
> >> >> > > MessageQueueProvider in the community providers. [1] You’re
> >> >> > > more
> >> than
> >> >> > > welcome to contribute one!
> >> >> > >
> >> >> > > The only blocker to adding IBM MQ support to
> >> >> > > MessageQueueProvider
> >> is
> >> >> "the
> >> >> > > addition of a new IBM provider". We need to follow the
> >> >> > > adoption
> >> path
> >> >> > > (AIP-95), similar to the recent Informatica provider. [2]
> >> >> > >
> >> >> > > > Whether there is interest in such a provider If yes, whether
> >> >> > > > it
> >> >> should
> >> >> > > > live under apache-airflow-providers-ibm And if we formalize
> >> >> > > > this
> >> as
> >> >> an
> >> >> > > > AIP or draft PR
> >> >> > >
> >> >> > > So, from my perspective, the IBM Hook, Trigger, and
> >> >> MessageQueueProvider
> >> >> > > would be better placed under the IBM provider, and perhaps you
> >> could
> >> >> > start
> >> >> > > by opening a draft PR and then initiating a voting thread on
> >> >> > > the
> >> dev
> >> >> > > mailing list.
> >> >> > >
> >> >> > > Thanks!
> >> >> > >
> >> >> > > [1]
> >> >> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2
> >> >> > > F%2Fgithub.com%2Fapache%2Fairflow%2Fissues%2F52712&data=05%7C0
> >> >> > > 2%7Cdavid.blain%40infrabel.be%7C753b70a0859d495498a408de7bc1e3
> >> >> > > 4f%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C63908426984520
> >> >> > > 2381%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwL
> >> >> > > jAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C
> >> >> > > 0%7C%7C%7C&sdata=zSpuj%2B%2Fx7UAUmDXHM0se6QYtpL6DOHBjGWXmvgAST
> >> >> > > lc%3D&reserved=0
> >> >> > > [2]
> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flis
> >> ts.apache.org%2Fthread%2Fwsfgh23jm6hkrly4lx1m21ftllqshpgo&data=05%7C0
> >> 2%7Cdavid.blain%40infrabel.be%7C753b70a0859d495498a408de7bc1e34f%7Cb8
> >> 2bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639084269845219729%7CUnknown
> >> %7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW
> >> 4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Ct8WnY45AQJ
> >> fUm8%2BcufmaUsRbYtBAK2tdBCeNanqLK0%3D&reserved=0
> >> >> > >
> >> >> > > Best regards,
> >> >> > >
> >> >> > > Jason
> >> >> > >
> >> >> > >
> >> >> > > On Tue, Mar 3, 2026 at 4:32 PM Blain David <
> >> david.blain@infrabel.be>
> >> >> > > wrote:
> >> >> > >
> >> >> > > > Hi all,
> >> >> > > >
> >> >> > > > At our company we recently implemented an IBM MQ integration
> >> >> > > > for Airflow and I would like to gauge interest in
> >> >> > > > contributing this
> >> as a
> >> >> > > > new provider package.
> >> >> > > >
> >> >> > > > Motivation
> >> >> > > >
> >> >> > > > With the introduction of event-driven scheduling and the
> >> >> > > > MessageQueueProvider abstraction in Airflow, it has become
> >> >> > > > significantly easier to trigger DAGs from external message
> >> brokers
> >> >> (as
> >> >> > > > described in Astronomer's guide on event-driven scheduling):
> >> >> > > >
> >> >> > > > https://www/.
> >> >> > > > astronomer.io
> >> >> %2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05
> >> >> > > > %7C02%7Cdavid.blain%40infrabel.be
> >> >> %7C5d12469000204dfffcaa08de7908795c%7
> >> >> > > >
> >> >> Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CU
> >> >> nkno
> >> >> > > >
> >> >> wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiO
> >> >> iJXa
> >> >> > > >
> >> >> W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOn
> >> >> epYK
> >> >> > > > MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0
> >> >> > > >
> >> >> > > > Many enterprises still rely heavily on IBM MQ as their
> >> >> > > > primary enterprise messaging backbone. However, at the
> >> >> > > > moment there is no official Airflow provider supporting IBM MQ.
> >> >> > > >
> >> >> > > > Our implementation enables:
> >> >> > > >
> >> >> > > > * An IBMMQHook
> >> >> > > > * A MessageQueueProvider implementation for IBM MQ
> >> >> > > > * The ability to trigger DAGs from IBM MQ events
> >> >> > > > * Standard producer/consumer patterns from within tasks
> >> >> > > >
> >> >> > > > This allows IBM MQ to function similarly to Kafka, SQS,
> >> >> > > > etc.,
> >> within
> >> >> > > > the Airflow event-driven scheduling framework.
> >> >> > > >
> >> >> > > > Technical Details
> >> >> > > >
> >> >> > > > The implementation is built on top of the open-source IBM MQ
> >> Python
> >> >> > > > wrapper:
> >> >> > > >
> >> >> > > > * IBM MQ Python (ibmmq) library:
> >> >> > > > https://gith/
> >> >> > > > ub.com
> >> >> %2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in
> >> >> > > > frabel.be
> >> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f
> >> >> > > >
> >> >> 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJ
> >> >> FbXB
> >> >> > > >
> >> >> 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpb
> >> >> CIsI
> >> >> > > >
> >> >> ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKf
> >> >> jzI7
> >> >> > > > BILZDNmg%3D&reserved=0
> >> >> > > >
> >> >> > > > IBM has recently released and documented their modern Python
> >> binding
> >> >> > > here:
> >> >> > > >
> >> >> > > > https://comm/
> >> >> > > > unity.ibm.com
> >> >> %2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1
> >> >> > > >
> >> >> 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40inf
> >> >> rabe
> >> >> > > > l.be
> >> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f
> >> >> > > >
> >> >> 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0
> >> >> eU1h
> >> >> > > >
> >> >> cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIl
> >> >> dUIj
> >> >> > > >
> >> >> oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9
> >> >> F3sB
> >> >> > > > abTMg%3D&reserved=0
> >> >> > > >
> >> >> > > > The hook supports:
> >> >> > > >
> >> >> > > > * Secure connections (TLS)
> >> >> > > > * Queue get/put operations
> >> >> > > > * Configurable polling behavior
> >> >> > > > * Transaction handling where applicable
> >> >> > > >
> >> >> > > > The MessageQueueProvider implementation integrates with
> >> >> > > > Airflow's event-driven scheduling so that DAGs can be
> >> >> > > > triggered based on
> >> IBM
> >> >> MQ
> >> >> > > > messages.
> >> >> > > >
> >> >> > > > Why this might make sense:
> >> >> > > >
> >> >> > > > * IBM MQ is still widely used in regulated industries
> >> (banking,
> >> >> > > > insurance, government).
> >> >> > > > * Many enterprises using Airflow also run IBM MQ.
> >> >> > > > * This would allow IBM MQ to be a first-class citizen in
> >> >> Airflow's
> >> >> > > > event-driven ecosystem.
> >> >> > > > * The dependency is officially maintained by IBM and open
> >> >> source.
> >> >> > > >
> >> >> > > > I am willing to act as initial maintainer and code owner, of
> >> course
> >> >> > > > this is purely a proposition.
> >> >> > > >
> >> >> > > > I would appreciate feedback on:
> >> >> > > >
> >> >> > > > * Whether there is interest in such a provider
> >> >> > > > * If yes, whether it should live under
> >> >> apache-airflow-providers-ibm
> >> >> > > > * And if we formalize this as an AIP or draft PR
> >> >> > > >
> >> >> > > > Happy to share a draft implementation through a PR if there
> >> >> > > > is
> >> >> > interest.
> >> >> > > >
> >> >> > > > Thanks!
> >> >> > > > David
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >
> >>
> >
>
RE: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Posted by Blain David <da...@infrabel.be>.
Indeed, I did the same as with the Microsoft providers, as those are the one I was familiar with.
-----Original Message-----
From: Jarek Potiuk <ja...@potiuk.com>
Sent: 06 March 2026 21:49
To: Elad Kalif <el...@apache.org>
Cc: dev@airflow.apache.org; Vikram Koka <vi...@astronomer.io>
Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze niet vertrouwt, klik niet op een link of open geen bijlages. Bij twijfel, stuur deze e-mail als bijlage naar abuse@infrabel.be<ma...@infrabel.be>.
Actually Vikram, Elad - you are right, I was initially under the impression that -mq is "our message queue" -> but since this is about "IBM MQ" as a separate product/entity in IBM you are completely right it should be a separate provider.
On Fri, Mar 6, 2026 at 7:07 PM Elad Kalif <el...@apache.org> wrote:
> If IBM services are independent from one another we should follow how
> we do it on microsoft.
>
> While there is no apache-airflow-providers-microsoft provider.
> We do have:
> apache-airflow-providers-microsoft-mssql
> apache-airflow-providers-microsoft-azure
> and others... azure is not the same division as mssql.
>
> The google provider is a unique case. We have
> apache-airflow-providers-google due to historical reasons (google ads
> has nothing to do with google cloud) and splitting the provider is not
> easy task
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2Fapache%2Fairflow%2Fissues%2F15933&data=05%7C02%7Cdavid.blain%
> 40infrabel.be%7C753b70a0859d495498a408de7bc1e34f%7Cb82bc314ab8e4d6fb18
> 946f02e1f27f2%7C0%7C0%7C639084269845151921%7CUnknown%7CTWFpbGZsb3d8eyJ
> FbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpb
> CIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=I2QJFAYkyMpe8ytFByC3omVDvtz1drY
> rnNL5NTf4SIw%3D&reserved=0 I would also argue that
> apache-airflow-providers-amazon is not that good.
> If someone would like to add integration of Amazon marketplaces APIs
> (retail) that has nothing to do with AWS.
>
>
> On Fri, Mar 6, 2026 at 5:47 PM Vikram Koka via dev
> <de...@airflow.apache.org>
> wrote:
>
>> I am not sure about that Kaxil and Jarek.
>>
>> I understand the prior model of Google and Amazon, but those had
>> teams responsible for all the integrations with that service.
>> In the new governance model, the support burden for an interested
>> individual or individuals seems too high when the team is not a
>> service provider.
>>
>> Why not let individuals or SIs (system integrators) take the
>> "individual service supported" approach?
>> This is similar to the Apache services model.
>>
>> Vikram
>>
>>
>> On Wed, Mar 4, 2026 at 1:20 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>> > +1
>> >
>> > On Wed, Mar 4, 2026 at 10:17 PM Kaxil Naik <ka...@gmail.com> wrote:
>> >
>> >> It should just be called apache-airflow-providers-ibm, in future
>> >> folks might want to add more IBM hooks that are not just message queues.
>> >>
>> >> Similar to how Amazon & Google provider treats PubSub or Kinesis
>> >>
>> >> On Tue, 3 Mar 2026 at 16:57, Vikram Koka via dev <
>> dev@airflow.apache.org>
>> >> wrote:
>> >>
>> >> > Thanks David.
>> >> > I agree with Jason, that a PR is sufficient for this, and an AIP
>> >> > is
>> not
>> >> > required.
>> >> >
>> >> > Looking forward to this,
>> >> > Vikram
>> >> >
>> >> > On Tue, Mar 3, 2026 at 7:28 AM Blain David
>> >> > <da...@infrabel.be>
>> >> > wrote:
>> >> >
>> >> > > Hello Jason,
>> >> > >
>> >> > > Thanks for you reply, I've created a draft PR for this provider:
>> >> > >
>> >> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2
>> >> > > F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F62790&data=05%7C02%
>> >> > > 7Cdavid.blain%40infrabel.be%7C753b70a0859d495498a408de7bc1e34f
>> >> > > %7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C6390842698451855
>> >> > > 21%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjA
>> >> > > uMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%
>> >> > > 7C%7C%7C&sdata=TcqQfxqzMsZ%2BJME%2BZz1F1J0imMpaU30A5pgvN%2BkhX
>> >> > > KE%3D&reserved=0
>> >> > >
>> >> > > Kind regards,
>> >> > > David
>> >> > >
>> >> > > -----Original Message-----
>> >> > > From: Zhe-You(Jason) Liu <ja...@apache.org>
>> >> > > Sent: 03 March 2026 10:37
>> >> > > To: dev@airflow.apache.org
>> >> > > Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider
>> >> > > (Hook
>> +
>> >> > > MessageQueueProvider) to Airflow
>> >> > >
>> >> > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent
>> >> > > en
>> deze
>> >> > > niet vertrouwt, klik niet op een link of open geen bijlages.
>> >> > > Bij
>> >> twijfel,
>> >> > > stuur deze e-mail als bijlage naar abuse@infrabel.be<mailto:
>> >> > > abuse@infrabel.be>.
>> >> > >
>> >> > > Hi David,
>> >> > >
>> >> > > Thank you for your interest in adding a new
>> >> > > MessageQueueProvider to Airflow!
>> >> > >
>> >> > > I previously opened an issue about adding more providers that
>> support
>> >> > > MessageQueueProvider in the community providers. [1] You’re
>> >> > > more
>> than
>> >> > > welcome to contribute one!
>> >> > >
>> >> > > The only blocker to adding IBM MQ support to
>> >> > > MessageQueueProvider
>> is
>> >> "the
>> >> > > addition of a new IBM provider". We need to follow the
>> >> > > adoption
>> path
>> >> > > (AIP-95), similar to the recent Informatica provider. [2]
>> >> > >
>> >> > > > Whether there is interest in such a provider If yes, whether
>> >> > > > it
>> >> should
>> >> > > > live under apache-airflow-providers-ibm And if we formalize
>> >> > > > this
>> as
>> >> an
>> >> > > > AIP or draft PR
>> >> > >
>> >> > > So, from my perspective, the IBM Hook, Trigger, and
>> >> MessageQueueProvider
>> >> > > would be better placed under the IBM provider, and perhaps you
>> could
>> >> > start
>> >> > > by opening a draft PR and then initiating a voting thread on
>> >> > > the
>> dev
>> >> > > mailing list.
>> >> > >
>> >> > > Thanks!
>> >> > >
>> >> > > [1]
>> >> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2
>> >> > > F%2Fgithub.com%2Fapache%2Fairflow%2Fissues%2F52712&data=05%7C0
>> >> > > 2%7Cdavid.blain%40infrabel.be%7C753b70a0859d495498a408de7bc1e3
>> >> > > 4f%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C63908426984520
>> >> > > 2381%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwL
>> >> > > jAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C
>> >> > > 0%7C%7C%7C&sdata=zSpuj%2B%2Fx7UAUmDXHM0se6QYtpL6DOHBjGWXmvgAST
>> >> > > lc%3D&reserved=0
>> >> > > [2]
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flis
>> ts.apache.org%2Fthread%2Fwsfgh23jm6hkrly4lx1m21ftllqshpgo&data=05%7C0
>> 2%7Cdavid.blain%40infrabel.be%7C753b70a0859d495498a408de7bc1e34f%7Cb8
>> 2bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639084269845219729%7CUnknown
>> %7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW
>> 4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Ct8WnY45AQJ
>> fUm8%2BcufmaUsRbYtBAK2tdBCeNanqLK0%3D&reserved=0
>> >> > >
>> >> > > Best regards,
>> >> > >
>> >> > > Jason
>> >> > >
>> >> > >
>> >> > > On Tue, Mar 3, 2026 at 4:32 PM Blain David <
>> david.blain@infrabel.be>
>> >> > > wrote:
>> >> > >
>> >> > > > Hi all,
>> >> > > >
>> >> > > > At our company we recently implemented an IBM MQ integration
>> >> > > > for Airflow and I would like to gauge interest in
>> >> > > > contributing this
>> as a
>> >> > > > new provider package.
>> >> > > >
>> >> > > > Motivation
>> >> > > >
>> >> > > > With the introduction of event-driven scheduling and the
>> >> > > > MessageQueueProvider abstraction in Airflow, it has become
>> >> > > > significantly easier to trigger DAGs from external message
>> brokers
>> >> (as
>> >> > > > described in Astronomer's guide on event-driven scheduling):
>> >> > > >
>> >> > > > https://www/.
>> >> > > > astronomer.io
>> >> %2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05
>> >> > > > %7C02%7Cdavid.blain%40infrabel.be
>> >> %7C5d12469000204dfffcaa08de7908795c%7
>> >> > > >
>> >> Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CU
>> >> nkno
>> >> > > >
>> >> wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiO
>> >> iJXa
>> >> > > >
>> >> W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOn
>> >> epYK
>> >> > > > MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0
>> >> > > >
>> >> > > > Many enterprises still rely heavily on IBM MQ as their
>> >> > > > primary enterprise messaging backbone. However, at the
>> >> > > > moment there is no official Airflow provider supporting IBM MQ.
>> >> > > >
>> >> > > > Our implementation enables:
>> >> > > >
>> >> > > > * An IBMMQHook
>> >> > > > * A MessageQueueProvider implementation for IBM MQ
>> >> > > > * The ability to trigger DAGs from IBM MQ events
>> >> > > > * Standard producer/consumer patterns from within tasks
>> >> > > >
>> >> > > > This allows IBM MQ to function similarly to Kafka, SQS,
>> >> > > > etc.,
>> within
>> >> > > > the Airflow event-driven scheduling framework.
>> >> > > >
>> >> > > > Technical Details
>> >> > > >
>> >> > > > The implementation is built on top of the open-source IBM MQ
>> Python
>> >> > > > wrapper:
>> >> > > >
>> >> > > > * IBM MQ Python (ibmmq) library:
>> >> > > > https://gith/
>> >> > > > ub.com
>> >> %2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in
>> >> > > > frabel.be
>> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f
>> >> > > >
>> >> 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJ
>> >> FbXB
>> >> > > >
>> >> 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpb
>> >> CIsI
>> >> > > >
>> >> ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKf
>> >> jzI7
>> >> > > > BILZDNmg%3D&reserved=0
>> >> > > >
>> >> > > > IBM has recently released and documented their modern Python
>> binding
>> >> > > here:
>> >> > > >
>> >> > > > https://comm/
>> >> > > > unity.ibm.com
>> >> %2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1
>> >> > > >
>> >> 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40inf
>> >> rabe
>> >> > > > l.be
>> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f
>> >> > > >
>> >> 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0
>> >> eU1h
>> >> > > >
>> >> cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIl
>> >> dUIj
>> >> > > >
>> >> oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9
>> >> F3sB
>> >> > > > abTMg%3D&reserved=0
>> >> > > >
>> >> > > > The hook supports:
>> >> > > >
>> >> > > > * Secure connections (TLS)
>> >> > > > * Queue get/put operations
>> >> > > > * Configurable polling behavior
>> >> > > > * Transaction handling where applicable
>> >> > > >
>> >> > > > The MessageQueueProvider implementation integrates with
>> >> > > > Airflow's event-driven scheduling so that DAGs can be
>> >> > > > triggered based on
>> IBM
>> >> MQ
>> >> > > > messages.
>> >> > > >
>> >> > > > Why this might make sense:
>> >> > > >
>> >> > > > * IBM MQ is still widely used in regulated industries
>> (banking,
>> >> > > > insurance, government).
>> >> > > > * Many enterprises using Airflow also run IBM MQ.
>> >> > > > * This would allow IBM MQ to be a first-class citizen in
>> >> Airflow's
>> >> > > > event-driven ecosystem.
>> >> > > > * The dependency is officially maintained by IBM and open
>> >> source.
>> >> > > >
>> >> > > > I am willing to act as initial maintainer and code owner, of
>> course
>> >> > > > this is purely a proposition.
>> >> > > >
>> >> > > > I would appreciate feedback on:
>> >> > > >
>> >> > > > * Whether there is interest in such a provider
>> >> > > > * If yes, whether it should live under
>> >> apache-airflow-providers-ibm
>> >> > > > * And if we formalize this as an AIP or draft PR
>> >> > > >
>> >> > > > Happy to share a draft implementation through a PR if there
>> >> > > > is
>> >> > interest.
>> >> > > >
>> >> > > > Thanks!
>> >> > > > David
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>>
>
Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Posted by Jarek Potiuk <ja...@potiuk.com>.
Actually Vikram, Elad - you are right, I was initially under the impression
that -mq is "our message queue" -> but since this is about "IBM MQ" as a
separate product/entity in IBM you are completely right it should be a
separate provider.
On Fri, Mar 6, 2026 at 7:07 PM Elad Kalif <el...@apache.org> wrote:
> If IBM services are independent from one another we should follow how we
> do it on microsoft.
>
> While there is no apache-airflow-providers-microsoft provider.
> We do have:
> apache-airflow-providers-microsoft-mssql
> apache-airflow-providers-microsoft-azure
> and others... azure is not the same division as mssql.
>
> The google provider is a unique case. We have
> apache-airflow-providers-google due to historical reasons (google ads has
> nothing to do with google cloud) and splitting the provider is not easy
> task https://github.com/apache/airflow/issues/15933
> I would also argue that apache-airflow-providers-amazon is not that good.
> If someone would like to add integration of Amazon marketplaces APIs
> (retail) that has nothing to do with AWS.
>
>
> On Fri, Mar 6, 2026 at 5:47 PM Vikram Koka via dev <de...@airflow.apache.org>
> wrote:
>
>> I am not sure about that Kaxil and Jarek.
>>
>> I understand the prior model of Google and Amazon, but those had teams
>> responsible for all the integrations with that service.
>> In the new governance model, the support burden for an interested
>> individual or individuals seems too high when the team is not a service
>> provider.
>>
>> Why not let individuals or SIs (system integrators) take the "individual
>> service supported" approach?
>> This is similar to the Apache services model.
>>
>> Vikram
>>
>>
>> On Wed, Mar 4, 2026 at 1:20 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>> > +1
>> >
>> > On Wed, Mar 4, 2026 at 10:17 PM Kaxil Naik <ka...@gmail.com> wrote:
>> >
>> >> It should just be called apache-airflow-providers-ibm, in future folks
>> >> might want to add more IBM hooks that are not just message queues.
>> >>
>> >> Similar to how Amazon & Google provider treats PubSub or Kinesis
>> >>
>> >> On Tue, 3 Mar 2026 at 16:57, Vikram Koka via dev <
>> dev@airflow.apache.org>
>> >> wrote:
>> >>
>> >> > Thanks David.
>> >> > I agree with Jason, that a PR is sufficient for this, and an AIP is
>> not
>> >> > required.
>> >> >
>> >> > Looking forward to this,
>> >> > Vikram
>> >> >
>> >> > On Tue, Mar 3, 2026 at 7:28 AM Blain David <da...@infrabel.be>
>> >> > wrote:
>> >> >
>> >> > > Hello Jason,
>> >> > >
>> >> > > Thanks for you reply, I've created a draft PR for this provider:
>> >> > >
>> >> > > https://github.com/apache/airflow/pull/62790
>> >> > >
>> >> > > Kind regards,
>> >> > > David
>> >> > >
>> >> > > -----Original Message-----
>> >> > > From: Zhe-You(Jason) Liu <ja...@apache.org>
>> >> > > Sent: 03 March 2026 10:37
>> >> > > To: dev@airflow.apache.org
>> >> > > Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook
>> +
>> >> > > MessageQueueProvider) to Airflow
>> >> > >
>> >> > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en
>> deze
>> >> > > niet vertrouwt, klik niet op een link of open geen bijlages. Bij
>> >> twijfel,
>> >> > > stuur deze e-mail als bijlage naar abuse@infrabel.be<mailto:
>> >> > > abuse@infrabel.be>.
>> >> > >
>> >> > > Hi David,
>> >> > >
>> >> > > Thank you for your interest in adding a new MessageQueueProvider to
>> >> > > Airflow!
>> >> > >
>> >> > > I previously opened an issue about adding more providers that
>> support
>> >> > > MessageQueueProvider in the community providers. [1] You’re more
>> than
>> >> > > welcome to contribute one!
>> >> > >
>> >> > > The only blocker to adding IBM MQ support to MessageQueueProvider
>> is
>> >> "the
>> >> > > addition of a new IBM provider". We need to follow the adoption
>> path
>> >> > > (AIP-95), similar to the recent Informatica provider. [2]
>> >> > >
>> >> > > > Whether there is interest in such a provider If yes, whether it
>> >> should
>> >> > > > live under apache-airflow-providers-ibm And if we formalize this
>> as
>> >> an
>> >> > > > AIP or draft PR
>> >> > >
>> >> > > So, from my perspective, the IBM Hook, Trigger, and
>> >> MessageQueueProvider
>> >> > > would be better placed under the IBM provider, and perhaps you
>> could
>> >> > start
>> >> > > by opening a draft PR and then initiating a voting thread on the
>> dev
>> >> > > mailing list.
>> >> > >
>> >> > > Thanks!
>> >> > >
>> >> > > [1] https://github.com/apache/airflow/issues/52712
>> >> > > [2]
>> https://lists.apache.org/thread/wsfgh23jm6hkrly4lx1m21ftllqshpgo
>> >> > >
>> >> > > Best regards,
>> >> > >
>> >> > > Jason
>> >> > >
>> >> > >
>> >> > > On Tue, Mar 3, 2026 at 4:32 PM Blain David <
>> david.blain@infrabel.be>
>> >> > > wrote:
>> >> > >
>> >> > > > Hi all,
>> >> > > >
>> >> > > > At our company we recently implemented an IBM MQ integration for
>> >> > > > Airflow and I would like to gauge interest in contributing this
>> as a
>> >> > > > new provider package.
>> >> > > >
>> >> > > > Motivation
>> >> > > >
>> >> > > > With the introduction of event-driven scheduling and the
>> >> > > > MessageQueueProvider abstraction in Airflow, it has become
>> >> > > > significantly easier to trigger DAGs from external message
>> brokers
>> >> (as
>> >> > > > described in Astronomer's guide on event-driven scheduling):
>> >> > > >
>> >> > > > https://www/.
>> >> > > > astronomer.io
>> >> %2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05
>> >> > > > %7C02%7Cdavid.blain%40infrabel.be
>> >> %7C5d12469000204dfffcaa08de7908795c%7
>> >> > > >
>> >> Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CUnkno
>> >> > > >
>> >> wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXa
>> >> > > >
>> >> W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOnepYK
>> >> > > > MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0
>> >> > > >
>> >> > > > Many enterprises still rely heavily on IBM MQ as their primary
>> >> > > > enterprise messaging backbone. However, at the moment there is no
>> >> > > > official Airflow provider supporting IBM MQ.
>> >> > > >
>> >> > > > Our implementation enables:
>> >> > > >
>> >> > > > * An IBMMQHook
>> >> > > > * A MessageQueueProvider implementation for IBM MQ
>> >> > > > * The ability to trigger DAGs from IBM MQ events
>> >> > > > * Standard producer/consumer patterns from within tasks
>> >> > > >
>> >> > > > This allows IBM MQ to function similarly to Kafka, SQS, etc.,
>> within
>> >> > > > the Airflow event-driven scheduling framework.
>> >> > > >
>> >> > > > Technical Details
>> >> > > >
>> >> > > > The implementation is built on top of the open-source IBM MQ
>> Python
>> >> > > > wrapper:
>> >> > > >
>> >> > > > * IBM MQ Python (ibmmq) library:
>> >> > > > https://gith/
>> >> > > > ub.com
>> >> %2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in
>> >> > > > frabel.be
>> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f
>> >> > > >
>> >> 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJFbXB
>> >> > > >
>> >> 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsI
>> >> > > >
>> >> ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKfjzI7
>> >> > > > BILZDNmg%3D&reserved=0
>> >> > > >
>> >> > > > IBM has recently released and documented their modern Python
>> binding
>> >> > > here:
>> >> > > >
>> >> > > > https://comm/
>> >> > > > unity.ibm.com
>> >> %2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1
>> >> > > >
>> >> 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40infrabe
>> >> > > > l.be
>> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f
>> >> > > >
>> >> 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1h
>> >> > > >
>> >> cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIj
>> >> > > >
>> >> oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9F3sB
>> >> > > > abTMg%3D&reserved=0
>> >> > > >
>> >> > > > The hook supports:
>> >> > > >
>> >> > > > * Secure connections (TLS)
>> >> > > > * Queue get/put operations
>> >> > > > * Configurable polling behavior
>> >> > > > * Transaction handling where applicable
>> >> > > >
>> >> > > > The MessageQueueProvider implementation integrates with Airflow's
>> >> > > > event-driven scheduling so that DAGs can be triggered based on
>> IBM
>> >> MQ
>> >> > > > messages.
>> >> > > >
>> >> > > > Why this might make sense:
>> >> > > >
>> >> > > > * IBM MQ is still widely used in regulated industries
>> (banking,
>> >> > > > insurance, government).
>> >> > > > * Many enterprises using Airflow also run IBM MQ.
>> >> > > > * This would allow IBM MQ to be a first-class citizen in
>> >> Airflow's
>> >> > > > event-driven ecosystem.
>> >> > > > * The dependency is officially maintained by IBM and open
>> >> source.
>> >> > > >
>> >> > > > I am willing to act as initial maintainer and code owner, of
>> course
>> >> > > > this is purely a proposition.
>> >> > > >
>> >> > > > I would appreciate feedback on:
>> >> > > >
>> >> > > > * Whether there is interest in such a provider
>> >> > > > * If yes, whether it should live under
>> >> apache-airflow-providers-ibm
>> >> > > > * And if we formalize this as an AIP or draft PR
>> >> > > >
>> >> > > > Happy to share a draft implementation through a PR if there is
>> >> > interest.
>> >> > > >
>> >> > > > Thanks!
>> >> > > > David
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>>
>
Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Posted by Elad Kalif <el...@apache.org>.
If IBM services are independent from one another we should follow how we do
it on microsoft.
While there is no apache-airflow-providers-microsoft provider.
We do have:
apache-airflow-providers-microsoft-mssql
apache-airflow-providers-microsoft-azure
and others... azure is not the same division as mssql.
The google provider is a unique case. We have
apache-airflow-providers-google due to historical reasons (google ads has
nothing to do with google cloud) and splitting the provider is not easy
task https://github.com/apache/airflow/issues/15933
I would also argue that apache-airflow-providers-amazon is not that good.
If someone would like to add integration of Amazon marketplaces APIs
(retail) that has nothing to do with AWS.
On Fri, Mar 6, 2026 at 5:47 PM Vikram Koka via dev <de...@airflow.apache.org>
wrote:
> I am not sure about that Kaxil and Jarek.
>
> I understand the prior model of Google and Amazon, but those had teams
> responsible for all the integrations with that service.
> In the new governance model, the support burden for an interested
> individual or individuals seems too high when the team is not a service
> provider.
>
> Why not let individuals or SIs (system integrators) take the "individual
> service supported" approach?
> This is similar to the Apache services model.
>
> Vikram
>
>
> On Wed, Mar 4, 2026 at 1:20 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
> > +1
> >
> > On Wed, Mar 4, 2026 at 10:17 PM Kaxil Naik <ka...@gmail.com> wrote:
> >
> >> It should just be called apache-airflow-providers-ibm, in future folks
> >> might want to add more IBM hooks that are not just message queues.
> >>
> >> Similar to how Amazon & Google provider treats PubSub or Kinesis
> >>
> >> On Tue, 3 Mar 2026 at 16:57, Vikram Koka via dev <
> dev@airflow.apache.org>
> >> wrote:
> >>
> >> > Thanks David.
> >> > I agree with Jason, that a PR is sufficient for this, and an AIP is
> not
> >> > required.
> >> >
> >> > Looking forward to this,
> >> > Vikram
> >> >
> >> > On Tue, Mar 3, 2026 at 7:28 AM Blain David <da...@infrabel.be>
> >> > wrote:
> >> >
> >> > > Hello Jason,
> >> > >
> >> > > Thanks for you reply, I've created a draft PR for this provider:
> >> > >
> >> > > https://github.com/apache/airflow/pull/62790
> >> > >
> >> > > Kind regards,
> >> > > David
> >> > >
> >> > > -----Original Message-----
> >> > > From: Zhe-You(Jason) Liu <ja...@apache.org>
> >> > > Sent: 03 March 2026 10:37
> >> > > To: dev@airflow.apache.org
> >> > > Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook +
> >> > > MessageQueueProvider) to Airflow
> >> > >
> >> > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en
> deze
> >> > > niet vertrouwt, klik niet op een link of open geen bijlages. Bij
> >> twijfel,
> >> > > stuur deze e-mail als bijlage naar abuse@infrabel.be<mailto:
> >> > > abuse@infrabel.be>.
> >> > >
> >> > > Hi David,
> >> > >
> >> > > Thank you for your interest in adding a new MessageQueueProvider to
> >> > > Airflow!
> >> > >
> >> > > I previously opened an issue about adding more providers that
> support
> >> > > MessageQueueProvider in the community providers. [1] You’re more
> than
> >> > > welcome to contribute one!
> >> > >
> >> > > The only blocker to adding IBM MQ support to MessageQueueProvider is
> >> "the
> >> > > addition of a new IBM provider". We need to follow the adoption path
> >> > > (AIP-95), similar to the recent Informatica provider. [2]
> >> > >
> >> > > > Whether there is interest in such a provider If yes, whether it
> >> should
> >> > > > live under apache-airflow-providers-ibm And if we formalize this
> as
> >> an
> >> > > > AIP or draft PR
> >> > >
> >> > > So, from my perspective, the IBM Hook, Trigger, and
> >> MessageQueueProvider
> >> > > would be better placed under the IBM provider, and perhaps you could
> >> > start
> >> > > by opening a draft PR and then initiating a voting thread on the dev
> >> > > mailing list.
> >> > >
> >> > > Thanks!
> >> > >
> >> > > [1] https://github.com/apache/airflow/issues/52712
> >> > > [2]
> https://lists.apache.org/thread/wsfgh23jm6hkrly4lx1m21ftllqshpgo
> >> > >
> >> > > Best regards,
> >> > >
> >> > > Jason
> >> > >
> >> > >
> >> > > On Tue, Mar 3, 2026 at 4:32 PM Blain David <david.blain@infrabel.be
> >
> >> > > wrote:
> >> > >
> >> > > > Hi all,
> >> > > >
> >> > > > At our company we recently implemented an IBM MQ integration for
> >> > > > Airflow and I would like to gauge interest in contributing this
> as a
> >> > > > new provider package.
> >> > > >
> >> > > > Motivation
> >> > > >
> >> > > > With the introduction of event-driven scheduling and the
> >> > > > MessageQueueProvider abstraction in Airflow, it has become
> >> > > > significantly easier to trigger DAGs from external message brokers
> >> (as
> >> > > > described in Astronomer's guide on event-driven scheduling):
> >> > > >
> >> > > > https://www/.
> >> > > > astronomer.io
> >> %2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05
> >> > > > %7C02%7Cdavid.blain%40infrabel.be
> >> %7C5d12469000204dfffcaa08de7908795c%7
> >> > > >
> >> Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CUnkno
> >> > > >
> >> wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXa
> >> > > >
> >> W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOnepYK
> >> > > > MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0
> >> > > >
> >> > > > Many enterprises still rely heavily on IBM MQ as their primary
> >> > > > enterprise messaging backbone. However, at the moment there is no
> >> > > > official Airflow provider supporting IBM MQ.
> >> > > >
> >> > > > Our implementation enables:
> >> > > >
> >> > > > * An IBMMQHook
> >> > > > * A MessageQueueProvider implementation for IBM MQ
> >> > > > * The ability to trigger DAGs from IBM MQ events
> >> > > > * Standard producer/consumer patterns from within tasks
> >> > > >
> >> > > > This allows IBM MQ to function similarly to Kafka, SQS, etc.,
> within
> >> > > > the Airflow event-driven scheduling framework.
> >> > > >
> >> > > > Technical Details
> >> > > >
> >> > > > The implementation is built on top of the open-source IBM MQ
> Python
> >> > > > wrapper:
> >> > > >
> >> > > > * IBM MQ Python (ibmmq) library:
> >> > > > https://gith/
> >> > > > ub.com
> >> %2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in
> >> > > > frabel.be
> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f
> >> > > >
> >> 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJFbXB
> >> > > >
> >> 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsI
> >> > > >
> >> ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKfjzI7
> >> > > > BILZDNmg%3D&reserved=0
> >> > > >
> >> > > > IBM has recently released and documented their modern Python
> binding
> >> > > here:
> >> > > >
> >> > > > https://comm/
> >> > > > unity.ibm.com
> >> %2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1
> >> > > >
> >> 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40infrabe
> >> > > > l.be
> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f
> >> > > >
> >> 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1h
> >> > > >
> >> cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIj
> >> > > >
> >> oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9F3sB
> >> > > > abTMg%3D&reserved=0
> >> > > >
> >> > > > The hook supports:
> >> > > >
> >> > > > * Secure connections (TLS)
> >> > > > * Queue get/put operations
> >> > > > * Configurable polling behavior
> >> > > > * Transaction handling where applicable
> >> > > >
> >> > > > The MessageQueueProvider implementation integrates with Airflow's
> >> > > > event-driven scheduling so that DAGs can be triggered based on IBM
> >> MQ
> >> > > > messages.
> >> > > >
> >> > > > Why this might make sense:
> >> > > >
> >> > > > * IBM MQ is still widely used in regulated industries
> (banking,
> >> > > > insurance, government).
> >> > > > * Many enterprises using Airflow also run IBM MQ.
> >> > > > * This would allow IBM MQ to be a first-class citizen in
> >> Airflow's
> >> > > > event-driven ecosystem.
> >> > > > * The dependency is officially maintained by IBM and open
> >> source.
> >> > > >
> >> > > > I am willing to act as initial maintainer and code owner, of
> course
> >> > > > this is purely a proposition.
> >> > > >
> >> > > > I would appreciate feedback on:
> >> > > >
> >> > > > * Whether there is interest in such a provider
> >> > > > * If yes, whether it should live under
> >> apache-airflow-providers-ibm
> >> > > > * And if we formalize this as an AIP or draft PR
> >> > > >
> >> > > > Happy to share a draft implementation through a PR if there is
> >> > interest.
> >> > > >
> >> > > > Thanks!
> >> > > > David
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Posted by Vikram Koka via dev <de...@airflow.apache.org>.
I am not sure about that Kaxil and Jarek.
I understand the prior model of Google and Amazon, but those had teams
responsible for all the integrations with that service.
In the new governance model, the support burden for an interested
individual or individuals seems too high when the team is not a service
provider.
Why not let individuals or SIs (system integrators) take the "individual
service supported" approach?
This is similar to the Apache services model.
Vikram
On Wed, Mar 4, 2026 at 1:20 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> +1
>
> On Wed, Mar 4, 2026 at 10:17 PM Kaxil Naik <ka...@gmail.com> wrote:
>
>> It should just be called apache-airflow-providers-ibm, in future folks
>> might want to add more IBM hooks that are not just message queues.
>>
>> Similar to how Amazon & Google provider treats PubSub or Kinesis
>>
>> On Tue, 3 Mar 2026 at 16:57, Vikram Koka via dev <de...@airflow.apache.org>
>> wrote:
>>
>> > Thanks David.
>> > I agree with Jason, that a PR is sufficient for this, and an AIP is not
>> > required.
>> >
>> > Looking forward to this,
>> > Vikram
>> >
>> > On Tue, Mar 3, 2026 at 7:28 AM Blain David <da...@infrabel.be>
>> > wrote:
>> >
>> > > Hello Jason,
>> > >
>> > > Thanks for you reply, I've created a draft PR for this provider:
>> > >
>> > > https://github.com/apache/airflow/pull/62790
>> > >
>> > > Kind regards,
>> > > David
>> > >
>> > > -----Original Message-----
>> > > From: Zhe-You(Jason) Liu <ja...@apache.org>
>> > > Sent: 03 March 2026 10:37
>> > > To: dev@airflow.apache.org
>> > > Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook +
>> > > MessageQueueProvider) to Airflow
>> > >
>> > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze
>> > > niet vertrouwt, klik niet op een link of open geen bijlages. Bij
>> twijfel,
>> > > stuur deze e-mail als bijlage naar abuse@infrabel.be<mailto:
>> > > abuse@infrabel.be>.
>> > >
>> > > Hi David,
>> > >
>> > > Thank you for your interest in adding a new MessageQueueProvider to
>> > > Airflow!
>> > >
>> > > I previously opened an issue about adding more providers that support
>> > > MessageQueueProvider in the community providers. [1] You’re more than
>> > > welcome to contribute one!
>> > >
>> > > The only blocker to adding IBM MQ support to MessageQueueProvider is
>> "the
>> > > addition of a new IBM provider". We need to follow the adoption path
>> > > (AIP-95), similar to the recent Informatica provider. [2]
>> > >
>> > > > Whether there is interest in such a provider If yes, whether it
>> should
>> > > > live under apache-airflow-providers-ibm And if we formalize this as
>> an
>> > > > AIP or draft PR
>> > >
>> > > So, from my perspective, the IBM Hook, Trigger, and
>> MessageQueueProvider
>> > > would be better placed under the IBM provider, and perhaps you could
>> > start
>> > > by opening a draft PR and then initiating a voting thread on the dev
>> > > mailing list.
>> > >
>> > > Thanks!
>> > >
>> > > [1] https://github.com/apache/airflow/issues/52712
>> > > [2] https://lists.apache.org/thread/wsfgh23jm6hkrly4lx1m21ftllqshpgo
>> > >
>> > > Best regards,
>> > >
>> > > Jason
>> > >
>> > >
>> > > On Tue, Mar 3, 2026 at 4:32 PM Blain David <da...@infrabel.be>
>> > > wrote:
>> > >
>> > > > Hi all,
>> > > >
>> > > > At our company we recently implemented an IBM MQ integration for
>> > > > Airflow and I would like to gauge interest in contributing this as a
>> > > > new provider package.
>> > > >
>> > > > Motivation
>> > > >
>> > > > With the introduction of event-driven scheduling and the
>> > > > MessageQueueProvider abstraction in Airflow, it has become
>> > > > significantly easier to trigger DAGs from external message brokers
>> (as
>> > > > described in Astronomer's guide on event-driven scheduling):
>> > > >
>> > > > https://www/.
>> > > > astronomer.io
>> %2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05
>> > > > %7C02%7Cdavid.blain%40infrabel.be
>> %7C5d12469000204dfffcaa08de7908795c%7
>> > > >
>> Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CUnkno
>> > > >
>> wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXa
>> > > >
>> W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOnepYK
>> > > > MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0
>> > > >
>> > > > Many enterprises still rely heavily on IBM MQ as their primary
>> > > > enterprise messaging backbone. However, at the moment there is no
>> > > > official Airflow provider supporting IBM MQ.
>> > > >
>> > > > Our implementation enables:
>> > > >
>> > > > * An IBMMQHook
>> > > > * A MessageQueueProvider implementation for IBM MQ
>> > > > * The ability to trigger DAGs from IBM MQ events
>> > > > * Standard producer/consumer patterns from within tasks
>> > > >
>> > > > This allows IBM MQ to function similarly to Kafka, SQS, etc., within
>> > > > the Airflow event-driven scheduling framework.
>> > > >
>> > > > Technical Details
>> > > >
>> > > > The implementation is built on top of the open-source IBM MQ Python
>> > > > wrapper:
>> > > >
>> > > > * IBM MQ Python (ibmmq) library:
>> > > > https://gith/
>> > > > ub.com
>> %2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in
>> > > > frabel.be
>> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f
>> > > >
>> 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJFbXB
>> > > >
>> 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsI
>> > > >
>> ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKfjzI7
>> > > > BILZDNmg%3D&reserved=0
>> > > >
>> > > > IBM has recently released and documented their modern Python binding
>> > > here:
>> > > >
>> > > > https://comm/
>> > > > unity.ibm.com
>> %2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1
>> > > >
>> 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40infrabe
>> > > > l.be
>> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f
>> > > >
>> 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1h
>> > > >
>> cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIj
>> > > >
>> oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9F3sB
>> > > > abTMg%3D&reserved=0
>> > > >
>> > > > The hook supports:
>> > > >
>> > > > * Secure connections (TLS)
>> > > > * Queue get/put operations
>> > > > * Configurable polling behavior
>> > > > * Transaction handling where applicable
>> > > >
>> > > > The MessageQueueProvider implementation integrates with Airflow's
>> > > > event-driven scheduling so that DAGs can be triggered based on IBM
>> MQ
>> > > > messages.
>> > > >
>> > > > Why this might make sense:
>> > > >
>> > > > * IBM MQ is still widely used in regulated industries (banking,
>> > > > insurance, government).
>> > > > * Many enterprises using Airflow also run IBM MQ.
>> > > > * This would allow IBM MQ to be a first-class citizen in
>> Airflow's
>> > > > event-driven ecosystem.
>> > > > * The dependency is officially maintained by IBM and open
>> source.
>> > > >
>> > > > I am willing to act as initial maintainer and code owner, of course
>> > > > this is purely a proposition.
>> > > >
>> > > > I would appreciate feedback on:
>> > > >
>> > > > * Whether there is interest in such a provider
>> > > > * If yes, whether it should live under
>> apache-airflow-providers-ibm
>> > > > * And if we formalize this as an AIP or draft PR
>> > > >
>> > > > Happy to share a draft implementation through a PR if there is
>> > interest.
>> > > >
>> > > > Thanks!
>> > > > David
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Posted by Jarek Potiuk <ja...@potiuk.com>.
+1
On Wed, Mar 4, 2026 at 10:17 PM Kaxil Naik <ka...@gmail.com> wrote:
> It should just be called apache-airflow-providers-ibm, in future folks
> might want to add more IBM hooks that are not just message queues.
>
> Similar to how Amazon & Google provider treats PubSub or Kinesis
>
> On Tue, 3 Mar 2026 at 16:57, Vikram Koka via dev <de...@airflow.apache.org>
> wrote:
>
> > Thanks David.
> > I agree with Jason, that a PR is sufficient for this, and an AIP is not
> > required.
> >
> > Looking forward to this,
> > Vikram
> >
> > On Tue, Mar 3, 2026 at 7:28 AM Blain David <da...@infrabel.be>
> > wrote:
> >
> > > Hello Jason,
> > >
> > > Thanks for you reply, I've created a draft PR for this provider:
> > >
> > > https://github.com/apache/airflow/pull/62790
> > >
> > > Kind regards,
> > > David
> > >
> > > -----Original Message-----
> > > From: Zhe-You(Jason) Liu <ja...@apache.org>
> > > Sent: 03 March 2026 10:37
> > > To: dev@airflow.apache.org
> > > Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook +
> > > MessageQueueProvider) to Airflow
> > >
> > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze
> > > niet vertrouwt, klik niet op een link of open geen bijlages. Bij
> twijfel,
> > > stuur deze e-mail als bijlage naar abuse@infrabel.be<mailto:
> > > abuse@infrabel.be>.
> > >
> > > Hi David,
> > >
> > > Thank you for your interest in adding a new MessageQueueProvider to
> > > Airflow!
> > >
> > > I previously opened an issue about adding more providers that support
> > > MessageQueueProvider in the community providers. [1] You’re more than
> > > welcome to contribute one!
> > >
> > > The only blocker to adding IBM MQ support to MessageQueueProvider is
> "the
> > > addition of a new IBM provider". We need to follow the adoption path
> > > (AIP-95), similar to the recent Informatica provider. [2]
> > >
> > > > Whether there is interest in such a provider If yes, whether it
> should
> > > > live under apache-airflow-providers-ibm And if we formalize this as
> an
> > > > AIP or draft PR
> > >
> > > So, from my perspective, the IBM Hook, Trigger, and
> MessageQueueProvider
> > > would be better placed under the IBM provider, and perhaps you could
> > start
> > > by opening a draft PR and then initiating a voting thread on the dev
> > > mailing list.
> > >
> > > Thanks!
> > >
> > > [1] https://github.com/apache/airflow/issues/52712
> > > [2] https://lists.apache.org/thread/wsfgh23jm6hkrly4lx1m21ftllqshpgo
> > >
> > > Best regards,
> > >
> > > Jason
> > >
> > >
> > > On Tue, Mar 3, 2026 at 4:32 PM Blain David <da...@infrabel.be>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > At our company we recently implemented an IBM MQ integration for
> > > > Airflow and I would like to gauge interest in contributing this as a
> > > > new provider package.
> > > >
> > > > Motivation
> > > >
> > > > With the introduction of event-driven scheduling and the
> > > > MessageQueueProvider abstraction in Airflow, it has become
> > > > significantly easier to trigger DAGs from external message brokers
> (as
> > > > described in Astronomer's guide on event-driven scheduling):
> > > >
> > > > https://www/.
> > > > astronomer.io
> %2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05
> > > > %7C02%7Cdavid.blain%40infrabel.be
> %7C5d12469000204dfffcaa08de7908795c%7
> > > >
> Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CUnkno
> > > >
> wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXa
> > > >
> W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOnepYK
> > > > MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0
> > > >
> > > > Many enterprises still rely heavily on IBM MQ as their primary
> > > > enterprise messaging backbone. However, at the moment there is no
> > > > official Airflow provider supporting IBM MQ.
> > > >
> > > > Our implementation enables:
> > > >
> > > > * An IBMMQHook
> > > > * A MessageQueueProvider implementation for IBM MQ
> > > > * The ability to trigger DAGs from IBM MQ events
> > > > * Standard producer/consumer patterns from within tasks
> > > >
> > > > This allows IBM MQ to function similarly to Kafka, SQS, etc., within
> > > > the Airflow event-driven scheduling framework.
> > > >
> > > > Technical Details
> > > >
> > > > The implementation is built on top of the open-source IBM MQ Python
> > > > wrapper:
> > > >
> > > > * IBM MQ Python (ibmmq) library:
> > > > https://gith/
> > > > ub.com
> %2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in
> > > > frabel.be
> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f
> > > >
> 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJFbXB
> > > >
> 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsI
> > > >
> ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKfjzI7
> > > > BILZDNmg%3D&reserved=0
> > > >
> > > > IBM has recently released and documented their modern Python binding
> > > here:
> > > >
> > > > https://comm/
> > > > unity.ibm.com
> %2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1
> > > >
> 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40infrabe
> > > > l.be
> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f
> > > >
> 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1h
> > > >
> cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIj
> > > >
> oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9F3sB
> > > > abTMg%3D&reserved=0
> > > >
> > > > The hook supports:
> > > >
> > > > * Secure connections (TLS)
> > > > * Queue get/put operations
> > > > * Configurable polling behavior
> > > > * Transaction handling where applicable
> > > >
> > > > The MessageQueueProvider implementation integrates with Airflow's
> > > > event-driven scheduling so that DAGs can be triggered based on IBM MQ
> > > > messages.
> > > >
> > > > Why this might make sense:
> > > >
> > > > * IBM MQ is still widely used in regulated industries (banking,
> > > > insurance, government).
> > > > * Many enterprises using Airflow also run IBM MQ.
> > > > * This would allow IBM MQ to be a first-class citizen in
> Airflow's
> > > > event-driven ecosystem.
> > > > * The dependency is officially maintained by IBM and open source.
> > > >
> > > > I am willing to act as initial maintainer and code owner, of course
> > > > this is purely a proposition.
> > > >
> > > > I would appreciate feedback on:
> > > >
> > > > * Whether there is interest in such a provider
> > > > * If yes, whether it should live under
> apache-airflow-providers-ibm
> > > > * And if we formalize this as an AIP or draft PR
> > > >
> > > > Happy to share a draft implementation through a PR if there is
> > interest.
> > > >
> > > > Thanks!
> > > > David
> > > >
> > >
> >
>
Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Posted by Kaxil Naik <ka...@gmail.com>.
It should just be called apache-airflow-providers-ibm, in future folks
might want to add more IBM hooks that are not just message queues.
Similar to how Amazon & Google provider treats PubSub or Kinesis
On Tue, 3 Mar 2026 at 16:57, Vikram Koka via dev <de...@airflow.apache.org>
wrote:
> Thanks David.
> I agree with Jason, that a PR is sufficient for this, and an AIP is not
> required.
>
> Looking forward to this,
> Vikram
>
> On Tue, Mar 3, 2026 at 7:28 AM Blain David <da...@infrabel.be>
> wrote:
>
> > Hello Jason,
> >
> > Thanks for you reply, I've created a draft PR for this provider:
> >
> > https://github.com/apache/airflow/pull/62790
> >
> > Kind regards,
> > David
> >
> > -----Original Message-----
> > From: Zhe-You(Jason) Liu <ja...@apache.org>
> > Sent: 03 March 2026 10:37
> > To: dev@airflow.apache.org
> > Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook +
> > MessageQueueProvider) to Airflow
> >
> > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze
> > niet vertrouwt, klik niet op een link of open geen bijlages. Bij twijfel,
> > stuur deze e-mail als bijlage naar abuse@infrabel.be<mailto:
> > abuse@infrabel.be>.
> >
> > Hi David,
> >
> > Thank you for your interest in adding a new MessageQueueProvider to
> > Airflow!
> >
> > I previously opened an issue about adding more providers that support
> > MessageQueueProvider in the community providers. [1] You’re more than
> > welcome to contribute one!
> >
> > The only blocker to adding IBM MQ support to MessageQueueProvider is "the
> > addition of a new IBM provider". We need to follow the adoption path
> > (AIP-95), similar to the recent Informatica provider. [2]
> >
> > > Whether there is interest in such a provider If yes, whether it should
> > > live under apache-airflow-providers-ibm And if we formalize this as an
> > > AIP or draft PR
> >
> > So, from my perspective, the IBM Hook, Trigger, and MessageQueueProvider
> > would be better placed under the IBM provider, and perhaps you could
> start
> > by opening a draft PR and then initiating a voting thread on the dev
> > mailing list.
> >
> > Thanks!
> >
> > [1] https://github.com/apache/airflow/issues/52712
> > [2] https://lists.apache.org/thread/wsfgh23jm6hkrly4lx1m21ftllqshpgo
> >
> > Best regards,
> >
> > Jason
> >
> >
> > On Tue, Mar 3, 2026 at 4:32 PM Blain David <da...@infrabel.be>
> > wrote:
> >
> > > Hi all,
> > >
> > > At our company we recently implemented an IBM MQ integration for
> > > Airflow and I would like to gauge interest in contributing this as a
> > > new provider package.
> > >
> > > Motivation
> > >
> > > With the introduction of event-driven scheduling and the
> > > MessageQueueProvider abstraction in Airflow, it has become
> > > significantly easier to trigger DAGs from external message brokers (as
> > > described in Astronomer's guide on event-driven scheduling):
> > >
> > > https://www/.
> > > astronomer.io%2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05
> > > %7C02%7Cdavid.blain%40infrabel.be%7C5d12469000204dfffcaa08de7908795c%7
> > > Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CUnkno
> > > wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXa
> > > W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOnepYK
> > > MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0
> > >
> > > Many enterprises still rely heavily on IBM MQ as their primary
> > > enterprise messaging backbone. However, at the moment there is no
> > > official Airflow provider supporting IBM MQ.
> > >
> > > Our implementation enables:
> > >
> > > * An IBMMQHook
> > > * A MessageQueueProvider implementation for IBM MQ
> > > * The ability to trigger DAGs from IBM MQ events
> > > * Standard producer/consumer patterns from within tasks
> > >
> > > This allows IBM MQ to function similarly to Kafka, SQS, etc., within
> > > the Airflow event-driven scheduling framework.
> > >
> > > Technical Details
> > >
> > > The implementation is built on top of the open-source IBM MQ Python
> > > wrapper:
> > >
> > > * IBM MQ Python (ibmmq) library:
> > > https://gith/
> > > ub.com%2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in
> > > frabel.be%7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f
> > > 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJFbXB
> > > 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsI
> > > ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKfjzI7
> > > BILZDNmg%3D&reserved=0
> > >
> > > IBM has recently released and documented their modern Python binding
> > here:
> > >
> > > https://comm/
> > > unity.ibm.com%2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1
> > > 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40infrabe
> > > l.be%7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f
> > > 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1h
> > > cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIj
> > > oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9F3sB
> > > abTMg%3D&reserved=0
> > >
> > > The hook supports:
> > >
> > > * Secure connections (TLS)
> > > * Queue get/put operations
> > > * Configurable polling behavior
> > > * Transaction handling where applicable
> > >
> > > The MessageQueueProvider implementation integrates with Airflow's
> > > event-driven scheduling so that DAGs can be triggered based on IBM MQ
> > > messages.
> > >
> > > Why this might make sense:
> > >
> > > * IBM MQ is still widely used in regulated industries (banking,
> > > insurance, government).
> > > * Many enterprises using Airflow also run IBM MQ.
> > > * This would allow IBM MQ to be a first-class citizen in Airflow's
> > > event-driven ecosystem.
> > > * The dependency is officially maintained by IBM and open source.
> > >
> > > I am willing to act as initial maintainer and code owner, of course
> > > this is purely a proposition.
> > >
> > > I would appreciate feedback on:
> > >
> > > * Whether there is interest in such a provider
> > > * If yes, whether it should live under apache-airflow-providers-ibm
> > > * And if we formalize this as an AIP or draft PR
> > >
> > > Happy to share a draft implementation through a PR if there is
> interest.
> > >
> > > Thanks!
> > > David
> > >
> >
>
Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Posted by Vikram Koka via dev <de...@airflow.apache.org>.
Thanks David.
I agree with Jason, that a PR is sufficient for this, and an AIP is not
required.
Looking forward to this,
Vikram
On Tue, Mar 3, 2026 at 7:28 AM Blain David <da...@infrabel.be> wrote:
> Hello Jason,
>
> Thanks for you reply, I've created a draft PR for this provider:
>
> https://github.com/apache/airflow/pull/62790
>
> Kind regards,
> David
>
> -----Original Message-----
> From: Zhe-You(Jason) Liu <ja...@apache.org>
> Sent: 03 March 2026 10:37
> To: dev@airflow.apache.org
> Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook +
> MessageQueueProvider) to Airflow
>
> EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze
> niet vertrouwt, klik niet op een link of open geen bijlages. Bij twijfel,
> stuur deze e-mail als bijlage naar abuse@infrabel.be<mailto:
> abuse@infrabel.be>.
>
> Hi David,
>
> Thank you for your interest in adding a new MessageQueueProvider to
> Airflow!
>
> I previously opened an issue about adding more providers that support
> MessageQueueProvider in the community providers. [1] You’re more than
> welcome to contribute one!
>
> The only blocker to adding IBM MQ support to MessageQueueProvider is "the
> addition of a new IBM provider". We need to follow the adoption path
> (AIP-95), similar to the recent Informatica provider. [2]
>
> > Whether there is interest in such a provider If yes, whether it should
> > live under apache-airflow-providers-ibm And if we formalize this as an
> > AIP or draft PR
>
> So, from my perspective, the IBM Hook, Trigger, and MessageQueueProvider
> would be better placed under the IBM provider, and perhaps you could start
> by opening a draft PR and then initiating a voting thread on the dev
> mailing list.
>
> Thanks!
>
> [1] https://github.com/apache/airflow/issues/52712
> [2] https://lists.apache.org/thread/wsfgh23jm6hkrly4lx1m21ftllqshpgo
>
> Best regards,
>
> Jason
>
>
> On Tue, Mar 3, 2026 at 4:32 PM Blain David <da...@infrabel.be>
> wrote:
>
> > Hi all,
> >
> > At our company we recently implemented an IBM MQ integration for
> > Airflow and I would like to gauge interest in contributing this as a
> > new provider package.
> >
> > Motivation
> >
> > With the introduction of event-driven scheduling and the
> > MessageQueueProvider abstraction in Airflow, it has become
> > significantly easier to trigger DAGs from external message brokers (as
> > described in Astronomer's guide on event-driven scheduling):
> >
> > https://www/.
> > astronomer.io%2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05
> > %7C02%7Cdavid.blain%40infrabel.be%7C5d12469000204dfffcaa08de7908795c%7
> > Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CUnkno
> > wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXa
> > W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOnepYK
> > MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0
> >
> > Many enterprises still rely heavily on IBM MQ as their primary
> > enterprise messaging backbone. However, at the moment there is no
> > official Airflow provider supporting IBM MQ.
> >
> > Our implementation enables:
> >
> > * An IBMMQHook
> > * A MessageQueueProvider implementation for IBM MQ
> > * The ability to trigger DAGs from IBM MQ events
> > * Standard producer/consumer patterns from within tasks
> >
> > This allows IBM MQ to function similarly to Kafka, SQS, etc., within
> > the Airflow event-driven scheduling framework.
> >
> > Technical Details
> >
> > The implementation is built on top of the open-source IBM MQ Python
> > wrapper:
> >
> > * IBM MQ Python (ibmmq) library:
> > https://gith/
> > ub.com%2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in
> > frabel.be%7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f
> > 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJFbXB
> > 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsI
> > ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKfjzI7
> > BILZDNmg%3D&reserved=0
> >
> > IBM has recently released and documented their modern Python binding
> here:
> >
> > https://comm/
> > unity.ibm.com%2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1
> > 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40infrabe
> > l.be%7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f
> > 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1h
> > cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIj
> > oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9F3sB
> > abTMg%3D&reserved=0
> >
> > The hook supports:
> >
> > * Secure connections (TLS)
> > * Queue get/put operations
> > * Configurable polling behavior
> > * Transaction handling where applicable
> >
> > The MessageQueueProvider implementation integrates with Airflow's
> > event-driven scheduling so that DAGs can be triggered based on IBM MQ
> > messages.
> >
> > Why this might make sense:
> >
> > * IBM MQ is still widely used in regulated industries (banking,
> > insurance, government).
> > * Many enterprises using Airflow also run IBM MQ.
> > * This would allow IBM MQ to be a first-class citizen in Airflow's
> > event-driven ecosystem.
> > * The dependency is officially maintained by IBM and open source.
> >
> > I am willing to act as initial maintainer and code owner, of course
> > this is purely a proposition.
> >
> > I would appreciate feedback on:
> >
> > * Whether there is interest in such a provider
> > * If yes, whether it should live under apache-airflow-providers-ibm
> > * And if we formalize this as an AIP or draft PR
> >
> > Happy to share a draft implementation through a PR if there is interest.
> >
> > Thanks!
> > David
> >
>
RE: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Posted by Blain David <da...@infrabel.be>.
Hello Jason,
Thanks for you reply, I've created a draft PR for this provider:
https://github.com/apache/airflow/pull/62790
Kind regards,
David
-----Original Message-----
From: Zhe-You(Jason) Liu <ja...@apache.org>
Sent: 03 March 2026 10:37
To: dev@airflow.apache.org
Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze niet vertrouwt, klik niet op een link of open geen bijlages. Bij twijfel, stuur deze e-mail als bijlage naar abuse@infrabel.be<ma...@infrabel.be>.
Hi David,
Thank you for your interest in adding a new MessageQueueProvider to Airflow!
I previously opened an issue about adding more providers that support MessageQueueProvider in the community providers. [1] You’re more than welcome to contribute one!
The only blocker to adding IBM MQ support to MessageQueueProvider is "the addition of a new IBM provider". We need to follow the adoption path (AIP-95), similar to the recent Informatica provider. [2]
> Whether there is interest in such a provider If yes, whether it should
> live under apache-airflow-providers-ibm And if we formalize this as an
> AIP or draft PR
So, from my perspective, the IBM Hook, Trigger, and MessageQueueProvider would be better placed under the IBM provider, and perhaps you could start by opening a draft PR and then initiating a voting thread on the dev mailing list.
Thanks!
[1] https://github.com/apache/airflow/issues/52712
[2] https://lists.apache.org/thread/wsfgh23jm6hkrly4lx1m21ftllqshpgo
Best regards,
Jason
On Tue, Mar 3, 2026 at 4:32 PM Blain David <da...@infrabel.be> wrote:
> Hi all,
>
> At our company we recently implemented an IBM MQ integration for
> Airflow and I would like to gauge interest in contributing this as a
> new provider package.
>
> Motivation
>
> With the introduction of event-driven scheduling and the
> MessageQueueProvider abstraction in Airflow, it has become
> significantly easier to trigger DAGs from external message brokers (as
> described in Astronomer's guide on event-driven scheduling):
>
> https://www/.
> astronomer.io%2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05
> %7C02%7Cdavid.blain%40infrabel.be%7C5d12469000204dfffcaa08de7908795c%7
> Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CUnkno
> wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXa
> W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOnepYK
> MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0
>
> Many enterprises still rely heavily on IBM MQ as their primary
> enterprise messaging backbone. However, at the moment there is no
> official Airflow provider supporting IBM MQ.
>
> Our implementation enables:
>
> * An IBMMQHook
> * A MessageQueueProvider implementation for IBM MQ
> * The ability to trigger DAGs from IBM MQ events
> * Standard producer/consumer patterns from within tasks
>
> This allows IBM MQ to function similarly to Kafka, SQS, etc., within
> the Airflow event-driven scheduling framework.
>
> Technical Details
>
> The implementation is built on top of the open-source IBM MQ Python
> wrapper:
>
> * IBM MQ Python (ibmmq) library:
> https://gith/
> ub.com%2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in
> frabel.be%7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f
> 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJFbXB
> 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsI
> ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKfjzI7
> BILZDNmg%3D&reserved=0
>
> IBM has recently released and documented their modern Python binding here:
>
> https://comm/
> unity.ibm.com%2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1
> 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40infrabe
> l.be%7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f
> 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1h
> cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIj
> oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9F3sB
> abTMg%3D&reserved=0
>
> The hook supports:
>
> * Secure connections (TLS)
> * Queue get/put operations
> * Configurable polling behavior
> * Transaction handling where applicable
>
> The MessageQueueProvider implementation integrates with Airflow's
> event-driven scheduling so that DAGs can be triggered based on IBM MQ
> messages.
>
> Why this might make sense:
>
> * IBM MQ is still widely used in regulated industries (banking,
> insurance, government).
> * Many enterprises using Airflow also run IBM MQ.
> * This would allow IBM MQ to be a first-class citizen in Airflow's
> event-driven ecosystem.
> * The dependency is officially maintained by IBM and open source.
>
> I am willing to act as initial maintainer and code owner, of course
> this is purely a proposition.
>
> I would appreciate feedback on:
>
> * Whether there is interest in such a provider
> * If yes, whether it should live under apache-airflow-providers-ibm
> * And if we formalize this as an AIP or draft PR
>
> Happy to share a draft implementation through a PR if there is interest.
>
> Thanks!
> David
>
Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + MessageQueueProvider) to Airflow
Posted by "Zhe-You(Jason) Liu" <ja...@apache.org>.
Hi David,
Thank you for your interest in adding a new MessageQueueProvider to Airflow!
I previously opened an issue about adding more providers that support
MessageQueueProvider in the community providers. [1] You’re more than
welcome to contribute one!
The only blocker to adding IBM MQ support to MessageQueueProvider is "the
addition of a new IBM provider". We need to follow the adoption path
(AIP-95), similar to the recent Informatica provider. [2]
> Whether there is interest in such a provider
> If yes, whether it should live under apache-airflow-providers-ibm
> And if we formalize this as an AIP or draft PR
So, from my perspective, the IBM Hook, Trigger, and MessageQueueProvider
would be better placed under the IBM provider, and perhaps you could start
by opening a draft PR and then initiating a voting thread on the dev
mailing list.
Thanks!
[1] https://github.com/apache/airflow/issues/52712
[2] https://lists.apache.org/thread/wsfgh23jm6hkrly4lx1m21ftllqshpgo
Best regards,
Jason
On Tue, Mar 3, 2026 at 4:32 PM Blain David <da...@infrabel.be> wrote:
> Hi all,
>
> At our company we recently implemented an IBM MQ integration for Airflow
> and I would like to gauge interest in contributing this as a new provider
> package.
>
> Motivation
>
> With the introduction of event-driven scheduling and the
> MessageQueueProvider abstraction in Airflow, it has become significantly
> easier to trigger DAGs from external message brokers (as described in
> Astronomer's guide on event-driven scheduling):
>
> https://www.astronomer.io/docs/learn/airflow-event-driven-scheduling
>
> Many enterprises still rely heavily on IBM MQ as their primary enterprise
> messaging backbone. However, at the moment there is no official Airflow
> provider supporting IBM MQ.
>
> Our implementation enables:
>
> * An IBMMQHook
> * A MessageQueueProvider implementation for IBM MQ
> * The ability to trigger DAGs from IBM MQ events
> * Standard producer/consumer patterns from within tasks
>
> This allows IBM MQ to function similarly to Kafka, SQS, etc., within the
> Airflow event-driven scheduling framework.
>
> Technical Details
>
> The implementation is built on top of the open-source IBM MQ Python
> wrapper:
>
> * IBM MQ Python (ibmmq) library:
> https://github.com/ibm-messaging/mq-mqi-python
>
> IBM has recently released and documented their modern Python binding here:
>
> https://community.ibm.com/community/user/blogs/dylan-goode/2025/10/16/new-python-binding-for-ibm-mq
>
> The hook supports:
>
> * Secure connections (TLS)
> * Queue get/put operations
> * Configurable polling behavior
> * Transaction handling where applicable
>
> The MessageQueueProvider implementation integrates with Airflow's
> event-driven scheduling so that DAGs can be triggered based on IBM MQ
> messages.
>
> Why this might make sense:
>
> * IBM MQ is still widely used in regulated industries (banking,
> insurance, government).
> * Many enterprises using Airflow also run IBM MQ.
> * This would allow IBM MQ to be a first-class citizen in Airflow's
> event-driven ecosystem.
> * The dependency is officially maintained by IBM and open source.
>
> I am willing to act as initial maintainer and code owner, of course this
> is purely a proposition.
>
> I would appreciate feedback on:
>
> * Whether there is interest in such a provider
> * If yes, whether it should live under apache-airflow-providers-ibm
> * And if we formalize this as an AIP or draft PR
>
> Happy to share a draft implementation through a PR if there is interest.
>
> Thanks!
> David
>