You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Igor Kholopov via dev <de...@airflow.apache.org> on 2026/05/10 12:25:57 UTC
[DISCUSS] New AIP-XX: AIP-85-1: DAG importer
---------- Forwarded message ---------
From: Dilnaz Amanzholova <ad...@google.com>
Date: Fri, May 8, 2026 at 4:59 PM
Subject: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
Dear Airflow Community,
I would like to start a discussion regarding AIP-85-1: DAG importer.
To clarify, this is not a new proposal but a scoped-down, modular part of
the existing AIP-85. Original AIP-85 was created in 2024, and since then,
Airflow 3 has developed new integral functionalities (like already
implemented AIP-66 and AIP-72). We decided to divide the parent AIP into
two independent parts to better manage complexity. While AIP-85 covers the
overarching architectural framework for parsing controls, this specific
part (AIP-85-1) focuses strictly on implementing the DAG importer and its
configuration.
The goal is to allow alternative formats like YAML to be first-class
citizens in Airflow by decoupling DAG discovery from parsing. This modular
approach will allow us to implement the core importer mechanism faster and
independently of broader architectural updates.
NOTE: There is an ongoing discussion around AIP-108, which is proposing
coordinator for DAG parsing and they are slightly overlapping. I think they
need to be aligned.
Please see the attached discussion document for your reference: link
<https://docs.google.com/document/d/1K6-4cGoZItXGQHZjOydNbc7rGtOp_XKfFurMnFptKe0/edit?usp=sharing>
I look forward to your feedback.
Kind regards,
Dilnaz Amanzholova
Re: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
Posted by Jens Scheffler <js...@apache.org>.
Hi,
thanks for raising the dicussion. Can I please encourage to place the
AIP document into cwiki/Confluence which is the official place to
discuss AIP documents?
Technically yes can also be hosted as a Google doc but we can not assume
verybody has a Google account to view and comment.
Jens
On 10.05.26 14:25, Igor Kholopov via dev wrote:
> ---------- Forwarded message ---------
> From: Dilnaz Amanzholova <ad...@google.com>
> Date: Fri, May 8, 2026 at 4:59 PM
> Subject: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
>
>
> Dear Airflow Community,
>
> I would like to start a discussion regarding AIP-85-1: DAG importer.
>
> To clarify, this is not a new proposal but a scoped-down, modular part of
> the existing AIP-85. Original AIP-85 was created in 2024, and since then,
> Airflow 3 has developed new integral functionalities (like already
> implemented AIP-66 and AIP-72). We decided to divide the parent AIP into
> two independent parts to better manage complexity. While AIP-85 covers the
> overarching architectural framework for parsing controls, this specific
> part (AIP-85-1) focuses strictly on implementing the DAG importer and its
> configuration.
>
> The goal is to allow alternative formats like YAML to be first-class
> citizens in Airflow by decoupling DAG discovery from parsing. This modular
> approach will allow us to implement the core importer mechanism faster and
> independently of broader architectural updates.
>
> NOTE: There is an ongoing discussion around AIP-108, which is proposing
> coordinator for DAG parsing and they are slightly overlapping. I think they
> need to be aligned.
>
> Please see the attached discussion document for your reference: link
> <https://docs.google.com/document/d/1K6-4cGoZItXGQHZjOydNbc7rGtOp_XKfFurMnFptKe0/edit?usp=sharing>
>
> I look forward to your feedback.
>
> Kind regards,
> Dilnaz Amanzholova
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org
Re: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
Posted by Dilnaz Amanzholova via dev <de...@airflow.apache.org>.
Hi all,
As discussed during the dev call, I updated the "Out of scope" section of
AIP-85. Please take a look and let me know if I missed anything
Thanks,
Dilnaz
On Wed, Aug 12, 2026 at 5:28 PM Dilnaz Amanzholova <ad...@google.com>
wrote:
> Hi all,
>
> As there have been no comments on AIP-85 for some time, I would like to
> actively ask for any feedback in this conversation and drive the proposal
> to a conclusion.Please share any questions (or LGTMs are also welcome) so
> we can move this forward to the voting phase.
>
> Thank you,
> Dilnaz
>
>
> On Wed, Jul 29, 2026 at 11:32 AM Dilnaz Amanzholova <ad...@google.com>
> wrote:
>
>> Dear Airflow Community,
>>
>> This is the main discussion thread for AIP-85: DAG Importer.
>>
>> I would like to actively facilitate this conversation and drive the AIP
>> to a conclusion, as this process has been ongoing for some time. I highly
>> appreciate any feedback, especially since there may still be some
>> uncertainty regarding the final state of this proposed change.
>>
>> Please use this thread to bring up any unclear points or questions. I am
>> open to all discussions and look forward to your input.
>>
>> Kind regards,
>> Dilnaz
>>
>>
>> On Mon, Jun 1, 2026 at 10:16 AM Dilnaz Amanzholova <ad...@google.com>
>> wrote:
>>
>>> Dear Airflow Community reviewers,
>>>
>>> I would like to facilitate a discussion about AIP-85. If there are no
>>> objections, I will move it to the Vote phase by the end of this week.
>>>
>>> Kind regards,
>>> Dilnaz Amanzholova
>>>
>>> On Tue, May 26, 2026 at 11:06 AM Dilnaz Amanzholova <ad...@google.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> AIP-85 <https://cwiki.apache.org/confluence/x/_Q7OEg> was modified to
>>>> include only the *DAG importer* context from the design doc, and
>>>> AIP-110 <https://cwiki.apache.org/confluence/x/boCnGQ> was created for
>>>> the interactive DAG processor(original AIP-85: Track 2).
>>>>
>>>> On Sun, May 10, 2026 at 2:26 PM Igor Kholopov <ik...@google.com>
>>>> wrote:
>>>>
>>>>> ---------- Forwarded message ---------
>>>>> From: Dilnaz Amanzholova <ad...@google.com>
>>>>> Date: Fri, May 8, 2026 at 4:59 PM
>>>>> Subject: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
>>>>>
>>>>>
>>>>> Dear Airflow Community,
>>>>>
>>>>> I would like to start a discussion regarding AIP-85-1: DAG importer.
>>>>>
>>>>> To clarify, this is not a new proposal but a scoped-down, modular part
>>>>> of the existing AIP-85. Original AIP-85 was created in 2024, and since
>>>>> then, Airflow 3 has developed new integral functionalities (like already
>>>>> implemented AIP-66 and AIP-72). We decided to divide the parent AIP into
>>>>> two independent parts to better manage complexity. While AIP-85 covers the
>>>>> overarching architectural framework for parsing controls, this specific
>>>>> part (AIP-85-1) focuses strictly on implementing the DAG importer and its
>>>>> configuration.
>>>>>
>>>>> The goal is to allow alternative formats like YAML to be first-class
>>>>> citizens in Airflow by decoupling DAG discovery from parsing. This modular
>>>>> approach will allow us to implement the core importer mechanism faster and
>>>>> independently of broader architectural updates.
>>>>>
>>>>> NOTE: There is an ongoing discussion around AIP-108, which
>>>>> is proposing coordinator for DAG parsing and they are slightly overlapping.
>>>>> I think they need to be aligned.
>>>>>
>>>>> Please see the attached discussion document for your reference: link
>>>>> <https://docs.google.com/document/d/1K6-4cGoZItXGQHZjOydNbc7rGtOp_XKfFurMnFptKe0/edit?usp=sharing>
>>>>>
>>>>> I look forward to your feedback.
>>>>>
>>>>> Kind regards,
>>>>> Dilnaz Amanzholova
>>>>>
>>>>>
Re: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
Posted by Dilnaz Amanzholova via dev <de...@airflow.apache.org>.
Hi all,
As there have been no comments on AIP-85 for some time, I would like to
actively ask for any feedback in this conversation and drive the proposal
to a conclusion.Please share any questions (or LGTMs are also welcome) so
we can move this forward to the voting phase.
Thank you,
Dilnaz
On Wed, Jul 29, 2026 at 11:32 AM Dilnaz Amanzholova <ad...@google.com>
wrote:
> Dear Airflow Community,
>
> This is the main discussion thread for AIP-85: DAG Importer.
>
> I would like to actively facilitate this conversation and drive the AIP to
> a conclusion, as this process has been ongoing for some time. I highly
> appreciate any feedback, especially since there may still be some
> uncertainty regarding the final state of this proposed change.
>
> Please use this thread to bring up any unclear points or questions. I am
> open to all discussions and look forward to your input.
>
> Kind regards,
> Dilnaz
>
>
> On Mon, Jun 1, 2026 at 10:16 AM Dilnaz Amanzholova <ad...@google.com>
> wrote:
>
>> Dear Airflow Community reviewers,
>>
>> I would like to facilitate a discussion about AIP-85. If there are no
>> objections, I will move it to the Vote phase by the end of this week.
>>
>> Kind regards,
>> Dilnaz Amanzholova
>>
>> On Tue, May 26, 2026 at 11:06 AM Dilnaz Amanzholova <ad...@google.com>
>> wrote:
>>
>>> Hi,
>>>
>>> AIP-85 <https://cwiki.apache.org/confluence/x/_Q7OEg> was modified to
>>> include only the *DAG importer* context from the design doc, and AIP-110
>>> <https://cwiki.apache.org/confluence/x/boCnGQ> was created for the
>>> interactive DAG processor(original AIP-85: Track 2).
>>>
>>> On Sun, May 10, 2026 at 2:26 PM Igor Kholopov <ik...@google.com>
>>> wrote:
>>>
>>>> ---------- Forwarded message ---------
>>>> From: Dilnaz Amanzholova <ad...@google.com>
>>>> Date: Fri, May 8, 2026 at 4:59 PM
>>>> Subject: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
>>>>
>>>>
>>>> Dear Airflow Community,
>>>>
>>>> I would like to start a discussion regarding AIP-85-1: DAG importer.
>>>>
>>>> To clarify, this is not a new proposal but a scoped-down, modular part
>>>> of the existing AIP-85. Original AIP-85 was created in 2024, and since
>>>> then, Airflow 3 has developed new integral functionalities (like already
>>>> implemented AIP-66 and AIP-72). We decided to divide the parent AIP into
>>>> two independent parts to better manage complexity. While AIP-85 covers the
>>>> overarching architectural framework for parsing controls, this specific
>>>> part (AIP-85-1) focuses strictly on implementing the DAG importer and its
>>>> configuration.
>>>>
>>>> The goal is to allow alternative formats like YAML to be first-class
>>>> citizens in Airflow by decoupling DAG discovery from parsing. This modular
>>>> approach will allow us to implement the core importer mechanism faster and
>>>> independently of broader architectural updates.
>>>>
>>>> NOTE: There is an ongoing discussion around AIP-108, which is proposing
>>>> coordinator for DAG parsing and they are slightly overlapping. I think they
>>>> need to be aligned.
>>>>
>>>> Please see the attached discussion document for your reference: link
>>>> <https://docs.google.com/document/d/1K6-4cGoZItXGQHZjOydNbc7rGtOp_XKfFurMnFptKe0/edit?usp=sharing>
>>>>
>>>> I look forward to your feedback.
>>>>
>>>> Kind regards,
>>>> Dilnaz Amanzholova
>>>>
>>>>
Re: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
Posted by Dilnaz Amanzholova via dev <de...@airflow.apache.org>.
Dear Airflow Community,
This is the main discussion thread for AIP-85: DAG Importer.
I would like to actively facilitate this conversation and drive the AIP to
a conclusion, as this process has been ongoing for some time. I highly
appreciate any feedback, especially since there may still be some
uncertainty regarding the final state of this proposed change.
Please use this thread to bring up any unclear points or questions. I am
open to all discussions and look forward to your input.
Kind regards,
Dilnaz
On Mon, Jun 1, 2026 at 10:16 AM Dilnaz Amanzholova <ad...@google.com>
wrote:
> Dear Airflow Community reviewers,
>
> I would like to facilitate a discussion about AIP-85. If there are no
> objections, I will move it to the Vote phase by the end of this week.
>
> Kind regards,
> Dilnaz Amanzholova
>
> On Tue, May 26, 2026 at 11:06 AM Dilnaz Amanzholova <ad...@google.com>
> wrote:
>
>> Hi,
>>
>> AIP-85 <https://cwiki.apache.org/confluence/x/_Q7OEg> was modified to
>> include only the *DAG importer* context from the design doc, and AIP-110
>> <https://cwiki.apache.org/confluence/x/boCnGQ> was created for the
>> interactive DAG processor(original AIP-85: Track 2).
>>
>> On Sun, May 10, 2026 at 2:26 PM Igor Kholopov <ik...@google.com>
>> wrote:
>>
>>> ---------- Forwarded message ---------
>>> From: Dilnaz Amanzholova <ad...@google.com>
>>> Date: Fri, May 8, 2026 at 4:59 PM
>>> Subject: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
>>>
>>>
>>> Dear Airflow Community,
>>>
>>> I would like to start a discussion regarding AIP-85-1: DAG importer.
>>>
>>> To clarify, this is not a new proposal but a scoped-down, modular part
>>> of the existing AIP-85. Original AIP-85 was created in 2024, and since
>>> then, Airflow 3 has developed new integral functionalities (like already
>>> implemented AIP-66 and AIP-72). We decided to divide the parent AIP into
>>> two independent parts to better manage complexity. While AIP-85 covers the
>>> overarching architectural framework for parsing controls, this specific
>>> part (AIP-85-1) focuses strictly on implementing the DAG importer and its
>>> configuration.
>>>
>>> The goal is to allow alternative formats like YAML to be first-class
>>> citizens in Airflow by decoupling DAG discovery from parsing. This modular
>>> approach will allow us to implement the core importer mechanism faster and
>>> independently of broader architectural updates.
>>>
>>> NOTE: There is an ongoing discussion around AIP-108, which is proposing
>>> coordinator for DAG parsing and they are slightly overlapping. I think they
>>> need to be aligned.
>>>
>>> Please see the attached discussion document for your reference: link
>>> <https://docs.google.com/document/d/1K6-4cGoZItXGQHZjOydNbc7rGtOp_XKfFurMnFptKe0/edit?usp=sharing>
>>>
>>> I look forward to your feedback.
>>>
>>> Kind regards,
>>> Dilnaz Amanzholova
>>>
>>>
Re: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
Posted by Dilnaz Amanzholova via dev <de...@airflow.apache.org>.
Dear Airflow Community reviewers,
I would like to facilitate a discussion about AIP-85. If there are no
objections, I will move it to the Vote phase by the end of this week.
Kind regards,
Dilnaz Amanzholova
On Tue, May 26, 2026 at 11:06 AM Dilnaz Amanzholova <ad...@google.com>
wrote:
> Hi,
>
> AIP-85 <https://cwiki.apache.org/confluence/x/_Q7OEg> was modified to
> include only the *DAG importer* context from the design doc, and AIP-110
> <https://cwiki.apache.org/confluence/x/boCnGQ> was created for the
> interactive DAG processor(original AIP-85: Track 2).
>
> On Sun, May 10, 2026 at 2:26 PM Igor Kholopov <ik...@google.com>
> wrote:
>
>> ---------- Forwarded message ---------
>> From: Dilnaz Amanzholova <ad...@google.com>
>> Date: Fri, May 8, 2026 at 4:59 PM
>> Subject: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
>>
>>
>> Dear Airflow Community,
>>
>> I would like to start a discussion regarding AIP-85-1: DAG importer.
>>
>> To clarify, this is not a new proposal but a scoped-down, modular part of
>> the existing AIP-85. Original AIP-85 was created in 2024, and since then,
>> Airflow 3 has developed new integral functionalities (like already
>> implemented AIP-66 and AIP-72). We decided to divide the parent AIP into
>> two independent parts to better manage complexity. While AIP-85 covers the
>> overarching architectural framework for parsing controls, this specific
>> part (AIP-85-1) focuses strictly on implementing the DAG importer and its
>> configuration.
>>
>> The goal is to allow alternative formats like YAML to be first-class
>> citizens in Airflow by decoupling DAG discovery from parsing. This modular
>> approach will allow us to implement the core importer mechanism faster and
>> independently of broader architectural updates.
>>
>> NOTE: There is an ongoing discussion around AIP-108, which is proposing
>> coordinator for DAG parsing and they are slightly overlapping. I think they
>> need to be aligned.
>>
>> Please see the attached discussion document for your reference: link
>> <https://docs.google.com/document/d/1K6-4cGoZItXGQHZjOydNbc7rGtOp_XKfFurMnFptKe0/edit?usp=sharing>
>>
>> I look forward to your feedback.
>>
>> Kind regards,
>> Dilnaz Amanzholova
>>
>>
Re: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
Posted by Dilnaz Amanzholova via dev <de...@airflow.apache.org>.
Hi,
AIP-85 <https://cwiki.apache.org/confluence/x/_Q7OEg> was modified to
include only the *DAG importer* context from the design doc, and AIP-110
<https://cwiki.apache.org/confluence/x/boCnGQ> was created for the
interactive DAG processor(original AIP-85: Track 2).
On Sun, May 10, 2026 at 2:26 PM Igor Kholopov <ik...@google.com> wrote:
> ---------- Forwarded message ---------
> From: Dilnaz Amanzholova <ad...@google.com>
> Date: Fri, May 8, 2026 at 4:59 PM
> Subject: [DISCUSS] New AIP-XX: AIP-85-1: DAG importer
>
>
> Dear Airflow Community,
>
> I would like to start a discussion regarding AIP-85-1: DAG importer.
>
> To clarify, this is not a new proposal but a scoped-down, modular part of
> the existing AIP-85. Original AIP-85 was created in 2024, and since then,
> Airflow 3 has developed new integral functionalities (like already
> implemented AIP-66 and AIP-72). We decided to divide the parent AIP into
> two independent parts to better manage complexity. While AIP-85 covers the
> overarching architectural framework for parsing controls, this specific
> part (AIP-85-1) focuses strictly on implementing the DAG importer and its
> configuration.
>
> The goal is to allow alternative formats like YAML to be first-class
> citizens in Airflow by decoupling DAG discovery from parsing. This modular
> approach will allow us to implement the core importer mechanism faster and
> independently of broader architectural updates.
>
> NOTE: There is an ongoing discussion around AIP-108, which is proposing
> coordinator for DAG parsing and they are slightly overlapping. I think they
> need to be aligned.
>
> Please see the attached discussion document for your reference: link
> <https://docs.google.com/document/d/1K6-4cGoZItXGQHZjOydNbc7rGtOp_XKfFurMnFptKe0/edit?usp=sharing>
>
> I look forward to your feedback.
>
> Kind regards,
> Dilnaz Amanzholova
>
>