[GitHub] [airflow] mobuchowski commented on issue #17984: Add possibility to LineageBackend to be notified of task instance execution start and failure

[GitBox <gi...@apache.org>]

mobuchowski commented on issue #17984:
URL: https://github.com/apache/airflow/issues/17984#issuecomment-966505971


   I think that having something global rather than per operator makes more sense in our case.
   Conceptually, it's easier to inject - Airflow user specifies in config that they want to use `OpenLineagePlugin`.
   The plugin registers what events it wants to listen for.
   Airflow notifies plugin when one of those events fire.
   
   Looks like the `pluggy` approach works like that - just with different implementation, calls instead of events.
   
   The `on_state_change` approach might work better when you want to specify different behavior for different operators - or just as a hook interface for pluggy?
   
   >What do you want in context?
   
   Besides particular task and task instance, we use `DagRun` for stuff like execution date and id, and `Dag` for getting id and description. Generally the more metadata we can have about it, the better.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org