You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "dstandish (via GitHub)" <gi...@apache.org> on 2025/11/14 01:50:04 UTC
[I] Solve for on_asset_changed re asset partitioning [airflow]
dstandish opened a new issue, #58290:
URL: https://github.com/apache/airflow/issues/58290
### Body
The hook on_asset_changed is invoked whenever there is an asset event. The problem is we pass the asset and not the event. But, this means that we have no way to know what happened to the asset. We only know it was "updated" (and not for example what partition may have been updated)
Maybe we should change the signature to use asset event instead? Or maybe we should add asset event? Or maybe we should make a new hook, `on_asset_event`
cc @Lee-W @uranusjr
### Committer
- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Solve for on_asset_changed re asset partitioning [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr closed issue #58290: Solve for on_asset_changed re asset partitioning
URL: https://github.com/apache/airflow/issues/58290
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Solve for on_asset_changed re asset partitioning [airflow]
Posted by "profgrammer (via GitHub)" <gi...@apache.org>.
profgrammer commented on issue #58290:
URL: https://github.com/apache/airflow/issues/58290#issuecomment-3531500198
Hi, I would like to work on this feature, can it be assigned to me?
IIUC we want to add the asset change event itself to the signature of on_asset_changed hook. This will require a refactor across usages of this hook in the codebase.
Before starting, I would like to understand how the event would be used inside the hook. Is it for logging? Or is there some other use for the event inside the hook?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Solve for on_asset_changed re asset partitioning [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on issue #58290:
URL: https://github.com/apache/airflow/issues/58290#issuecomment-3539641722
Or maybe we should just add a new `on_asset_event_created`? (IDK)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Solve for on_asset_changed re asset partitioning [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on issue #58290:
URL: https://github.com/apache/airflow/issues/58290#issuecomment-3544644585
@profgrammer yep, sure, but let's confirm how we want to implement before stepping in 🙂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Solve for on_asset_changed re asset partitioning [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on issue #58290:
URL: https://github.com/apache/airflow/issues/58290#issuecomment-3531445988
> Or maybe we should add asset event?
I feel this way might be better 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Solve for on_asset_changed re asset partitioning [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on issue #58290:
URL: https://github.com/apache/airflow/issues/58290#issuecomment-3551480808
> I'd like to understand the use case for the feature in parallel. How will we be using the event inside the hook?
We only pass the event, the user decide how they want to use it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Solve for on_asset_changed re asset partitioning [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on issue #58290:
URL: https://github.com/apache/airflow/issues/58290#issuecomment-3551488959
If we're to have both `on_asset_changed` and `on_asset_event_created`, we probably can repurpose `on_asset_changed` to detect an asset becomes inactive or an extra change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Solve for on_asset_changed re asset partitioning [airflow]
Posted by "profgrammer (via GitHub)" <gi...@apache.org>.
profgrammer commented on issue #58290:
URL: https://github.com/apache/airflow/issues/58290#issuecomment-3548885785
> Before starting, I would like to understand how the event would be used inside the hook. Is it for logging? Or is there some other use for the event inside the hook?
I'd like to understand the use case for the feature in parallel. How will we be using the event inside the hook?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Solve for on_asset_changed re asset partitioning [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on issue #58290:
URL: https://github.com/apache/airflow/issues/58290#issuecomment-3597140798
see @Lee-W's comment here https://github.com/apache/airflow/pull/58289#discussion_r2570609713
see also https://github.com/apache/airflow/issues/42254
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org