You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/02 02:02:41 UTC
[GitHub] [airflow] KevinJiao opened a new issue #21923: Programmatic customization of run_id for scheduled DagRuns
KevinJiao opened a new issue #21923:
URL: https://github.com/apache/airflow/issues/21923
### Description
Allow DAG authors to control how `run_id`'s are generated for created DagRuns. Currently the only way to specify a DagRun's `run_id` is through the manual trigger workflow either through the CLI or API and passing in `run_id`. It would be great if DAG authors are able to write a custom logic to generate `run_id`'s from scheduled `DagRunInterval`'s.
### Use case/motivation
In Airflow 1.x, the semantics of `execution_date` were burdensome enough for users that DAG authors would subclass DAG to override `create_dagrun` so that when new DagRuns were created, they were created with `run_id`'s that provided context into semantics about the DagRun. For example,
```
def create_dagrun(self, **kwargs):
kwargs['run_id'] = kwargs['execution_date'] + self.following_schedule(kwargs['execution_date']).date()
return super().create_dagrun(kwargs)
```
would result in the UI DagRun dropdown to display the weekday of when the Dag actually ran.
<img width="528" alt="image001" src="https://user-images.githubusercontent.com/9851473/156280393-e261d7fa-dfe0-41db-9887-941510f4070f.png">
After upgrading to Airflow 2.0 and with Dag serialization in the scheduler overridden methods are no longer there in the SerializedDAG, so we are back to having `scheduled__<execution_date>` values in the UI dropdown. It would be great if some functionality could be exposed either through the DAG or just in the UI to display meaningful values in the DagRun dropdown.
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hterik commented on issue #21923: Programmatic customization of run_id for scheduled DagRuns
Posted by GitBox <gi...@apache.org>.
hterik commented on issue #21923:
URL: https://github.com/apache/airflow/issues/21923#issuecomment-1056360644
Agree that navigating the UI based on dagrun-id is very confusing today. But i don't think generating custom `run_id` is necessarily the solution.
It would be better if the UI could format the id and even augment more data from the dagrun into a more meaningful description, for example reading parameters from the `conf` or reading output produced by selected `xcom` parameters. For most of our use cases the execution date itself means nothing, it is mostly the input provided via parameters that are used to identify a run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #21923: Programmatic customization of run_id for scheduled DagRuns
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #21923:
URL: https://github.com/apache/airflow/issues/21923#issuecomment-1056057490
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org