You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/02/04 21:10:12 UTC
[GitHub] [airflow] NilsJPWerner opened a new issue #21336: Override the dag run_id from within the ui
NilsJPWerner opened a new issue #21336:
URL: https://github.com/apache/airflow/issues/21336
### Description
It would be great to have the ability to override the generated run_ids like `scheduled__2022-01-27T14:00:00+00:00` so that it is easier to find specific dag runs in the ui. I know the rest api allows you to specify a run id, but it would be great if ui users could also specify a run_id using for example dag_run conf.
### Use case/motivation
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] talnagar commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
talnagar commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1046581337
hi I would like to be assigned this issue. I've never contributed before. what's the protocol for doing it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031879570
> In the [CONTRIBUTING.rst](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#submit-feedback) there is a section about creating feature requests, and the issue page has a dedicated button for feature requests, so I believed that posting a feature request was the way to go about these things.
I did not say it's WRONG to add feature request, I just said you do not need it if you want open a PR :) . It was not a complaint - just a statement :D .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031806711
Since the API already has it, and we already have a "trigger" screen where we could add this I've marked it as good-first-issue too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] NilsJPWerner commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
NilsJPWerner commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1060166676
That's probably true. I will create another issue for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #21336:
URL: https://github.com/apache/airflow/issues/21336
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] NilsJPWerner commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
NilsJPWerner commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1058017663
1) I would argue that a separate field is the way to go since the api treats it separately from dag run conf.
2) I would personally love that option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] talnagar commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
talnagar commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1058038841
regarding the option to add a name for scheduled tasks - I didn't see an option to manually trigger scheduled tasks from the ui.
Is there such an option? @NilsJPWerner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031816503
> I'm confused, why does this not require a feature request?
Because we work on PR base. If the PR is accepted, it is merged. This is how a governance in an open source world (and in Airflow in particular) work. There is no-one to 'accept' feature requests so it cannot be 'accepted'. It can be rejected and closed if there is is a justified -1 from a committer though.
And even if people think a feature is good and there is a consensus about it, l it might never be implemented if no-one, well, implement it.
So the most certain way to implement a feature if you want it is to ... Implement it as a PR yourself. Or pick someone's interest and convince them to implement it. Or mark it as a good first issue and hope someone will pick it as their first issue if this is an easy fix.
That's why you do not need issues - good PR is enough. But if a change is big you want to discuss it before PR to check if there is a consensus on this being good direction. And it might turn you need to create and vote on Airflow Improvement Proposal, when both change is really big.
Airflow has more than 1900 contributors BTW.
You can take a look at our CONTRIBUTING.rst for more details
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] NilsJPWerner edited a comment on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
NilsJPWerner edited a comment on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031876060
> Or pick someone's interest and convince them to implement it. Or mark it as a good first issue and hope someone will pick it as their first issue if this is an easy fix.
That was my goal when I opened the feature request. I could potentially attempt the PR myself, but I am not familiar with the code base so would probably need a bit of direction. I definitely wouldn't want to create a pr if I didn't have an indication of whether it would be acceptable to the community, so an issue seems like a good way to gauge that.
In the [CONTRIBUTING.rst](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#submit-feedback ) there is a section about creating feature requests, and the issue page has a dedicated button for feature requests, so I believed that posting a feature request was the way to go about these things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] NilsJPWerner commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
NilsJPWerner commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031664717
I'm confused, why does this not require a feature request?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] NilsJPWerner commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
NilsJPWerner commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031881493
I understand now. I got the indication from @ashb's tag that a PR for this issue would likely be accepted so I will look into what it would take to make the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] NilsJPWerner commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
NilsJPWerner commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031876060
> Or pick someone's interest and convince them to implement it. Or mark it as a good first issue and hope someone will pick it as their first issue if this is an easy fix.
That was my goal when I opened the feature request. I could potentially attempt the PR myself, but I am not familiar with the code base so would probably need a bit of direction.
In the [CONTRIBUTING.rst](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#submit-feedback ) there is a section about creating feature requests, and the issue page has a dedicated button for feature requests, so I believed that posting a feature request was the way to go about these things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] NilsJPWerner edited a comment on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
NilsJPWerner edited a comment on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1058232926
I think it would have to be setup through some argument to the dag initializer. For example:
```python
def custom_run_id_naming_function(context):
execution_date = context.get("data_interval_start")
return f"custom_name_{execution_date}"
with DAG(
"test_dag",
default_args=default_args,
schedule_interval="0 3 * * *",
run_id_namer=custom_run_id_naming_function,
) as dag:
pass
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1046650727
Assignd you . See CONTRIBUTING.rst
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031816503
> I'm confused, why does this not require a feature request?
Because we work on PR base. If the PR is accepted, it is merged. This how a governance in an open source world (and in Airflow in particular) there is no-one to 'accept' feature requests so it cannot be 'accepted'. It can be rejected and closed if there is is a justified -1 from a committer though.
And even if people think a feature is good and there is a consensus about it, l it might never be implemented if no-one, well, implement it.
So the most certain way to implement a feature if you want it is to ... Implement it as a PR yourself. Or pick someone's interest and convince them to implement it. Or mark it as a good first issue and hope someone will pick it as their first issue if this is an easy fix.
That's why you do not need issues - good PR is enough. But if a change is big you want to discuss it before PR to check if there is a consensus on this being good direction. And it might turn you need to create and vote on Airflow Improvement Proposal, when both change is really big.
Airflow has more than 1900 contributors BTW.
You can take a look at our CONTRIBUTING.rst for more details
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1030910202
Feel free to propose PR on that. It does not require a feature request. PR is more than enough.,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] talnagar commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
talnagar commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1059959960
In this case I think it should be in a separate PR since it doesn't rely on existing API capabilities like adding a manual run_id.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] NilsJPWerner edited a comment on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
NilsJPWerner edited a comment on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1058232926
I think it would have to be setup through some argument to the dag initializer. For example:
```python
def custom_run_id_naming_function(context):
execution_date = context.get("data_interval_start")
return f"custom_name_{execution_date}"
with DAG(
"test_dag",
default_args=default_args,
schedule_interval="0 3 * * *",
run_id_namer=custom_run_id_naming_function,
) as dag:
pass
```
Maybe @ashb has an opinion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] NilsJPWerner commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
NilsJPWerner commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1058232926
I think it would have to be setup through some argument to the dag initializer. For example:
```python
def custom_run_id_name(context):
execution_date = context.get("data_interval_start")
return f"custom_name_{execution_date}"
with DAG(
"test_dag",
default_args=default_args,
schedule_interval="0 3 * * *",
run_id_namer=custom_run_id_name,
) as dag:
pass
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] talnagar commented on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
talnagar commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1053361280
hi
some questions before I start implementing it:
1. Is it preferable to have a separate input for 'dag run id' or should it be part of the configuration json
2. should there be an option to set a base name for scheduled tasks and than the run id will be something like 'myrun__2022-01-27T14:00:00+00:00'?
would love any input on this.
thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] NilsJPWerner edited a comment on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
NilsJPWerner edited a comment on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1058232926
I think it would have to be setup through some argument to the dag initializer. For example:
```python
def custom_run_id_naming_function(context):
execution_date = context.get("data_interval_start")
return f"custom_name_{execution_date}"
with DAG(
"test_dag",
default_args=default_args,
schedule_interval="0 3 * * *",
run_id_namer=custom_run_id_naming_function,
) as dag:
pass
```
Maybe @uranusjr has an opinion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031816503
> I'm confused, why does this not require a feature request?
Because we work on PR base. If the PR is accepted, it is merged. This is how a governance in an open source world (and in Airflow in particular) work. There is no-one to 'accept' feature requests so it cannot be 'accepted'. It can be rejected and closed if there is is a justified -1 from a committer though.
And even if people think a feature is good and there is a consensus about it, l it might never be implemented if no-one, well, implement it.
So the most certain way to implement a feature if you want it is to ... Implement it as a PR yourself. Or pick someone's interest and convince them to implement it. Or mark it as a good first issue and hope someone will pick it as their first issue if this is an easy fix.
That's why you do not need issues - good PR is enough. But if a change is big you want to discuss it before PR to check if there is a consensus on this being good direction. And it might turn you need to create and vote on Airflow Improvement Proposal, when the change is really big.
Airflow has more than 1900 contributors BTW.
You can take a look at our CONTRIBUTING.rst for more details
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #21336: Override the dag run_id from within the ui
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #21336:
URL: https://github.com/apache/airflow/issues/21336
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org