You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/05/15 13:55:12 UTC
[GitHub] [airflow] realradical opened a new pull request #8875: [AIRFLOW-8842] fix Dag Run UI execution date with timezone cannot be saved issue
realradical opened a new pull request #8875:
URL: https://github.com/apache/airflow/pull/8875
---
Make sure to mark the boxes below before creating PR: [x]
- [ ] Description above provides context of the change
- [ ] Unit tests coverage for changes (not needed for documentation changes)
- [ ] Target Github ISSUE in description if exists
- [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [ ] Relevant documentation is updated including usage instructions.
- [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#discussion_r426065144
##########
File path: tests/www/test_views.py
##########
@@ -2601,14 +2601,14 @@ def test_create_dagrun(self):
dr = self.session.query(models.DagRun).one()
- self.assertEqual(dr.execution_date, timezone.convert_to_utc(datetime(2018, 7, 6, 5, 4, 3)))
+ self.assertEqual(dr.execution_date, dt.fromisoformat('2018-07-06 05:04:03-02:00'))
def test_create_dagrun_valid_conf(self):
conf_value = dict(Valid=True)
data = {
"state": "running",
"dag_id": "example_bash_operator",
- "execution_date": "2018-07-06 05:05:03",
+ "execution_date": "2018-07-06 05:05:03-02:00",
Review comment:
Is it still possible to create it without specifying a timezone too?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] realradical commented on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
realradical commented on pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#issuecomment-629928724
@ashb the quarantined CI build job fails.. any idea?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#discussion_r426477404
##########
File path: tests/www/test_views.py
##########
@@ -2601,14 +2601,14 @@ def test_create_dagrun(self):
dr = self.session.query(models.DagRun).one()
- self.assertEqual(dr.execution_date, timezone.convert_to_utc(datetime(2018, 7, 6, 5, 4, 3)))
+ self.assertEqual(dr.execution_date, dt.fromisoformat('2018-07-06 05:04:03-02:00'))
def test_create_dagrun_valid_conf(self):
conf_value = dict(Valid=True)
data = {
"state": "running",
"dag_id": "example_bash_operator",
- "execution_date": "2018-07-06 05:05:03",
+ "execution_date": "2018-07-06 05:05:03-02:00",
Review comment:
I guess the html form would allow it, but the JS that populates/updates that field doesn't allow it.
This is probably fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] realradical commented on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
realradical commented on pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#issuecomment-630209425
@ashb i accidentally closed the pr. will create another one now. sorry about it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#discussion_r426064945
##########
File path: airflow/www/forms.py
##########
@@ -82,7 +82,8 @@ class DagRunForm(DynamicForm):
widget=Select2Widget())
execution_date = DateTimeField(
lazy_gettext('Execution Date'),
- widget=AirflowDateTimePickerWidget())
+ widget=AirflowDateTimePickerWidget(),
+ format='%Y-%m-%d %H:%M:%S%z')
Review comment:
It may be more work, but the ideal would be to use Pendulum to parse these dates -- that's what we use almost everywhere else in Airflow `airflow.utils.timezone.parse()`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] realradical commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
realradical commented on a change in pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#discussion_r426478960
##########
File path: tests/www/test_views.py
##########
@@ -2601,14 +2601,14 @@ def test_create_dagrun(self):
dr = self.session.query(models.DagRun).one()
- self.assertEqual(dr.execution_date, timezone.convert_to_utc(datetime(2018, 7, 6, 5, 4, 3)))
+ self.assertEqual(dr.execution_date, dt.fromisoformat('2018-07-06 05:04:03-02:00'))
def test_create_dagrun_valid_conf(self):
conf_value = dict(Valid=True)
data = {
"state": "running",
"dag_id": "example_bash_operator",
- "execution_date": "2018-07-06 05:05:03",
+ "execution_date": "2018-07-06 05:05:03-02:00",
Review comment:
yeah i poked around on the dag run page, seems like bootstrap JS makes sure a timezone is populated which was currently causing the issue
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#issuecomment-630051088
Looking better @realradical!
Can you test that changing the timezone in the UI updates this field while the page is open? (If it doesn't it means it's missing an html class)
I also think that we should test more than one timezone, and for completeness, test what happens when the form is submitted without a TZ (it should use the "default" configured TZ. So we should also use `@conf_vars` for that test to change the default for that one test to not be UTC.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] realradical commented on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
realradical commented on pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#issuecomment-630081091
> @conf_vars
@ashb
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] realradical commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
realradical commented on a change in pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#discussion_r426286484
##########
File path: tests/www/test_views.py
##########
@@ -2601,14 +2601,14 @@ def test_create_dagrun(self):
dr = self.session.query(models.DagRun).one()
- self.assertEqual(dr.execution_date, timezone.convert_to_utc(datetime(2018, 7, 6, 5, 4, 3)))
+ self.assertEqual(dr.execution_date, dt.fromisoformat('2018-07-06 05:04:03-02:00'))
def test_create_dagrun_valid_conf(self):
conf_value = dict(Valid=True)
data = {
"state": "running",
"dag_id": "example_bash_operator",
- "execution_date": "2018-07-06 05:05:03",
+ "execution_date": "2018-07-06 05:05:03-02:00",
Review comment:
I don't think so. correct me if i am wrong
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] realradical closed pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
realradical closed pull request #8875:
URL: https://github.com/apache/airflow/pull/8875
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8875: [AIRFLOW-8842] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#issuecomment-629249520
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (flake8, pylint and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/master/docs/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for testing locally, itโs a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#coding-style-and-best-practices).
Apache Airflow is a community-driven project and together we are making it better ๐.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://apache-airflow-slack.herokuapp.com/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] realradical edited a comment on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
realradical edited a comment on pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#issuecomment-630081091
> @conf_vars
@ashb I tried changing the timezone in the UI and it works!
I used `@conf_vars({("core", "default_timezone"): "America/Toronto"})` in the test without timezone. However, the timezone setting used in `timezone.make_aware` utility function is initialized before the config overriding in my test. https://github.com/apache/airflow/blob/master/airflow/settings.py
Personally I don't think we need to cover this case, the frontend should make sure that the datetime submitted always with a timezone.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#issuecomment-630127673
@realradical I think the backend change you have here is good! With it it should accept any datetime that the UI displays, but without it it would only accept TZ in the current TZ.
i.e. the change as you had it meant you could copy-and-paste a date-time with TZ and have it work, so I'd still think this is a good change to have.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] realradical commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
realradical commented on a change in pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#discussion_r426287007
##########
File path: airflow/www/forms.py
##########
@@ -82,7 +82,8 @@ class DagRunForm(DynamicForm):
widget=Select2Widget())
execution_date = DateTimeField(
lazy_gettext('Execution Date'),
- widget=AirflowDateTimePickerWidget())
+ widget=AirflowDateTimePickerWidget(),
+ format='%Y-%m-%d %H:%M:%S%z')
Review comment:
@ashb thing is `execution_date ` is a flask field. I am not sure how we can make flask use Pendulum.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] realradical commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
realradical commented on a change in pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#discussion_r426297022
##########
File path: airflow/www/forms.py
##########
@@ -82,7 +82,8 @@ class DagRunForm(DynamicForm):
widget=Select2Widget())
execution_date = DateTimeField(
lazy_gettext('Execution Date'),
- widget=AirflowDateTimePickerWidget())
+ widget=AirflowDateTimePickerWidget(),
+ format='%Y-%m-%d %H:%M:%S%z')
Review comment:
@ashb created a custom field and use Pendulum now. Please check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #8875: [AIRFLOW-8875] fix Dag Run UI execution date with timezone cannot be saved issue
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #8875:
URL: https://github.com/apache/airflow/pull/8875#discussion_r426287475
##########
File path: airflow/www/forms.py
##########
@@ -82,7 +82,8 @@ class DagRunForm(DynamicForm):
widget=Select2Widget())
execution_date = DateTimeField(
lazy_gettext('Execution Date'),
- widget=AirflowDateTimePickerWidget())
+ widget=AirflowDateTimePickerWidget(),
+ format='%Y-%m-%d %H:%M:%S%z')
Review comment:
It _may_ be doable, but is not documented/clear. I think when I added timezones I got it working for display side (rather i got it using my custom control to decorate the fields so they get converted)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org