You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/10/13 11:35:05 UTC
[GitHub] [airflow] zbstof opened a new pull request #18938: Jinja templates should be rendered in dict keys
zbstof opened a new pull request #18938:
URL: https://github.com/apache/airflow/pull/18938
Currently keys stay unrendered which is confusing for the users and require workaround like explicitly using `Variable.get`, which is evaluated eagerly during dag rendering, leading to further need for workarounds.
Tested by running tasks locally.
I could add unit tests, if necessary, but please advise the best place for doing so.
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zbstof edited a comment on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
zbstof edited a comment on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-998098248
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-943435358
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zbstof commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
zbstof commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-947716186
I've fixed static check, should I rebase to fix broken tests in `Quarantined tests`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] BasPH commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
BasPH commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-942275322
It wasn't me :) This was added a long time ago -> https://github.com/apache/airflow/pull/4292/files
Technically I don't see anything wrong with also templating the keys 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-942218073
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zbstof commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
zbstof commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-942236636
Turns out there's already a test that this doesn't work added by @BasPH , but I think it was codifying existing behaviour, not enforcing a contract.
I fixed that test
Please clarify if you want me to add more test cases?
> maybe there was a reason it was implemented like that ?
Given how dicts are just fancy list of 2-tuples, and we already handle explicit lists and tuples, I don't see why we shouldn't handle dictionary keys
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] houqp commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
houqp commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-1086403357
@zbstof , there is a formatting issue that needs to be fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-942218073
I wonder if it can have some unintended side effects, I cannot think of any, but maybe there was a reason it was implemented like that ?
In any case @zbstof - you will need to add test cases covering that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zbstof commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
zbstof commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-1037989301
Ping
нд, 13 лют. 2022, 02:08 користувач github-actions[bot] <
***@***.***> пише:
> This pull request has been automatically marked as stale because it has
> not had recent activity. It will be closed in 5 days if no further activity
> occurs. Thank you for your contributions.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/airflow/pull/18938#issuecomment-1037598181>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKPUA5YQLUDFE7A4BFSBQLU23ZBLANCNFSM5F465FCQ>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
> You are receiving this because you modified the open/close state.Message
> ID: ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-1037598181
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-942241999
Wel. The fact that there was a test for that indicates that there **could** be a reason. I cannot think of any as well, but It's better to check it. This is quite crucial part of the code, so I would not like to spoil something.
@BasPH, @kaxil - I'd love to hear your thoughts here. Maybe there is a good reason for not rendering the keys?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zbstof commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
zbstof commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-942236636
Turns out there's already a test that this doesn't work added by @BasPH , but I think it was codifying existing behaviour, not enforcing a contract.
I fixed that test
Please clarify if you want me to add more test cases?
> maybe there was a reason it was implemented like that ?
Given how dicts are just fancy list of 2-tuples, and we already handle explicit lists and tuples, I don't see why we shouldn't handle dictionary keys
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-943435474
Works for me then - some static checks need to be fixed though (and I think we should hold-off with merging that until we release 2.2.1 as there are potential conflicts I think).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-1037598181
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-942349084
Static checks are failing - the other failing tests is irrelevant - flaky test that will be fixed by #18947
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zbstof edited a comment on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
zbstof edited a comment on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-942236636
Turns out there's already a test that dict keys are not rendered, added by @BasPH , but I think it was codifying existing behaviour, not enforcing a contract.
I fixed that test
Please clarify if you want me to add more test cases?
> maybe there was a reason it was implemented like that ?
Given how dicts are just fancy list of 2-tuples, and we already handle explicit lists and tuples, I don't see why we shouldn't handle dictionary keys
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zbstof commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
zbstof commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-1037989301
Ping
нд, 13 лют. 2022, 02:08 користувач github-actions[bot] <
***@***.***> пише:
> This pull request has been automatically marked as stale because it has
> not had recent activity. It will be closed in 5 days if no further activity
> occurs. Thank you for your contributions.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/airflow/pull/18938#issuecomment-1037598181>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKPUA5YQLUDFE7A4BFSBQLU23ZBLANCNFSM5F465FCQ>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
> You are receiving this because you modified the open/close state.Message
> ID: ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-1085254947
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
eladkal commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-997442039
@zbstof can you please address above comment and rebase?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
uranusjr commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-948534575
That would be best (although I don't think it'd help; Quarantined failed due to the task being killed because it ran for too long, not any concrete test failures)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zbstof commented on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
zbstof commented on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-998098248
@eladkal, @kaxil
I've accidentally closed PR, please re-open
I've also added an entry in `UPDATING.md`, please check if it's good enough
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zbstof closed pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
zbstof closed pull request #18938:
URL: https://github.com/apache/airflow/pull/18938
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zbstof edited a comment on pull request #18938: Jinja templates should be rendered in dict keys
Posted by GitBox <gi...@apache.org>.
zbstof edited a comment on pull request #18938:
URL: https://github.com/apache/airflow/pull/18938#issuecomment-942236636
Turns out there's already a test that dict keys are not rendered, added by @BasPH , but I think it was codifying existing behaviour, not enforcing a contract.
I fixed that test
Please clarify if you want me to add more test cases?
> maybe there was a reason it was implemented like that ?
Given how dicts are just fancy list of 2-tuples, and we already handle explicit lists and tuples, I don't see why we shouldn't handle dictionary keys
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org