You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/09/01 15:29:13 UTC
[GitHub] [airflow] ashb opened a new pull request, #26109: Prepare release of amazon provider out of band
ashb opened a new pull request, #26109:
URL: https://github.com/apache/airflow/pull/26109
There is an circular import error that means 5.0.0 can't be used with
the SecretsManager - worth releaseing a fix for this now
See #26107
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r960827131
##########
airflow/providers/amazon/CHANGELOG.rst:
##########
@@ -24,6 +24,45 @@
Changelog
---------
+5.1.0
+.....
+
+Breaking changes
Review Comment:
```suggestion
Breaking changes
```
This should be removed (I planned to improve that a bit to not generate that part if needed :) )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a diff in pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961075279
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,21 @@
Changelog
---------
+1.1.1
Review Comment:
See https://github.com/apache/airflow/pull/26109/files#diff-e1c023651fed5dc86ec9d3303308052dd32afa65a80bfbc6d6c4554d9aae7c1d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961986957
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
My thought is that if someone ran in to the pandas issue there would have been at least one issue opened by it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r962004956
##########
airflow/providers/sqlite/CHANGELOG.rst:
##########
@@ -24,6 +24,26 @@
Changelog
---------
+3.2.1
+.....
+
+Breaking changes
Review Comment:
Likewise (this is why I want to automate it BTW :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1234782551
Why? If we REALLY want to split providers, we need to look at them in isolation. Each provider should be treated separately, independently from the others (this is one of the reasons we added common.sql to try various scenarios).
If you look at common.sql in isolation - adding placeholder is a new feature, that trino, presto, sqlite might use to produce a bugfix should depend on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1235336821
Also one more "hint" and suggestion - the output of "prepare-provider-documentation" is optimised to allow to make fast decisions on bugfix/feature/breaking/doc-only by the release manager. For every provider you get list of commits printed together with the URL, so if after looking at the list of commits, you are not sure what decision to make you can ctrl+click on the URL of particular commit and you will see it in the browser and will be able to asses if the change is fix/doc/feature/breaking change.
And I would heartily recommend (if you have not done so) to run this doc generation for all providers at once. Then we can release all the changed at once, and the automation is written in the way that it will only prepare packages for those providers that need release (so if you marked something as doc-only change, the provider will not get prepared. And those providers that did not have any change whatsoever are automatically skipped.
It usually takes me 10 minutes or so to go though all the providers and make decisions about the providers - and then keeping the PR rebased and incrementally update newly merged PRs is couple of minutes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
eladkal commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r960863961
##########
airflow/providers/amazon/CHANGELOG.rst:
##########
@@ -24,6 +24,45 @@
Changelog
---------
+5.1.0
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+
+Features
+~~~~~~~~
+
+* ``Additional mask aws credentials (#26014)``
+* ``Add RedshiftDeleteClusterSnapshotOperator (#25975)``
+* ``Add redshift create cluster snapshot operator (#25857)``
+* ``Add common-sql lower bound for common-sql (#25789)``
+* ``Allow AWS Secrets Backends use AWS Connection capabilities (#25628)``
+* ``Implement 'EmrEksCreateClusterOperator' (#25816)``
+* ``Improve error handling/messaging around bucket exist check (#25805)``
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix display aws connection info (#26025)``
+* ``Fix 'EcsBaseOperator' and 'EcsBaseSensor' arguments (#25989)``
+* ``Fix EMR serverless system test (#25969)``
Review Comment:
I think system test fixes should be skipped from change log.. no real value to users going over these entries. Its internal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
eladkal commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961072885
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,21 @@
Changelog
---------
+1.1.1
+.....
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix placeholders in 'TrinoHook', 'PrestoHook', 'SqliteHook' (#25939)``
+* ``Fix (and test) SQLTableCheckOperator on postgresql (#25821)``
Review Comment:
To avoid this bug for future postgres releases don't we need to set
`apache-airflow-providers-common-sql>=1.1.1` in postgres provider yaml and include release for postgres provider as well?
https://github.com/apache/airflow/blob/a2db8fcb7df1a266e82e17b937c9c1cf01a16a42/airflow/providers/postgres/provider.yaml#L43
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,21 @@
Changelog
---------
+1.1.1
Review Comment:
wait.. if we are releasing `1.1.1` we have a problem
https://github.com/apache/airflow/pull/25939 set minimum sql provider 1.2.0 for trino/sqlite/presto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961779926
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,26 @@
Changelog
---------
+1.2.0
+.....
+
+
+Features
+~~~~~~~~
+
+* ``Make placeholder style configurable (#25939)``
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix (and test) SQLTableCheckOperator on postgresql (#25821)``
+* ``Don't use Pandas for SQLTableCheckOperator (#25822)``
+
+.. Below changes are excluded from the changelog. Move them to
+ appropriate section above if needed. Do not delete the lines(!):
+ * ``Better error messsage for pre-common-sql providers (#26051)``
Review Comment:
I'd move those two also to the Bug-fixes section (maybe the first one to Misc section, but the second is a bug-fix for sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1234746174
I have added the other providers here and dropped common.sql down to 1.1.1 (instead of 1.2.0) as it felt like a bugfix release to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a diff in pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961075096
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,21 @@
Changelog
---------
+1.1.1
Review Comment:
In this pr I (tried) to change the minimum to those to be 1.1.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
eladkal commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961075108
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,21 @@
Changelog
---------
+1.1.1
Review Comment:
Ah you already fixed it :)
resolving this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1234470065
I actually wanted to make a new wave much quicker (I can even do it this evening). I think it's worth it because of few other fixes in comon.sql. I wanted to make all the providers that depend on it to be >=1.2.0 because of that:
* #25821
* #25822
* #25855
* #25939
The overhead involved in voting and testing (communication/release management) is very comparable when you do one versus all of them. Any reason to speed up just Amazon).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1235877715
> The >=1.2.0 is important to fix.
Oh yeah, I fluffed that up somehow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1234479673
BTW. The main reason to bundle them is that currently we have no way to have two "providers" votings running at the same time - there is only one directory where we keep the "voted" providers and it is also used in the automation process, so trying to run two votes in parallel will break things (and process of verification for PMCs will be different).
So I am ok with releasing Amazon separately too (if you want to go through the process), but this will mean that we won't release the new wave until the Amazon one is completed.
Up to you to decide.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r960944080
##########
airflow/providers/amazon/CHANGELOG.rst:
##########
@@ -24,6 +24,45 @@
Changelog
---------
+5.1.0
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+
+Features
+~~~~~~~~
+
+* ``Additional mask aws credentials (#26014)``
+* ``Add RedshiftDeleteClusterSnapshotOperator (#25975)``
+* ``Add redshift create cluster snapshot operator (#25857)``
+* ``Add common-sql lower bound for common-sql (#25789)``
+* ``Allow AWS Secrets Backends use AWS Connection capabilities (#25628)``
+* ``Implement 'EmrEksCreateClusterOperator' (#25816)``
+* ``Improve error handling/messaging around bucket exist check (#25805)``
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix display aws connection info (#26025)``
+* ``Fix 'EcsBaseOperator' and 'EcsBaseSensor' arguments (#25989)``
+* ``Fix EMR serverless system test (#25969)``
Review Comment:
Right. I usually put them in "ignored" section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1235610082
> For my first time I'd like to just do the bug fixes that we know are critical/urgent
That's OK :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r960827131
##########
airflow/providers/amazon/CHANGELOG.rst:
##########
@@ -24,6 +24,45 @@
Changelog
---------
+5.1.0
+.....
+
+Breaking changes
Review Comment:
```suggestion
Breaking changes
```
This should be removed (I planned to improve that a bit to not generate that part if not needed :) ). I recently added the feature of just choosing type of change so we can now decide which of the headers should be generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1234486435
Oh yeah, the common SQL bug _I_ fixed 🤦🏻
I'll re-work this to do a bundle of all those providers. Is it okay if I do this batch of releases?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961780583
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
Same in all other places where common-sql is referred to (those bugs in 1.1.0 were rather annoying and could impact more scenarios than what we are aware of - for example the Pandas thingie - so I think moving all our providers to >1.2.0 should happen).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r962018541
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
It's just saving a lot of time of maintainers for having to analyse every possible commbination comparing to automated update of comon dependency. Even if it is a noise, I simply don't want to bother about those intricacies and it makes MY life easier when relesing new version. Is that good enough reason ? I think so. A lot of my time might be freed to do other stuff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r962019670
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
What I am really talkiong is a little empathy and understanding of what consequences of not upgrading is. That's it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1236789923
Also it's worth to re-run `breeze relase-management prepare-providers-documentation --yes` after rebasing this one on top of main to make sure some recen merges were not missed in the changelog in the providers being released.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1235324827
> > Why? If we REALLY want to split providers, we need to look at them in isolation. Each provider should be treated separately, independently from the others (this is one of the reasons we added common.sql to try various scenarios).
> > If you look at common.sql in isolation - adding placeholder is a new feature, that trino, presto, sqlite might use to produce a bugfix release - and they should depend on this feature.
>
> Oh yeah very good point. It was late last night and I didn't think it through.
Also. The current automation is very well prepared to handle this case. I am not sure if you are aware (I described it in the "procedure" but you can keep on rebasing this PR and regenerate the docs after each rebase and it should be cool even if new changes get merged over time. This is pretty normal to have the "docs" PR to go through several iterations and get some new PRs merged, so in order not to loose any new changes, always after rebasing it, you should regenerate the docs using:
```
breeze release-management prepare-provider-documentation --answer yes [packages]
```
This one will update the docs (and you will have to manually review the changed files and possibly move around the changelog entries, but the previous entries you already added are untouched so it works incrementally.
This is based on the PR# not PR description so this gives also opportunity to update descriptions of the changelog in more meaninggful way (as long as you don't change (PR #) in parenthesis.
For example in this case, it is perfectly ok (and that's what I'd do) to update the description of the PR in common.sql:
In Trino/Presto/SQlite you would then have:
```
Bug Fixes
~~~~~~~~~
* ``Fix placeholders in 'TrinoHook', 'PrestoHook', 'SqliteHook' (#25939)``
```
But in common SQL:
```
Features
~~~~~~~~~
* ``Add placeholders for SQL parameters (#25939)``
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1236787550
`breeze static-checks --last-commit` Is the easiest way to re-run necessary pre-commit on the last commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1236785466
> Approved - pending the last remaining >=1.1.0
Gah how did I _still_ miss one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961780171
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
```suggestion
- apache-airflow-providers-common-sql>=1.2.0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r962004604
##########
airflow/providers/presto/CHANGELOG.rst:
##########
@@ -24,6 +24,26 @@
Changelog
---------
+4.0.1
+.....
+
+Breaking changes
Review Comment:
Should be removed :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961097020
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,21 @@
Changelog
---------
+1.1.1
Review Comment:
Nope. Common.sql provider has the new feature (placeholder) so all was fine there. This was a bugfix to trino/presto/sqlite but they dependent on new feature added to common.sql
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,21 @@
Changelog
---------
+1.1.1
Review Comment:
Nope. Common.sql provider has the new feature (placeholder) so all was fine there. This was a bugfix to trino/presto/sqlite but they depended on new feature added to common.sql
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1235328709
BTW. I found the "changed files" view of IntelliJ super helpful while reviewing the changed Docs. This is a view that only shows files actually changed locally and it allows to review multiple changed CHANGELOGS after running the `prepare-provider-documentation` really, realy quickly and efficiently. That's where old good vim and terminal (without some extra plugins) are beaten by IDEs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1235517973
For my first time I'd like to just do the bug fixes that we know are critical/urgent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r962015482
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
> I disagree -- for anything but postgres provider the update is essentially noise. We're making users think about updating/work out what has changed , but odds are the update would have no effect on most of them
I think it makes no impact on users - they will just bump the common.sql provider automatically without even knowing about that (that's the power of requirements), it's largely invisible for the users, because users will never upgrade the common.sql on their own and we should do it for them. My view on the "common" providers (and similarly for any other common functionality we might add - we already discussed taskflow for comon taksflow functions) is that this is the way to enable new "common" code without bumping Airflow version, and without incurirng the current penalty on backwards compatibilityl
The problem if we DON't bump the common version is that we have no way to test it easily. We are not able to test if the new postgres provider does not use any of the 1.2 features implicitly or by accident. So updating the common.sql to >=1.2.0 is our own sanity to not having to run tests of the new provider versionn with 1.1.0 and 1.0.0
Are you sure that the new apache provider is going to work with common.sql 1.0.0 and 1.1.0 ? I am not . How are you going to test it if someone adds new code to it between now and next release? We currently have no way of testin it. We could add more test matrix but I think it makes little sense.
I treat common.sql as a playground to test and see what kind of problems it might introduce and this is the result of observing similar issue.
That's why I think with the common providers, pretty much always we:
* not have backwards incompatible changes 1.* version for as long as we can
* add >=1.N.0 every time we release 1..N.0 for all providers that depend on them
Do you think there are other ways we can make sure 1.0.0 and 1..1..0 common.sql compatibility can be tested in the nest release?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961984980
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
I disagree -- for anything but postgres provider the update is essentially noise. We're making users think about updating/work out what has changed , but odds are the update would have no effect on most of them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r962342429
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,26 @@
Changelog
---------
+1.2.0
+.....
+
+
+Features
+~~~~~~~~
+
+* ``Make placeholder style configurable (#25939)``
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix (and test) SQLTableCheckOperator on postgresql (#25821)``
+* ``Don't use Pandas for SQLTableCheckOperator (#25822)``
+
+.. Below changes are excluded from the changelog. Move them to
+ appropriate section above if needed. Do not delete the lines(!):
+ * ``Better error messsage for pre-common-sql providers (#26051)``
+ * ``Discard semicolon stripping in SQL hook (#25855)``
Review Comment:
```suggestion
Features
~~~~~~~~
* ``Make placeholder style configurable (#25939)``
* ``Better error messsage for pre-common-sql providers (#26051)``
Bug Fixes
~~~~~~~~~
* ``Fix (and test) SQLTableCheckOperator on postgresql (#25821)``
* ``Don't use Pandas for SQLTableCheckOperator (#25822)``
* ``Discard semicolon stripping in SQL hook (#25855)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
```
##########
airflow/providers/presto/CHANGELOG.rst:
##########
@@ -24,6 +24,26 @@
Changelog
---------
+4.0.1
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+
Review Comment:
```suggestion
```
##########
airflow/providers/sqlite/CHANGELOG.rst:
##########
@@ -24,6 +24,26 @@
Changelog
---------
+3.2.1
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+
Review Comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961539814
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,21 @@
Changelog
---------
+1.1.1
+.....
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix placeholders in 'TrinoHook', 'PrestoHook', 'SqliteHook' (#25939)``
+* ``Fix (and test) SQLTableCheckOperator on postgresql (#25821)``
Review Comment:
Yes. We should bump all the providers that use common.sql and add >=1.2.0 for all of them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
eladkal commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961074846
##########
airflow/providers/common/sql/CHANGELOG.rst:
##########
@@ -24,6 +24,21 @@
Changelog
---------
+1.1.1
+.....
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix placeholders in 'TrinoHook', 'PrestoHook', 'SqliteHook' (#25939)``
+* ``Fix (and test) SQLTableCheckOperator on postgresql (#25821)``
Review Comment:
Ah you already fixed it :)
resolving this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1235246394
> Why? If we REALLY want to split providers, we need to look at them in isolation. Each provider should be treated separately, independently from the others (this is one of the reasons we added common.sql to try various scenarios).
>
> If you look at common.sql in isolation - adding placeholder is a new feature, that trino, presto, sqlite might use to produce a bugfix release - and they should depend on this feature.
Oh yeah very good point. It was late last night and I didn't think it through.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1234474438
BTW. It looks good.
I looked at it and there is only one thing to fix (and I planned to automate the header generation a bit better :D).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a diff in pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r960875848
##########
airflow/providers/amazon/CHANGELOG.rst:
##########
@@ -24,6 +24,45 @@
Changelog
---------
+5.1.0
+.....
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+
+Features
+~~~~~~~~
+
+* ``Additional mask aws credentials (#26014)``
+* ``Add RedshiftDeleteClusterSnapshotOperator (#25975)``
+* ``Add redshift create cluster snapshot operator (#25857)``
+* ``Add common-sql lower bound for common-sql (#25789)``
+* ``Allow AWS Secrets Backends use AWS Connection capabilities (#25628)``
+* ``Implement 'EmrEksCreateClusterOperator' (#25816)``
+* ``Improve error handling/messaging around bucket exist check (#25805)``
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix display aws connection info (#26025)``
+* ``Fix 'EcsBaseOperator' and 'EcsBaseSensor' arguments (#25989)``
+* ``Fix EMR serverless system test (#25969)``
Review Comment:
Oh yes agreed. The others are
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1236850359
> Also it's worth to re-run `breeze relase-management prepare-providers-documentation --answer yes` after rebasing this one on top of main to make sure some recent merges were not missed in the changelog in the providers being released.
Yup, I have been doing that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1236786774
Also I think you will need to run pre-commits, the dependencies.json will need to be regenerated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1234589345
> Oh yeah, the common SQL bug _I_ fixed 🤦🏻
>
> I'll re-work this to do a bundle of all those providers. Is it okay if I do this batch of releases?
Sure. I'd love that :). BTW. If you want to fix the "prepare-provider_packages.py" script to conditionally add "Breaking changes" / "Features" depending on the choice of the user - might be a good opportunity to do it :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #26109: Prepare release of amazon provider out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on PR #26109:
URL: https://github.com/apache/airflow/pull/26109#issuecomment-1234442664
First time I've done a provider release Jarek, so please let me know if you spot any mistakes I've made
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961780583
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
Same in all other places where common-sql is referred to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
ashb commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r961989101
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
Oh also for this diff, yeah I _meant_ to do that but failed in the diff somehow.
Anyway, separate PR for postgres is here https://github.com/apache/airflow/pull/26143 -- I made it a separate PR as that seemed the easiest way (somehow!) to get this showing up in the change log for the postgres provider.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r962015482
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
> I disagree -- for anything but postgres provider the update is essentially noise. We're making users think about updating/work out what has changed , but odds are the update would have no effect on most of them
I think it makes no impact on users - they will just bump the common.sql provider automatically without even knowing about that (that's the power of requirements), it's largely invisible for the users, because users will never upgrade the common.sql on their own and we should do it for them. My view on the "common" providers (and similarly for any other common functionality we might add - we already discussed taskflow for comon taksflow functions) is that this is the way to enable new "common" code without bumping Airflow version, and without incurirng the current penalty on backwards compatibilityl
The problem if we DON't bump the common version is that we have no way to test it easily. We are not able to test if the new postgres provider does not use any of the 1.2 features implicitly or by accident. So updating the common.sql to >=1.2.0 is our own sanity to not having to run tests of the new provider versionn with 1.1.0 and 1.0.0
Are you sure that the new postgres provider is goign to work with common.sql 1.0.0 and 1.1.0 ? I am not . How are you going to test it if someone adds new code to postgres provider? We currently have no way of testin it. We could add more test matrix but I think it makes little sense.
I treat common.sql as a playground to test and see what kind of problems it might introduce and this is the result of observing similar issue.
That's why I think with the common providers, pretty much always we:
* not have backwards incompatible changes 1.* version for as long as we can
* add >=1.N.0 every time we release 1..N.0 for all providers that depend on them
Do you think there are other ways we can make sure 1.0.0 and 1..1..0 common.sql compatibility can be tested in the nest release?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r962015482
##########
airflow/providers/presto/provider.yaml:
##########
@@ -38,7 +39,7 @@ versions:
dependencies:
- apache-airflow>=2.2.0
- - apache-airflow-providers-common-sql>=1.2.0
+ - apache-airflow-providers-common-sql>=1.1.1
Review Comment:
> I disagree -- for anything but postgres provider the update is essentially noise. We're making users think about updating/work out what has changed , but odds are the update would have no effect on most of them
I think it makes no impact on users - they will just bump the common.sql provider automatically without even knowing about that (that's the power of requirements), it's largely invisible for the users, because users will never upgrade the common.sql on their own and we should do it for them. My view on the "common" providers (and similarly for any other common functionality we might add - we already discussed taskflow for comon taksflow functions) is that this is the way to enable new "common" code without bumping Airflow version, and without incurirng the current penalty on backwards compatibilityl
The problem if we DON't bump the common version is that we have no way to test it easily. We are not able to test if the new postgres provider does not use any of the 1.2 features implicitly or by accident. So updating the common.sql to >=1.2.0 is our own sanity to not having to run tests of the new provider versionn with 1.1.0 and 1.0.0
Are you sure that the new apache provider is going to work with common.sql 1.0.0 and 1.1.0 ? I am not . How are you going to test it if someone adds new code to postgres provider? We currently have no way of testin it. We could add more test matrix but I think it makes little sense.
I treat common.sql as a playground to test and see what kind of problems it might introduce and this is the result of observing similar issue.
That's why I think with the common providers, pretty much always we:
* not have backwards incompatible changes 1.* version for as long as we can
* add >=1.N.0 every time we release 1..N.0 for all providers that depend on them
Do you think there are other ways we can make sure 1.0.0 and 1..1..0 common.sql compatibility can be tested in the nest release?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb merged pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
ashb merged PR #26109:
URL: https://github.com/apache/airflow/pull/26109
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #26109: Prepare bug-fix release of providers out of band
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #26109:
URL: https://github.com/apache/airflow/pull/26109#discussion_r962718228
##########
docs/apache-airflow-providers-amazon/index.rst:
##########
@@ -88,7 +88,7 @@ Requirements
PIP package Version required
======================================= ==================
``apache-airflow`` ``>=2.2.0``
-``apache-airflow-providers-common-sql``
+``apache-airflow-providers-common-sql`` ``>=1.1.0``
Review Comment:
that one is still left
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org