You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/12/29 06:48:09 UTC

[GitHub] [airflow] jedcunningham opened a new pull request #20549: Remove unnecessary python 3.6 conditionals

jedcunningham opened a new pull request #20549:
URL: https://github.com/apache/airflow/pull/20549


   Since Python 3.7 is now the lowest supported version, we no longer need
   to have conditionals to support 3.6.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk commented on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002684393

> I'm not opposed to keeping ones that are intended to be cherry-picked back, but why wouldn't we want to remove all the others? And why _after_ 2.3.0, why would we need/want them shipped in that version?

We could do it even now - but only if we accept the risk that things won't work for Python 3.6 when we cherry-pick something to 2.2.4 (or 2.2.5 if we have it). We do not yet know all the stuff we will be cherry-picking (and we can decide tomorrow that we cherry-pick something new). This is connected with the risk, that during cherry-picking some of those 'if PY36' will be removed and we will not realize that because the cherry-pick will cleanly apply and our tests might not catch it.

Why waiting until 2.3.0 might help ?

Because (at least so far) releasing 2.N.0 is the time we stop "mass-cherry-picking" to 2.N-1 branch. So far I think we have not done even once a release for previous minor release once we released 2.N.0. This also decreases the risk of such accidental Python 3.6 removal if we decide to release an urgent bugfix in 2.N-1. We do not yet know how soon we will release 2.3.0 - it depends on many factors, and we do not know if we will have 2.2.4 or maybe even 2.2.5 and how many cherry-picked commits we will have there - it really depends on whether we will find and fix more bugs.

So this is really a question of how confident we are that this will not be a problem. I think it's up to you to decide - you are the release manager. But you should take the risk into account - and decide consciously to accept the risk. And if you are fine with that I am also fine. I just put "request changes" so that this is not merged accidentally, the case is discussed and decision is made with the risks in mind :).

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk edited a comment on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002684393

We could do it even now - but only if we accept the risk that things won't work for Python 3.6 when we cherry-pick something to 2.2.4 (or 2.2.5 if we have it). We do not yet know all the stuff we will be cherry-picking (and we can decide tomorrow or in a week that we cherry-pick something new). This is connected with the risk, that during cherry-picking some of those 'if PY36' will be removed and we will not realize that because the cherry-pick will cleanly apply and our tests might not catch it. As a result we might have failing 2.2.4 on Python 3.6 in some cases.

Why waiting until 2.3.0 might help ?

Because (at least so far) releasing 2.N.0 is the time we stop "mass-cherry-picking" to 2.N-1 branch. So far I think we have not done even once a release for previous minor release once we released 2.N.0. This also decreases the risk of such accidental Python 3.6 removal if we decide to release an urgent bugfix in 2.N-1. (because we will be very selective with cherry-picks). We do not yet know how soon we will release 2.3.0 - it depends on many factors, and we do not know if we will have 2.2.4 or maybe even 2.2.5 and how many cherry-picked commits we will have there - it really depends on whether we will find and fix more important bugs until 2.3.0 is released.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk edited a comment on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002684393

We could do it even now - but only if we accept the risk that things won't work for Python 3.6 when we cherry-pick something to 2.2.4 (or 2.2.5 if we have it). We do not yet know all the stuff we will be cherry-picking (and we can decide tomorrow or in a week that we cherry-pick something new). This is connected with the risk, that during cherry-picking some of those 'if PY36' will be removed and we will not realize that because the cherry-pick will cleanly apply and our tests might not catch it. As a result we might have failing 2.2.4 on Python 3.6 in some cases.

Why waiting until 2.3.0 might help ?

Because (at least so far) releasing 2.N.0 is the time we stop "mass-cherry-picking" to 2.N-1 branch. So far I think we have not done even once a release for previous minor release once we released 2.N.0. This also decreases the risk of such accidental Python 3.6 removal if we decide to release an urgent bugfix in 2.N-1. (because we will be very selective with cherry-picks). We do not yet know how soon we will release 2.3.0 - it depends on many factors, and we do not know if we will have 2.2.4 or maybe even 2.2.5 and how many cherry-picked commits we will have there - it really depends on whether we will find and fix more important bugs until 2.3.0 is released.

So this is really a question of how confident we are that this will not be a problem. I think it's up to you to decide - you are the release manager :D.

But you should take the risk into account - and decide consciously to accept the risk. And if you are fine with that I am also fine. I just put "request changes" so that this is not merged accidentally, the case is discussed and decision is made with the risks in mind :).

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk edited a comment on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002684393

We could do it even now - but only if we accept the risk that things won't work for Python 3.6 when we cherry-pick something to 2.2.4 (or 2.2.5 if we have it). We do not yet know all the stuff we will be cherry-picking (and we can decide tomorrow or in a week that we cherry-pick something new). This is connected with the risk, that during cherry-picking some of those 'if PY36' will be removed by the cherry-pick to 2-2 branch and we will not realize that because the cherry-pick will cleanly apply and our tests might not catch it. As a result we might have failing 2.2.4 on Python 3.6 in some cases.

Example:

If tomorrow we find an error in `airflow/configuration.py` and have a bugfix and cherry-pick that change, it might also remove the PEP662 removal (depending how close the fix is to those removed lines (might also result with conflict).

Why waiting until 2.3.0 might help ?

Because (at least so far) releasing 2.N.0 is the time we stop "mass-cherry-picking" to 2.N-1 branch. So far I think we have not done even once a release for previous minor release once we released 2.N.0. This also decreases the risk of such accidental Python 3.6 removal if we decide to release an urgent bugfix in 2.N-1. (because we will be very selective with cherry-picks). We do not yet know how soon we will release 2.3.0 - it depends on many factors, and we do not know if we will have 2.2.4 or maybe even 2.2.5 and how many cherry-picked commits we will have there - it really depends on whether we will find and fix more important bugs until 2.3.0 is released.

So this is really a question of how confident we are that this will not be a problem. I think it's up to you to decide - you are the release manager :D.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk edited a comment on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002684393

We could do it even now - but only if we accept the risk that things won't work for Python 3.6 when we cherry-pick something to 2.2.4 (or 2.2.5 if we have it). We do not yet know all the stuff we will be cherry-picking (and we can decide tomorrow or in a week that we cherry-pick something new). This is connected with the risk, that during cherry-picking some of those 'if PY36' will be removed and we will not realize that because the cherry-pick will cleanly apply and our tests might not catch it. As a result we might have failing 2.2.4 on Python 3.6 in some cases.

Why waiting until 2.3.0 might help ?

Because (at least so far) releasing 2.N.0 is the time we stop "mass-cherry-picking" to 2.N-1 branch. So far I think we have not done even once a release for previous minor release once we released 2.N.0. This also decreases the risk of such accidental Python 3.6 removal if we decide to release an urgent bugfix in 2.N-1. (because we will be very selective with cherry-picks). We do not yet know how soon we will release 2.3.0 - it depends on many factors, and we do not know if we will have 2.2.4 or maybe even 2.2.5 and how many cherry-picked commits we will have there - it really depends on whether we will find and fix more bugs.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1046353838


   This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk edited a comment on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002684393

We could do it even now - but only if we accept the risk that things won't work for Python 3.6 when we cherry-pick something to 2.2.4 (or 2.2.5 if we have it). We do not yet know all the stuff we will be cherry-picking (and we can decide tomorrow or in a week that we cherry-pick something new). This is connected with the risk, that during cherry-picking some of those 'if PY36' will be removed by the cherry-pick to 2-2 branch and we will not realize that because the cherry-pick will cleanly apply and our tests might not catch it. As a result we might have failing 2.2.4 on Python 3.6 in some cases.

Why waiting until 2.3.0 might help ?

Because (at least so far) releasing 2.N.0 is the time we stop "mass-cherry-picking" to 2.N-1 branch. So far I think we have not done even once a release for previous minor release once we released 2.N.0. This also decreases the risk of such accidental Python 3.6 removal if we decide to release an urgent bugfix in 2.N-1. (because we will be very selective with cherry-picks). We do not yet know how soon we will release 2.3.0 - it depends on many factors, and we do not know if we will have 2.2.4 or maybe even 2.2.5 and how many cherry-picked commits we will have there - it really depends on whether we will find and fix more important bugs until 2.3.0 is released.

So this is really a question of how confident we are that this will not be a problem. I think it's up to you to decide - you are the release manager :D.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on a change in pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk commented on a change in pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#discussion_r776415408



##########
File path: Dockerfile.ci
##########
@@ -25,7 +25,7 @@ ARG AIRFLOW_VERSION="2.2.0.dev0"
 ARG AIRFLOW_IMAGE_REPOSITORY="https://github.com/apache/airflow"
 
 # By increasing this number we can do force build of all dependencies
-ARG DEPENDENCIES_EPOCH_NUMBER="6"

Review comment:
       Ah I see :). 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on a change in pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk commented on a change in pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#discussion_r776289357



##########
File path: setup.cfg
##########
@@ -131,7 +131,6 @@ install_requires =
     openapi-spec-validator>=0.2.4
     packaging>=14.0
     pendulum~=2.0
-    pep562~=1.0;python_version<"3.7"

Review comment:
       This one we can likely remove even now.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk edited a comment on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002684393

We could do it even now - but only if we accept the risk that things won't work for Python 3.6 when we cherry-pick something to 2.2.4 (or 2.2.5 if we have it). We do not yet know all the stuff we will be cherry-picking (and we can decide tomorrow or in a week that we cherry-pick something new). This is connected with the risk, that during cherry-picking some of those 'if PY36' will be removed by the cherry-pick to 2-2 branch and we will not realize that because the cherry-pick will cleanly apply and our tests might not catch it. As a result we might have failing 2.2.4 on Python 3.6 in some cases.

Example:

If tomorrow we find an error in `airflow/configuration.py` and have a bugfix and cherry-pick that change to v2_2_test, it might also remove the PEP662 removal (depending how close the fix is to those removed lines (might also result with conflict).

Why waiting until 2.3.0 might help ?

Because (at least so far) releasing 2.N.0 is the time we stop "mass-cherry-picking" to 2.N-1 branch. So far I think we have not done even once a release for previous minor release once we released 2.N.0. This also decreases the risk of such accidental Python 3.6 removal if we decide to release an urgent bugfix in 2.N-1. (because we will be very selective with cherry-picks). We do not yet know how soon we will release 2.3.0 - it depends on many factors, and we do not know if we will have 2.2.4 or maybe even 2.2.5 and how many cherry-picked commits we will have there - it really depends on whether we will find and fix more important bugs until 2.3.0 is released.

So this is really a question of how confident we are that this will not be a problem. I think it's up to you to decide - you are the release manager :D.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk edited a comment on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002684393

We could do it even now - but only if we accept the risk that things won't work for Python 3.6 when we cherry-pick something to 2.2.4 (or 2.2.5 if we have it). We do not yet know all the stuff we will be cherry-picking (and we can decide tomorrow or in a week that we cherry-pick something new). This is connected with the risk, that during cherry-picking some of those 'if PY36' will be removed and we will not realize that because the cherry-pick will cleanly apply and our tests might not catch it. As a result we might have failing 2.2.4 on Python 3.6 in some cases.

Why waiting until 2.3.0 might help ?

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr commented on a change in pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

uranusjr commented on a change in pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#discussion_r779336877



##########
File path: Dockerfile.ci
##########
@@ -25,7 +25,7 @@ ARG AIRFLOW_VERSION="2.2.0.dev0"
 ARG AIRFLOW_IMAGE_REPOSITORY="https://github.com/apache/airflow"
 
 # By increasing this number we can do force build of all dependencies
-ARG DEPENDENCIES_EPOCH_NUMBER="6"

Review comment:
       Actually since `pep562` is only installed on `python_version < '3.7'`, I don’t think we even need to rebuild the dependencies because it is never actually installed in any of the images right now.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk edited a comment on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002684393

We could do it even now - but only if we accept the risk that things won't work for Python 3.6 when we cherry-pick something to 2.2.4 (or 2.2.5 if we have it). We do not yet know all the stuff we will be cherry-picking (and we can decide tomorrow or in a week that we cherry-pick something new). This is connected with the risk, that during cherry-picking some of those 'if PY36' will be removed and we will not realize that because the cherry-pick will cleanly apply and our tests might not catch it. As a result we might have failing 2.3.4 on Python 3.6 in some cases.

Why waiting until 2.3.0 might help ?

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk edited a comment on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002684393

We could do it even now - but only if we accept the risk that things won't work for Python 3.6 when we cherry-pick something to 2.2.4 (or 2.2.5 if we have it). We do not yet know all the stuff we will be cherry-picking (and we can decide tomorrow or in a week that we cherry-pick something new). This is connected with the risk, that during cherry-picking some of those 'if PY36' will be removed and we will not realize that because the cherry-pick will cleanly apply and our tests might not catch it.

Why waiting until 2.3.0 might help ?

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk commented on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1046551090


   We should be able to merge it already I think 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on a change in pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

potiuk commented on a change in pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#discussion_r776289123



##########
File path: Dockerfile.ci
##########
@@ -25,7 +25,7 @@ ARG AIRFLOW_VERSION="2.2.0.dev0"
 ARG AIRFLOW_IMAGE_REPOSITORY="https://github.com/apache/airflow"
 
 # By increasing this number we can do force build of all dependencies
-ARG DEPENDENCIES_EPOCH_NUMBER="6"

Review comment:
       That was accidental I think :)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] jedcunningham commented on pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

jedcunningham commented on pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#issuecomment-1002670187


   I'm not opposed to keeping ones that are intended to be cherry-picked back, but why wouldn't we want to remove all the others? And why _after_ 2.3.0, why would we need/want them shipped in that version?
   
   We can't remove `pep562` until we drop the conditionals, so I'm confused 🤷‍♂️


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] jedcunningham commented on a change in pull request #20549: Remove unnecessary python 3.6 conditionals

Posted by GitBox <gi...@apache.org>.

jedcunningham commented on a change in pull request #20549:
URL: https://github.com/apache/airflow/pull/20549#discussion_r776400313



##########
File path: Dockerfile.ci
##########
@@ -25,7 +25,7 @@ ARG AIRFLOW_VERSION="2.2.0.dev0"
 ARG AIRFLOW_IMAGE_REPOSITORY="https://github.com/apache/airflow"
 
 # By increasing this number we can do force build of all dependencies
-ARG DEPENDENCIES_EPOCH_NUMBER="6"

Review comment:
       This was bumped because `pep562` was removed.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org