You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2023/04/16 08:38:59 UTC

[GitHub] [airflow] potiuk opened a new pull request, #30667: Suspend Yandex provider due to protobuf limitation

potiuk opened a new pull request, #30667:
URL: https://github.com/apache/airflow/pull/30667

   Yandex provider brings protobuf dependency down to <4 and we are gearing up to updating it everywhere else. Protobuf3 support ends in Q2 2023 for Python https://protobuf.dev/support/version-support/#python
   
   Yandex is the last provider that we do not closely collaborate on fixing
   * Gogle provider dependencies are actively upgraded to latest version by Google led team: #30067 (some of the libraries are already updated) with target to update all dependencies by mid-May
   * Apache-Beam has already merged protobuf4 support https://github.com/apache/beam/pull/25874 with the target of relasing it in 2.47.0 mid-May
   * The mysql-connector-python in MySQL provider is already turned into optional dependency: #30487
   
   The only remaining dependency limiting us to protobuf 3 (<3.21) is yandexcloud. We've opened an issue to yandexcloud
   https://github.com/yandex-cloud/python-sdk/issues/71 3 weeks ago and while there was an initial interest, there is no progress on the issue, therefore - in order to prepare for running all the tests and final migration to protobuf4 we need to suspend Yandex provider - following the suspension process we agreed and got a LAZY CONSENSUS on in
   the https://lists.apache.org/thread/g8b3k028qhzgw6c3yz4jvmlc67kcr9hj mailing list discussion.
   
   The yandex provider can be removed from suspension by a PR reverting this change once yandexcloud dependency removes the protobuf limitation in their release and PR reverting this change (and fixing all tests and static check that will be needed) is the way it can be done.
   
   <!--
   Thank you for contributing! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   In case of an existing issue, reference it using one of the following:
   
   closes: #ISSUE
   related: #ISSUE
   
   How to write a good git commit message:
   http://chris.beams.io/posts/git-commit/
   -->
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk merged pull request #30667: Suspend Yandex provider due to protobuf limitation

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #30667:
URL: https://github.com/apache/airflow/pull/30667


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] eladkal commented on a diff in pull request #30667: Suspend Yandex provider due to protobuf limitation

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #30667:
URL: https://github.com/apache/airflow/pull/30667#discussion_r1167825757


##########
airflow/providers/yandex/provider.yaml:
##########
@@ -20,7 +20,7 @@ package-name: apache-airflow-providers-yandex
 name: Yandex
 description: |
     Yandex including `Yandex.Cloud <https://cloud.yandex.com/>`__
-suspended: false
+suspended: true

Review Comment:
   ```suggestion
   suspended: true # see https://github.com/apache/airflow/pull/30667/
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on pull request #30667: Suspend Yandex provider due to protobuf limitation

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #30667:
URL: https://github.com/apache/airflow/pull/30667#issuecomment-1510428774

   > wow, it shows how much #30422 has simplified the suspension process! LGTM
   
   Yep. All with a single flag flipped. That's the beauty of  having everything under control of our CI/Breeze/pre-commits framework.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on a diff in pull request #30667: Suspend Yandex provider due to protobuf limitation

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #30667:
URL: https://github.com/apache/airflow/pull/30667#discussion_r1167970919


##########
airflow/providers/yandex/provider.yaml:
##########
@@ -20,7 +20,7 @@ package-name: apache-airflow-providers-yandex
 name: Yandex
 description: |
     Yandex including `Yandex.Cloud <https://cloud.yandex.com/>`__
-suspended: false
+suspended: true

Review Comment:
   Yep.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org