You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "amoghrajesh (via GitHub)" <gi...@apache.org> on 2025/03/25 05:29:07 UTC

[PR] Disallowing kylinpy 2.7.0 as it contains invalid pypi metadata [airflow]

amoghrajesh opened a new pull request, #48226:
URL: https://github.com/apache/airflow/pull/48226

   <!--
    Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at
   
      http://www.apache.org/licenses/LICENSE-2.0
   
    Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.
    -->
   
   <!--
   Thank you for contributing! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   In case of an existing issue, reference it using one of the following:
   
   closes: #ISSUE
   related: #ISSUE
   
   How to write a good git commit message:
   http://chris.beams.io/posts/git-commit/
   -->
   
   kylinpy 2.7.0 has a bad metadata on pypi as reported here: https://github.com/apache/airflow/actions/runs/14044194114/job/39321485583
   
   Diasllowing that version. I also wonder if we need to bump since this version was from May 2020.
   
   
   <!-- Please keep an empty line above the dashes. -->
   ---
   **^ Add meaningful description above**
   Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Disallowing kylinpy 2.7.0 as it contains invalid pypi metadata [airflow]

Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #48226:
URL: https://github.com/apache/airflow/pull/48226#issuecomment-2750212206

   > Just wondering >2.7.0 is existing and how was it working before? Did somebody update the previously released 2.7.0 with bad meta data but it was working before?!?
   
   Its actually quite coincidental, the version is a range, so it can be picked up based on other constraints, seems it is just coincidence that this version gets picked up this time!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Disallowing kylinpy 2.7.0 as it contains invalid pypi metadata [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #48226:
URL: https://github.com/apache/airflow/pull/48226#issuecomment-2750196515

   See https://github.com/Kyligence/kylinpy/issues/57#issuecomment-2749127711
   
   > I also wonder if we need to bump since this version was from May 2020.
   
   Not sure if it matters. Latest version is also from 2020. It seems like there is not much to do with this SDK (also only 2 open issues there) so it might make sense that it's oldish release but probably just fine as is.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Disallowing kylinpy 2.7.0 as it contains invalid pypi metadata [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #48226:
URL: https://github.com/apache/airflow/pull/48226#issuecomment-2750980418

   Just to clarify - no need to revert it BTW, it will work now with setuptools 78.0.2 where they brought back handling of "invalid" metadata, but this PR would not change anything if 78.0.1 was still the latest version.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Disallowing kylinpy 2.7.0 as it contains invalid pypi metadata [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #48226:
URL: https://github.com/apache/airflow/pull/48226#issuecomment-2750976221

   Well. This was not the reason in this case. The reason was really the new `setuptools` -  78.0.1 - started to complain about the invalid metadata. But even latest kylinpy 2.8.4 has this invalid metadata:
   
   ```cfg
   [aliases]
   test = pytest
   
   [metadata]
   description-file = README.rst    # !!!! THIS IS WRONG. Should be description_file !!!!
   
   [egg_info]
   tag_build = 
   tag_date = 0
   ```
   
   The problem with `setuptools` is that this is a default build-requirement for many packages. No matter what version you have locally, if your project has `pyproject.toml`, when a package needs to be built, it is built in `isolation_mode` by default - which means that whatever frontend you use (`pip`, `hatch`, `uv`) - it will create a separate, NEW virtualenv to build such project (`kylinpy` in this case) - it will install there LATEST requirements that the project declares as build dependencies and attempt to build the project in that venv.
   
   There are MANY projects like `kylinpy` that do not pin their build requirements, This means tha whenever new setuptools is released, unless you specifiy `--no-build-isolation` for specific package (there are ways to individually specify which packages should be build without isolation, they will have a separately installed venv with latest setuptools - and when the latest setuptools does not like bad metadata, such package cannot be built.
   
   There are several ways to deal with it:
   
   1) --no-build-isolation + pinning setuptools in your main venv (then such build will use whatever version of setuptools you already have in your local, shared venv where you run `pip install` or `uv pip install`
   
   2) when you use `pip` you can set PIP_CONSTRAINTS env variable and point it to a constraint file where you specify which version of setuptools (and other dependencies) should be used. Note - that only work via env variable, when you pass constraints with `--constraints` , those constrints are NOT passed to the isolated, new virtualenv
   
   3) when you use `uv` you can pass `--build-constraints` flag and pass constratints this way to the isolated venv where package is built. But then that flag applies to the whole `uv pip install` or `uv sync` or whatever command - so those constraints apply to all the packages that `uv` installs that needs building (i.e. all packages that have no .whl dstribution that matches your machine / architecture and so on.
   
   So in this case - when next time setuptools maintainers will attempt to get rid of old, deprecated and "wrong" metadata packges, we either have to workaround it in one of the ways above (but then our users will have to do the same when installing airflow from `PyPI` - which is not ideal. Or better - we need to work with those dependencies of ours that have that problem and either help to (F)ix them or (F)ork them (I did it with json-merge-patch where I vendored in the 70 lines of code) or (F)orego them - i.e. switch to another dependency that is modern.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Disallowing kylinpy 2.7.0 as it contains invalid pypi metadata [airflow]

Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh merged PR #48226:
URL: https://github.com/apache/airflow/pull/48226


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org