You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/07/22 09:14:06 UTC
[GitHub] [arrow-datafusion-python] andygrove opened a new issue, #7: Release version 0.7.0
andygrove opened a new issue, #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7
I would like to propose releasing version 0.7.0 shortly after DataFusion 11.0.0 is released.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] francis-du commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
francis-du commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1308127863
I'm interested in doing some work on this, anyone who needs my help please assign me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1327784914
I started a vote on the mailing list: https://lists.apache.org/thread/1nh72rdvywbxyyjhgqs3jd5xrnhx6n5f
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] Jimexist commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
Jimexist commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1319435708
last time I did publish, I used twine and upload it like any other python packages. maturin was used in prior steps to build the artifacts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] Jimexist commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
Jimexist commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1306589917
I used to do release while it is hosted in datafusion-contrib org, if help is needed I can share more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1328070630
I am now trying to upload to testpypi using twine (based on the instructions at https://packaging.python.org/en/latest/tutorials/packaging-projects/) but am running into an error.
```bash
$ python3 -m twine upload --repository testpypi --verbose datafusion-0.7.0-cp37-abi3*.whl
Uploading distributions to https://test.pypi.org/legacy/
INFO datafusion-0.7.0-cp37-abi3-macosx_10_7_x86_64.whl (9.4 MB)
INFO datafusion-0.7.0-cp37-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (11.9 MB)
INFO datafusion-0.7.0-cp37-abi3-win_amd64.whl (10.0 MB)
INFO Querying keyring for username
Enter your username: __token__
INFO Querying keyring for password
Enter your password:
INFO username: __token__
INFO password: <hidden>
Uploading datafusion-0.7.0-cp37-abi3-macosx_10_7_x86_64.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.9/9.9 MB • 00:06 • 1.7 MB/s
INFO Response from https://test.pypi.org/legacy/:
400 Invalid value for project_urls. Error: Use valid URL.
INFO <html>
<head>
<title>400 Invalid value for project_urls. Error: Use valid URL.</title>
</head>
<body>
<h1>400 Invalid value for project_urls. Error: Use valid URL.</h1>
The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>
Invalid value for project_urls. Error: Use valid URL.
</body>
</html>
```
Any idea what I am doing wrong @Jimexist @isidentical?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove closed issue #7: Release version 0.7.0
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #7: Release version 0.7.0
URL: https://github.com/apache/arrow-datafusion-python/issues/7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] francis-du commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
francis-du commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1314697946
> If we are ok with just running `maturin publish` for now, I think we could go ahead and create a release candidate, vote on it, then run that command to publish?
I think these are enough, does @isidentical has anything else to add?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1328071856
I think the issue may be that in https://github.com/apache/arrow-datafusion-python/commit/f0d565912cd1cb86e5f268ff41bf1118e9743690#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711 we removed the `https://` prefix from the URLs in `project.url`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] francis-du commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
francis-du commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1240316797
I would like to participate in the release process to familiarize myself with the ASF specification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1314183499
`maturin publish` sounds good. Is publishing to PyPi enough for this release? How important is it to publish wheels?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] isidentical commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
isidentical commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1315595659
`maturin publish` publishes (with the right set of arguments) wheels only for the system you are running it in. So we still have to do something like polars where we would repeatedly call `maturin publish` on multiple different environments to package wheels for everyone. I think the most important one would be MacOS + manylinux2010/manylinux2014 but if windows is easy we can also include it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1317681710
@isidentical @francis-du @Jimexist Let me know what you think of the previous statement/question :point_up:. Any reason I should not go ahead and cut a release candidate now? It has been more than 6 months since the last release so I am keen to get some momentum going again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1236422658
I ran into problems releasing this and could use help from someone who is more familiar with the process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1327760163
I pushed a 0.7.0-rc1 tag, and it built the wheels. The artifacts are available for download at the bottom of this page: https://github.com/apache/arrow-datafusion-python/actions/runs/3549837792
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] isidentical commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
isidentical commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1317707647
By the way, source distribution hash can also be used to verify it on PyPI directly (so I assume that satisfies the voting guidelines in the sense that the actual release is now shipped, and the wheels are just auxiliary artefacts)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1307647514
> I used to do release while it is hosted in datafusion-contrib org, if help is needed I can share more.
Thanks @Jimexist. Is there any documentation on how to do the release? I guess the first step is getting this kind of documentation into the repo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1314186590
If we are ok with just running `maturin publish` for now, I think we could go ahead and create a release candidate, vote on it, then run that command to publish?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] isidentical commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
isidentical commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1315514638
> maturin publish sounds good. Is publishing to PyPi enough for this release? How important is it to publish wheels?
I am not super sure if `maturin publish` has an internal mechanism to allow this, but in general wheels are a great way to save a lot of trouble for users (since otherwise, each installation would essentially need to recompile & link datafusion; which in turn would require the installation system to have a proper rust tooling ready).
Another idea is checking out cross compilation from maturin. I haven't used it and maybe it won't help with such a complex dependency chain as ours but it might be worth a shot. https://www.maturin.rs/distribution.html#build-wheels & https://www.maturin.rs/distribution.html#cross-compile-to-linuxmacos
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove closed issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove closed issue #7: Release version 0.7.0
URL: https://github.com/apache/arrow-datafusion-python/issues/7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] isidentical commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
isidentical commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1317691238
> To make sure I understand this correctly... we already have a GitHub workflow to build the wheels when we tag the repo, so if we go ahead with a release now, we can always run maturin publish manually on Mac & Linux after the release is tagged, and with appropriate votes on releasing?
That seems to be my understanding. Wheels are basically pre-packaged versions of source distributions; so if there is no formal guidelines on binary publishing in ASF I think you can start the voting process directly on the source distributions and once we cut it we should be able to build wheels for it separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] Jimexist commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
Jimexist commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1319435047
> candid
one way to have a thing for people to test out is instead of publishing a release candidate, publish to testpypi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] isidentical commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
isidentical commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1307973778
I am quite unfamiliar with the release procedure of this repo, but here are my observations on how the basic flow of a release might look like:
- It seems like the last release was at July with 0.6.0, so we probably need to bump the version tag for this one?
- There is already some automation in place for [building the wheels](https://github.com/apache/arrow-datafusion-python/blob/72f06006ca78ab4bcfcfc10ea3d3f2c2e9e515e9/.github/workflows/build.yml#L18-L22) when an rc version tag is pushed. So we can trigger it and use the artifacts for the release candidate process.
- Once the vote passes, we probably need to repeat these two steps but with the final version (that'd be `0.7.0`). For building wheels for all platforms, I think it makes sense to change the build action to actually be ran on every tag push.
- Downloading the artifacts from the build of 0.7.0, and uploading them to [PyPI using twine](https://github.com/pypa/twine).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] francis-du commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
francis-du commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1314702972
> `maturin publish` sounds good. Is publishing to PyPi enough for this release? How important is it to publish wheels?
I think `maturin publish` should upload wheel directly. If we use `twine + GitHub Action`, we may needs to build the wheel .`maturin publish` should not need to build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1306465820
I would like to see us start releasing this project again, but I don't have sufficient knowledge about Python. I would love to pair with someone to get a release out.
@isidentical I was curious - do you have any interest in the DataFusion Python bindings?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] isidentical commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
isidentical commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1307966622
@andygrove @Jimexist also let me know if there is anything I can help with.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] francis-du commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
francis-du commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1308161472
> I am quite unfamiliar with the release procedure of this repo, but here are my observations on how the basic flow of a release might look like:
>
> * It seems like the last release was at July with 0.6.0, so we probably need to bump the version tag for this one?
> * There is already some automation in place for [building the wheels](https://github.com/apache/arrow-datafusion-python/blob/72f06006ca78ab4bcfcfc10ea3d3f2c2e9e515e9/.github/workflows/build.yml#L18-L22) when an rc version tag is pushed. So we can trigger it and use the artifacts for the release candidate process.
> * Once the vote passes, we probably need to repeat these two steps but with the final version (that'd be `0.7.0`). For building wheels for all platforms, I think it makes sense to change the build action to actually be ran on every tag push.
> * Downloading the artifacts from the build of 0.7.0, and uploading them to [PyPI using twine](https://github.com/pypa/twine).
We don't seem to need to publish to PyPI with `twine`, we can publish directly with [`maturin publish`](https://www.maturin.rs/tutorial.html).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1315520745
Maybe we can learn from how Polars does this. It looks like they use `maturin publish` to publish the wheels?
https://github.com/pola-rs/polars/blob/master/.github/workflows/create-py-release-manylinux.yaml
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on issue #7: Release version 0.7.0
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #7:
URL: https://github.com/apache/arrow-datafusion-python/issues/7#issuecomment-1317678817
To make sure I understand this correctly... we already have a GitHub workflow to build the wheels when we tag the repo, so if we go ahead with a release now, we can always run `maturin publish` manually on Mac & Linux after the release is tagged, and with appropriate votes on releasing? I would rather release soon and learn what we can do to improve the process next time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org