You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2025/09/14 11:07:49 UTC
[I] Release DataFusion `50.0.0` (Nov 2025) [datafusion]
alamb opened a new issue, #17558:
URL: https://github.com/apache/datafusion/issues/17558
### Is your feature request related to a problem or challenge?
_No response_
### Describe the solution you'd like
_No response_
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3508262110
I have made a PR to revert the coalesce/nvl/nvl2 rewrites
- https://github.com/apache/datafusion/pull/18567
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3516426820
FYI, Just started the vote process
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3503601696
Starting to test with Comet: https://github.com/apache/datafusion-comet/pull/2729
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3536552300
@alamb thank you, just started the vote process https://lists.apache.org/thread/k2ygp0c1rp56ww44lljf0yg6qf60q5f7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3530545418
> Time to start the RC2
I think we still need to update changelog before releasing, will do later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3493378609
ballista tested with commit 32d26187a358aaa0803867a7963cd149ec1fcd1e and looks ok 👍🏻
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3505053700
Given the current state of tests, I think we should cut `branch-51` tomorrow (and backport anything else we need to get the release out).
I think @xudong963 said he was away this weekend, so I'll plan to make the branch tomorrow morning unless I hear otherwise
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `50.0.0` (Nov 2025) [datafusion]
Posted by "mjgarton (via GitHub)" <gi...@apache.org>.
mjgarton commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3289921549
Should the title of this issue say 51.0.0 instead of 50.0.0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3482510743
@Omega359 do we have additional work needed for the default timezone fixes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "pepijnve (via GitHub)" <gi...@apache.org>.
pepijnve commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3481496267
> FYI, I plan to cut the 51.0.0 in the next two days. Do we have some high-priority PRs waiting for review?
#18444 and #18183 are the remaining two for the `CASE` epic. Would be nice if we could still get those in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3506390216
I also filed a ticket for a blog post 🎣
- https://github.com/apache/datafusion/issues/18548
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3533155365
>
> Could anyone help do this? My network has been bad for a long time:
>
> ```
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Request GET /repos/apache/datafusion/commits/e14322da196a75bea2de2be560b93ea7c8e83d11/pulls failed with 403: rate limit exceeded
> Setting next backoff to 3503.394145s
> ```
Done!
- https://github.com/apache/datafusion/pull/18705
BTW this happens to me when I don't use my github token (the rate limit is apparently much stricter)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3455847002
If we could get the following arrow fix merged and released as a patch and upgrade to that patch before the upcoming release it would be sweet:
- https://github.com/apache/arrow-rs/pull/8496
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3455817991
I think we could start working on the release next week?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3487110060
> I'd like to get the first one in for 51.0.0. The other two while nice to have I do not see as critical
Thanks @Omega359 -- added it to the list above
I also think we need to fix this regression:
- https://github.com/apache/datafusion/issues/17801
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3499042036
I made a PR to upgrade delta.rs and I didn't find anything show stopping
- https://github.com/delta-io/delta-rs/pull/3921
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3495756289
mv repo looks good: https://github.com/datafusion-contrib/datafusion-materialized-views/pull/104
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3456065741
> If we could get the following arrow fix merged and released as a patch and upgrade to that patch before the upcoming release it would be sweet:
>
> * [fix(parquet): converting parquet schema with backward compatible repeated struct/primitive with provided arrow schema arrow-rs#8496](https://github.com/apache/arrow-rs/pull/8496)
I think that one will be released in arrow 57.1.0 (a minor update that won't require a datafusion upgrade).
So once we have released datafusion 51 we'll be able to do a minor update to arrow 57.1.0
(in other words, I don't think the release timeline of DataFusion will impact the flow of that PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3509550989
Thank you @alamb, I followed the current status, looks like the only blocker is https://github.com/apache/datafusion/issues/18527?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3490395287
I'm testing with mv repo tomorrow.
It would be nice to cut and vote at Friday (I'm off on the weekend)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3499311437
> I made a PR to upgrade delta.rs and I didn't find anything show stopping
>
> * [feat: update to DataFusion 51.0.0 delta-io/delta-rs#3921](https://github.com/delta-io/delta-rs/pull/3921)
How did you pick that commit? There's no branch-51 yet, so unsure what will make the cut. Should I test Comet with latest main commit?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3479604755
FYI, I plan to cut the 51.0.0 in the next two days. Do we have some high-priority PRs waiting for review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 closed issue #17558: Release DataFusion `51.0.0` (Nov 2025)
URL: https://github.com/apache/datafusion/issues/17558
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3505092896
I started [testing with iceberg-rust](https://github.com/apache/iceberg-rust/pull/1830) and the blast radius is pretty big for their page index tests, so Comet might lag behind DF 51 a bit to help get iceberg-rust sorted next week or the week after. That's not a -1 or anything from me though. DF 51 looks good from Comet's end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "pepijnve (via GitHub)" <gi...@apache.org>.
pepijnve commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3497445645
> I also think we need to fix this regression:
> https://github.com/apache/datafusion/issues/17801
Regarding that regression, I'm happy to help out with getting this fixed but I got a bit stuck in figuring out what the most appropriate solution is. I'll write up what I've tried so far in the comment thread of that issue. If I could get some help in deciding how to proceed we should be able to resolve this fairly quickly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3499385817
> How did you pick that commit? There's no branch-51 yet, so unsure what will make the cut. Should I test Comet with latest main commit?
Yes I just picked the latest main commit 🤷
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3520070011
> > FYI, Just started the vote process
>
> [@xudong963](https://github.com/xudong963) please share the vote link?
Here, https://lists.apache.org/thread/32dcnyrhcjxg6scx2wjv9m6m22skft32.
But I think the RC1 failed, let's wait for RC2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3553993889
Thank you @xudong963 -- amazing as alwasy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3518198693
> FYI, Just started the vote process
@xudong963 please share the vote link?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "findepi (via GitHub)" <gi...@apache.org>.
findepi commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3291011666
I removed https://github.com/apache/datafusion/pull/16842 from "Changes to add to upgrade guide". I think it was completed in 50.0.0. If not, let's add it back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3501407529
> > How did you pick that commit? There's no branch-51 yet, so unsure what will make the cut. Should I test Comet with latest main commit?
>
> Yes I just picked the latest main commit 🤷
Random sample, cornerstone of good quality control 😀 (just kidding)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3506332710
I have created the branch here:
- https://github.com/apache/datafusion/tree/branch-51
I think this issue needs to be fixed prior to release (I will make a PR to revert the offending commits shortly)
- [ ] https://github.com/apache/datafusion/issues/17801
This one (raised by @shehabgamin) would be nice to look at as well
- [ ] https://github.com/apache/datafusion/issues/18527
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3516917264
Found another regression I think: https://github.com/apache/datafusion/issues/18622
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3531704481
> > Time to start the RC2
>
> I think we still need to update changelog before releasing, will do later
Could anyone help do this? My network has been bad for a long time:
```
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Request GET /repos/apache/datafusion/commits/e14322da196a75bea2de2be560b93ea7c8e83d11/pulls failed with 403: rate limit exceeded
Setting next backoff to 3503.394145s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3511173272
> Thank you [@alamb](https://github.com/alamb), I followed the current status, looks like the only blocker is [#18527](https://github.com/apache/datafusion/issues/18527)?
Maybe -- I don't think that ticket is actionable at the moment (it doesn't have enough information to reproduce / understand what is potentially wrong)
Unless @shehabgamin or someone from his team can help triage / provide more information I think we should not treat https://github.com/apache/datafusion/issues/18527 as a blocker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3552321597
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "shehabgamin (via GitHub)" <gi...@apache.org>.
shehabgamin commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3499697794
> I made a PR to upgrade delta.rs and I didn't find anything show stopping
>
> * [feat: update to DataFusion 51.0.0 delta-io/delta-rs#3921](https://github.com/delta-io/delta-rs/pull/3921)
@alamb We should coordinate on the next release. It looks like we can save some duplicated effort:
https://github.com/lakehq/delta-rs/pull/2
## Testing With Sail
@lonless9 made a PR to upgrade to DataFusion 51:
https://github.com/lakehq/sail/pull/1031
### Breaking Changes
- PyO3
- Tonic
### Regressions
One of our tests related to window functions is failing with the error:
```
Expects PARTITION BY expression to be ordered
```
Here is the test in question:
https://github.com/lakehq/sail/blob/8bc2a05d595f02aba3a23fcd26aef81df36a7b96/python/pysail/tests/spark/test_execution.py#L124-L148
We likely won’t have the bandwidth to dig deeper in time for the release, but it’s something worth looking into, IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3502046516
@shehabgamin -- thank you
> > I made a PR to upgrade delta.rs and I didn't find anything show stopping
> > ```
> > * [feat: update to DataFusion 51.0.0 delta-io/delta-rs#3921](https://github.com/delta-io/delta-rs/pull/3921)
> > ```
>
> [@alamb](https://github.com/alamb) We should coordinate on the next release. It looks like we can save some duplicated effort: [lakehq/delta-rs#2](https://github.com/lakehq/delta-rs/pull/2)
Thanks! Yes that would be great -- it would be super helpful. I started my testing about the same time as @lonless9 filed https://github.com/delta-io/delta-rs/pull/3921 -- hopefully next time we'll avoid the duplication
> ## Testing With Sail
> [@lonless9](https://github.com/lonless9) made a PR to upgrade to DataFusion 51: [lakehq/sail#1031](https://github.com/lakehq/sail/pull/1031)
❤️
> ```
> Expects PARTITION BY expression to be ordered
> ```
>
> Here is the test in question: https://github.com/lakehq/sail/blob/8bc2a05d595f02aba3a23fcd26aef81df36a7b96/python/pysail/tests/spark/test_execution.py#L124-L148
Thanks -- I looked at that briefly at that test and it isn't clear where "Expects PARTITION BY expression to be ordered" is coming from. While the window function does have a `partition by` expression, I don't see why the partition by would be ordered.
I filed a ticket to track it
- https://github.com/apache/datafusion/issues/18527
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3482779938
@alamb - the outstanding ones I've got on my radar are
- https://github.com/apache/datafusion/pull/18025
- https://github.com/apache/datafusion/pull/18408 (new feature, not in critical path)
- https://github.com/apache/datafusion/pull/18417 (been broken for quite a while now)
I'd like to get the first one in for 51.0.0. The other two while nice to have I do not see as critical
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3511919357
> Unless [@shehabgamin](https://github.com/shehabgamin) or someone from his team can help triage / provide more information I think we should not treat [#18527](https://github.com/apache/datafusion/issues/18527) as a blocker.
Let's wait for one day, I may start the vote process tomorrow (UTC+8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3518185925
@hareshkh wanna include https://github.com/apache/datafusion/pull/18596 to the release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3290775287
> Should the title of this issue say 51.0.0 instead of 50.0.0?
yes, thank you, i changed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3527291411
It seems like all PRs are added to branch-51, cc @comphead @timsaucer
Time to start the RC2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "shehabgamin (via GitHub)" <gi...@apache.org>.
shehabgamin commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3487645051
We're working on testing Sail on our end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Release DataFusion `51.0.0` (Nov 2025) [datafusion]
Posted by "AdamGS (via GitHub)" <gi...@apache.org>.
AdamGS commented on issue #17558:
URL: https://github.com/apache/datafusion/issues/17558#issuecomment-3487738588
Testing with Vortex [here](https://github.com/vortex-data/vortex/pull/5189)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org