You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "wiedld (via GitHub)" <gi...@apache.org> on 2024/04/01 17:42:48 UTC
[PR] WIP: df patched upgrade to 2024-03-05, requiring new DF fixes [arrow-datafusion]
wiedld opened a new pull request, #9901:
URL: https://github.com/apache/arrow-datafusion/pull/9901
⚠️ **WIP: will not be merged.** ⚠️
Below is edited, since a patch was merged into main
## What's (was) in this branch:
When testing against iox, we have been finding patches needed in DF. This is a branch for datafusion through EOD 2024-03-05, and then layering on patches needed.
1. Starting at datafusion main branch commit from March 5th 2024:
```
commit ea01e56c3341dd4308a24e94091b86ee475ce224
Author: Marko Milenković <mi...@users.noreply.github.com>
Date: Tue Mar 5 23:05:19 2024 +0000
Add plugable handler for `CREATE FUNCTION` (#9333)
```
1. Then we added these commits:
* Fix incorrect results with multiple `COUNT(DISTINCT..)` aggregates on dictionaries (#9679), merged into datafusion on 2024-03-19.
* fix(9678): short circuiting prevented population of visited stack, for common subexpr elimination optimization (#9685), merged into datafusion on 2024-03-21.
1. And a new patch, based upon a newly found bug:
* original fix, in 2 commits (d59a8debab0a63ceaf862097ab474d38949ce056 and 049bf0927d8653c91703e1ce02b38c806925ac16)
* reverted those 2 commits, then:
* added [actual fix merged into DF main](https://github.com/apache/arrow-datafusion/commit/cd7a00b08309f7229073e4bba686d6271726ab1c):
```
commit 4a387c8cb53994ae4a41afc14c7a73fafb7ffcf6
Author: wiedld <wi...@users.noreply.github.com>
Date: Sun Mar 31 05:09:06 2024 -0700
fix(9870): common expression elimination optimization, should always re-find the correct expression during re-write. (#9871)
```
* reason for this patch is [detailed here](https://github.com/influxdata/influxdb_iox/pull/
10463#issuecomment-2024334683).
1. This new patch^^, merged into DF main on 2024-03-31, no longer had 2 methods which existed at 2024-03-05. Therefore, those two methods were patched (just for this 2024-03-05 branch):
```
commit 9af8ac27e1567ea6e7bf76643d398d71979f56a4
Author: wiedld <dl...@gmail.com>
Date: Mon Apr 1 08:53:20 2024 -0700
refactor: incorporate ExprSet changes into the methods that no longer exist on main, but do exist at 2024-03-05
```
1. The new DF patch ([merged into main](https://github.com/apache/arrow-datafusion/commit/cd7a00b08309f7229073e4bba686d6271726ab1c)) also included a test using coalesce. This test relies upon a bug fix merged into main on March 7th (and not available on this 2024-03-05 branch). Added that patch too:
```
commit 581e74785b876615d6a63db8c2e5ba372bf78828 (HEAD -> iox-10349/df-upgrade-with-patches, origin/iox-10349/df-upgrade-with-patches)
Author: comphead <co...@users.noreply.github.com>
Date: Thu Mar 21 11:07:40 2024 -0700
build: modify code to comply with latest clippy requirement (#9725)
```
1. add the clippy build fix
```
commit 581e74785b876615d6a63db8c2e5ba372bf78828 (HEAD -> iox-10349/df-upgrade-with-patches, origin/iox-10349/df-upgrade-with-patches)
Author: comphead <co...@users.noreply.github.com>
Date: Thu Mar 21 11:07:40 2024 -0700
build: modify code to comply with latest clippy requirement (#9725)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] WIP: df patched upgrade to 2024-03-05, requiring new DF fixes [arrow-datafusion]
Posted by "wiedld (via GitHub)" <gi...@apache.org>.
wiedld closed pull request #9901: WIP: df patched upgrade to 2024-03-05, requiring new DF fixes
URL: https://github.com/apache/arrow-datafusion/pull/9901
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] WIP: df patched upgrade to 2024-03-05, requiring new DF fixes [arrow-datafusion]
Posted by "wiedld (via GitHub)" <gi...@apache.org>.
wiedld commented on PR #9901:
URL: https://github.com/apache/arrow-datafusion/pull/9901#issuecomment-2030233015
Opened on wrong base. Closing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org