You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datafusion.apache.org by Andrew Lamb <al...@influxdata.com> on 2026/09/16 17:59:42 UTC
Notes from Sync Call 2026-09-16
We had our sync call today; I think we should be better at writing down
what was discussed and I took some notes on the google doc [1].
Here they are in case you are interested
Andrew
Topics
-
Adam GS – decorrelation subquery;
-
Many issues being filed / proposed to fix
-
How to do more isolated testing
-
Issue on general decorrelation:
https://github.com/apache/datafusion/issues/5492
-
Ideas on testing different optimizer passes
-
Follow on: post to the existing issues, to try and get enough
momentum to implement the project
-
Raz – Blocked Aggregation Update
-
Issue: https://github.com/apache/datafusion/issues/24704
-
Prototype of a new proposed approach is available
-
Open questions: do we need a new Trait to help migration, how much
performance slowdown we are willing to accept.
-
Request: Please read the issue and provide feedback, would the new
proposed API work for your aggregates
-
Haresh – CI Automation for more active automation
-
Would like to automatically identify and backport PRs
-
Would be willing to help out
-
Discussed various tradeoffs and ways to automate the process and the
criteria on which to backport
-
Andrew also discussed LTS in
https://github.com/apache/datafusion/issues/16622
-
Ask: please review and comment on the issue if you are interested
-
Martin – “DAVE” stack idea.
-
Blog:
https://sequenceandsilicon.substack.com/p/the-dave-stack-the-age-of-domain
-
Stu – if time, range partition
[1]:
https://docs.google.com/document/d/1vHWQOal-P3jh2A_XuLcLxSQ3VYd0yAexAgWSCNeprWQ/edit?tab=t.0
Re: Notes from Sync Call 2026-09-16
Posted by ku...@gmail.com.
Hi Andrew & Kumar,
Andrew thank you for confirming the information is still accurate. I was trying to send that email as a question/point out my own confusion, I hope the tone wasn’t harsh. I’ll make a calendar event on my side.
Kumar, thank you for the Google link. The Parquet, Arrow, & Iceberg communities have iCal links that can add a calendar to your Gmail or Apple calendar. So I was looking for something similar and just sent the previous email to make sure I didn’t miss something simple.
> On Sep 17, 2026, at 04:16, Kumar Ujjawal <uj...@gmail.com> wrote:
>
> Hi Kurtis,
>
> We do have a link for google meet if that’s what you meant.
> https://meet.google.com/nfg-eviu-qrm
>
>
> Kumar Ujjawal
>
>> On Thu, 17 Sep 2026 at 4:31 PM, Andrew Lamb <al...@influxdata.com> wrote:
>>
>> Hi Kurtis,
>>
>>> Is there a calendar invite for community syncs?
>>
>> I don't know of any calendar invite.
>>
>>> When I checked
>> https://datafusion.apache.org/contributor-guide/communication.html I found
>> this doc link
>>
>> https://docs.google.com/document/d/1NBpkIAuU7O9h8Br5CbFksDhX-L9TyO9wmGLPMe0Plc8
>> which I just realized said the meeting notes were no longer updated,
>>
>> That google doc has the correct information, namely
>>
>>> The Apache DataFusion Community meeting is held every Wednesday at 11:00
>> a.m. Eastern Time, which is 16:00 UTC during Eastern Standard Time and
>> 15:00 UTC during Eastern Daylight Time.
>>> ...
>>> The Google Meet link for the video call:
>> http://meet.google.com/nfg-eviu-qrm
>>
>> Yes it seems no one has taken notes at the sync for a while. I started
>> taking notes a few weeks ago because about 10 people regularly on the call,
>> Matt is no longer MC'ing, and we have topics of general interest. I seem to
>> have put the notes into a different document for last two weeks. I have
>> moved them into the public one now
>>
>>> but I initially read as the sync was no longer happening.
>>
>> Yes, I can see how that would be the case. I am sorry for that and
>> hopefully we can keep the notes more updated going forward
>>
>> Andrew
>>
>>> On Wed, Sep 16, 2026 at 5:29 PM <ku...@gmail.com> wrote:
>>>
>>> Hi Andrew & Datafusion Community,
>>>
>>> Is there a calendar invite for community syncs? When I checked
>>> https://datafusion.apache.org/contributor-guide/communication.html I
>>> found this doc link
>>>
>> https://docs.google.com/document/d/1NBpkIAuU7O9h8Br5CbFksDhX-L9TyO9wmGLPMe0Plc8
>>> which I just realized said the meeting notes were no longer updated, but
>> I
>>> initially read as the sync was no longer happening.
>>>
>>> Best Regards,
>>> Kurtis Wright
>>>
>>>> On Sep 16, 2026, at 11:00, Andrew Lamb <al...@influxdata.com> wrote:
>>>>
>>>> We had our sync call today; I think we should be better at writing
>> down
>>>> what was discussed and I took some notes on the google doc [1].
>>>>
>>>> Here they are in case you are interested
>>>>
>>>> Andrew
>>>>
>>>>
>>>>
>>>> Topics
>>>>
>>>> -
>>>>
>>>> Adam GS – decorrelation subquery;
>>>> -
>>>>
>>>> Many issues being filed / proposed to fix
>>>> -
>>>>
>>>> How to do more isolated testing
>>>> -
>>>>
>>>> Issue on general decorrelation:
>>>> https://github.com/apache/datafusion/issues/5492
>>>> -
>>>>
>>>> Ideas on testing different optimizer passes
>>>> -
>>>>
>>>> Follow on: post to the existing issues, to try and get enough
>>>> momentum to implement the project
>>>> -
>>>>
>>>> Raz – Blocked Aggregation Update
>>>> -
>>>>
>>>> Issue: https://github.com/apache/datafusion/issues/24704
>>>> -
>>>>
>>>> Prototype of a new proposed approach is available
>>>> -
>>>>
>>>> Open questions: do we need a new Trait to help migration, how much
>>>> performance slowdown we are willing to accept.
>>>> -
>>>>
>>>> Request: Please read the issue and provide feedback, would the new
>>>> proposed API work for your aggregates
>>>> -
>>>>
>>>> Haresh – CI Automation for more active automation
>>>> -
>>>>
>>>> Would like to automatically identify and backport PRs
>>>> -
>>>>
>>>> Would be willing to help out
>>>> -
>>>>
>>>> Discussed various tradeoffs and ways to automate the process and
>> the
>>>> criteria on which to backport
>>>> -
>>>>
>>>> Andrew also discussed LTS in
>>>> https://github.com/apache/datafusion/issues/16622
>>>> -
>>>>
>>>> Ask: please review and comment on the issue if you are interested
>>>> -
>>>>
>>>> Martin – “DAVE” stack idea.
>>>> -
>>>>
>>>> Blog:
>>>>
>>>
>> https://sequenceandsilicon.substack.com/p/the-dave-stack-the-age-of-domain
>>>>
>>>> -
>>>>
>>>> Stu – if time, range partition
>>>>
>>>>
>>>> [1]:
>>>>
>>>
>> https://docs.google.com/document/d/1vHWQOal-P3jh2A_XuLcLxSQ3VYd0yAexAgWSCNeprWQ/edit?tab=t.0
>>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: Notes from Sync Call 2026-09-16
Posted by Kumar Ujjawal <uj...@gmail.com>.
Hi Kurtis,
We do have a link for google meet if that’s what you meant.
https://meet.google.com/nfg-eviu-qrm
Kumar Ujjawal
On Thu, 17 Sep 2026 at 4:31 PM, Andrew Lamb <al...@influxdata.com> wrote:
> Hi Kurtis,
>
> > Is there a calendar invite for community syncs?
>
> I don't know of any calendar invite.
>
> > When I checked
> https://datafusion.apache.org/contributor-guide/communication.html I found
> this doc link
>
> https://docs.google.com/document/d/1NBpkIAuU7O9h8Br5CbFksDhX-L9TyO9wmGLPMe0Plc8
> which I just realized said the meeting notes were no longer updated,
>
> That google doc has the correct information, namely
>
> > The Apache DataFusion Community meeting is held every Wednesday at 11:00
> a.m. Eastern Time, which is 16:00 UTC during Eastern Standard Time and
> 15:00 UTC during Eastern Daylight Time.
> > ...
> > The Google Meet link for the video call:
> http://meet.google.com/nfg-eviu-qrm
>
> Yes it seems no one has taken notes at the sync for a while. I started
> taking notes a few weeks ago because about 10 people regularly on the call,
> Matt is no longer MC'ing, and we have topics of general interest. I seem to
> have put the notes into a different document for last two weeks. I have
> moved them into the public one now
>
> > but I initially read as the sync was no longer happening.
>
> Yes, I can see how that would be the case. I am sorry for that and
> hopefully we can keep the notes more updated going forward
>
> Andrew
>
> On Wed, Sep 16, 2026 at 5:29 PM <ku...@gmail.com> wrote:
>
> > Hi Andrew & Datafusion Community,
> >
> > Is there a calendar invite for community syncs? When I checked
> > https://datafusion.apache.org/contributor-guide/communication.html I
> > found this doc link
> >
> https://docs.google.com/document/d/1NBpkIAuU7O9h8Br5CbFksDhX-L9TyO9wmGLPMe0Plc8
> > which I just realized said the meeting notes were no longer updated, but
> I
> > initially read as the sync was no longer happening.
> >
> > Best Regards,
> > Kurtis Wright
> >
> > > On Sep 16, 2026, at 11:00, Andrew Lamb <al...@influxdata.com> wrote:
> > >
> > > We had our sync call today; I think we should be better at writing
> down
> > > what was discussed and I took some notes on the google doc [1].
> > >
> > > Here they are in case you are interested
> > >
> > > Andrew
> > >
> > >
> > >
> > > Topics
> > >
> > > -
> > >
> > > Adam GS – decorrelation subquery;
> > > -
> > >
> > > Many issues being filed / proposed to fix
> > > -
> > >
> > > How to do more isolated testing
> > > -
> > >
> > > Issue on general decorrelation:
> > > https://github.com/apache/datafusion/issues/5492
> > > -
> > >
> > > Ideas on testing different optimizer passes
> > > -
> > >
> > > Follow on: post to the existing issues, to try and get enough
> > > momentum to implement the project
> > > -
> > >
> > > Raz – Blocked Aggregation Update
> > > -
> > >
> > > Issue: https://github.com/apache/datafusion/issues/24704
> > > -
> > >
> > > Prototype of a new proposed approach is available
> > > -
> > >
> > > Open questions: do we need a new Trait to help migration, how much
> > > performance slowdown we are willing to accept.
> > > -
> > >
> > > Request: Please read the issue and provide feedback, would the new
> > > proposed API work for your aggregates
> > > -
> > >
> > > Haresh – CI Automation for more active automation
> > > -
> > >
> > > Would like to automatically identify and backport PRs
> > > -
> > >
> > > Would be willing to help out
> > > -
> > >
> > > Discussed various tradeoffs and ways to automate the process and
> the
> > > criteria on which to backport
> > > -
> > >
> > > Andrew also discussed LTS in
> > > https://github.com/apache/datafusion/issues/16622
> > > -
> > >
> > > Ask: please review and comment on the issue if you are interested
> > > -
> > >
> > > Martin – “DAVE” stack idea.
> > > -
> > >
> > > Blog:
> > >
> >
> https://sequenceandsilicon.substack.com/p/the-dave-stack-the-age-of-domain
> > >
> > > -
> > >
> > > Stu – if time, range partition
> > >
> > >
> > > [1]:
> > >
> >
> https://docs.google.com/document/d/1vHWQOal-P3jh2A_XuLcLxSQ3VYd0yAexAgWSCNeprWQ/edit?tab=t.0
> >
>
Re: Notes from Sync Call 2026-09-16
Posted by Andrew Lamb <al...@influxdata.com>.
Hi Kurtis,
> Is there a calendar invite for community syncs?
I don't know of any calendar invite.
> When I checked
https://datafusion.apache.org/contributor-guide/communication.html I found
this doc link
https://docs.google.com/document/d/1NBpkIAuU7O9h8Br5CbFksDhX-L9TyO9wmGLPMe0Plc8
which I just realized said the meeting notes were no longer updated,
That google doc has the correct information, namely
> The Apache DataFusion Community meeting is held every Wednesday at 11:00
a.m. Eastern Time, which is 16:00 UTC during Eastern Standard Time and
15:00 UTC during Eastern Daylight Time.
> ...
> The Google Meet link for the video call:
http://meet.google.com/nfg-eviu-qrm
Yes it seems no one has taken notes at the sync for a while. I started
taking notes a few weeks ago because about 10 people regularly on the call,
Matt is no longer MC'ing, and we have topics of general interest. I seem to
have put the notes into a different document for last two weeks. I have
moved them into the public one now
> but I initially read as the sync was no longer happening.
Yes, I can see how that would be the case. I am sorry for that and
hopefully we can keep the notes more updated going forward
Andrew
On Wed, Sep 16, 2026 at 5:29 PM <ku...@gmail.com> wrote:
> Hi Andrew & Datafusion Community,
>
> Is there a calendar invite for community syncs? When I checked
> https://datafusion.apache.org/contributor-guide/communication.html I
> found this doc link
> https://docs.google.com/document/d/1NBpkIAuU7O9h8Br5CbFksDhX-L9TyO9wmGLPMe0Plc8
> which I just realized said the meeting notes were no longer updated, but I
> initially read as the sync was no longer happening.
>
> Best Regards,
> Kurtis Wright
>
> > On Sep 16, 2026, at 11:00, Andrew Lamb <al...@influxdata.com> wrote:
> >
> > We had our sync call today; I think we should be better at writing down
> > what was discussed and I took some notes on the google doc [1].
> >
> > Here they are in case you are interested
> >
> > Andrew
> >
> >
> >
> > Topics
> >
> > -
> >
> > Adam GS – decorrelation subquery;
> > -
> >
> > Many issues being filed / proposed to fix
> > -
> >
> > How to do more isolated testing
> > -
> >
> > Issue on general decorrelation:
> > https://github.com/apache/datafusion/issues/5492
> > -
> >
> > Ideas on testing different optimizer passes
> > -
> >
> > Follow on: post to the existing issues, to try and get enough
> > momentum to implement the project
> > -
> >
> > Raz – Blocked Aggregation Update
> > -
> >
> > Issue: https://github.com/apache/datafusion/issues/24704
> > -
> >
> > Prototype of a new proposed approach is available
> > -
> >
> > Open questions: do we need a new Trait to help migration, how much
> > performance slowdown we are willing to accept.
> > -
> >
> > Request: Please read the issue and provide feedback, would the new
> > proposed API work for your aggregates
> > -
> >
> > Haresh – CI Automation for more active automation
> > -
> >
> > Would like to automatically identify and backport PRs
> > -
> >
> > Would be willing to help out
> > -
> >
> > Discussed various tradeoffs and ways to automate the process and the
> > criteria on which to backport
> > -
> >
> > Andrew also discussed LTS in
> > https://github.com/apache/datafusion/issues/16622
> > -
> >
> > Ask: please review and comment on the issue if you are interested
> > -
> >
> > Martin – “DAVE” stack idea.
> > -
> >
> > Blog:
> >
> https://sequenceandsilicon.substack.com/p/the-dave-stack-the-age-of-domain
> >
> > -
> >
> > Stu – if time, range partition
> >
> >
> > [1]:
> >
> https://docs.google.com/document/d/1vHWQOal-P3jh2A_XuLcLxSQ3VYd0yAexAgWSCNeprWQ/edit?tab=t.0
>
Re: Notes from Sync Call 2026-09-16
Posted by ku...@gmail.com.
Hi Andrew & Datafusion Community,
Is there a calendar invite for community syncs? When I checked https://datafusion.apache.org/contributor-guide/communication.html I found this doc link https://docs.google.com/document/d/1NBpkIAuU7O9h8Br5CbFksDhX-L9TyO9wmGLPMe0Plc8 which I just realized said the meeting notes were no longer updated, but I initially read as the sync was no longer happening.
Best Regards,
Kurtis Wright
> On Sep 16, 2026, at 11:00, Andrew Lamb <al...@influxdata.com> wrote:
>
> We had our sync call today; I think we should be better at writing down
> what was discussed and I took some notes on the google doc [1].
>
> Here they are in case you are interested
>
> Andrew
>
>
>
> Topics
>
> -
>
> Adam GS – decorrelation subquery;
> -
>
> Many issues being filed / proposed to fix
> -
>
> How to do more isolated testing
> -
>
> Issue on general decorrelation:
> https://github.com/apache/datafusion/issues/5492
> -
>
> Ideas on testing different optimizer passes
> -
>
> Follow on: post to the existing issues, to try and get enough
> momentum to implement the project
> -
>
> Raz – Blocked Aggregation Update
> -
>
> Issue: https://github.com/apache/datafusion/issues/24704
> -
>
> Prototype of a new proposed approach is available
> -
>
> Open questions: do we need a new Trait to help migration, how much
> performance slowdown we are willing to accept.
> -
>
> Request: Please read the issue and provide feedback, would the new
> proposed API work for your aggregates
> -
>
> Haresh – CI Automation for more active automation
> -
>
> Would like to automatically identify and backport PRs
> -
>
> Would be willing to help out
> -
>
> Discussed various tradeoffs and ways to automate the process and the
> criteria on which to backport
> -
>
> Andrew also discussed LTS in
> https://github.com/apache/datafusion/issues/16622
> -
>
> Ask: please review and comment on the issue if you are interested
> -
>
> Martin – “DAVE” stack idea.
> -
>
> Blog:
> https://sequenceandsilicon.substack.com/p/the-dave-stack-the-age-of-domain
>
> -
>
> Stu – if time, range partition
>
>
> [1]:
> https://docs.google.com/document/d/1vHWQOal-P3jh2A_XuLcLxSQ3VYd0yAexAgWSCNeprWQ/edit?tab=t.0