You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by 953396112 <13...@qq.com> on 2020/11/10 12:47:37 UTC
回复：[DISCUSS] State of the project 2020
Hi Stamatis,
Thanks for your great work!
Calcite is very good at semantic transformation of relational algebra. In the process of the project, users can optimize relational algebra by implementing RelOptRule or RelShuttle and so on, so as to bring relational algebra into physical query engine. Another point is the construction of materialized view recognition framework, which realizes the ability of materialized view recognition of common relational algebra. The expression of relational algebra is very flexible, and it can do better in identifying various materialized views.
+1 for voting Haisheng.
Best,
Zhaohui Xu
------------------ 原始邮件 ------------------
发件人: "dev" <zabetak@gmail.com>;
发送时间: 2020年11月5日(星期四) 上午6:26
收件人: "dev"<dev@calcite.apache.org>;
主题: [DISCUSS] State of the project 2020
Hi Calcite community members,
A bit more than five years ago (October 22, 2015) Calcite graduated to a
top-level Apache project[1]. At that time, the community decided to have an
annual “state of the project” discussion and to vote for a new PMC
chair/VP[2]. So, I’m kicking off both of those discussions.
I think it was an excellent year so far in many aspects.
We were lucky to have many high quality contributions including: notable
improvements in the Volcano planner (for speed, plan quality,
extensibility) bringing it a bit closer to Cascades and Columbia [6, 7, 8,
9]; easier and more extensible parameterization of rules [3]; new dialects
such as ClickHouse [4], and Presto [5]; support for SQL hints [10]; new
adapters for querying Redis [11] and InnoDB [12] through Calcite; various
enhancements in streaming SQL. The previous list is by no means exhaustive.
Apart from the new features, certainly worth mentioning is the
modernization of the build and test infrastructure (for both Calcite and
Avatica), with the migration from maven to gradle, JUnit4 to JUnit5, and
the introduction of GitHub actions as part of the CI.
In terms of CI, I am happy to see a few more integration tests (IT) running
on a regular basis on GitHub. Eventually, it will be nice to have even more
IT tests to help us catch regressions early on and improve the quality of
our releases.
We wouldn’t have so many great contributions, if we didn’t also have
prolific contributors.
Our community has grown with Danny, Haisheng, Ruben, joining the PMC,
Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers, and many
more people chiming in discussions, reviews, and submitting pull requests,
who are not yet committers but I’m sure some of whom will become in the
near future.
We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica release
(1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think that is a
great tempo that we should strive to maintain in the years to come. One
thing to improve is the poor implication of other people than Francis on
the Avatica side; the rest of us, putting myself first, should try to be
more involved by reviewing PRs, preparing releases, voting etc.
It was nice to see our community members giving talks to conferences such
as ApacheCon, and Flink Forward presenting Calcite and/or its application.
Some of us have also done presentations in universities in order to
introduce Calcite to the next generation of computer engineers. One or two
conferences per year is a good number but it would be even better if we
could increase this frequency. There are still many people, especially
younger engineers, who are not aware of Calcite (at least this is the
impression that I get by speaking with people in Europe) and we should be
more active on the project’s dissemination.
Calcite is a very versatile library/framework that can be used in many
contexts. On one side, it is used in many production systems and utility
apps with the most recent adopters being Hazelcast, Ignite, SuperSQL
(Tencent), and NeuroBlade. On the other side, its adoption in research
projects and teaching could be boosted. Every university has multiple
projects and courses around databases and data integration where Calcite
could be a good fit.
Over the past few years we always had problems with reviewing pull requests
and I don’t think we made much progress on this aspect. In our last
discussion around this topic, Julian suggested introducing some metrics and
giving credit to those people that are doing the most in this area, and we
all agreed to do so. Any ideas on improving this situation are highly
appreciated.
Calcite is a vivid community and we are lucky to participate in many
fruitful discussions. Of course, in every community there is some friction
from time to time and the same goes for Calcite. It is a bit unrealistic to
claim that we can eliminate it entirely but we can try to reduce it, by
being more attentive and patient.
Being PMC chair was a big learning experience for me and I am very grateful
for the opportunity that was given to me. It is certainly among the things
that I am most proud of and I would like to thank everyone who trusted and
helped me in this role.
Last but not least, we should discuss who should be the new PMC chair of
Calcite after I step down in December. I would like to nominate Haisheng
Yuan as the first candidate in the vote. Apart from many high quality
contributions, Haisheng has reviewed a big amount of PRs, and led many
technical discussions to consensus. Haisheng has been in the community for
a while and I believe he will be a great chair if he is willing to accept.
To conclude, I will repeat the questions from previous years:
1) What else are we doing well in the project?
2) What areas do we need to do better?
3) Which other candidates should we consider for PMC chair?
Please take some time to share your thoughts!
Note that this discussion is for everyone; even if you have never sent an
email to the list before now it is a good time to do so :)
Best,
Stamatis
[1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
[2]
http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E
[3] https://issues.apache.org/jira/browse/CALCITE-3923
[4] https://issues.apache.org/jira/browse/CALCITE-3724
[5] https://issues.apache.org/jira/browse/CALCITE-2157
[6] https://issues.apache.org/jira/browse/CALCITE-3916
[7] https://issues.apache.org/jira/browse/CALCITE-3896
[8] https://issues.apache.org/jira/browse/CALCITE-3753
[9] https://issues.apache.org/jira/browse/CALCITE-2970
[10] https://issues.apache.org/jira/browse/CALCITE-482
[11] https://issues.apache.org/jira/browse/CALCITE-3510
[12] https://issues.apache.org/jira/browse/CALCITE-4034
Re: [DISCUSS] State of the project 2020
Posted by Haisheng Yuan <hy...@apache.org>.
Thank you for serving as Calcite PMC chair, you are a great help for Calcite community.
I feel more than happy and honored to be nominated as candidate of chair for next year, I will do my best and continue serving the community if I were selected.
Thanks,
Haisheng Yuan
On 2020/11/18 18:53:08, Julian Hyde <jh...@apache.org> wrote:
> I think Haisheng is a great choice for chair, if he is willing.
>
> Thanks for serving as chair this year, Stamatis. You've done a great
> job keeping the community civilized and on track.
>
> Julian
>
> On Sat, Nov 14, 2020 at 10:23 PM Forward Xu <fo...@gmail.com> wrote:
> >
> > Hi Stamatis,
> >
> > Thanks for your great work.
> >
> >
> > Haisheng, Danny, and Ruben are all excellent. I believe they can do the
> > work.
> >
> > This time I will vote for Haisheng first.
> >
> >
> > Finally, +1 on Haisheng being our next PMC chair.
> >
> > Best,
> > Forward
> >
> >
> >
> > Stamatis Zampetakis <za...@gmail.com> 于2020年11月15日周日 上午7:15写道：
> >
> > > Thanks to all those who shared their thoughts so far!
> > >
> > > Most of the suggestions around reviews and PRs sound promising. Summarizing
> > > we have:
> > > * Adding component owners for reviews;
> > > * Cleaning up stale PRs;
> > > * Freezing master till every JIRA marked for the next release is reviewed;
> > >
> > > Having component owners has been discussed in the past [1, 2] but
> > > the general feeling was against.
> > > I don't know if now things are different but we could definitely follow-up
> > > in a separate discussion if people are interested.
> > >
> > > Stale PRs is an issue and becoming stale is what is more problematic. If we
> > > don't review PRs in a timely manner then
> > > there is a high chance that the contributor will lose interest and the PR
> > > will never merge. In the past [3], we discussed the possibility of adding a
> > > bot
> > > for helping us in this task but the idea was abandoned in the end. If done
> > > right, I think it can be useful. For instance, sending a reminder on our
> > > dev list
> > > if the PR does not have any activity for a certain time could help us maybe
> > > prioritize this over others.
> > >
> > > One simple thing that we can do without too much discussion is active
> > > committers to assign a number 5/10 PRs to themselves and we make sure
> > > that they are resolved (merged or closed) for the next release.
> > >
> > > Regarding the role of the PMC chair, there were three names mentioned so
> > > far: Haisheng, Danny, and Ruben. I believe in all of them!
> > > Depending on the interest from their side and the feedback from the rest of
> > > the community we will proceed to a vote.
> > >
> > > Best,
> > > Stamatis
> > >
> > > [1]
> > >
> > > https://lists.apache.org/thread.html/3b1f43b15306b2f1496fd73bda01dcb3a1dba08747b1267e1b5f8187%40%3Cdev.calcite.apache.org%3E
> > > [2]
> > >
> > > https://lists.apache.org/thread.html/7231a09fc9d1bad52c229664e8cb2a3631314b34464c2df10c594745%40%3Cdev.calcite.apache.org%3E
> > > [3]
> > >
> > > https://lists.apache.org/thread.html/e794ec7f36246ec918a409b80759efc30d5433eea08ee564d4ba0c67%40%3Cdev.calcite.apache.org%3E
> > >
> > >
> > > On Wed, Nov 11, 2020 at 10:19 PM Rui Wang <am...@apache.org> wrote:
> > >
> > > > Thanks Stamatis for serving as the PMC chair for Calcite in 2020!
> > > >
> > > > One of the highlights I want to give for the PR reviewing/merging area
> > > was
> > > > the practice we started to adopt in recent releases: we started to check
> > > > each JIRA marked with the right release version and see whether there is
> > > a
> > > > PR in a good shape that can be merged. And we have built a contract that
> > > > unless those promising PR are merged, we won't unlock the Calcite
> > > master. I
> > > > feel like this practice has proven to be useful (and of course thanks for
> > > > release managers who have been working on this!).
> > > >
> > > >
> > > > Finally, +1 on Haisheng being our next PMC chair.
> > > >
> > > >
> > > > -Rui
> > > >
> > > > On Wed, Nov 11, 2020 at 1:57 AM Ruben Q L <ru...@gmail.com> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > First of all, thanks Stamatis for this detailed "state of the project",
> > > > and
> > > > > thanks for your great work as PMC chair throughout this year, I take my
> > > > hat
> > > > > off.
> > > > >
> > > > > Regarding the PR issue, I agree with Chunwei. It is true that there is
> > > a
> > > > > big amount of pending PRs, but some of them are quite old (2 or even 3
> > > > > years old). Probably we should collectively spend some time to clean-up
> > > > our
> > > > > PR backlog, I'm pretty sure many of these PRs could be closed since
> > > they
> > > > > might no longer be relevant at this point. I know this is easier said
> > > > than
> > > > > done, but at some moment this is an effort that will have to be made.
> > > > >
> > > > > On the Avatica topic, personally speaking (but I guess other people
> > > might
> > > > > feel the same), this project is a bit of an "unknown", since I do not
> > > > work
> > > > > with it directly. Maybe a possible solution to try to "introduce"
> > > Avatica
> > > > > to the community and get more people involved could be via a talk in
> > > our
> > > > > coming online meetup.
> > > > >
> > > > > Finally, +1 on Haisheng being our next PMC chair.
> > > > >
> > > > > Best regards,
> > > > > Ruben
> > > > >
> > > > >
> > > > > On Tue, Nov 10, 2020 at 6:14 PM 953396112 <13...@qq.com> wrote:
> > > > >
> > > > > > Hi Stamatis,
> > > > > >
> > > > > > Thanks for your great work!
> > > > > >
> > > > > >
> > > > > > Calcite is very good at semantic transformation of relational
> > > algebra.
> > > > In
> > > > > > the process of the project, users can optimize relational algebra by
> > > > > > implementing RelOptRule or RelShuttle and so on, so as to bring
> > > > > relational
> > > > > > algebra into physical query engine. Another point is the construction
> > > > of
> > > > > > materialized view recognition framework, which realizes the ability
> > > of
> > > > > > materialized view recognition of common relational algebra. The
> > > > > expression
> > > > > > of relational algebra is very flexible, and it can do better in
> > > > > identifying
> > > > > > various materialized views.
> > > > > >
> > > > > >
> > > > > > +1 for voting Haisheng.
> > > > > >
> > > > > >
> > > > > > Best,
> > > > > > Zhaohui Xu
> > > > > >
> > > > > >
> > > > > > ------------------ 原始邮件 ------------------
> > > > > > 发件人:
> > > > > > "dev"
> > > > > > <
> > > > > > zabetak@gmail.com>;
> > > > > > 发送时间: 2020年11月5日(星期四) 上午6:26
> > > > > > 收件人: "dev"<de...@calcite.apache.org>;
> > > > > >
> > > > > > 主题: [DISCUSS] State of the project 2020
> > > > > >
> > > > > >
> > > > > >
> > > > > > Hi Calcite community members,
> > > > > >
> > > > > > A bit more than five years ago (October 22, 2015) Calcite graduated
> > > to
> > > > a
> > > > > > top-level Apache project[1]. At that time, the community decided to
> > > > have
> > > > > an
> > > > > > annual “state of the project” discussion and to vote for a new PMC
> > > > > > chair/VP[2]. So, I’m kicking off both of those discussions.
> > > > > >
> > > > > > I think it was an excellent year so far in many aspects.
> > > > > >
> > > > > > We were lucky to have many high quality contributions including:
> > > > notable
> > > > > > improvements in the Volcano planner (for speed, plan quality,
> > > > > > extensibility) bringing it a bit closer to Cascades and Columbia [6,
> > > 7,
> > > > > 8,
> > > > > > 9]; easier and more extensible parameterization of rules [3]; new
> > > > > dialects
> > > > > > such as ClickHouse [4], and Presto [5]; support for SQL hints [10];
> > > new
> > > > > > adapters for querying Redis [11] and InnoDB [12] through Calcite;
> > > > various
> > > > > > enhancements in streaming SQL. The previous list is by no means
> > > > > exhaustive.
> > > > > >
> > > > > > Apart from the new features, certainly worth mentioning is the
> > > > > > modernization of the build and test infrastructure (for both Calcite
> > > > and
> > > > > > Avatica), with the migration from maven to gradle, JUnit4 to JUnit5,
> > > > and
> > > > > > the introduction of GitHub actions as part of the CI.
> > > > > >
> > > > > > In terms of CI, I am happy to see a few more integration tests (IT)
> > > > > running
> > > > > > on a regular basis on GitHub. Eventually, it will be nice to have
> > > even
> > > > > more
> > > > > > IT tests to help us catch regressions early on and improve the
> > > quality
> > > > of
> > > > > > our releases.
> > > > > >
> > > > > > We wouldn’t have so many great contributions, if we didn’t also have
> > > > > > prolific contributors.
> > > > > > Our community has grown with Danny, Haisheng, Ruben, joining the PMC,
> > > > > > Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers, and
> > > > many
> > > > > > more people chiming in discussions, reviews, and submitting pull
> > > > > requests,
> > > > > > who are not yet committers but I’m sure some of whom will become in
> > > the
> > > > > > near future.
> > > > > >
> > > > > > We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica
> > > > release
> > > > > > (1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think that
> > > > is
> > > > > a
> > > > > > great tempo that we should strive to maintain in the years to come.
> > > One
> > > > > > thing to improve is the poor implication of other people than Francis
> > > > on
> > > > > > the Avatica side; the rest of us, putting myself first, should try to
> > > > be
> > > > > > more involved by reviewing PRs, preparing releases, voting etc.
> > > > > >
> > > > > > It was nice to see our community members giving talks to conferences
> > > > such
> > > > > > as ApacheCon, and Flink Forward presenting Calcite and/or its
> > > > > application.
> > > > > > Some of us have also done presentations in universities in order to
> > > > > > introduce Calcite to the next generation of computer engineers. One
> > > or
> > > > > two
> > > > > > conferences per year is a good number but it would be even better if
> > > we
> > > > > > could increase this frequency. There are still many people,
> > > especially
> > > > > > younger engineers, who are not aware of Calcite (at least this is the
> > > > > > impression that I get by speaking with people in Europe) and we
> > > should
> > > > be
> > > > > > more active on the project’s dissemination.
> > > > > >
> > > > > > Calcite is a very versatile library/framework that can be used in
> > > many
> > > > > > contexts. On one side, it is used in many production systems and
> > > > utility
> > > > > > apps with the most recent adopters being Hazelcast, Ignite, SuperSQL
> > > > > > (Tencent), and NeuroBlade. On the other side, its adoption in
> > > research
> > > > > > projects and teaching could be boosted. Every university has multiple
> > > > > > projects and courses around databases and data integration where
> > > > Calcite
> > > > > > could be a good fit.
> > > > > >
> > > > > > Over the past few years we always had problems with reviewing pull
> > > > > requests
> > > > > > and I don’t think we made much progress on this aspect. In our last
> > > > > > discussion around this topic, Julian suggested introducing some
> > > metrics
> > > > > and
> > > > > > giving credit to those people that are doing the most in this area,
> > > and
> > > > > we
> > > > > > all agreed to do so. Any ideas on improving this situation are highly
> > > > > > appreciated.
> > > > > >
> > > > > > Calcite is a vivid community and we are lucky to participate in many
> > > > > > fruitful discussions. Of course, in every community there is some
> > > > > friction
> > > > > > from time to time and the same goes for Calcite. It is a bit
> > > > unrealistic
> > > > > to
> > > > > > claim that we can eliminate it entirely but we can try to reduce it,
> > > by
> > > > > > being more attentive and patient.
> > > > > >
> > > > > > Being PMC chair was a big learning experience for me and I am very
> > > > > grateful
> > > > > > for the opportunity that was given to me. It is certainly among the
> > > > > things
> > > > > > that I am most proud of and I would like to thank everyone who
> > > trusted
> > > > > and
> > > > > > helped me in this role.
> > > > > >
> > > > > > Last but not least, we should discuss who should be the new PMC chair
> > > > of
> > > > > > Calcite after I step down in December. I would like to nominate
> > > > Haisheng
> > > > > > Yuan as the first candidate in the vote. Apart from many high quality
> > > > > > contributions, Haisheng has reviewed a big amount of PRs, and led
> > > many
> > > > > > technical discussions to consensus. Haisheng has been in the
> > > community
> > > > > for
> > > > > > a while and I believe he will be a great chair if he is willing to
> > > > > accept.
> > > > > >
> > > > > > To conclude, I will repeat the questions from previous years:
> > > > > >
> > > > > > 1) What else are we doing well in the project?
> > > > > > 2) What areas do we need to do better?
> > > > > > 3) Which other candidates should we consider for PMC chair?
> > > > > >
> > > > > > Please take some time to share your thoughts!
> > > > > >
> > > > > > Note that this discussion is for everyone; even if you have never
> > > sent
> > > > an
> > > > > > email to the list before now it is a good time to do so :)
> > > > > >
> > > > > > Best,
> > > > > > Stamatis
> > > > > >
> > > > > > [1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
> > > > > > [2]
> > > > > >
> > > > > >
> > > > >
> > > >
> > > http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E
> > > > > > [3] https://issues.apache.org/jira/browse/CALCITE-3923
> > > > > > [4] https://issues.apache.org/jira/browse/CALCITE-3724
> > > > > > [5] https://issues.apache.org/jira/browse/CALCITE-2157
> > > > > > [6] https://issues.apache.org/jira/browse/CALCITE-3916
> > > > > > [7] https://issues.apache.org/jira/browse/CALCITE-3896
> > > > > > [8] https://issues.apache.org/jira/browse/CALCITE-3753
> > > > > > [9] https://issues.apache.org/jira/browse/CALCITE-2970
> > > > > > [10] https://issues.apache.org/jira/browse/CALCITE-482
> > > > > > [11] https://issues.apache.org/jira/browse/CALCITE-3510
> > > > > > [12] https://issues.apache.org/jira/browse/CALCITE-4034
> > > > >
> > > >
> > >
>
Re: [DISCUSS] State of the project 2020
Posted by Julian Hyde <jh...@apache.org>.
I think Haisheng is a great choice for chair, if he is willing.
Thanks for serving as chair this year, Stamatis. You've done a great
job keeping the community civilized and on track.
Julian
On Sat, Nov 14, 2020 at 10:23 PM Forward Xu <fo...@gmail.com> wrote:
>
> Hi Stamatis,
>
> Thanks for your great work.
>
>
> Haisheng, Danny, and Ruben are all excellent. I believe they can do the
> work.
>
> This time I will vote for Haisheng first.
>
>
> Finally, +1 on Haisheng being our next PMC chair.
>
> Best,
> Forward
>
>
>
> Stamatis Zampetakis <za...@gmail.com> 于2020年11月15日周日 上午7:15写道：
>
> > Thanks to all those who shared their thoughts so far!
> >
> > Most of the suggestions around reviews and PRs sound promising. Summarizing
> > we have:
> > * Adding component owners for reviews;
> > * Cleaning up stale PRs;
> > * Freezing master till every JIRA marked for the next release is reviewed;
> >
> > Having component owners has been discussed in the past [1, 2] but
> > the general feeling was against.
> > I don't know if now things are different but we could definitely follow-up
> > in a separate discussion if people are interested.
> >
> > Stale PRs is an issue and becoming stale is what is more problematic. If we
> > don't review PRs in a timely manner then
> > there is a high chance that the contributor will lose interest and the PR
> > will never merge. In the past [3], we discussed the possibility of adding a
> > bot
> > for helping us in this task but the idea was abandoned in the end. If done
> > right, I think it can be useful. For instance, sending a reminder on our
> > dev list
> > if the PR does not have any activity for a certain time could help us maybe
> > prioritize this over others.
> >
> > One simple thing that we can do without too much discussion is active
> > committers to assign a number 5/10 PRs to themselves and we make sure
> > that they are resolved (merged or closed) for the next release.
> >
> > Regarding the role of the PMC chair, there were three names mentioned so
> > far: Haisheng, Danny, and Ruben. I believe in all of them!
> > Depending on the interest from their side and the feedback from the rest of
> > the community we will proceed to a vote.
> >
> > Best,
> > Stamatis
> >
> > [1]
> >
> > https://lists.apache.org/thread.html/3b1f43b15306b2f1496fd73bda01dcb3a1dba08747b1267e1b5f8187%40%3Cdev.calcite.apache.org%3E
> > [2]
> >
> > https://lists.apache.org/thread.html/7231a09fc9d1bad52c229664e8cb2a3631314b34464c2df10c594745%40%3Cdev.calcite.apache.org%3E
> > [3]
> >
> > https://lists.apache.org/thread.html/e794ec7f36246ec918a409b80759efc30d5433eea08ee564d4ba0c67%40%3Cdev.calcite.apache.org%3E
> >
> >
> > On Wed, Nov 11, 2020 at 10:19 PM Rui Wang <am...@apache.org> wrote:
> >
> > > Thanks Stamatis for serving as the PMC chair for Calcite in 2020!
> > >
> > > One of the highlights I want to give for the PR reviewing/merging area
> > was
> > > the practice we started to adopt in recent releases: we started to check
> > > each JIRA marked with the right release version and see whether there is
> > a
> > > PR in a good shape that can be merged. And we have built a contract that
> > > unless those promising PR are merged, we won't unlock the Calcite
> > master. I
> > > feel like this practice has proven to be useful (and of course thanks for
> > > release managers who have been working on this!).
> > >
> > >
> > > Finally, +1 on Haisheng being our next PMC chair.
> > >
> > >
> > > -Rui
> > >
> > > On Wed, Nov 11, 2020 at 1:57 AM Ruben Q L <ru...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > First of all, thanks Stamatis for this detailed "state of the project",
> > > and
> > > > thanks for your great work as PMC chair throughout this year, I take my
> > > hat
> > > > off.
> > > >
> > > > Regarding the PR issue, I agree with Chunwei. It is true that there is
> > a
> > > > big amount of pending PRs, but some of them are quite old (2 or even 3
> > > > years old). Probably we should collectively spend some time to clean-up
> > > our
> > > > PR backlog, I'm pretty sure many of these PRs could be closed since
> > they
> > > > might no longer be relevant at this point. I know this is easier said
> > > than
> > > > done, but at some moment this is an effort that will have to be made.
> > > >
> > > > On the Avatica topic, personally speaking (but I guess other people
> > might
> > > > feel the same), this project is a bit of an "unknown", since I do not
> > > work
> > > > with it directly. Maybe a possible solution to try to "introduce"
> > Avatica
> > > > to the community and get more people involved could be via a talk in
> > our
> > > > coming online meetup.
> > > >
> > > > Finally, +1 on Haisheng being our next PMC chair.
> > > >
> > > > Best regards,
> > > > Ruben
> > > >
> > > >
> > > > On Tue, Nov 10, 2020 at 6:14 PM 953396112 <13...@qq.com> wrote:
> > > >
> > > > > Hi Stamatis,
> > > > >
> > > > > Thanks for your great work!
> > > > >
> > > > >
> > > > > Calcite is very good at semantic transformation of relational
> > algebra.
> > > In
> > > > > the process of the project, users can optimize relational algebra by
> > > > > implementing RelOptRule or RelShuttle and so on, so as to bring
> > > > relational
> > > > > algebra into physical query engine. Another point is the construction
> > > of
> > > > > materialized view recognition framework, which realizes the ability
> > of
> > > > > materialized view recognition of common relational algebra. The
> > > > expression
> > > > > of relational algebra is very flexible, and it can do better in
> > > > identifying
> > > > > various materialized views.
> > > > >
> > > > >
> > > > > +1 for voting Haisheng.
> > > > >
> > > > >
> > > > > Best,
> > > > > Zhaohui Xu
> > > > >
> > > > >
> > > > > ------------------ 原始邮件 ------------------
> > > > > 发件人:
> > > > > "dev"
> > > > > <
> > > > > zabetak@gmail.com>;
> > > > > 发送时间: 2020年11月5日(星期四) 上午6:26
> > > > > 收件人: "dev"<dev@calcite.apache.org>;
> > > > >
> > > > > 主题: [DISCUSS] State of the project 2020
> > > > >
> > > > >
> > > > >
> > > > > Hi Calcite community members,
> > > > >
> > > > > A bit more than five years ago (October 22, 2015) Calcite graduated
> > to
> > > a
> > > > > top-level Apache project[1]. At that time, the community decided to
> > > have
> > > > an
> > > > > annual “state of the project” discussion and to vote for a new PMC
> > > > > chair/VP[2]. So, I’m kicking off both of those discussions.
> > > > >
> > > > > I think it was an excellent year so far in many aspects.
> > > > >
> > > > > We were lucky to have many high quality contributions including:
> > > notable
> > > > > improvements in the Volcano planner (for speed, plan quality,
> > > > > extensibility) bringing it a bit closer to Cascades and Columbia [6,
> > 7,
> > > > 8,
> > > > > 9]; easier and more extensible parameterization of rules [3]; new
> > > > dialects
> > > > > such as ClickHouse [4], and Presto [5]; support for SQL hints [10];
> > new
> > > > > adapters for querying Redis [11] and InnoDB [12] through Calcite;
> > > various
> > > > > enhancements in streaming SQL. The previous list is by no means
> > > > exhaustive.
> > > > >
> > > > > Apart from the new features, certainly worth mentioning is the
> > > > > modernization of the build and test infrastructure (for both Calcite
> > > and
> > > > > Avatica), with the migration from maven to gradle, JUnit4 to JUnit5,
> > > and
> > > > > the introduction of GitHub actions as part of the CI.
> > > > >
> > > > > In terms of CI, I am happy to see a few more integration tests (IT)
> > > > running
> > > > > on a regular basis on GitHub. Eventually, it will be nice to have
> > even
> > > > more
> > > > > IT tests to help us catch regressions early on and improve the
> > quality
> > > of
> > > > > our releases.
> > > > >
> > > > > We wouldn’t have so many great contributions, if we didn’t also have
> > > > > prolific contributors.
> > > > > Our community has grown with Danny, Haisheng, Ruben, joining the PMC,
> > > > > Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers, and
> > > many
> > > > > more people chiming in discussions, reviews, and submitting pull
> > > > requests,
> > > > > who are not yet committers but I’m sure some of whom will become in
> > the
> > > > > near future.
> > > > >
> > > > > We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica
> > > release
> > > > > (1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think that
> > > is
> > > > a
> > > > > great tempo that we should strive to maintain in the years to come.
> > One
> > > > > thing to improve is the poor implication of other people than Francis
> > > on
> > > > > the Avatica side; the rest of us, putting myself first, should try to
> > > be
> > > > > more involved by reviewing PRs, preparing releases, voting etc.
> > > > >
> > > > > It was nice to see our community members giving talks to conferences
> > > such
> > > > > as ApacheCon, and Flink Forward presenting Calcite and/or its
> > > > application.
> > > > > Some of us have also done presentations in universities in order to
> > > > > introduce Calcite to the next generation of computer engineers. One
> > or
> > > > two
> > > > > conferences per year is a good number but it would be even better if
> > we
> > > > > could increase this frequency. There are still many people,
> > especially
> > > > > younger engineers, who are not aware of Calcite (at least this is the
> > > > > impression that I get by speaking with people in Europe) and we
> > should
> > > be
> > > > > more active on the project’s dissemination.
> > > > >
> > > > > Calcite is a very versatile library/framework that can be used in
> > many
> > > > > contexts. On one side, it is used in many production systems and
> > > utility
> > > > > apps with the most recent adopters being Hazelcast, Ignite, SuperSQL
> > > > > (Tencent), and NeuroBlade. On the other side, its adoption in
> > research
> > > > > projects and teaching could be boosted. Every university has multiple
> > > > > projects and courses around databases and data integration where
> > > Calcite
> > > > > could be a good fit.
> > > > >
> > > > > Over the past few years we always had problems with reviewing pull
> > > > requests
> > > > > and I don’t think we made much progress on this aspect. In our last
> > > > > discussion around this topic, Julian suggested introducing some
> > metrics
> > > > and
> > > > > giving credit to those people that are doing the most in this area,
> > and
> > > > we
> > > > > all agreed to do so. Any ideas on improving this situation are highly
> > > > > appreciated.
> > > > >
> > > > > Calcite is a vivid community and we are lucky to participate in many
> > > > > fruitful discussions. Of course, in every community there is some
> > > > friction
> > > > > from time to time and the same goes for Calcite. It is a bit
> > > unrealistic
> > > > to
> > > > > claim that we can eliminate it entirely but we can try to reduce it,
> > by
> > > > > being more attentive and patient.
> > > > >
> > > > > Being PMC chair was a big learning experience for me and I am very
> > > > grateful
> > > > > for the opportunity that was given to me. It is certainly among the
> > > > things
> > > > > that I am most proud of and I would like to thank everyone who
> > trusted
> > > > and
> > > > > helped me in this role.
> > > > >
> > > > > Last but not least, we should discuss who should be the new PMC chair
> > > of
> > > > > Calcite after I step down in December. I would like to nominate
> > > Haisheng
> > > > > Yuan as the first candidate in the vote. Apart from many high quality
> > > > > contributions, Haisheng has reviewed a big amount of PRs, and led
> > many
> > > > > technical discussions to consensus. Haisheng has been in the
> > community
> > > > for
> > > > > a while and I believe he will be a great chair if he is willing to
> > > > accept.
> > > > >
> > > > > To conclude, I will repeat the questions from previous years:
> > > > >
> > > > > 1) What else are we doing well in the project?
> > > > > 2) What areas do we need to do better?
> > > > > 3) Which other candidates should we consider for PMC chair?
> > > > >
> > > > > Please take some time to share your thoughts!
> > > > >
> > > > > Note that this discussion is for everyone; even if you have never
> > sent
> > > an
> > > > > email to the list before now it is a good time to do so :)
> > > > >
> > > > > Best,
> > > > > Stamatis
> > > > >
> > > > > [1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
> > > > > [2]
> > > > >
> > > > >
> > > >
> > >
> > http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E
> > > > > [3] https://issues.apache.org/jira/browse/CALCITE-3923
> > > > > [4] https://issues.apache.org/jira/browse/CALCITE-3724
> > > > > [5] https://issues.apache.org/jira/browse/CALCITE-2157
> > > > > [6] https://issues.apache.org/jira/browse/CALCITE-3916
> > > > > [7] https://issues.apache.org/jira/browse/CALCITE-3896
> > > > > [8] https://issues.apache.org/jira/browse/CALCITE-3753
> > > > > [9] https://issues.apache.org/jira/browse/CALCITE-2970
> > > > > [10] https://issues.apache.org/jira/browse/CALCITE-482
> > > > > [11] https://issues.apache.org/jira/browse/CALCITE-3510
> > > > > [12] https://issues.apache.org/jira/browse/CALCITE-4034
> > > >
> > >
> >
Re: [DISCUSS] State of the project 2020
Posted by Forward Xu <fo...@gmail.com>.
Hi Stamatis,
Thanks for your great work.
Haisheng, Danny, and Ruben are all excellent. I believe they can do the
work.
This time I will vote for Haisheng first.
Finally, +1 on Haisheng being our next PMC chair.
Best,
Forward
Stamatis Zampetakis <za...@gmail.com> 于2020年11月15日周日 上午7:15写道：
> Thanks to all those who shared their thoughts so far!
>
> Most of the suggestions around reviews and PRs sound promising. Summarizing
> we have:
> * Adding component owners for reviews;
> * Cleaning up stale PRs;
> * Freezing master till every JIRA marked for the next release is reviewed;
>
> Having component owners has been discussed in the past [1, 2] but
> the general feeling was against.
> I don't know if now things are different but we could definitely follow-up
> in a separate discussion if people are interested.
>
> Stale PRs is an issue and becoming stale is what is more problematic. If we
> don't review PRs in a timely manner then
> there is a high chance that the contributor will lose interest and the PR
> will never merge. In the past [3], we discussed the possibility of adding a
> bot
> for helping us in this task but the idea was abandoned in the end. If done
> right, I think it can be useful. For instance, sending a reminder on our
> dev list
> if the PR does not have any activity for a certain time could help us maybe
> prioritize this over others.
>
> One simple thing that we can do without too much discussion is active
> committers to assign a number 5/10 PRs to themselves and we make sure
> that they are resolved (merged or closed) for the next release.
>
> Regarding the role of the PMC chair, there were three names mentioned so
> far: Haisheng, Danny, and Ruben. I believe in all of them!
> Depending on the interest from their side and the feedback from the rest of
> the community we will proceed to a vote.
>
> Best,
> Stamatis
>
> [1]
>
> https://lists.apache.org/thread.html/3b1f43b15306b2f1496fd73bda01dcb3a1dba08747b1267e1b5f8187%40%3Cdev.calcite.apache.org%3E
> [2]
>
> https://lists.apache.org/thread.html/7231a09fc9d1bad52c229664e8cb2a3631314b34464c2df10c594745%40%3Cdev.calcite.apache.org%3E
> [3]
>
> https://lists.apache.org/thread.html/e794ec7f36246ec918a409b80759efc30d5433eea08ee564d4ba0c67%40%3Cdev.calcite.apache.org%3E
>
>
> On Wed, Nov 11, 2020 at 10:19 PM Rui Wang <am...@apache.org> wrote:
>
> > Thanks Stamatis for serving as the PMC chair for Calcite in 2020!
> >
> > One of the highlights I want to give for the PR reviewing/merging area
> was
> > the practice we started to adopt in recent releases: we started to check
> > each JIRA marked with the right release version and see whether there is
> a
> > PR in a good shape that can be merged. And we have built a contract that
> > unless those promising PR are merged, we won't unlock the Calcite
> master. I
> > feel like this practice has proven to be useful (and of course thanks for
> > release managers who have been working on this!).
> >
> >
> > Finally, +1 on Haisheng being our next PMC chair.
> >
> >
> > -Rui
> >
> > On Wed, Nov 11, 2020 at 1:57 AM Ruben Q L <ru...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > First of all, thanks Stamatis for this detailed "state of the project",
> > and
> > > thanks for your great work as PMC chair throughout this year, I take my
> > hat
> > > off.
> > >
> > > Regarding the PR issue, I agree with Chunwei. It is true that there is
> a
> > > big amount of pending PRs, but some of them are quite old (2 or even 3
> > > years old). Probably we should collectively spend some time to clean-up
> > our
> > > PR backlog, I'm pretty sure many of these PRs could be closed since
> they
> > > might no longer be relevant at this point. I know this is easier said
> > than
> > > done, but at some moment this is an effort that will have to be made.
> > >
> > > On the Avatica topic, personally speaking (but I guess other people
> might
> > > feel the same), this project is a bit of an "unknown", since I do not
> > work
> > > with it directly. Maybe a possible solution to try to "introduce"
> Avatica
> > > to the community and get more people involved could be via a talk in
> our
> > > coming online meetup.
> > >
> > > Finally, +1 on Haisheng being our next PMC chair.
> > >
> > > Best regards,
> > > Ruben
> > >
> > >
> > > On Tue, Nov 10, 2020 at 6:14 PM 953396112 <13...@qq.com> wrote:
> > >
> > > > Hi Stamatis,
> > > >
> > > > Thanks for your great work!
> > > >
> > > >
> > > > Calcite is very good at semantic transformation of relational
> algebra.
> > In
> > > > the process of the project, users can optimize relational algebra by
> > > > implementing RelOptRule or RelShuttle and so on, so as to bring
> > > relational
> > > > algebra into physical query engine. Another point is the construction
> > of
> > > > materialized view recognition framework, which realizes the ability
> of
> > > > materialized view recognition of common relational algebra. The
> > > expression
> > > > of relational algebra is very flexible, and it can do better in
> > > identifying
> > > > various materialized views.
> > > >
> > > >
> > > > +1 for voting Haisheng.
> > > >
> > > >
> > > > Best,
> > > > Zhaohui Xu
> > > >
> > > >
> > > > ------------------ 原始邮件 ------------------
> > > > 发件人:
> > > > "dev"
> > > > <
> > > > zabetak@gmail.com>;
> > > > 发送时间: 2020年11月5日(星期四) 上午6:26
> > > > 收件人: "dev"<dev@calcite.apache.org>;
> > > >
> > > > 主题: [DISCUSS] State of the project 2020
> > > >
> > > >
> > > >
> > > > Hi Calcite community members,
> > > >
> > > > A bit more than five years ago (October 22, 2015) Calcite graduated
> to
> > a
> > > > top-level Apache project[1]. At that time, the community decided to
> > have
> > > an
> > > > annual “state of the project” discussion and to vote for a new PMC
> > > > chair/VP[2]. So, I’m kicking off both of those discussions.
> > > >
> > > > I think it was an excellent year so far in many aspects.
> > > >
> > > > We were lucky to have many high quality contributions including:
> > notable
> > > > improvements in the Volcano planner (for speed, plan quality,
> > > > extensibility) bringing it a bit closer to Cascades and Columbia [6,
> 7,
> > > 8,
> > > > 9]; easier and more extensible parameterization of rules [3]; new
> > > dialects
> > > > such as ClickHouse [4], and Presto [5]; support for SQL hints [10];
> new
> > > > adapters for querying Redis [11] and InnoDB [12] through Calcite;
> > various
> > > > enhancements in streaming SQL. The previous list is by no means
> > > exhaustive.
> > > >
> > > > Apart from the new features, certainly worth mentioning is the
> > > > modernization of the build and test infrastructure (for both Calcite
> > and
> > > > Avatica), with the migration from maven to gradle, JUnit4 to JUnit5,
> > and
> > > > the introduction of GitHub actions as part of the CI.
> > > >
> > > > In terms of CI, I am happy to see a few more integration tests (IT)
> > > running
> > > > on a regular basis on GitHub. Eventually, it will be nice to have
> even
> > > more
> > > > IT tests to help us catch regressions early on and improve the
> quality
> > of
> > > > our releases.
> > > >
> > > > We wouldn’t have so many great contributions, if we didn’t also have
> > > > prolific contributors.
> > > > Our community has grown with Danny, Haisheng, Ruben, joining the PMC,
> > > > Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers, and
> > many
> > > > more people chiming in discussions, reviews, and submitting pull
> > > requests,
> > > > who are not yet committers but I’m sure some of whom will become in
> the
> > > > near future.
> > > >
> > > > We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica
> > release
> > > > (1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think that
> > is
> > > a
> > > > great tempo that we should strive to maintain in the years to come.
> One
> > > > thing to improve is the poor implication of other people than Francis
> > on
> > > > the Avatica side; the rest of us, putting myself first, should try to
> > be
> > > > more involved by reviewing PRs, preparing releases, voting etc.
> > > >
> > > > It was nice to see our community members giving talks to conferences
> > such
> > > > as ApacheCon, and Flink Forward presenting Calcite and/or its
> > > application.
> > > > Some of us have also done presentations in universities in order to
> > > > introduce Calcite to the next generation of computer engineers. One
> or
> > > two
> > > > conferences per year is a good number but it would be even better if
> we
> > > > could increase this frequency. There are still many people,
> especially
> > > > younger engineers, who are not aware of Calcite (at least this is the
> > > > impression that I get by speaking with people in Europe) and we
> should
> > be
> > > > more active on the project’s dissemination.
> > > >
> > > > Calcite is a very versatile library/framework that can be used in
> many
> > > > contexts. On one side, it is used in many production systems and
> > utility
> > > > apps with the most recent adopters being Hazelcast, Ignite, SuperSQL
> > > > (Tencent), and NeuroBlade. On the other side, its adoption in
> research
> > > > projects and teaching could be boosted. Every university has multiple
> > > > projects and courses around databases and data integration where
> > Calcite
> > > > could be a good fit.
> > > >
> > > > Over the past few years we always had problems with reviewing pull
> > > requests
> > > > and I don’t think we made much progress on this aspect. In our last
> > > > discussion around this topic, Julian suggested introducing some
> metrics
> > > and
> > > > giving credit to those people that are doing the most in this area,
> and
> > > we
> > > > all agreed to do so. Any ideas on improving this situation are highly
> > > > appreciated.
> > > >
> > > > Calcite is a vivid community and we are lucky to participate in many
> > > > fruitful discussions. Of course, in every community there is some
> > > friction
> > > > from time to time and the same goes for Calcite. It is a bit
> > unrealistic
> > > to
> > > > claim that we can eliminate it entirely but we can try to reduce it,
> by
> > > > being more attentive and patient.
> > > >
> > > > Being PMC chair was a big learning experience for me and I am very
> > > grateful
> > > > for the opportunity that was given to me. It is certainly among the
> > > things
> > > > that I am most proud of and I would like to thank everyone who
> trusted
> > > and
> > > > helped me in this role.
> > > >
> > > > Last but not least, we should discuss who should be the new PMC chair
> > of
> > > > Calcite after I step down in December. I would like to nominate
> > Haisheng
> > > > Yuan as the first candidate in the vote. Apart from many high quality
> > > > contributions, Haisheng has reviewed a big amount of PRs, and led
> many
> > > > technical discussions to consensus. Haisheng has been in the
> community
> > > for
> > > > a while and I believe he will be a great chair if he is willing to
> > > accept.
> > > >
> > > > To conclude, I will repeat the questions from previous years:
> > > >
> > > > 1) What else are we doing well in the project?
> > > > 2) What areas do we need to do better?
> > > > 3) Which other candidates should we consider for PMC chair?
> > > >
> > > > Please take some time to share your thoughts!
> > > >
> > > > Note that this discussion is for everyone; even if you have never
> sent
> > an
> > > > email to the list before now it is a good time to do so :)
> > > >
> > > > Best,
> > > > Stamatis
> > > >
> > > > [1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
> > > > [2]
> > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E
> > > > [3] https://issues.apache.org/jira/browse/CALCITE-3923
> > > > [4] https://issues.apache.org/jira/browse/CALCITE-3724
> > > > [5] https://issues.apache.org/jira/browse/CALCITE-2157
> > > > [6] https://issues.apache.org/jira/browse/CALCITE-3916
> > > > [7] https://issues.apache.org/jira/browse/CALCITE-3896
> > > > [8] https://issues.apache.org/jira/browse/CALCITE-3753
> > > > [9] https://issues.apache.org/jira/browse/CALCITE-2970
> > > > [10] https://issues.apache.org/jira/browse/CALCITE-482
> > > > [11] https://issues.apache.org/jira/browse/CALCITE-3510
> > > > [12] https://issues.apache.org/jira/browse/CALCITE-4034
> > >
> >
>
Re: [DISCUSS] State of the project 2020
Posted by Danny Chan <da...@apache.org>.
Oops, i just saw the mail ~
Thanks Stamatis Zampetakis for the whole year nice work, you are always
warm and nice to work with.
I'm also +1 for Haisheng being the PMC chair, he is experienced in
optimizer and i believe he would take Calcite to a better direction in the
following year ~
Also thanks all the contributors to Calcite, Calcite becomes more and more
popular because of your active contributions ~
Stamatis Zampetakis <za...@gmail.com> 于2020年11月25日周三 上午7:19写道：
> It seems that there is consensus to appoint Haisheng as the new chair. I
> will start a vote soon!
>
> There were many good ideas throughout this thread about the future of
> Calcite.
> I encourage people to take initiatives and start separate threads in order
> to push them forward.
>
> Best,
> Stamatis
>
> On Sat, Nov 21, 2020 at 4:08 AM Michael Mior <mm...@apache.org> wrote:
>
> > Thanks Stamatis for serving as chair! I would also be happy to have
> > Haisheng for next year. I wonder if there are some additional
> > automated ways we can help with improving the likelihood of merging
> > tests. For example, auto commenting on PRs with suspected issues such
> > as missing tests, poor code style, etc. This would be possible to
> > implement with GitHub actions and could save some committer reviewing
> > time by allowing contributors to fix some issues before anyone looks
> > at it. This would certainly require some thought to be a help rather
> > than a hindrance but I think there's some potential there.
> >
> > --
> > Michael Mior
> > mmior@apache.org
> >
> > Le sam. 14 nov. 2020 à 18:15, Stamatis Zampetakis <za...@gmail.com> a
> > écrit :
> > >
> > > Thanks to all those who shared their thoughts so far!
> > >
> > > Most of the suggestions around reviews and PRs sound promising.
> > Summarizing
> > > we have:
> > > * Adding component owners for reviews;
> > > * Cleaning up stale PRs;
> > > * Freezing master till every JIRA marked for the next release is
> > reviewed;
> > >
> > > Having component owners has been discussed in the past [1, 2] but
> > > the general feeling was against.
> > > I don't know if now things are different but we could definitely
> > follow-up
> > > in a separate discussion if people are interested.
> > >
> > > Stale PRs is an issue and becoming stale is what is more problematic.
> If
> > we
> > > don't review PRs in a timely manner then
> > > there is a high chance that the contributor will lose interest and the
> PR
> > > will never merge. In the past [3], we discussed the possibility of
> > adding a
> > > bot
> > > for helping us in this task but the idea was abandoned in the end. If
> > done
> > > right, I think it can be useful. For instance, sending a reminder on
> our
> > > dev list
> > > if the PR does not have any activity for a certain time could help us
> > maybe
> > > prioritize this over others.
> > >
> > > One simple thing that we can do without too much discussion is active
> > > committers to assign a number 5/10 PRs to themselves and we make sure
> > > that they are resolved (merged or closed) for the next release.
> > >
> > > Regarding the role of the PMC chair, there were three names mentioned
> so
> > > far: Haisheng, Danny, and Ruben. I believe in all of them!
> > > Depending on the interest from their side and the feedback from the
> rest
> > of
> > > the community we will proceed to a vote.
> > >
> > > Best,
> > > Stamatis
> > >
> > > [1]
> > >
> >
> https://lists.apache.org/thread.html/3b1f43b15306b2f1496fd73bda01dcb3a1dba08747b1267e1b5f8187%40%3Cdev.calcite.apache.org%3E
> > > [2]
> > >
> >
> https://lists.apache.org/thread.html/7231a09fc9d1bad52c229664e8cb2a3631314b34464c2df10c594745%40%3Cdev.calcite.apache.org%3E
> > > [3]
> > >
> >
> https://lists.apache.org/thread.html/e794ec7f36246ec918a409b80759efc30d5433eea08ee564d4ba0c67%40%3Cdev.calcite.apache.org%3E
> > >
> > >
> > > On Wed, Nov 11, 2020 at 10:19 PM Rui Wang <am...@apache.org>
> wrote:
> > >
> > > > Thanks Stamatis for serving as the PMC chair for Calcite in 2020!
> > > >
> > > > One of the highlights I want to give for the PR reviewing/merging
> area
> > was
> > > > the practice we started to adopt in recent releases: we started to
> > check
> > > > each JIRA marked with the right release version and see whether there
> > is a
> > > > PR in a good shape that can be merged. And we have built a contract
> > that
> > > > unless those promising PR are merged, we won't unlock the Calcite
> > master. I
> > > > feel like this practice has proven to be useful (and of course thanks
> > for
> > > > release managers who have been working on this!).
> > > >
> > > >
> > > > Finally, +1 on Haisheng being our next PMC chair.
> > > >
> > > >
> > > > -Rui
> > > >
> > > > On Wed, Nov 11, 2020 at 1:57 AM Ruben Q L <ru...@gmail.com> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > First of all, thanks Stamatis for this detailed "state of the
> > project",
> > > > and
> > > > > thanks for your great work as PMC chair throughout this year, I
> take
> > my
> > > > hat
> > > > > off.
> > > > >
> > > > > Regarding the PR issue, I agree with Chunwei. It is true that there
> > is a
> > > > > big amount of pending PRs, but some of them are quite old (2 or
> even
> > 3
> > > > > years old). Probably we should collectively spend some time to
> > clean-up
> > > > our
> > > > > PR backlog, I'm pretty sure many of these PRs could be closed since
> > they
> > > > > might no longer be relevant at this point. I know this is easier
> said
> > > > than
> > > > > done, but at some moment this is an effort that will have to be
> made.
> > > > >
> > > > > On the Avatica topic, personally speaking (but I guess other people
> > might
> > > > > feel the same), this project is a bit of an "unknown", since I do
> not
> > > > work
> > > > > with it directly. Maybe a possible solution to try to "introduce"
> > Avatica
> > > > > to the community and get more people involved could be via a talk
> in
> > our
> > > > > coming online meetup.
> > > > >
> > > > > Finally, +1 on Haisheng being our next PMC chair.
> > > > >
> > > > > Best regards,
> > > > > Ruben
> > > > >
> > > > >
> > > > > On Tue, Nov 10, 2020 at 6:14 PM 953396112 <13...@qq.com>
> > wrote:
> > > > >
> > > > > > Hi Stamatis,
> > > > > >
> > > > > > Thanks for your great work!
> > > > > >
> > > > > >
> > > > > > Calcite is very good at semantic transformation of relational
> > algebra.
> > > > In
> > > > > > the process of the project, users can optimize relational algebra
> > by
> > > > > > implementing RelOptRule or RelShuttle and so on, so as to bring
> > > > > relational
> > > > > > algebra into physical query engine. Another point is the
> > construction
> > > > of
> > > > > > materialized view recognition framework, which realizes the
> > ability of
> > > > > > materialized view recognition of common relational algebra. The
> > > > > expression
> > > > > > of relational algebra is very flexible, and it can do better in
> > > > > identifying
> > > > > > various materialized views.
> > > > > >
> > > > > >
> > > > > > +1 for voting Haisheng.
> > > > > >
> > > > > >
> > > > > > Best,
> > > > > > Zhaohui Xu
> > > > > >
> > > > > >
> > > > > > ------------------ 原始邮件 ------------------
> > > > > > 发件人:
> > > > > > "dev"
> > > > > > <
> > > > > > zabetak@gmail.com>;
> > > > > > 发送时间: 2020年11月5日(星期四) 上午6:26
> > > > > > 收件人: "dev"<dev@calcite.apache.org>;
> > > > > >
> > > > > > 主题: [DISCUSS] State of the project 2020
> > > > > >
> > > > > >
> > > > > >
> > > > > > Hi Calcite community members,
> > > > > >
> > > > > > A bit more than five years ago (October 22, 2015) Calcite
> > graduated to
> > > > a
> > > > > > top-level Apache project[1]. At that time, the community decided
> to
> > > > have
> > > > > an
> > > > > > annual “state of the project” discussion and to vote for a new
> PMC
> > > > > > chair/VP[2]. So, I’m kicking off both of those discussions.
> > > > > >
> > > > > > I think it was an excellent year so far in many aspects.
> > > > > >
> > > > > > We were lucky to have many high quality contributions including:
> > > > notable
> > > > > > improvements in the Volcano planner (for speed, plan quality,
> > > > > > extensibility) bringing it a bit closer to Cascades and Columbia
> > [6, 7,
> > > > > 8,
> > > > > > 9]; easier and more extensible parameterization of rules [3]; new
> > > > > dialects
> > > > > > such as ClickHouse [4], and Presto [5]; support for SQL hints
> > [10]; new
> > > > > > adapters for querying Redis [11] and InnoDB [12] through Calcite;
> > > > various
> > > > > > enhancements in streaming SQL. The previous list is by no means
> > > > > exhaustive.
> > > > > >
> > > > > > Apart from the new features, certainly worth mentioning is the
> > > > > > modernization of the build and test infrastructure (for both
> > Calcite
> > > > and
> > > > > > Avatica), with the migration from maven to gradle, JUnit4 to
> > JUnit5,
> > > > and
> > > > > > the introduction of GitHub actions as part of the CI.
> > > > > >
> > > > > > In terms of CI, I am happy to see a few more integration tests
> (IT)
> > > > > running
> > > > > > on a regular basis on GitHub. Eventually, it will be nice to have
> > even
> > > > > more
> > > > > > IT tests to help us catch regressions early on and improve the
> > quality
> > > > of
> > > > > > our releases.
> > > > > >
> > > > > > We wouldn’t have so many great contributions, if we didn’t also
> > have
> > > > > > prolific contributors.
> > > > > > Our community has grown with Danny, Haisheng, Ruben, joining the
> > PMC,
> > > > > > Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers,
> > and
> > > > many
> > > > > > more people chiming in discussions, reviews, and submitting pull
> > > > > requests,
> > > > > > who are not yet committers but I’m sure some of whom will become
> > in the
> > > > > > near future.
> > > > > >
> > > > > > We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica
> > > > release
> > > > > > (1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think
> > that
> > > > is
> > > > > a
> > > > > > great tempo that we should strive to maintain in the years to
> > come. One
> > > > > > thing to improve is the poor implication of other people than
> > Francis
> > > > on
> > > > > > the Avatica side; the rest of us, putting myself first, should
> try
> > to
> > > > be
> > > > > > more involved by reviewing PRs, preparing releases, voting etc.
> > > > > >
> > > > > > It was nice to see our community members giving talks to
> > conferences
> > > > such
> > > > > > as ApacheCon, and Flink Forward presenting Calcite and/or its
> > > > > application.
> > > > > > Some of us have also done presentations in universities in order
> to
> > > > > > introduce Calcite to the next generation of computer engineers.
> > One or
> > > > > two
> > > > > > conferences per year is a good number but it would be even better
> > if we
> > > > > > could increase this frequency. There are still many people,
> > especially
> > > > > > younger engineers, who are not aware of Calcite (at least this is
> > the
> > > > > > impression that I get by speaking with people in Europe) and we
> > should
> > > > be
> > > > > > more active on the project’s dissemination.
> > > > > >
> > > > > > Calcite is a very versatile library/framework that can be used in
> > many
> > > > > > contexts. On one side, it is used in many production systems and
> > > > utility
> > > > > > apps with the most recent adopters being Hazelcast, Ignite,
> > SuperSQL
> > > > > > (Tencent), and NeuroBlade. On the other side, its adoption in
> > research
> > > > > > projects and teaching could be boosted. Every university has
> > multiple
> > > > > > projects and courses around databases and data integration where
> > > > Calcite
> > > > > > could be a good fit.
> > > > > >
> > > > > > Over the past few years we always had problems with reviewing
> pull
> > > > > requests
> > > > > > and I don’t think we made much progress on this aspect. In our
> last
> > > > > > discussion around this topic, Julian suggested introducing some
> > metrics
> > > > > and
> > > > > > giving credit to those people that are doing the most in this
> > area, and
> > > > > we
> > > > > > all agreed to do so. Any ideas on improving this situation are
> > highly
> > > > > > appreciated.
> > > > > >
> > > > > > Calcite is a vivid community and we are lucky to participate in
> > many
> > > > > > fruitful discussions. Of course, in every community there is some
> > > > > friction
> > > > > > from time to time and the same goes for Calcite. It is a bit
> > > > unrealistic
> > > > > to
> > > > > > claim that we can eliminate it entirely but we can try to reduce
> > it, by
> > > > > > being more attentive and patient.
> > > > > >
> > > > > > Being PMC chair was a big learning experience for me and I am
> very
> > > > > grateful
> > > > > > for the opportunity that was given to me. It is certainly among
> the
> > > > > things
> > > > > > that I am most proud of and I would like to thank everyone who
> > trusted
> > > > > and
> > > > > > helped me in this role.
> > > > > >
> > > > > > Last but not least, we should discuss who should be the new PMC
> > chair
> > > > of
> > > > > > Calcite after I step down in December. I would like to nominate
> > > > Haisheng
> > > > > > Yuan as the first candidate in the vote. Apart from many high
> > quality
> > > > > > contributions, Haisheng has reviewed a big amount of PRs, and led
> > many
> > > > > > technical discussions to consensus. Haisheng has been in the
> > community
> > > > > for
> > > > > > a while and I believe he will be a great chair if he is willing
> to
> > > > > accept.
> > > > > >
> > > > > > To conclude, I will repeat the questions from previous years:
> > > > > >
> > > > > > 1) What else are we doing well in the project?
> > > > > > 2) What areas do we need to do better?
> > > > > > 3) Which other candidates should we consider for PMC chair?
> > > > > >
> > > > > > Please take some time to share your thoughts!
> > > > > >
> > > > > > Note that this discussion is for everyone; even if you have never
> > sent
> > > > an
> > > > > > email to the list before now it is a good time to do so :)
> > > > > >
> > > > > > Best,
> > > > > > Stamatis
> > > > > >
> > > > > > [1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
> > > > > > [2]
> > > > > >
> > > > > >
> > > > >
> > > >
> >
> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E
> > > > > > [3] https://issues.apache.org/jira/browse/CALCITE-3923
> > > > > > [4] https://issues.apache.org/jira/browse/CALCITE-3724
> > > > > > [5] https://issues.apache.org/jira/browse/CALCITE-2157
> > > > > > [6] https://issues.apache.org/jira/browse/CALCITE-3916
> > > > > > [7] https://issues.apache.org/jira/browse/CALCITE-3896
> > > > > > [8] https://issues.apache.org/jira/browse/CALCITE-3753
> > > > > > [9] https://issues.apache.org/jira/browse/CALCITE-2970
> > > > > > [10] https://issues.apache.org/jira/browse/CALCITE-482
> > > > > > [11] https://issues.apache.org/jira/browse/CALCITE-3510
> > > > > > [12] https://issues.apache.org/jira/browse/CALCITE-4034
> > > > >
> > > >
> >
>
Re: [DISCUSS] State of the project 2020
Posted by Stamatis Zampetakis <za...@gmail.com>.
It seems that there is consensus to appoint Haisheng as the new chair. I
will start a vote soon!
There were many good ideas throughout this thread about the future of
Calcite.
I encourage people to take initiatives and start separate threads in order
to push them forward.
Best,
Stamatis
On Sat, Nov 21, 2020 at 4:08 AM Michael Mior <mm...@apache.org> wrote:
> Thanks Stamatis for serving as chair! I would also be happy to have
> Haisheng for next year. I wonder if there are some additional
> automated ways we can help with improving the likelihood of merging
> tests. For example, auto commenting on PRs with suspected issues such
> as missing tests, poor code style, etc. This would be possible to
> implement with GitHub actions and could save some committer reviewing
> time by allowing contributors to fix some issues before anyone looks
> at it. This would certainly require some thought to be a help rather
> than a hindrance but I think there's some potential there.
>
> --
> Michael Mior
> mmior@apache.org
>
> Le sam. 14 nov. 2020 à 18:15, Stamatis Zampetakis <za...@gmail.com> a
> écrit :
> >
> > Thanks to all those who shared their thoughts so far!
> >
> > Most of the suggestions around reviews and PRs sound promising.
> Summarizing
> > we have:
> > * Adding component owners for reviews;
> > * Cleaning up stale PRs;
> > * Freezing master till every JIRA marked for the next release is
> reviewed;
> >
> > Having component owners has been discussed in the past [1, 2] but
> > the general feeling was against.
> > I don't know if now things are different but we could definitely
> follow-up
> > in a separate discussion if people are interested.
> >
> > Stale PRs is an issue and becoming stale is what is more problematic. If
> we
> > don't review PRs in a timely manner then
> > there is a high chance that the contributor will lose interest and the PR
> > will never merge. In the past [3], we discussed the possibility of
> adding a
> > bot
> > for helping us in this task but the idea was abandoned in the end. If
> done
> > right, I think it can be useful. For instance, sending a reminder on our
> > dev list
> > if the PR does not have any activity for a certain time could help us
> maybe
> > prioritize this over others.
> >
> > One simple thing that we can do without too much discussion is active
> > committers to assign a number 5/10 PRs to themselves and we make sure
> > that they are resolved (merged or closed) for the next release.
> >
> > Regarding the role of the PMC chair, there were three names mentioned so
> > far: Haisheng, Danny, and Ruben. I believe in all of them!
> > Depending on the interest from their side and the feedback from the rest
> of
> > the community we will proceed to a vote.
> >
> > Best,
> > Stamatis
> >
> > [1]
> >
> https://lists.apache.org/thread.html/3b1f43b15306b2f1496fd73bda01dcb3a1dba08747b1267e1b5f8187%40%3Cdev.calcite.apache.org%3E
> > [2]
> >
> https://lists.apache.org/thread.html/7231a09fc9d1bad52c229664e8cb2a3631314b34464c2df10c594745%40%3Cdev.calcite.apache.org%3E
> > [3]
> >
> https://lists.apache.org/thread.html/e794ec7f36246ec918a409b80759efc30d5433eea08ee564d4ba0c67%40%3Cdev.calcite.apache.org%3E
> >
> >
> > On Wed, Nov 11, 2020 at 10:19 PM Rui Wang <am...@apache.org> wrote:
> >
> > > Thanks Stamatis for serving as the PMC chair for Calcite in 2020!
> > >
> > > One of the highlights I want to give for the PR reviewing/merging area
> was
> > > the practice we started to adopt in recent releases: we started to
> check
> > > each JIRA marked with the right release version and see whether there
> is a
> > > PR in a good shape that can be merged. And we have built a contract
> that
> > > unless those promising PR are merged, we won't unlock the Calcite
> master. I
> > > feel like this practice has proven to be useful (and of course thanks
> for
> > > release managers who have been working on this!).
> > >
> > >
> > > Finally, +1 on Haisheng being our next PMC chair.
> > >
> > >
> > > -Rui
> > >
> > > On Wed, Nov 11, 2020 at 1:57 AM Ruben Q L <ru...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > First of all, thanks Stamatis for this detailed "state of the
> project",
> > > and
> > > > thanks for your great work as PMC chair throughout this year, I take
> my
> > > hat
> > > > off.
> > > >
> > > > Regarding the PR issue, I agree with Chunwei. It is true that there
> is a
> > > > big amount of pending PRs, but some of them are quite old (2 or even
> 3
> > > > years old). Probably we should collectively spend some time to
> clean-up
> > > our
> > > > PR backlog, I'm pretty sure many of these PRs could be closed since
> they
> > > > might no longer be relevant at this point. I know this is easier said
> > > than
> > > > done, but at some moment this is an effort that will have to be made.
> > > >
> > > > On the Avatica topic, personally speaking (but I guess other people
> might
> > > > feel the same), this project is a bit of an "unknown", since I do not
> > > work
> > > > with it directly. Maybe a possible solution to try to "introduce"
> Avatica
> > > > to the community and get more people involved could be via a talk in
> our
> > > > coming online meetup.
> > > >
> > > > Finally, +1 on Haisheng being our next PMC chair.
> > > >
> > > > Best regards,
> > > > Ruben
> > > >
> > > >
> > > > On Tue, Nov 10, 2020 at 6:14 PM 953396112 <13...@qq.com>
> wrote:
> > > >
> > > > > Hi Stamatis,
> > > > >
> > > > > Thanks for your great work!
> > > > >
> > > > >
> > > > > Calcite is very good at semantic transformation of relational
> algebra.
> > > In
> > > > > the process of the project, users can optimize relational algebra
> by
> > > > > implementing RelOptRule or RelShuttle and so on, so as to bring
> > > > relational
> > > > > algebra into physical query engine. Another point is the
> construction
> > > of
> > > > > materialized view recognition framework, which realizes the
> ability of
> > > > > materialized view recognition of common relational algebra. The
> > > > expression
> > > > > of relational algebra is very flexible, and it can do better in
> > > > identifying
> > > > > various materialized views.
> > > > >
> > > > >
> > > > > +1 for voting Haisheng.
> > > > >
> > > > >
> > > > > Best,
> > > > > Zhaohui Xu
> > > > >
> > > > >
> > > > > ------------------ 原始邮件 ------------------
> > > > > 发件人:
> > > > > "dev"
> > > > > <
> > > > > zabetak@gmail.com>;
> > > > > 发送时间: 2020年11月5日(星期四) 上午6:26
> > > > > 收件人: "dev"<dev@calcite.apache.org>;
> > > > >
> > > > > 主题: [DISCUSS] State of the project 2020
> > > > >
> > > > >
> > > > >
> > > > > Hi Calcite community members,
> > > > >
> > > > > A bit more than five years ago (October 22, 2015) Calcite
> graduated to
> > > a
> > > > > top-level Apache project[1]. At that time, the community decided to
> > > have
> > > > an
> > > > > annual “state of the project” discussion and to vote for a new PMC
> > > > > chair/VP[2]. So, I’m kicking off both of those discussions.
> > > > >
> > > > > I think it was an excellent year so far in many aspects.
> > > > >
> > > > > We were lucky to have many high quality contributions including:
> > > notable
> > > > > improvements in the Volcano planner (for speed, plan quality,
> > > > > extensibility) bringing it a bit closer to Cascades and Columbia
> [6, 7,
> > > > 8,
> > > > > 9]; easier and more extensible parameterization of rules [3]; new
> > > > dialects
> > > > > such as ClickHouse [4], and Presto [5]; support for SQL hints
> [10]; new
> > > > > adapters for querying Redis [11] and InnoDB [12] through Calcite;
> > > various
> > > > > enhancements in streaming SQL. The previous list is by no means
> > > > exhaustive.
> > > > >
> > > > > Apart from the new features, certainly worth mentioning is the
> > > > > modernization of the build and test infrastructure (for both
> Calcite
> > > and
> > > > > Avatica), with the migration from maven to gradle, JUnit4 to
> JUnit5,
> > > and
> > > > > the introduction of GitHub actions as part of the CI.
> > > > >
> > > > > In terms of CI, I am happy to see a few more integration tests (IT)
> > > > running
> > > > > on a regular basis on GitHub. Eventually, it will be nice to have
> even
> > > > more
> > > > > IT tests to help us catch regressions early on and improve the
> quality
> > > of
> > > > > our releases.
> > > > >
> > > > > We wouldn’t have so many great contributions, if we didn’t also
> have
> > > > > prolific contributors.
> > > > > Our community has grown with Danny, Haisheng, Ruben, joining the
> PMC,
> > > > > Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers,
> and
> > > many
> > > > > more people chiming in discussions, reviews, and submitting pull
> > > > requests,
> > > > > who are not yet committers but I’m sure some of whom will become
> in the
> > > > > near future.
> > > > >
> > > > > We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica
> > > release
> > > > > (1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think
> that
> > > is
> > > > a
> > > > > great tempo that we should strive to maintain in the years to
> come. One
> > > > > thing to improve is the poor implication of other people than
> Francis
> > > on
> > > > > the Avatica side; the rest of us, putting myself first, should try
> to
> > > be
> > > > > more involved by reviewing PRs, preparing releases, voting etc.
> > > > >
> > > > > It was nice to see our community members giving talks to
> conferences
> > > such
> > > > > as ApacheCon, and Flink Forward presenting Calcite and/or its
> > > > application.
> > > > > Some of us have also done presentations in universities in order to
> > > > > introduce Calcite to the next generation of computer engineers.
> One or
> > > > two
> > > > > conferences per year is a good number but it would be even better
> if we
> > > > > could increase this frequency. There are still many people,
> especially
> > > > > younger engineers, who are not aware of Calcite (at least this is
> the
> > > > > impression that I get by speaking with people in Europe) and we
> should
> > > be
> > > > > more active on the project’s dissemination.
> > > > >
> > > > > Calcite is a very versatile library/framework that can be used in
> many
> > > > > contexts. On one side, it is used in many production systems and
> > > utility
> > > > > apps with the most recent adopters being Hazelcast, Ignite,
> SuperSQL
> > > > > (Tencent), and NeuroBlade. On the other side, its adoption in
> research
> > > > > projects and teaching could be boosted. Every university has
> multiple
> > > > > projects and courses around databases and data integration where
> > > Calcite
> > > > > could be a good fit.
> > > > >
> > > > > Over the past few years we always had problems with reviewing pull
> > > > requests
> > > > > and I don’t think we made much progress on this aspect. In our last
> > > > > discussion around this topic, Julian suggested introducing some
> metrics
> > > > and
> > > > > giving credit to those people that are doing the most in this
> area, and
> > > > we
> > > > > all agreed to do so. Any ideas on improving this situation are
> highly
> > > > > appreciated.
> > > > >
> > > > > Calcite is a vivid community and we are lucky to participate in
> many
> > > > > fruitful discussions. Of course, in every community there is some
> > > > friction
> > > > > from time to time and the same goes for Calcite. It is a bit
> > > unrealistic
> > > > to
> > > > > claim that we can eliminate it entirely but we can try to reduce
> it, by
> > > > > being more attentive and patient.
> > > > >
> > > > > Being PMC chair was a big learning experience for me and I am very
> > > > grateful
> > > > > for the opportunity that was given to me. It is certainly among the
> > > > things
> > > > > that I am most proud of and I would like to thank everyone who
> trusted
> > > > and
> > > > > helped me in this role.
> > > > >
> > > > > Last but not least, we should discuss who should be the new PMC
> chair
> > > of
> > > > > Calcite after I step down in December. I would like to nominate
> > > Haisheng
> > > > > Yuan as the first candidate in the vote. Apart from many high
> quality
> > > > > contributions, Haisheng has reviewed a big amount of PRs, and led
> many
> > > > > technical discussions to consensus. Haisheng has been in the
> community
> > > > for
> > > > > a while and I believe he will be a great chair if he is willing to
> > > > accept.
> > > > >
> > > > > To conclude, I will repeat the questions from previous years:
> > > > >
> > > > > 1) What else are we doing well in the project?
> > > > > 2) What areas do we need to do better?
> > > > > 3) Which other candidates should we consider for PMC chair?
> > > > >
> > > > > Please take some time to share your thoughts!
> > > > >
> > > > > Note that this discussion is for everyone; even if you have never
> sent
> > > an
> > > > > email to the list before now it is a good time to do so :)
> > > > >
> > > > > Best,
> > > > > Stamatis
> > > > >
> > > > > [1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
> > > > > [2]
> > > > >
> > > > >
> > > >
> > >
> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E
> > > > > [3] https://issues.apache.org/jira/browse/CALCITE-3923
> > > > > [4] https://issues.apache.org/jira/browse/CALCITE-3724
> > > > > [5] https://issues.apache.org/jira/browse/CALCITE-2157
> > > > > [6] https://issues.apache.org/jira/browse/CALCITE-3916
> > > > > [7] https://issues.apache.org/jira/browse/CALCITE-3896
> > > > > [8] https://issues.apache.org/jira/browse/CALCITE-3753
> > > > > [9] https://issues.apache.org/jira/browse/CALCITE-2970
> > > > > [10] https://issues.apache.org/jira/browse/CALCITE-482
> > > > > [11] https://issues.apache.org/jira/browse/CALCITE-3510
> > > > > [12] https://issues.apache.org/jira/browse/CALCITE-4034
> > > >
> > >
>
Re: [DISCUSS] State of the project 2020
Posted by Michael Mior <mm...@apache.org>.
Thanks Stamatis for serving as chair! I would also be happy to have
Haisheng for next year. I wonder if there are some additional
automated ways we can help with improving the likelihood of merging
tests. For example, auto commenting on PRs with suspected issues such
as missing tests, poor code style, etc. This would be possible to
implement with GitHub actions and could save some committer reviewing
time by allowing contributors to fix some issues before anyone looks
at it. This would certainly require some thought to be a help rather
than a hindrance but I think there's some potential there.
--
Michael Mior
mmior@apache.org
Le sam. 14 nov. 2020 à 18:15, Stamatis Zampetakis <za...@gmail.com> a écrit :
>
> Thanks to all those who shared their thoughts so far!
>
> Most of the suggestions around reviews and PRs sound promising. Summarizing
> we have:
> * Adding component owners for reviews;
> * Cleaning up stale PRs;
> * Freezing master till every JIRA marked for the next release is reviewed;
>
> Having component owners has been discussed in the past [1, 2] but
> the general feeling was against.
> I don't know if now things are different but we could definitely follow-up
> in a separate discussion if people are interested.
>
> Stale PRs is an issue and becoming stale is what is more problematic. If we
> don't review PRs in a timely manner then
> there is a high chance that the contributor will lose interest and the PR
> will never merge. In the past [3], we discussed the possibility of adding a
> bot
> for helping us in this task but the idea was abandoned in the end. If done
> right, I think it can be useful. For instance, sending a reminder on our
> dev list
> if the PR does not have any activity for a certain time could help us maybe
> prioritize this over others.
>
> One simple thing that we can do without too much discussion is active
> committers to assign a number 5/10 PRs to themselves and we make sure
> that they are resolved (merged or closed) for the next release.
>
> Regarding the role of the PMC chair, there were three names mentioned so
> far: Haisheng, Danny, and Ruben. I believe in all of them!
> Depending on the interest from their side and the feedback from the rest of
> the community we will proceed to a vote.
>
> Best,
> Stamatis
>
> [1]
> https://lists.apache.org/thread.html/3b1f43b15306b2f1496fd73bda01dcb3a1dba08747b1267e1b5f8187%40%3Cdev.calcite.apache.org%3E
> [2]
> https://lists.apache.org/thread.html/7231a09fc9d1bad52c229664e8cb2a3631314b34464c2df10c594745%40%3Cdev.calcite.apache.org%3E
> [3]
> https://lists.apache.org/thread.html/e794ec7f36246ec918a409b80759efc30d5433eea08ee564d4ba0c67%40%3Cdev.calcite.apache.org%3E
>
>
> On Wed, Nov 11, 2020 at 10:19 PM Rui Wang <am...@apache.org> wrote:
>
> > Thanks Stamatis for serving as the PMC chair for Calcite in 2020!
> >
> > One of the highlights I want to give for the PR reviewing/merging area was
> > the practice we started to adopt in recent releases: we started to check
> > each JIRA marked with the right release version and see whether there is a
> > PR in a good shape that can be merged. And we have built a contract that
> > unless those promising PR are merged, we won't unlock the Calcite master. I
> > feel like this practice has proven to be useful (and of course thanks for
> > release managers who have been working on this!).
> >
> >
> > Finally, +1 on Haisheng being our next PMC chair.
> >
> >
> > -Rui
> >
> > On Wed, Nov 11, 2020 at 1:57 AM Ruben Q L <ru...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > First of all, thanks Stamatis for this detailed "state of the project",
> > and
> > > thanks for your great work as PMC chair throughout this year, I take my
> > hat
> > > off.
> > >
> > > Regarding the PR issue, I agree with Chunwei. It is true that there is a
> > > big amount of pending PRs, but some of them are quite old (2 or even 3
> > > years old). Probably we should collectively spend some time to clean-up
> > our
> > > PR backlog, I'm pretty sure many of these PRs could be closed since they
> > > might no longer be relevant at this point. I know this is easier said
> > than
> > > done, but at some moment this is an effort that will have to be made.
> > >
> > > On the Avatica topic, personally speaking (but I guess other people might
> > > feel the same), this project is a bit of an "unknown", since I do not
> > work
> > > with it directly. Maybe a possible solution to try to "introduce" Avatica
> > > to the community and get more people involved could be via a talk in our
> > > coming online meetup.
> > >
> > > Finally, +1 on Haisheng being our next PMC chair.
> > >
> > > Best regards,
> > > Ruben
> > >
> > >
> > > On Tue, Nov 10, 2020 at 6:14 PM 953396112 <13...@qq.com> wrote:
> > >
> > > > Hi Stamatis,
> > > >
> > > > Thanks for your great work!
> > > >
> > > >
> > > > Calcite is very good at semantic transformation of relational algebra.
> > In
> > > > the process of the project, users can optimize relational algebra by
> > > > implementing RelOptRule or RelShuttle and so on, so as to bring
> > > relational
> > > > algebra into physical query engine. Another point is the construction
> > of
> > > > materialized view recognition framework, which realizes the ability of
> > > > materialized view recognition of common relational algebra. The
> > > expression
> > > > of relational algebra is very flexible, and it can do better in
> > > identifying
> > > > various materialized views.
> > > >
> > > >
> > > > +1 for voting Haisheng.
> > > >
> > > >
> > > > Best,
> > > > Zhaohui Xu
> > > >
> > > >
> > > > ------------------ 原始邮件 ------------------
> > > > 发件人:
> > > > "dev"
> > > > <
> > > > zabetak@gmail.com>;
> > > > 发送时间: 2020年11月5日(星期四) 上午6:26
> > > > 收件人: "dev"<dev@calcite.apache.org>;
> > > >
> > > > 主题: [DISCUSS] State of the project 2020
> > > >
> > > >
> > > >
> > > > Hi Calcite community members,
> > > >
> > > > A bit more than five years ago (October 22, 2015) Calcite graduated to
> > a
> > > > top-level Apache project[1]. At that time, the community decided to
> > have
> > > an
> > > > annual “state of the project” discussion and to vote for a new PMC
> > > > chair/VP[2]. So, I’m kicking off both of those discussions.
> > > >
> > > > I think it was an excellent year so far in many aspects.
> > > >
> > > > We were lucky to have many high quality contributions including:
> > notable
> > > > improvements in the Volcano planner (for speed, plan quality,
> > > > extensibility) bringing it a bit closer to Cascades and Columbia [6, 7,
> > > 8,
> > > > 9]; easier and more extensible parameterization of rules [3]; new
> > > dialects
> > > > such as ClickHouse [4], and Presto [5]; support for SQL hints [10]; new
> > > > adapters for querying Redis [11] and InnoDB [12] through Calcite;
> > various
> > > > enhancements in streaming SQL. The previous list is by no means
> > > exhaustive.
> > > >
> > > > Apart from the new features, certainly worth mentioning is the
> > > > modernization of the build and test infrastructure (for both Calcite
> > and
> > > > Avatica), with the migration from maven to gradle, JUnit4 to JUnit5,
> > and
> > > > the introduction of GitHub actions as part of the CI.
> > > >
> > > > In terms of CI, I am happy to see a few more integration tests (IT)
> > > running
> > > > on a regular basis on GitHub. Eventually, it will be nice to have even
> > > more
> > > > IT tests to help us catch regressions early on and improve the quality
> > of
> > > > our releases.
> > > >
> > > > We wouldn’t have so many great contributions, if we didn’t also have
> > > > prolific contributors.
> > > > Our community has grown with Danny, Haisheng, Ruben, joining the PMC,
> > > > Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers, and
> > many
> > > > more people chiming in discussions, reviews, and submitting pull
> > > requests,
> > > > who are not yet committers but I’m sure some of whom will become in the
> > > > near future.
> > > >
> > > > We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica
> > release
> > > > (1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think that
> > is
> > > a
> > > > great tempo that we should strive to maintain in the years to come. One
> > > > thing to improve is the poor implication of other people than Francis
> > on
> > > > the Avatica side; the rest of us, putting myself first, should try to
> > be
> > > > more involved by reviewing PRs, preparing releases, voting etc.
> > > >
> > > > It was nice to see our community members giving talks to conferences
> > such
> > > > as ApacheCon, and Flink Forward presenting Calcite and/or its
> > > application.
> > > > Some of us have also done presentations in universities in order to
> > > > introduce Calcite to the next generation of computer engineers. One or
> > > two
> > > > conferences per year is a good number but it would be even better if we
> > > > could increase this frequency. There are still many people, especially
> > > > younger engineers, who are not aware of Calcite (at least this is the
> > > > impression that I get by speaking with people in Europe) and we should
> > be
> > > > more active on the project’s dissemination.
> > > >
> > > > Calcite is a very versatile library/framework that can be used in many
> > > > contexts. On one side, it is used in many production systems and
> > utility
> > > > apps with the most recent adopters being Hazelcast, Ignite, SuperSQL
> > > > (Tencent), and NeuroBlade. On the other side, its adoption in research
> > > > projects and teaching could be boosted. Every university has multiple
> > > > projects and courses around databases and data integration where
> > Calcite
> > > > could be a good fit.
> > > >
> > > > Over the past few years we always had problems with reviewing pull
> > > requests
> > > > and I don’t think we made much progress on this aspect. In our last
> > > > discussion around this topic, Julian suggested introducing some metrics
> > > and
> > > > giving credit to those people that are doing the most in this area, and
> > > we
> > > > all agreed to do so. Any ideas on improving this situation are highly
> > > > appreciated.
> > > >
> > > > Calcite is a vivid community and we are lucky to participate in many
> > > > fruitful discussions. Of course, in every community there is some
> > > friction
> > > > from time to time and the same goes for Calcite. It is a bit
> > unrealistic
> > > to
> > > > claim that we can eliminate it entirely but we can try to reduce it, by
> > > > being more attentive and patient.
> > > >
> > > > Being PMC chair was a big learning experience for me and I am very
> > > grateful
> > > > for the opportunity that was given to me. It is certainly among the
> > > things
> > > > that I am most proud of and I would like to thank everyone who trusted
> > > and
> > > > helped me in this role.
> > > >
> > > > Last but not least, we should discuss who should be the new PMC chair
> > of
> > > > Calcite after I step down in December. I would like to nominate
> > Haisheng
> > > > Yuan as the first candidate in the vote. Apart from many high quality
> > > > contributions, Haisheng has reviewed a big amount of PRs, and led many
> > > > technical discussions to consensus. Haisheng has been in the community
> > > for
> > > > a while and I believe he will be a great chair if he is willing to
> > > accept.
> > > >
> > > > To conclude, I will repeat the questions from previous years:
> > > >
> > > > 1) What else are we doing well in the project?
> > > > 2) What areas do we need to do better?
> > > > 3) Which other candidates should we consider for PMC chair?
> > > >
> > > > Please take some time to share your thoughts!
> > > >
> > > > Note that this discussion is for everyone; even if you have never sent
> > an
> > > > email to the list before now it is a good time to do so :)
> > > >
> > > > Best,
> > > > Stamatis
> > > >
> > > > [1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
> > > > [2]
> > > >
> > > >
> > >
> > http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E
> > > > [3] https://issues.apache.org/jira/browse/CALCITE-3923
> > > > [4] https://issues.apache.org/jira/browse/CALCITE-3724
> > > > [5] https://issues.apache.org/jira/browse/CALCITE-2157
> > > > [6] https://issues.apache.org/jira/browse/CALCITE-3916
> > > > [7] https://issues.apache.org/jira/browse/CALCITE-3896
> > > > [8] https://issues.apache.org/jira/browse/CALCITE-3753
> > > > [9] https://issues.apache.org/jira/browse/CALCITE-2970
> > > > [10] https://issues.apache.org/jira/browse/CALCITE-482
> > > > [11] https://issues.apache.org/jira/browse/CALCITE-3510
> > > > [12] https://issues.apache.org/jira/browse/CALCITE-4034
> > >
> >
Re: [DISCUSS] State of the project 2020
Posted by Stamatis Zampetakis <za...@gmail.com>.
Thanks to all those who shared their thoughts so far!
Most of the suggestions around reviews and PRs sound promising. Summarizing
we have:
* Adding component owners for reviews;
* Cleaning up stale PRs;
* Freezing master till every JIRA marked for the next release is reviewed;
Having component owners has been discussed in the past [1, 2] but
the general feeling was against.
I don't know if now things are different but we could definitely follow-up
in a separate discussion if people are interested.
Stale PRs is an issue and becoming stale is what is more problematic. If we
don't review PRs in a timely manner then
there is a high chance that the contributor will lose interest and the PR
will never merge. In the past [3], we discussed the possibility of adding a
bot
for helping us in this task but the idea was abandoned in the end. If done
right, I think it can be useful. For instance, sending a reminder on our
dev list
if the PR does not have any activity for a certain time could help us maybe
prioritize this over others.
One simple thing that we can do without too much discussion is active
committers to assign a number 5/10 PRs to themselves and we make sure
that they are resolved (merged or closed) for the next release.
Regarding the role of the PMC chair, there were three names mentioned so
far: Haisheng, Danny, and Ruben. I believe in all of them!
Depending on the interest from their side and the feedback from the rest of
the community we will proceed to a vote.
Best,
Stamatis
[1]
https://lists.apache.org/thread.html/3b1f43b15306b2f1496fd73bda01dcb3a1dba08747b1267e1b5f8187%40%3Cdev.calcite.apache.org%3E
[2]
https://lists.apache.org/thread.html/7231a09fc9d1bad52c229664e8cb2a3631314b34464c2df10c594745%40%3Cdev.calcite.apache.org%3E
[3]
https://lists.apache.org/thread.html/e794ec7f36246ec918a409b80759efc30d5433eea08ee564d4ba0c67%40%3Cdev.calcite.apache.org%3E
On Wed, Nov 11, 2020 at 10:19 PM Rui Wang <am...@apache.org> wrote:
> Thanks Stamatis for serving as the PMC chair for Calcite in 2020!
>
> One of the highlights I want to give for the PR reviewing/merging area was
> the practice we started to adopt in recent releases: we started to check
> each JIRA marked with the right release version and see whether there is a
> PR in a good shape that can be merged. And we have built a contract that
> unless those promising PR are merged, we won't unlock the Calcite master. I
> feel like this practice has proven to be useful (and of course thanks for
> release managers who have been working on this!).
>
>
> Finally, +1 on Haisheng being our next PMC chair.
>
>
> -Rui
>
> On Wed, Nov 11, 2020 at 1:57 AM Ruben Q L <ru...@gmail.com> wrote:
>
> > Hello,
> >
> > First of all, thanks Stamatis for this detailed "state of the project",
> and
> > thanks for your great work as PMC chair throughout this year, I take my
> hat
> > off.
> >
> > Regarding the PR issue, I agree with Chunwei. It is true that there is a
> > big amount of pending PRs, but some of them are quite old (2 or even 3
> > years old). Probably we should collectively spend some time to clean-up
> our
> > PR backlog, I'm pretty sure many of these PRs could be closed since they
> > might no longer be relevant at this point. I know this is easier said
> than
> > done, but at some moment this is an effort that will have to be made.
> >
> > On the Avatica topic, personally speaking (but I guess other people might
> > feel the same), this project is a bit of an "unknown", since I do not
> work
> > with it directly. Maybe a possible solution to try to "introduce" Avatica
> > to the community and get more people involved could be via a talk in our
> > coming online meetup.
> >
> > Finally, +1 on Haisheng being our next PMC chair.
> >
> > Best regards,
> > Ruben
> >
> >
> > On Tue, Nov 10, 2020 at 6:14 PM 953396112 <13...@qq.com> wrote:
> >
> > > Hi Stamatis,
> > >
> > > Thanks for your great work!
> > >
> > >
> > > Calcite is very good at semantic transformation of relational algebra.
> In
> > > the process of the project, users can optimize relational algebra by
> > > implementing RelOptRule or RelShuttle and so on, so as to bring
> > relational
> > > algebra into physical query engine. Another point is the construction
> of
> > > materialized view recognition framework, which realizes the ability of
> > > materialized view recognition of common relational algebra. The
> > expression
> > > of relational algebra is very flexible, and it can do better in
> > identifying
> > > various materialized views.
> > >
> > >
> > > +1 for voting Haisheng.
> > >
> > >
> > > Best,
> > > Zhaohui Xu
> > >
> > >
> > > ------------------ 原始邮件 ------------------
> > > 发件人:
> > > "dev"
> > > <
> > > zabetak@gmail.com>;
> > > 发送时间: 2020年11月5日(星期四) 上午6:26
> > > 收件人: "dev"<dev@calcite.apache.org>;
> > >
> > > 主题: [DISCUSS] State of the project 2020
> > >
> > >
> > >
> > > Hi Calcite community members,
> > >
> > > A bit more than five years ago (October 22, 2015) Calcite graduated to
> a
> > > top-level Apache project[1]. At that time, the community decided to
> have
> > an
> > > annual “state of the project” discussion and to vote for a new PMC
> > > chair/VP[2]. So, I’m kicking off both of those discussions.
> > >
> > > I think it was an excellent year so far in many aspects.
> > >
> > > We were lucky to have many high quality contributions including:
> notable
> > > improvements in the Volcano planner (for speed, plan quality,
> > > extensibility) bringing it a bit closer to Cascades and Columbia [6, 7,
> > 8,
> > > 9]; easier and more extensible parameterization of rules [3]; new
> > dialects
> > > such as ClickHouse [4], and Presto [5]; support for SQL hints [10]; new
> > > adapters for querying Redis [11] and InnoDB [12] through Calcite;
> various
> > > enhancements in streaming SQL. The previous list is by no means
> > exhaustive.
> > >
> > > Apart from the new features, certainly worth mentioning is the
> > > modernization of the build and test infrastructure (for both Calcite
> and
> > > Avatica), with the migration from maven to gradle, JUnit4 to JUnit5,
> and
> > > the introduction of GitHub actions as part of the CI.
> > >
> > > In terms of CI, I am happy to see a few more integration tests (IT)
> > running
> > > on a regular basis on GitHub. Eventually, it will be nice to have even
> > more
> > > IT tests to help us catch regressions early on and improve the quality
> of
> > > our releases.
> > >
> > > We wouldn’t have so many great contributions, if we didn’t also have
> > > prolific contributors.
> > > Our community has grown with Danny, Haisheng, Ruben, joining the PMC,
> > > Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers, and
> many
> > > more people chiming in discussions, reviews, and submitting pull
> > requests,
> > > who are not yet committers but I’m sure some of whom will become in the
> > > near future.
> > >
> > > We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica
> release
> > > (1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think that
> is
> > a
> > > great tempo that we should strive to maintain in the years to come. One
> > > thing to improve is the poor implication of other people than Francis
> on
> > > the Avatica side; the rest of us, putting myself first, should try to
> be
> > > more involved by reviewing PRs, preparing releases, voting etc.
> > >
> > > It was nice to see our community members giving talks to conferences
> such
> > > as ApacheCon, and Flink Forward presenting Calcite and/or its
> > application.
> > > Some of us have also done presentations in universities in order to
> > > introduce Calcite to the next generation of computer engineers. One or
> > two
> > > conferences per year is a good number but it would be even better if we
> > > could increase this frequency. There are still many people, especially
> > > younger engineers, who are not aware of Calcite (at least this is the
> > > impression that I get by speaking with people in Europe) and we should
> be
> > > more active on the project’s dissemination.
> > >
> > > Calcite is a very versatile library/framework that can be used in many
> > > contexts. On one side, it is used in many production systems and
> utility
> > > apps with the most recent adopters being Hazelcast, Ignite, SuperSQL
> > > (Tencent), and NeuroBlade. On the other side, its adoption in research
> > > projects and teaching could be boosted. Every university has multiple
> > > projects and courses around databases and data integration where
> Calcite
> > > could be a good fit.
> > >
> > > Over the past few years we always had problems with reviewing pull
> > requests
> > > and I don’t think we made much progress on this aspect. In our last
> > > discussion around this topic, Julian suggested introducing some metrics
> > and
> > > giving credit to those people that are doing the most in this area, and
> > we
> > > all agreed to do so. Any ideas on improving this situation are highly
> > > appreciated.
> > >
> > > Calcite is a vivid community and we are lucky to participate in many
> > > fruitful discussions. Of course, in every community there is some
> > friction
> > > from time to time and the same goes for Calcite. It is a bit
> unrealistic
> > to
> > > claim that we can eliminate it entirely but we can try to reduce it, by
> > > being more attentive and patient.
> > >
> > > Being PMC chair was a big learning experience for me and I am very
> > grateful
> > > for the opportunity that was given to me. It is certainly among the
> > things
> > > that I am most proud of and I would like to thank everyone who trusted
> > and
> > > helped me in this role.
> > >
> > > Last but not least, we should discuss who should be the new PMC chair
> of
> > > Calcite after I step down in December. I would like to nominate
> Haisheng
> > > Yuan as the first candidate in the vote. Apart from many high quality
> > > contributions, Haisheng has reviewed a big amount of PRs, and led many
> > > technical discussions to consensus. Haisheng has been in the community
> > for
> > > a while and I believe he will be a great chair if he is willing to
> > accept.
> > >
> > > To conclude, I will repeat the questions from previous years:
> > >
> > > 1) What else are we doing well in the project?
> > > 2) What areas do we need to do better?
> > > 3) Which other candidates should we consider for PMC chair?
> > >
> > > Please take some time to share your thoughts!
> > >
> > > Note that this discussion is for everyone; even if you have never sent
> an
> > > email to the list before now it is a good time to do so :)
> > >
> > > Best,
> > > Stamatis
> > >
> > > [1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
> > > [2]
> > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E
> > > [3] https://issues.apache.org/jira/browse/CALCITE-3923
> > > [4] https://issues.apache.org/jira/browse/CALCITE-3724
> > > [5] https://issues.apache.org/jira/browse/CALCITE-2157
> > > [6] https://issues.apache.org/jira/browse/CALCITE-3916
> > > [7] https://issues.apache.org/jira/browse/CALCITE-3896
> > > [8] https://issues.apache.org/jira/browse/CALCITE-3753
> > > [9] https://issues.apache.org/jira/browse/CALCITE-2970
> > > [10] https://issues.apache.org/jira/browse/CALCITE-482
> > > [11] https://issues.apache.org/jira/browse/CALCITE-3510
> > > [12] https://issues.apache.org/jira/browse/CALCITE-4034
> >
>
Re: [DISCUSS] State of the project 2020
Posted by Rui Wang <am...@apache.org>.
Thanks Stamatis for serving as the PMC chair for Calcite in 2020!
One of the highlights I want to give for the PR reviewing/merging area was
the practice we started to adopt in recent releases: we started to check
each JIRA marked with the right release version and see whether there is a
PR in a good shape that can be merged. And we have built a contract that
unless those promising PR are merged, we won't unlock the Calcite master. I
feel like this practice has proven to be useful (and of course thanks for
release managers who have been working on this!).
Finally, +1 on Haisheng being our next PMC chair.
-Rui
On Wed, Nov 11, 2020 at 1:57 AM Ruben Q L <ru...@gmail.com> wrote:
> Hello,
>
> First of all, thanks Stamatis for this detailed "state of the project", and
> thanks for your great work as PMC chair throughout this year, I take my hat
> off.
>
> Regarding the PR issue, I agree with Chunwei. It is true that there is a
> big amount of pending PRs, but some of them are quite old (2 or even 3
> years old). Probably we should collectively spend some time to clean-up our
> PR backlog, I'm pretty sure many of these PRs could be closed since they
> might no longer be relevant at this point. I know this is easier said than
> done, but at some moment this is an effort that will have to be made.
>
> On the Avatica topic, personally speaking (but I guess other people might
> feel the same), this project is a bit of an "unknown", since I do not work
> with it directly. Maybe a possible solution to try to "introduce" Avatica
> to the community and get more people involved could be via a talk in our
> coming online meetup.
>
> Finally, +1 on Haisheng being our next PMC chair.
>
> Best regards,
> Ruben
>
>
> On Tue, Nov 10, 2020 at 6:14 PM 953396112 <13...@qq.com> wrote:
>
> > Hi Stamatis,
> >
> > Thanks for your great work!
> >
> >
> > Calcite is very good at semantic transformation of relational algebra. In
> > the process of the project, users can optimize relational algebra by
> > implementing RelOptRule or RelShuttle and so on, so as to bring
> relational
> > algebra into physical query engine. Another point is the construction of
> > materialized view recognition framework, which realizes the ability of
> > materialized view recognition of common relational algebra. The
> expression
> > of relational algebra is very flexible, and it can do better in
> identifying
> > various materialized views.
> >
> >
> > +1 for voting Haisheng.
> >
> >
> > Best,
> > Zhaohui Xu
> >
> >
> > ------------------ 原始邮件 ------------------
> > 发件人:
> > "dev"
> > <
> > zabetak@gmail.com>;
> > 发送时间: 2020年11月5日(星期四) 上午6:26
> > 收件人: "dev"<dev@calcite.apache.org>;
> >
> > 主题: [DISCUSS] State of the project 2020
> >
> >
> >
> > Hi Calcite community members,
> >
> > A bit more than five years ago (October 22, 2015) Calcite graduated to a
> > top-level Apache project[1]. At that time, the community decided to have
> an
> > annual “state of the project” discussion and to vote for a new PMC
> > chair/VP[2]. So, I’m kicking off both of those discussions.
> >
> > I think it was an excellent year so far in many aspects.
> >
> > We were lucky to have many high quality contributions including: notable
> > improvements in the Volcano planner (for speed, plan quality,
> > extensibility) bringing it a bit closer to Cascades and Columbia [6, 7,
> 8,
> > 9]; easier and more extensible parameterization of rules [3]; new
> dialects
> > such as ClickHouse [4], and Presto [5]; support for SQL hints [10]; new
> > adapters for querying Redis [11] and InnoDB [12] through Calcite; various
> > enhancements in streaming SQL. The previous list is by no means
> exhaustive.
> >
> > Apart from the new features, certainly worth mentioning is the
> > modernization of the build and test infrastructure (for both Calcite and
> > Avatica), with the migration from maven to gradle, JUnit4 to JUnit5, and
> > the introduction of GitHub actions as part of the CI.
> >
> > In terms of CI, I am happy to see a few more integration tests (IT)
> running
> > on a regular basis on GitHub. Eventually, it will be nice to have even
> more
> > IT tests to help us catch regressions early on and improve the quality of
> > our releases.
> >
> > We wouldn’t have so many great contributions, if we didn’t also have
> > prolific contributors.
> > Our community has grown with Danny, Haisheng, Ruben, joining the PMC,
> > Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers, and many
> > more people chiming in discussions, reviews, and submitting pull
> requests,
> > who are not yet committers but I’m sure some of whom will become in the
> > near future.
> >
> > We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica release
> > (1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think that is
> a
> > great tempo that we should strive to maintain in the years to come. One
> > thing to improve is the poor implication of other people than Francis on
> > the Avatica side; the rest of us, putting myself first, should try to be
> > more involved by reviewing PRs, preparing releases, voting etc.
> >
> > It was nice to see our community members giving talks to conferences such
> > as ApacheCon, and Flink Forward presenting Calcite and/or its
> application.
> > Some of us have also done presentations in universities in order to
> > introduce Calcite to the next generation of computer engineers. One or
> two
> > conferences per year is a good number but it would be even better if we
> > could increase this frequency. There are still many people, especially
> > younger engineers, who are not aware of Calcite (at least this is the
> > impression that I get by speaking with people in Europe) and we should be
> > more active on the project’s dissemination.
> >
> > Calcite is a very versatile library/framework that can be used in many
> > contexts. On one side, it is used in many production systems and utility
> > apps with the most recent adopters being Hazelcast, Ignite, SuperSQL
> > (Tencent), and NeuroBlade. On the other side, its adoption in research
> > projects and teaching could be boosted. Every university has multiple
> > projects and courses around databases and data integration where Calcite
> > could be a good fit.
> >
> > Over the past few years we always had problems with reviewing pull
> requests
> > and I don’t think we made much progress on this aspect. In our last
> > discussion around this topic, Julian suggested introducing some metrics
> and
> > giving credit to those people that are doing the most in this area, and
> we
> > all agreed to do so. Any ideas on improving this situation are highly
> > appreciated.
> >
> > Calcite is a vivid community and we are lucky to participate in many
> > fruitful discussions. Of course, in every community there is some
> friction
> > from time to time and the same goes for Calcite. It is a bit unrealistic
> to
> > claim that we can eliminate it entirely but we can try to reduce it, by
> > being more attentive and patient.
> >
> > Being PMC chair was a big learning experience for me and I am very
> grateful
> > for the opportunity that was given to me. It is certainly among the
> things
> > that I am most proud of and I would like to thank everyone who trusted
> and
> > helped me in this role.
> >
> > Last but not least, we should discuss who should be the new PMC chair of
> > Calcite after I step down in December. I would like to nominate Haisheng
> > Yuan as the first candidate in the vote. Apart from many high quality
> > contributions, Haisheng has reviewed a big amount of PRs, and led many
> > technical discussions to consensus. Haisheng has been in the community
> for
> > a while and I believe he will be a great chair if he is willing to
> accept.
> >
> > To conclude, I will repeat the questions from previous years:
> >
> > 1) What else are we doing well in the project?
> > 2) What areas do we need to do better?
> > 3) Which other candidates should we consider for PMC chair?
> >
> > Please take some time to share your thoughts!
> >
> > Note that this discussion is for everyone; even if you have never sent an
> > email to the list before now it is a good time to do so :)
> >
> > Best,
> > Stamatis
> >
> > [1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
> > [2]
> >
> >
> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E
> > [3] https://issues.apache.org/jira/browse/CALCITE-3923
> > [4] https://issues.apache.org/jira/browse/CALCITE-3724
> > [5] https://issues.apache.org/jira/browse/CALCITE-2157
> > [6] https://issues.apache.org/jira/browse/CALCITE-3916
> > [7] https://issues.apache.org/jira/browse/CALCITE-3896
> > [8] https://issues.apache.org/jira/browse/CALCITE-3753
> > [9] https://issues.apache.org/jira/browse/CALCITE-2970
> > [10] https://issues.apache.org/jira/browse/CALCITE-482
> > [11] https://issues.apache.org/jira/browse/CALCITE-3510
> > [12] https://issues.apache.org/jira/browse/CALCITE-4034
>
Re: [DISCUSS] State of the project 2020
Posted by Ruben Q L <ru...@gmail.com>.
Hello,
First of all, thanks Stamatis for this detailed "state of the project", and
thanks for your great work as PMC chair throughout this year, I take my hat
off.
Regarding the PR issue, I agree with Chunwei. It is true that there is a
big amount of pending PRs, but some of them are quite old (2 or even 3
years old). Probably we should collectively spend some time to clean-up our
PR backlog, I'm pretty sure many of these PRs could be closed since they
might no longer be relevant at this point. I know this is easier said than
done, but at some moment this is an effort that will have to be made.
On the Avatica topic, personally speaking (but I guess other people might
feel the same), this project is a bit of an "unknown", since I do not work
with it directly. Maybe a possible solution to try to "introduce" Avatica
to the community and get more people involved could be via a talk in our
coming online meetup.
Finally, +1 on Haisheng being our next PMC chair.
Best regards,
Ruben
On Tue, Nov 10, 2020 at 6:14 PM 953396112 <13...@qq.com> wrote:
> Hi Stamatis,
>
> Thanks for your great work!
>
>
> Calcite is very good at semantic transformation of relational algebra. In
> the process of the project, users can optimize relational algebra by
> implementing RelOptRule or RelShuttle and so on, so as to bring relational
> algebra into physical query engine. Another point is the construction of
> materialized view recognition framework, which realizes the ability of
> materialized view recognition of common relational algebra. The expression
> of relational algebra is very flexible, and it can do better in identifying
> various materialized views.
>
>
> +1 for voting Haisheng.
>
>
> Best,
> Zhaohui Xu
>
>
> ------------------ 原始邮件 ------------------
> 发件人:
> "dev"
> <
> zabetak@gmail.com>;
> 发送时间: 2020年11月5日(星期四) 上午6:26
> 收件人: "dev"<dev@calcite.apache.org>;
>
> 主题: [DISCUSS] State of the project 2020
>
>
>
> Hi Calcite community members,
>
> A bit more than five years ago (October 22, 2015) Calcite graduated to a
> top-level Apache project[1]. At that time, the community decided to have an
> annual “state of the project” discussion and to vote for a new PMC
> chair/VP[2]. So, I’m kicking off both of those discussions.
>
> I think it was an excellent year so far in many aspects.
>
> We were lucky to have many high quality contributions including: notable
> improvements in the Volcano planner (for speed, plan quality,
> extensibility) bringing it a bit closer to Cascades and Columbia [6, 7, 8,
> 9]; easier and more extensible parameterization of rules [3]; new dialects
> such as ClickHouse [4], and Presto [5]; support for SQL hints [10]; new
> adapters for querying Redis [11] and InnoDB [12] through Calcite; various
> enhancements in streaming SQL. The previous list is by no means exhaustive.
>
> Apart from the new features, certainly worth mentioning is the
> modernization of the build and test infrastructure (for both Calcite and
> Avatica), with the migration from maven to gradle, JUnit4 to JUnit5, and
> the introduction of GitHub actions as part of the CI.
>
> In terms of CI, I am happy to see a few more integration tests (IT) running
> on a regular basis on GitHub. Eventually, it will be nice to have even more
> IT tests to help us catch regressions early on and improve the quality of
> our releases.
>
> We wouldn’t have so many great contributions, if we didn’t also have
> prolific contributors.
> Our community has grown with Danny, Haisheng, Ruben, joining the PMC,
> Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers, and many
> more people chiming in discussions, reviews, and submitting pull requests,
> who are not yet committers but I’m sure some of whom will become in the
> near future.
>
> We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica release
> (1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think that is a
> great tempo that we should strive to maintain in the years to come. One
> thing to improve is the poor implication of other people than Francis on
> the Avatica side; the rest of us, putting myself first, should try to be
> more involved by reviewing PRs, preparing releases, voting etc.
>
> It was nice to see our community members giving talks to conferences such
> as ApacheCon, and Flink Forward presenting Calcite and/or its application.
> Some of us have also done presentations in universities in order to
> introduce Calcite to the next generation of computer engineers. One or two
> conferences per year is a good number but it would be even better if we
> could increase this frequency. There are still many people, especially
> younger engineers, who are not aware of Calcite (at least this is the
> impression that I get by speaking with people in Europe) and we should be
> more active on the project’s dissemination.
>
> Calcite is a very versatile library/framework that can be used in many
> contexts. On one side, it is used in many production systems and utility
> apps with the most recent adopters being Hazelcast, Ignite, SuperSQL
> (Tencent), and NeuroBlade. On the other side, its adoption in research
> projects and teaching could be boosted. Every university has multiple
> projects and courses around databases and data integration where Calcite
> could be a good fit.
>
> Over the past few years we always had problems with reviewing pull requests
> and I don’t think we made much progress on this aspect. In our last
> discussion around this topic, Julian suggested introducing some metrics and
> giving credit to those people that are doing the most in this area, and we
> all agreed to do so. Any ideas on improving this situation are highly
> appreciated.
>
> Calcite is a vivid community and we are lucky to participate in many
> fruitful discussions. Of course, in every community there is some friction
> from time to time and the same goes for Calcite. It is a bit unrealistic to
> claim that we can eliminate it entirely but we can try to reduce it, by
> being more attentive and patient.
>
> Being PMC chair was a big learning experience for me and I am very grateful
> for the opportunity that was given to me. It is certainly among the things
> that I am most proud of and I would like to thank everyone who trusted and
> helped me in this role.
>
> Last but not least, we should discuss who should be the new PMC chair of
> Calcite after I step down in December. I would like to nominate Haisheng
> Yuan as the first candidate in the vote. Apart from many high quality
> contributions, Haisheng has reviewed a big amount of PRs, and led many
> technical discussions to consensus. Haisheng has been in the community for
> a while and I believe he will be a great chair if he is willing to accept.
>
> To conclude, I will repeat the questions from previous years:
>
> 1) What else are we doing well in the project?
> 2) What areas do we need to do better?
> 3) Which other candidates should we consider for PMC chair?
>
> Please take some time to share your thoughts!
>
> Note that this discussion is for everyone; even if you have never sent an
> email to the list before now it is a good time to do so :)
>
> Best,
> Stamatis
>
> [1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
> [2]
>
> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E
> [3] https://issues.apache.org/jira/browse/CALCITE-3923
> [4] https://issues.apache.org/jira/browse/CALCITE-3724
> [5] https://issues.apache.org/jira/browse/CALCITE-2157
> [6] https://issues.apache.org/jira/browse/CALCITE-3916
> [7] https://issues.apache.org/jira/browse/CALCITE-3896
> [8] https://issues.apache.org/jira/browse/CALCITE-3753
> [9] https://issues.apache.org/jira/browse/CALCITE-2970
> [10] https://issues.apache.org/jira/browse/CALCITE-482
> [11] https://issues.apache.org/jira/browse/CALCITE-3510
> [12] https://issues.apache.org/jira/browse/CALCITE-4034