You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Alex Plehanov <pl...@gmail.com> on 2022/03/06 10:03:40 UTC
Re: [PROPOSAL] Release Calcite-based SQL engine as an experimental feature
Hello Igniters,
I've prepared the pull request [1] and have plans to merge it to the master
branch in about two weeks, if there is no objection.
[1]: https://github.com/apache/ignite/pull/9855
чт, 30 дек. 2021 г. в 13:43, Anton Vinogradov <av...@apache.org>:
> > it would be great to release a new SQL engine in 2.13 as an
> experimental feature.
> ++1
>
> On Thu, Dec 30, 2021 at 12:55 PM Alex Plehanov <pl...@gmail.com>
> wrote:
>
> > Andrey,
> >
> > > Is this [1] a full scope of the tickets that MUST be resolved before
> the
> > engine could be merged?
> > Yes, we must resolve at least these tickets before merging. If you see
> any
> > other release blockers fill free to attach them to this ticket.
> >
> > > I think we have to add instructions to the readme file on how to turn a
> > new SQL engine on.
> > Sure, I think it should be the part of documentation ticket.
> >
> > > Also, I don't like the module name "ignite-calcite", because Calcite is
> > an independent project.
> > Personally, I see no problems here (but it's discussable). We have a lot
> of
> > modules where the name is an independent project: "ignite-kafka",
> > "ignite-spring", "ignite-kubernetes", "ignite-log4j", "ignite-zookeeper",
> > etc.
> >
> > > So, would you mind renaming the module to e.g. "ignite-sql-engine" or
> > "ignite-sql"?
> > Module "ignite-indexing" also contains SQL engine, so names like
> > "ignite-sql-engine" or "ignite-sql" will be ambiguous.
> >
> > чт, 30 дек. 2021 г. в 13:54, Andrey Mashenkov <
> andrey.mashenkov@gmail.com
> > >:
> >
> > > Alex,
> > > it would be great to release a new SQL engine in 2.13 as an
> > > experimental feature.
> > >
> > > Is this [1] a full scope of the tickets that MUST be resolved before
> the
> > > engine could be merged?
> > > I think we have to add instructions to the readme file on how to turn a
> > new
> > > SQL engine on.
> > >
> > > Also, I don't like the module name "ignite-calcite", because Calcite is
> > an
> > > independent project.
> > > and Ignite just uses it.
> > > So, would you mind renaming the module to e.g. "ignite-sql-engine" or
> > > "ignite-sql"?
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-15436
> > >
> > > On Thu, Dec 30, 2021 at 11:10 AM Zhenya Stanilovsky
> > > <ar...@mail.ru.invalid> wrote:
> > >
> > > >
> > > > Alex, great !
> > > > If someone wants to touch codebase somehow plz use this branch [1]
> > > > Test passed can be found here [2] [3]
> > > >
> > > > [1]
> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite
> > > > [2]
> > > >
> > >
> >
> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite
> > > > [3]
> > > >
> > >
> >
> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/sql
> > > >
> > > > >
> > > > >>
> > > > >>>Hello, Igniters!
> > > > >>>
> > > > >>>As you may already know there is the new Ignite SQL engine based
> on
> > > > Apache
> > > > >>>Calcite currently under development.
> > > > >>>
> > > > >>>Reasons to move from H2-based engine and motivation for creating
> the
> > > new
> > > > >>>one in details described in IEP-37 [1].
> > > > >>>
> > > > >>>You can find all related to the new engine source code changes in
> > the
> > > > >>>"sql-calcite" branch [2].
> > > > >>>
> > > > >>>Calcite-based SQL engine is not production-ready yet and has a lot
> > of
> > > > known
> > > > >>>issues. In the future, the new engine should be fully independent
> of
> > > > >>>"ignite-indexing" and H2, but now it relies on schema management
> and
> > > > >>>indexes implemented in the "ignite-indexing" module and can't work
> > > > without
> > > > >>>the old engine. Despite all of the above mentioned, in the current
> > > > state,
> > > > >>>it has its own parsing, planning and execution flow and is almost
> as
> > > > >>>functional as the H2-based SQL engine.
> > > > >>>
> > > > >>>Some users are already interested in the Calcite-based engine and
> > > asking
> > > > >>>about the development status and release dates. Calcite-based SQL
> > > engine
> > > > >>>will be the only SQL engine in Ignite 3.0. Perhaps even in 2.x we
> > can
> > > > get
> > > > >>>rid of the H2-based engine at some time in the future. There is
> some
> > > > syntax
> > > > >>>difference between Calcite and H2 (Calcite is closer to SQL
> > standards
> > > > than
> > > > >>>H2) and a totally new execution flow. After the release of this
> > > feature,
> > > > >>>users can try their queries and determine if any adaptation for
> them
> > > is
> > > > >>>required. With the new planning and execution flow, perhaps, some
> > > > queries
> > > > >>>will be executed more effectively, users can redirect such queries
> > to
> > > > the
> > > > >>>new engine.
> > > > >>>
> > > > >>>I think we can provide an opportunity to users to try the new
> engine
> > > and
> > > > >>>release it as an experimental feature with the next Apache Ignite
> > > > version
> > > > >>>(2.13).
> > > > >>>
> > > > >>>What do you think?
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrey V. Mashenkov
> > >
> >
>
Re: [PROPOSAL] Release Calcite-based SQL engine as an experimental feature
Posted by Anton Vinogradov <av...@apache.org>.
Great job!
On Thu, Mar 24, 2022 at 6:27 PM Nikita Amelchev <na...@apache.org>
wrote:
> Great news!
>
> I will cut the 2.13 release branch soon.
>
> чт, 24 мар. 2022 г. в 18:07, Alex Plehanov <pl...@gmail.com>:
>
> >
> > Hello Igniters,
> >
> > I've merged the pull request. The Calcite-based SQL engine is in the
> master
> > branch now.
> > If you desire to try it, you can find configuration instructions in
> > modules/calcite/README.txt file.
> >
> > вс, 6 мар. 2022 г. в 13:03, Alex Plehanov <pl...@gmail.com>:
> >
> > > Hello Igniters,
> > >
> > > I've prepared the pull request [1] and have plans to merge it to the
> > > master branch in about two weeks, if there is no objection.
> > >
> > > [1]: https://github.com/apache/ignite/pull/9855
> > >
> > > чт, 30 дек. 2021 г. в 13:43, Anton Vinogradov <av...@apache.org>:
> > >
> > >> > it would be great to release a new SQL engine in 2.13 as an
> > >> experimental feature.
> > >> ++1
> > >>
> > >> On Thu, Dec 30, 2021 at 12:55 PM Alex Plehanov <
> plehanov.alex@gmail.com>
> > >> wrote:
> > >>
> > >> > Andrey,
> > >> >
> > >> > > Is this [1] a full scope of the tickets that MUST be resolved
> before
> > >> the
> > >> > engine could be merged?
> > >> > Yes, we must resolve at least these tickets before merging. If you
> see
> > >> any
> > >> > other release blockers fill free to attach them to this ticket.
> > >> >
> > >> > > I think we have to add instructions to the readme file on how to
> turn
> > >> a
> > >> > new SQL engine on.
> > >> > Sure, I think it should be the part of documentation ticket.
> > >> >
> > >> > > Also, I don't like the module name "ignite-calcite", because
> Calcite
> > >> is
> > >> > an independent project.
> > >> > Personally, I see no problems here (but it's discussable). We have a
> > >> lot of
> > >> > modules where the name is an independent project: "ignite-kafka",
> > >> > "ignite-spring", "ignite-kubernetes", "ignite-log4j",
> > >> "ignite-zookeeper",
> > >> > etc.
> > >> >
> > >> > > So, would you mind renaming the module to e.g.
> "ignite-sql-engine" or
> > >> > "ignite-sql"?
> > >> > Module "ignite-indexing" also contains SQL engine, so names like
> > >> > "ignite-sql-engine" or "ignite-sql" will be ambiguous.
> > >> >
> > >> > чт, 30 дек. 2021 г. в 13:54, Andrey Mashenkov <
> > >> andrey.mashenkov@gmail.com
> > >> > >:
> > >> >
> > >> > > Alex,
> > >> > > it would be great to release a new SQL engine in 2.13 as an
> > >> > > experimental feature.
> > >> > >
> > >> > > Is this [1] a full scope of the tickets that MUST be resolved
> before
> > >> the
> > >> > > engine could be merged?
> > >> > > I think we have to add instructions to the readme file on how to
> turn
> > >> a
> > >> > new
> > >> > > SQL engine on.
> > >> > >
> > >> > > Also, I don't like the module name "ignite-calcite", because
> Calcite
> > >> is
> > >> > an
> > >> > > independent project.
> > >> > > and Ignite just uses it.
> > >> > > So, would you mind renaming the module to e.g.
> "ignite-sql-engine" or
> > >> > > "ignite-sql"?
> > >> > >
> > >> > > [1] https://issues.apache.org/jira/browse/IGNITE-15436
> > >> > >
> > >> > > On Thu, Dec 30, 2021 at 11:10 AM Zhenya Stanilovsky
> > >> > > <ar...@mail.ru.invalid> wrote:
> > >> > >
> > >> > > >
> > >> > > > Alex, great !
> > >> > > > If someone wants to touch codebase somehow plz use this branch
> [1]
> > >> > > > Test passed can be found here [2] [3]
> > >> > > >
> > >> > > > [1]
> > >> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite
> > >> > > > [2]
> > >> > > >
> > >> > >
> > >> >
> > >>
> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite
> > >> > > > [3]
> > >> > > >
> > >> > >
> > >> >
> > >>
> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/sql
> > >> > > >
> > >> > > > >
> > >> > > > >>
> > >> > > > >>>Hello, Igniters!
> > >> > > > >>>
> > >> > > > >>>As you may already know there is the new Ignite SQL engine
> based
> > >> on
> > >> > > > Apache
> > >> > > > >>>Calcite currently under development.
> > >> > > > >>>
> > >> > > > >>>Reasons to move from H2-based engine and motivation for
> creating
> > >> the
> > >> > > new
> > >> > > > >>>one in details described in IEP-37 [1].
> > >> > > > >>>
> > >> > > > >>>You can find all related to the new engine source code
> changes in
> > >> > the
> > >> > > > >>>"sql-calcite" branch [2].
> > >> > > > >>>
> > >> > > > >>>Calcite-based SQL engine is not production-ready yet and has
> a
> > >> lot
> > >> > of
> > >> > > > known
> > >> > > > >>>issues. In the future, the new engine should be fully
> > >> independent of
> > >> > > > >>>"ignite-indexing" and H2, but now it relies on schema
> management
> > >> and
> > >> > > > >>>indexes implemented in the "ignite-indexing" module and can't
> > >> work
> > >> > > > without
> > >> > > > >>>the old engine. Despite all of the above mentioned, in the
> > >> current
> > >> > > > state,
> > >> > > > >>>it has its own parsing, planning and execution flow and is
> > >> almost as
> > >> > > > >>>functional as the H2-based SQL engine.
> > >> > > > >>>
> > >> > > > >>>Some users are already interested in the Calcite-based
> engine and
> > >> > > asking
> > >> > > > >>>about the development status and release dates.
> Calcite-based SQL
> > >> > > engine
> > >> > > > >>>will be the only SQL engine in Ignite 3.0. Perhaps even in
> 2.x we
> > >> > can
> > >> > > > get
> > >> > > > >>>rid of the H2-based engine at some time in the future. There
> is
> > >> some
> > >> > > > syntax
> > >> > > > >>>difference between Calcite and H2 (Calcite is closer to SQL
> > >> > standards
> > >> > > > than
> > >> > > > >>>H2) and a totally new execution flow. After the release of
> this
> > >> > > feature,
> > >> > > > >>>users can try their queries and determine if any adaptation
> for
> > >> them
> > >> > > is
> > >> > > > >>>required. With the new planning and execution flow, perhaps,
> some
> > >> > > > queries
> > >> > > > >>>will be executed more effectively, users can redirect such
> > >> queries
> > >> > to
> > >> > > > the
> > >> > > > >>>new engine.
> > >> > > > >>>
> > >> > > > >>>I think we can provide an opportunity to users to try the new
> > >> engine
> > >> > > and
> > >> > > > >>>release it as an experimental feature with the next Apache
> Ignite
> > >> > > > version
> > >> > > > >>>(2.13).
> > >> > > > >>>
> > >> > > > >>>What do you think?
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Best regards,
> > >> > > Andrey V. Mashenkov
> > >> > >
> > >> >
> > >>
> > >
>
>
>
> --
> Best wishes,
> Amelchev Nikita
>
Re: [PROPOSAL] Release Calcite-based SQL engine as an experimental feature
Posted by Nikita Amelchev <na...@apache.org>.
Great news!
I will cut the 2.13 release branch soon.
чт, 24 мар. 2022 г. в 18:07, Alex Plehanov <pl...@gmail.com>:
>
> Hello Igniters,
>
> I've merged the pull request. The Calcite-based SQL engine is in the master
> branch now.
> If you desire to try it, you can find configuration instructions in
> modules/calcite/README.txt file.
>
> вс, 6 мар. 2022 г. в 13:03, Alex Plehanov <pl...@gmail.com>:
>
> > Hello Igniters,
> >
> > I've prepared the pull request [1] and have plans to merge it to the
> > master branch in about two weeks, if there is no objection.
> >
> > [1]: https://github.com/apache/ignite/pull/9855
> >
> > чт, 30 дек. 2021 г. в 13:43, Anton Vinogradov <av...@apache.org>:
> >
> >> > it would be great to release a new SQL engine in 2.13 as an
> >> experimental feature.
> >> ++1
> >>
> >> On Thu, Dec 30, 2021 at 12:55 PM Alex Plehanov <pl...@gmail.com>
> >> wrote:
> >>
> >> > Andrey,
> >> >
> >> > > Is this [1] a full scope of the tickets that MUST be resolved before
> >> the
> >> > engine could be merged?
> >> > Yes, we must resolve at least these tickets before merging. If you see
> >> any
> >> > other release blockers fill free to attach them to this ticket.
> >> >
> >> > > I think we have to add instructions to the readme file on how to turn
> >> a
> >> > new SQL engine on.
> >> > Sure, I think it should be the part of documentation ticket.
> >> >
> >> > > Also, I don't like the module name "ignite-calcite", because Calcite
> >> is
> >> > an independent project.
> >> > Personally, I see no problems here (but it's discussable). We have a
> >> lot of
> >> > modules where the name is an independent project: "ignite-kafka",
> >> > "ignite-spring", "ignite-kubernetes", "ignite-log4j",
> >> "ignite-zookeeper",
> >> > etc.
> >> >
> >> > > So, would you mind renaming the module to e.g. "ignite-sql-engine" or
> >> > "ignite-sql"?
> >> > Module "ignite-indexing" also contains SQL engine, so names like
> >> > "ignite-sql-engine" or "ignite-sql" will be ambiguous.
> >> >
> >> > чт, 30 дек. 2021 г. в 13:54, Andrey Mashenkov <
> >> andrey.mashenkov@gmail.com
> >> > >:
> >> >
> >> > > Alex,
> >> > > it would be great to release a new SQL engine in 2.13 as an
> >> > > experimental feature.
> >> > >
> >> > > Is this [1] a full scope of the tickets that MUST be resolved before
> >> the
> >> > > engine could be merged?
> >> > > I think we have to add instructions to the readme file on how to turn
> >> a
> >> > new
> >> > > SQL engine on.
> >> > >
> >> > > Also, I don't like the module name "ignite-calcite", because Calcite
> >> is
> >> > an
> >> > > independent project.
> >> > > and Ignite just uses it.
> >> > > So, would you mind renaming the module to e.g. "ignite-sql-engine" or
> >> > > "ignite-sql"?
> >> > >
> >> > > [1] https://issues.apache.org/jira/browse/IGNITE-15436
> >> > >
> >> > > On Thu, Dec 30, 2021 at 11:10 AM Zhenya Stanilovsky
> >> > > <ar...@mail.ru.invalid> wrote:
> >> > >
> >> > > >
> >> > > > Alex, great !
> >> > > > If someone wants to touch codebase somehow plz use this branch [1]
> >> > > > Test passed can be found here [2] [3]
> >> > > >
> >> > > > [1]
> >> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite
> >> > > > [2]
> >> > > >
> >> > >
> >> >
> >> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite
> >> > > > [3]
> >> > > >
> >> > >
> >> >
> >> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/sql
> >> > > >
> >> > > > >
> >> > > > >>
> >> > > > >>>Hello, Igniters!
> >> > > > >>>
> >> > > > >>>As you may already know there is the new Ignite SQL engine based
> >> on
> >> > > > Apache
> >> > > > >>>Calcite currently under development.
> >> > > > >>>
> >> > > > >>>Reasons to move from H2-based engine and motivation for creating
> >> the
> >> > > new
> >> > > > >>>one in details described in IEP-37 [1].
> >> > > > >>>
> >> > > > >>>You can find all related to the new engine source code changes in
> >> > the
> >> > > > >>>"sql-calcite" branch [2].
> >> > > > >>>
> >> > > > >>>Calcite-based SQL engine is not production-ready yet and has a
> >> lot
> >> > of
> >> > > > known
> >> > > > >>>issues. In the future, the new engine should be fully
> >> independent of
> >> > > > >>>"ignite-indexing" and H2, but now it relies on schema management
> >> and
> >> > > > >>>indexes implemented in the "ignite-indexing" module and can't
> >> work
> >> > > > without
> >> > > > >>>the old engine. Despite all of the above mentioned, in the
> >> current
> >> > > > state,
> >> > > > >>>it has its own parsing, planning and execution flow and is
> >> almost as
> >> > > > >>>functional as the H2-based SQL engine.
> >> > > > >>>
> >> > > > >>>Some users are already interested in the Calcite-based engine and
> >> > > asking
> >> > > > >>>about the development status and release dates. Calcite-based SQL
> >> > > engine
> >> > > > >>>will be the only SQL engine in Ignite 3.0. Perhaps even in 2.x we
> >> > can
> >> > > > get
> >> > > > >>>rid of the H2-based engine at some time in the future. There is
> >> some
> >> > > > syntax
> >> > > > >>>difference between Calcite and H2 (Calcite is closer to SQL
> >> > standards
> >> > > > than
> >> > > > >>>H2) and a totally new execution flow. After the release of this
> >> > > feature,
> >> > > > >>>users can try their queries and determine if any adaptation for
> >> them
> >> > > is
> >> > > > >>>required. With the new planning and execution flow, perhaps, some
> >> > > > queries
> >> > > > >>>will be executed more effectively, users can redirect such
> >> queries
> >> > to
> >> > > > the
> >> > > > >>>new engine.
> >> > > > >>>
> >> > > > >>>I think we can provide an opportunity to users to try the new
> >> engine
> >> > > and
> >> > > > >>>release it as an experimental feature with the next Apache Ignite
> >> > > > version
> >> > > > >>>(2.13).
> >> > > > >>>
> >> > > > >>>What do you think?
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Best regards,
> >> > > Andrey V. Mashenkov
> >> > >
> >> >
> >>
> >
--
Best wishes,
Amelchev Nikita
Re: [PROPOSAL] Release Calcite-based SQL engine as an experimental feature
Posted by Konstantin Orlov <ko...@gridgain.com>.
This is great news!
Alex, thanks for making it to the end!
--
Regards,
Konstantin Orlov
> On 24 Mar 2022, at 18:07, Alex Plehanov <pl...@gmail.com> wrote:
>
> Hello Igniters,
>
> I've merged the pull request. The Calcite-based SQL engine is in the master
> branch now.
> If you desire to try it, you can find configuration instructions in
> modules/calcite/README.txt file.
>
> вс, 6 мар. 2022 г. в 13:03, Alex Plehanov <pl...@gmail.com>:
>
>> Hello Igniters,
>>
>> I've prepared the pull request [1] and have plans to merge it to the
>> master branch in about two weeks, if there is no objection.
>>
>> [1]: https://github.com/apache/ignite/pull/9855
>>
>> чт, 30 дек. 2021 г. в 13:43, Anton Vinogradov <av...@apache.org>:
>>
>>>> it would be great to release a new SQL engine in 2.13 as an
>>> experimental feature.
>>> ++1
>>>
>>> On Thu, Dec 30, 2021 at 12:55 PM Alex Plehanov <pl...@gmail.com>
>>> wrote:
>>>
>>>> Andrey,
>>>>
>>>>> Is this [1] a full scope of the tickets that MUST be resolved before
>>> the
>>>> engine could be merged?
>>>> Yes, we must resolve at least these tickets before merging. If you see
>>> any
>>>> other release blockers fill free to attach them to this ticket.
>>>>
>>>>> I think we have to add instructions to the readme file on how to turn
>>> a
>>>> new SQL engine on.
>>>> Sure, I think it should be the part of documentation ticket.
>>>>
>>>>> Also, I don't like the module name "ignite-calcite", because Calcite
>>> is
>>>> an independent project.
>>>> Personally, I see no problems here (but it's discussable). We have a
>>> lot of
>>>> modules where the name is an independent project: "ignite-kafka",
>>>> "ignite-spring", "ignite-kubernetes", "ignite-log4j",
>>> "ignite-zookeeper",
>>>> etc.
>>>>
>>>>> So, would you mind renaming the module to e.g. "ignite-sql-engine" or
>>>> "ignite-sql"?
>>>> Module "ignite-indexing" also contains SQL engine, so names like
>>>> "ignite-sql-engine" or "ignite-sql" will be ambiguous.
>>>>
>>>> чт, 30 дек. 2021 г. в 13:54, Andrey Mashenkov <
>>> andrey.mashenkov@gmail.com
>>>>> :
>>>>
>>>>> Alex,
>>>>> it would be great to release a new SQL engine in 2.13 as an
>>>>> experimental feature.
>>>>>
>>>>> Is this [1] a full scope of the tickets that MUST be resolved before
>>> the
>>>>> engine could be merged?
>>>>> I think we have to add instructions to the readme file on how to turn
>>> a
>>>> new
>>>>> SQL engine on.
>>>>>
>>>>> Also, I don't like the module name "ignite-calcite", because Calcite
>>> is
>>>> an
>>>>> independent project.
>>>>> and Ignite just uses it.
>>>>> So, would you mind renaming the module to e.g. "ignite-sql-engine" or
>>>>> "ignite-sql"?
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-15436
>>>>>
>>>>> On Thu, Dec 30, 2021 at 11:10 AM Zhenya Stanilovsky
>>>>> <ar...@mail.ru.invalid> wrote:
>>>>>
>>>>>>
>>>>>> Alex, great !
>>>>>> If someone wants to touch codebase somehow plz use this branch [1]
>>>>>> Test passed can be found here [2] [3]
>>>>>>
>>>>>> [1]
>>> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite
>>>>>> [2]
>>>>>>
>>>>>
>>>>
>>> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite
>>>>>> [3]
>>>>>>
>>>>>
>>>>
>>> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/sql
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> Hello, Igniters!
>>>>>>>>>
>>>>>>>>> As you may already know there is the new Ignite SQL engine based
>>> on
>>>>>> Apache
>>>>>>>>> Calcite currently under development.
>>>>>>>>>
>>>>>>>>> Reasons to move from H2-based engine and motivation for creating
>>> the
>>>>> new
>>>>>>>>> one in details described in IEP-37 [1].
>>>>>>>>>
>>>>>>>>> You can find all related to the new engine source code changes in
>>>> the
>>>>>>>>> "sql-calcite" branch [2].
>>>>>>>>>
>>>>>>>>> Calcite-based SQL engine is not production-ready yet and has a
>>> lot
>>>> of
>>>>>> known
>>>>>>>>> issues. In the future, the new engine should be fully
>>> independent of
>>>>>>>>> "ignite-indexing" and H2, but now it relies on schema management
>>> and
>>>>>>>>> indexes implemented in the "ignite-indexing" module and can't
>>> work
>>>>>> without
>>>>>>>>> the old engine. Despite all of the above mentioned, in the
>>> current
>>>>>> state,
>>>>>>>>> it has its own parsing, planning and execution flow and is
>>> almost as
>>>>>>>>> functional as the H2-based SQL engine.
>>>>>>>>>
>>>>>>>>> Some users are already interested in the Calcite-based engine and
>>>>> asking
>>>>>>>>> about the development status and release dates. Calcite-based SQL
>>>>> engine
>>>>>>>>> will be the only SQL engine in Ignite 3.0. Perhaps even in 2.x we
>>>> can
>>>>>> get
>>>>>>>>> rid of the H2-based engine at some time in the future. There is
>>> some
>>>>>> syntax
>>>>>>>>> difference between Calcite and H2 (Calcite is closer to SQL
>>>> standards
>>>>>> than
>>>>>>>>> H2) and a totally new execution flow. After the release of this
>>>>> feature,
>>>>>>>>> users can try their queries and determine if any adaptation for
>>> them
>>>>> is
>>>>>>>>> required. With the new planning and execution flow, perhaps, some
>>>>>> queries
>>>>>>>>> will be executed more effectively, users can redirect such
>>> queries
>>>> to
>>>>>> the
>>>>>>>>> new engine.
>>>>>>>>>
>>>>>>>>> I think we can provide an opportunity to users to try the new
>>> engine
>>>>> and
>>>>>>>>> release it as an experimental feature with the next Apache Ignite
>>>>>> version
>>>>>>>>> (2.13).
>>>>>>>>>
>>>>>>>>> What do you think?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Andrey V. Mashenkov
>>>>>
>>>>
>>>
>>
Re: [PROPOSAL] Release Calcite-based SQL engine as an experimental feature
Posted by Alex Plehanov <pl...@gmail.com>.
Hello Igniters,
I've merged the pull request. The Calcite-based SQL engine is in the master
branch now.
If you desire to try it, you can find configuration instructions in
modules/calcite/README.txt file.
вс, 6 мар. 2022 г. в 13:03, Alex Plehanov <pl...@gmail.com>:
> Hello Igniters,
>
> I've prepared the pull request [1] and have plans to merge it to the
> master branch in about two weeks, if there is no objection.
>
> [1]: https://github.com/apache/ignite/pull/9855
>
> чт, 30 дек. 2021 г. в 13:43, Anton Vinogradov <av...@apache.org>:
>
>> > it would be great to release a new SQL engine in 2.13 as an
>> experimental feature.
>> ++1
>>
>> On Thu, Dec 30, 2021 at 12:55 PM Alex Plehanov <pl...@gmail.com>
>> wrote:
>>
>> > Andrey,
>> >
>> > > Is this [1] a full scope of the tickets that MUST be resolved before
>> the
>> > engine could be merged?
>> > Yes, we must resolve at least these tickets before merging. If you see
>> any
>> > other release blockers fill free to attach them to this ticket.
>> >
>> > > I think we have to add instructions to the readme file on how to turn
>> a
>> > new SQL engine on.
>> > Sure, I think it should be the part of documentation ticket.
>> >
>> > > Also, I don't like the module name "ignite-calcite", because Calcite
>> is
>> > an independent project.
>> > Personally, I see no problems here (but it's discussable). We have a
>> lot of
>> > modules where the name is an independent project: "ignite-kafka",
>> > "ignite-spring", "ignite-kubernetes", "ignite-log4j",
>> "ignite-zookeeper",
>> > etc.
>> >
>> > > So, would you mind renaming the module to e.g. "ignite-sql-engine" or
>> > "ignite-sql"?
>> > Module "ignite-indexing" also contains SQL engine, so names like
>> > "ignite-sql-engine" or "ignite-sql" will be ambiguous.
>> >
>> > чт, 30 дек. 2021 г. в 13:54, Andrey Mashenkov <
>> andrey.mashenkov@gmail.com
>> > >:
>> >
>> > > Alex,
>> > > it would be great to release a new SQL engine in 2.13 as an
>> > > experimental feature.
>> > >
>> > > Is this [1] a full scope of the tickets that MUST be resolved before
>> the
>> > > engine could be merged?
>> > > I think we have to add instructions to the readme file on how to turn
>> a
>> > new
>> > > SQL engine on.
>> > >
>> > > Also, I don't like the module name "ignite-calcite", because Calcite
>> is
>> > an
>> > > independent project.
>> > > and Ignite just uses it.
>> > > So, would you mind renaming the module to e.g. "ignite-sql-engine" or
>> > > "ignite-sql"?
>> > >
>> > > [1] https://issues.apache.org/jira/browse/IGNITE-15436
>> > >
>> > > On Thu, Dec 30, 2021 at 11:10 AM Zhenya Stanilovsky
>> > > <ar...@mail.ru.invalid> wrote:
>> > >
>> > > >
>> > > > Alex, great !
>> > > > If someone wants to touch codebase somehow plz use this branch [1]
>> > > > Test passed can be found here [2] [3]
>> > > >
>> > > > [1]
>> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite
>> > > > [2]
>> > > >
>> > >
>> >
>> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite
>> > > > [3]
>> > > >
>> > >
>> >
>> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/sql
>> > > >
>> > > > >
>> > > > >>
>> > > > >>>Hello, Igniters!
>> > > > >>>
>> > > > >>>As you may already know there is the new Ignite SQL engine based
>> on
>> > > > Apache
>> > > > >>>Calcite currently under development.
>> > > > >>>
>> > > > >>>Reasons to move from H2-based engine and motivation for creating
>> the
>> > > new
>> > > > >>>one in details described in IEP-37 [1].
>> > > > >>>
>> > > > >>>You can find all related to the new engine source code changes in
>> > the
>> > > > >>>"sql-calcite" branch [2].
>> > > > >>>
>> > > > >>>Calcite-based SQL engine is not production-ready yet and has a
>> lot
>> > of
>> > > > known
>> > > > >>>issues. In the future, the new engine should be fully
>> independent of
>> > > > >>>"ignite-indexing" and H2, but now it relies on schema management
>> and
>> > > > >>>indexes implemented in the "ignite-indexing" module and can't
>> work
>> > > > without
>> > > > >>>the old engine. Despite all of the above mentioned, in the
>> current
>> > > > state,
>> > > > >>>it has its own parsing, planning and execution flow and is
>> almost as
>> > > > >>>functional as the H2-based SQL engine.
>> > > > >>>
>> > > > >>>Some users are already interested in the Calcite-based engine and
>> > > asking
>> > > > >>>about the development status and release dates. Calcite-based SQL
>> > > engine
>> > > > >>>will be the only SQL engine in Ignite 3.0. Perhaps even in 2.x we
>> > can
>> > > > get
>> > > > >>>rid of the H2-based engine at some time in the future. There is
>> some
>> > > > syntax
>> > > > >>>difference between Calcite and H2 (Calcite is closer to SQL
>> > standards
>> > > > than
>> > > > >>>H2) and a totally new execution flow. After the release of this
>> > > feature,
>> > > > >>>users can try their queries and determine if any adaptation for
>> them
>> > > is
>> > > > >>>required. With the new planning and execution flow, perhaps, some
>> > > > queries
>> > > > >>>will be executed more effectively, users can redirect such
>> queries
>> > to
>> > > > the
>> > > > >>>new engine.
>> > > > >>>
>> > > > >>>I think we can provide an opportunity to users to try the new
>> engine
>> > > and
>> > > > >>>release it as an experimental feature with the next Apache Ignite
>> > > > version
>> > > > >>>(2.13).
>> > > > >>>
>> > > > >>>What do you think?
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > > Andrey V. Mashenkov
>> > >
>> >
>>
>