You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by eugene miretsky <eu...@gmail.com> on 2018/10/09 18:52:01 UTC
Re: Role of H2 datbase in Apache Ignite
Hello,
I have been struggling with this question myself for a while now too.
I think the documents are very ambiguous on how exactly H2 is being used.
The document that you linked say
"Apache Ignite leverages from H2's SQL query parser and optimizer as well
as the execution planner. Lastly, *H2 executes a query locally* on a
particular node and passes a local result to a distributed Ignite SQL
engine for further processing."
And
"However, *the data, as well as the indexes, are always stored in the
Ignite that executes queries* in a distributed and fault-tolerant manner
which is not supported by H2."
To me, this leaves a lot of ambiguity on how H2 is leveraged on a single
Ignite node. (I get that the Reduce stage, as well as distributed
transactions, are handled by Ignite, but how about the 'map' stage on a
single node).
How is a query executed on a single node?
Example query: Select count(customer_id) from user where (age > 20) group
by customer_id
What steps are taken?
1. execution plan: H2 creates an execution plan
2. data retrieval: Since data is stored off-heap, it has to be brought
into heap. Does H2 have anything to do with this step, or is it only
Ignite? When are indexes used for that?
3. Query execution: Once the data is on heap, what executes the Query
(the group_by, aggregations, filters that were not handled by indexes,
etc.)? H2 or Ignite?
On Fri, Sep 21, 2018 at 9:27 AM Mikhail <mi...@gmail.com> wrote:
> Hi,
>
> Could you please formulate your question? Because right not your message
> looks like a request for google.
> I think the following article has answer for your question:
> https://apacheignite-sql.readme.io/docs/how-ignite-sql-works
>
> Thanks,
> Mike.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
RE: Role of H2 datbase in Apache Ignite
Posted by Stanislav Lukyanov <st...@gmail.com>.
No, that one actually impacts how indexes are built, which is Ignite’s domain.
queryParallellism is the number of index trees that are created by Ignite.
In terms of Ignite-H2 communication, queryParallelism=N means that Ignite will send N parallel queries to H2,
each using a different index tree, then merge them.
Stan
From: eugene miretsky
Sent: 11 октября 2018 г. 20:58
To: user@ignite.apache.org
Subject: Re: Role of H2 datbase in Apache Ignite
Thanks!
So does it mean that CacheConfiguration.queryParallelism is really an H2 settings?
On Tue, Oct 9, 2018 at 4:27 PM Stanislav Lukyanov <st...@gmail.com> wrote:
In short, Ignite replaces H2’s storage level with its own.
For example, Ignite implements H2’s Index interface with its own off-heap data structures underneath.
When Ignite executes an SQL query, it will ask H2 to process it, then H2 will callback to Ignite’s implementations
of H2’s interfaces (such as Index) to actually retrieve the data.
I guess the on-heap processing is mostly H2, although there is a lot of work done by Ignite to make the distributed
map-reduce work like creating temporary tables for intermediate results.
Stan
From: eugene miretsky
Sent: 9 октября 2018 г. 21:52
To: user@ignite.apache.org
Subject: Re: Role of H2 datbase in Apache Ignite
Hello,
I have been struggling with this question myself for a while now too.
I think the documents are very ambiguous on how exactly H2 is being used.
The document that you linked say
"Apache Ignite leverages from H2's SQL query parser and optimizer as well as the execution planner. Lastly, H2 executes a query locally on a particular node and passes a local result to a distributed Ignite SQL engine for further processing."
And
"However, the data, as well as the indexes, are always stored in the Ignite that executes queries in a distributed and fault-tolerant manner which is not supported by H2."
To me, this leaves a lot of ambiguity on how H2 is leveraged on a single Ignite node. (I get that the Reduce stage, as well as distributed transactions, are handled by Ignite, but how about the 'map' stage on a single node).
How is a query executed on a single node?
Example query: Select count(customer_id) from user where (age > 20) group by customer_id
What steps are taken?
1. execution plan: H2 creates an execution plan
2. data retrieval: Since data is stored off-heap, it has to be brought into heap. Does H2 have anything to do with this step, or is it only Ignite? When are indexes used for that?
3. Query execution: Once the data is on heap, what executes the Query (the group_by, aggregations, filters that were not handled by indexes, etc.)? H2 or Ignite?
On Fri, Sep 21, 2018 at 9:27 AM Mikhail <mi...@gmail.com> wrote:
Hi,
Could you please formulate your question? Because right not your message
looks like a request for google.
I think the following article has answer for your question:
https://apacheignite-sql.readme.io/docs/how-ignite-sql-works
Thanks,
Mike.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Role of H2 datbase in Apache Ignite
Posted by eugene miretsky <eu...@gmail.com>.
Thanks!
So does it mean that CacheConfiguration.queryParallelism is really an H2
settings?
On Tue, Oct 9, 2018 at 4:27 PM Stanislav Lukyanov <st...@gmail.com>
wrote:
> In short, Ignite replaces H2’s storage level with its own.
>
> For example, Ignite implements H2’s Index interface with its own off-heap
> data structures underneath.
>
> When Ignite executes an SQL query, it will ask H2 to process it, then H2
> will callback to Ignite’s implementations
>
> of H2’s interfaces (such as Index) to actually retrieve the data.
>
> I guess the on-heap processing is mostly H2, although there is a lot of
> work done by Ignite to make the distributed
>
> map-reduce work like creating temporary tables for intermediate results.
>
>
>
> Stan
>
>
>
> *From: *eugene miretsky <eu...@gmail.com>
> *Sent: *9 октября 2018 г. 21:52
> *To: *user@ignite.apache.org
> *Subject: *Re: Role of H2 datbase in Apache Ignite
>
>
>
> Hello,
>
>
>
> I have been struggling with this question myself for a while now too.
>
> I think the documents are very ambiguous on how exactly H2 is being used.
>
>
>
> The document that you linked say
>
> "Apache Ignite leverages from H2's SQL query parser and optimizer as well
> as the execution planner. Lastly, *H2 executes a query locally* on a
> particular node and passes a local result to a distributed Ignite SQL
> engine for further processing."
>
>
>
> And
>
> "However, *the data, as well as the indexes, are always stored in the
> Ignite that executes queries* in a distributed and fault-tolerant manner
> which is not supported by H2."
>
>
>
> To me, this leaves a lot of ambiguity on how H2 is leveraged on a single
> Ignite node. (I get that the Reduce stage, as well as distributed
> transactions, are handled by Ignite, but how about the 'map' stage on a
> single node).
>
>
>
> How is a query executed on a single node?
>
> Example query: Select count(customer_id) from user where (age > 20) group
> by customer_id
>
>
>
> What steps are taken?
>
> 1. execution plan: H2 creates an execution plan
> 2. data retrieval: Since data is stored off-heap, it has to be
> brought into heap. Does H2 have anything to do with this step, or is it
> only Ignite? When are indexes used for that?
> 3. Query execution: Once the data is on heap, what executes the Query
> (the group_by, aggregations, filters that were not handled by indexes,
> etc.)? H2 or Ignite?
>
>
>
>
>
>
>
> On Fri, Sep 21, 2018 at 9:27 AM Mikhail <mi...@gmail.com>
> wrote:
>
> Hi,
>
> Could you please formulate your question? Because right not your message
> looks like a request for google.
> I think the following article has answer for your question:
> https://apacheignite-sql.readme.io/docs/how-ignite-sql-works
>
> Thanks,
> Mike.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
>
RE: Role of H2 datbase in Apache Ignite
Posted by Stanislav Lukyanov <st...@gmail.com>.
In short, Ignite replaces H2’s storage level with its own.
For example, Ignite implements H2’s Index interface with its own off-heap data structures underneath.
When Ignite executes an SQL query, it will ask H2 to process it, then H2 will callback to Ignite’s implementations
of H2’s interfaces (such as Index) to actually retrieve the data.
I guess the on-heap processing is mostly H2, although there is a lot of work done by Ignite to make the distributed
map-reduce work like creating temporary tables for intermediate results.
Stan
From: eugene miretsky
Sent: 9 октября 2018 г. 21:52
To: user@ignite.apache.org
Subject: Re: Role of H2 datbase in Apache Ignite
Hello,
I have been struggling with this question myself for a while now too.
I think the documents are very ambiguous on how exactly H2 is being used.
The document that you linked say
"Apache Ignite leverages from H2's SQL query parser and optimizer as well as the execution planner. Lastly, H2 executes a query locally on a particular node and passes a local result to a distributed Ignite SQL engine for further processing."
And
"However, the data, as well as the indexes, are always stored in the Ignite that executes queries in a distributed and fault-tolerant manner which is not supported by H2."
To me, this leaves a lot of ambiguity on how H2 is leveraged on a single Ignite node. (I get that the Reduce stage, as well as distributed transactions, are handled by Ignite, but how about the 'map' stage on a single node).
How is a query executed on a single node?
Example query: Select count(customer_id) from user where (age > 20) group by customer_id
What steps are taken?
1. execution plan: H2 creates an execution plan
2. data retrieval: Since data is stored off-heap, it has to be brought into heap. Does H2 have anything to do with this step, or is it only Ignite? When are indexes used for that?
3. Query execution: Once the data is on heap, what executes the Query (the group_by, aggregations, filters that were not handled by indexes, etc.)? H2 or Ignite?
On Fri, Sep 21, 2018 at 9:27 AM Mikhail <mi...@gmail.com> wrote:
Hi,
Could you please formulate your question? Because right not your message
looks like a request for google.
I think the following article has answer for your question:
https://apacheignite-sql.readme.io/docs/how-ignite-sql-works
Thanks,
Mike.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/