You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Shawn Du <sh...@neulion.com.cn> on 2017/01/05 07:20:40 UTC
compute SQL returned data.
Hi experts,
I want to compute the SQL returned data on the server side. Is there any
examples?
Thanks
Shawn
Re: compute SQL returned data.
Posted by dkarachentsev <dk...@gridgain.com>.
You can use compute tasks [1], in which invoke SQL query and process results.
[1] https://apacheignite.readme.io/docs/compute-grid
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/compute-SQL-returned-data-tp9890p9893.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: 答复: 答复: compute SQL returned data.
Posted by vkulichenko <va...@gmail.com>.
Shawn,
map() method is execute locally on the master node, so you can do all the
checks outside the task and then execute only if needed. Will this work?
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/compute-SQL-returned-data-tp9890p10016.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
答复: 答复: compute SQL returned data.
Posted by Shawn Du <sh...@neulion.com.cn>.
Yes. For my case, before submitting jobs, I need query data in task, so if
no data, I have to submit an empty job which does nothing, this is my
workaround.
Is there "normal" ways for this case?
Thanks
Shawn
-----邮件原件-----
发件人: vkulichenko [mailto:valentin.kulichenko@gmail.com]
发送时间: 2017年1月11日 3:31
收件人: user@ignite.apache.org
主题: Re: 答复: compute SQL returned data.
Hi Shawn,
This means what it says - ComputeTask.map() method returned no jobs, i.e.
null or empty map.
-Val
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/compute-SQL-returned-data-tp9
890p9999.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: 答复: compute SQL returned data.
Posted by vkulichenko <va...@gmail.com>.
Hi Shawn,
This means what it says - ComputeTask.map() method returned no jobs, i.e.
null or empty map.
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/compute-SQL-returned-data-tp9890p9999.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
答复: compute SQL returned data.
Posted by Shawn Du <sh...@neulion.com.cn>.
Hi,
When I run my compute task, it throws below exception. What it may be caused
by?
class org.apache.ignite.IgniteCheckedException: Task map operation produced
no mapped jobs
Thanks
Shawn
-----邮件原件-----
发件人: Shawn Du [mailto:shawn.du@neulion.com.cn]
发送时间: 2017年1月9日 17:31
收件人: user@ignite.apache.org
主题: 答复: compute SQL returned data.
Thanks, I already figure it out just As you mentioned.
-----邮件原件-----
发件人: dkarachentsev [mailto:dkarachentsev@gridgain.com]
发送时间: 2017年1月9日 17:24
收件人: user@ignite.apache.org
主题: Re: compute SQL returned data.
Hi Shawn,
For map-reduce operations is most suitable ComputeTask [1], which has
map/reduce methods. In map() method you may select nodes on which start
processing according to key (refer Ignite.affinity() [2,3]). Result will be
reduced and returned to the client.
In summary, client invokes SQL to collect keys and runs ComputeTask`s mapped
on nodes that has keys/values that should be processed (this will allow not
to transfer over network big values).
[1]
https://ignite.apache.org/releases/1.8.0/javadoc/org/apache/ignite/compute/C
omputeTask.html
[2]
https://ignite.apache.org/releases/1.8.0/javadoc/org/apache/ignite/Ignite.ht
ml#affinity(java.lang.String)
[3]
https://ignite.apache.org/releases/1.8.0/javadoc/org/apache/ignite/cache/aff
inity/Affinity.html
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/compute-SQL-returned-data-tp9
890p9967.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
答复: compute SQL returned data.
Posted by Shawn Du <sh...@neulion.com.cn>.
Thanks, I already figure it out just As you mentioned.
-----邮件原件-----
发件人: dkarachentsev [mailto:dkarachentsev@gridgain.com]
发送时间: 2017年1月9日 17:24
收件人: user@ignite.apache.org
主题: Re: compute SQL returned data.
Hi Shawn,
For map-reduce operations is most suitable ComputeTask [1], which has
map/reduce methods. In map() method you may select nodes on which start
processing according to key (refer Ignite.affinity() [2,3]). Result will be
reduced and returned to the client.
In summary, client invokes SQL to collect keys and runs ComputeTask`s mapped
on nodes that has keys/values that should be processed (this will allow not
to transfer over network big values).
[1]
https://ignite.apache.org/releases/1.8.0/javadoc/org/apache/ignite/compute/C
omputeTask.html
[2]
https://ignite.apache.org/releases/1.8.0/javadoc/org/apache/ignite/Ignite.ht
ml#affinity(java.lang.String)
[3]
https://ignite.apache.org/releases/1.8.0/javadoc/org/apache/ignite/cache/aff
inity/Affinity.html
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/compute-SQL-returned-data-tp9
890p9967.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: compute SQL returned data.
Posted by dkarachentsev <dk...@gridgain.com>.
Hi Shawn,
For map-reduce operations is most suitable ComputeTask [1], which has
map/reduce methods. In map() method you may select nodes on which start
processing according to key (refer Ignite.affinity() [2,3]). Result will be
reduced and returned to the client.
In summary, client invokes SQL to collect keys and runs ComputeTask`s mapped
on nodes that has keys/values that should be processed (this will allow not
to transfer over network big values).
[1]
https://ignite.apache.org/releases/1.8.0/javadoc/org/apache/ignite/compute/ComputeTask.html
[2]
https://ignite.apache.org/releases/1.8.0/javadoc/org/apache/ignite/Ignite.html#affinity(java.lang.String)
[3]
https://ignite.apache.org/releases/1.8.0/javadoc/org/apache/ignite/cache/affinity/Affinity.html
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/compute-SQL-returned-data-tp9890p9967.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: compute SQL returned data.
Posted by Denis Magda <dm...@apache.org>.
Keep in mind that if you’re going to execute local SQL queries inside of distributed computations then the computations have to be sent to the cluster with affinity based methods
https://apacheignite.readme.io/docs/collocate-compute-and-data#section-affinity-call-and-run-methods <https://apacheignite.readme.io/docs/collocate-compute-and-data#section-affinity-call-and-run-methods>
—
Denis
> On Jan 5, 2017, at 12:30 AM, Nikolai Tikhonov <nt...@apache.org> wrote:
>
> We can submite Compute Task which will be mapped jobs on nodes. Jobs can execute local SQL query (see Query#setLocal). Examples queries and compute task can be found here: https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples <https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples>
>
> On Thu, Jan 5, 2017 at 11:09 AM, Shawn Du <shawn.du@neulion.com.cn <ma...@neulion.com.cn>> wrote:
> Thanks!
>
>
>
> My case looks like this:
>
>
>
> We store big cache object to overcome the defects of ignite overhead of each cache entry.
>
> For these big cache object, we can’t use SQL to query/aggregate them. So we need compute by ourselves.
>
>
>
> We want to compute on the SQL returned data in ignite server side for several reasons:
>
> #1, for the SQL returned data is huge, we don’t want to transfer them from sever to client.
>
> #2, compute on the client server, we need write code to compute parallel. I assume ignite will do it for us.
>
> #3, in future, If we add new server nodes, we will benefit immediately.
>
>
>
> Thanks
>
> Shawn
>
>
>
>
>
> 发件人: Nikolai Tikhonov [mailto:ntikhonov@apache.org <ma...@apache.org>]
> 发送时间: 2017年1月5日 15:45
> 收件人: user@ignite.apache.org <ma...@ignite.apache.org>
> 主题: Re: compute SQL returned data.
>
>
>
> Hi,
>
>
>
> I'm not sure that undestrand what do you want. Could you describe your case more details?
>
> Also you can find examples on here: https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples <https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples>
>
>
> On Thu, Jan 5, 2017 at 10:20 AM, Shawn Du <shawn.du@neulion.com.cn <ma...@neulion.com.cn>> wrote:
>
> Hi experts,
>
>
>
> I want to compute the SQL returned data on the server side. Is there any examples?
>
>
>
>
>
> Thanks
>
> Shawn
>
>
>
>
Re: 答复: compute SQL returned data.
Posted by Nikolai Tikhonov <nt...@apache.org>.
We can submite Compute Task which will be mapped jobs on nodes. Jobs can
execute local SQL query (see Query#setLocal). Examples queries and compute
task can be found here: https://github.com/apache/ignite/tree/master/
examples/src/main/java/org/apache/ignite/examples
On Thu, Jan 5, 2017 at 11:09 AM, Shawn Du <sh...@neulion.com.cn> wrote:
> Thanks!
>
>
>
> My case looks like this:
>
>
>
> We store big cache object to overcome the defects of ignite overhead of
> each cache entry.
>
> For these big cache object, we can’t use SQL to query/aggregate them. So
> we need compute by ourselves.
>
>
>
> We want to compute on the SQL returned data in ignite server side for
> several reasons:
>
> #1, for the SQL returned data is huge, we don’t want to transfer them from
> sever to client.
>
> #2, compute on the client server, we need write code to compute parallel.
> I assume ignite will do it for us.
>
> #3, in future, If we add new server nodes, we will benefit immediately.
>
>
>
> Thanks
>
> Shawn
>
>
>
>
>
> *发件人:* Nikolai Tikhonov [mailto:ntikhonov@apache.org]
> *发送时间:* 2017年1月5日 15:45
> *收件人:* user@ignite.apache.org
> *主题:* Re: compute SQL returned data.
>
>
>
> Hi,
>
>
>
> I'm not sure that undestrand what do you want. Could you describe your
> case more details?
>
> Also you can find examples on here: https://github.com/
> apache/ignite/tree/master/examples/src/main/java/org/
> apache/ignite/examples
>
>
>
> On Thu, Jan 5, 2017 at 10:20 AM, Shawn Du <sh...@neulion.com.cn> wrote:
>
> Hi experts,
>
>
>
> I want to compute the SQL returned data on the server side. Is there any
> examples?
>
>
>
>
>
> Thanks
>
> Shawn
>
>
>
答复: compute SQL returned data.
Posted by Shawn Du <sh...@neulion.com.cn>.
Thanks!
My case looks like this:
We store big cache object to overcome the defects of ignite overhead of each cache entry.
For these big cache object, we can’t use SQL to query/aggregate them. So we need compute by ourselves.
We want to compute on the SQL returned data in ignite server side for several reasons:
#1, for the SQL returned data is huge, we don’t want to transfer them from sever to client.
#2, compute on the client server, we need write code to compute parallel. I assume ignite will do it for us.
#3, in future, If we add new server nodes, we will benefit immediately.
Thanks
Shawn
发件人: Nikolai Tikhonov [mailto:ntikhonov@apache.org]
发送时间: 2017年1月5日 15:45
收件人: user@ignite.apache.org
主题: Re: compute SQL returned data.
Hi,
I'm not sure that undestrand what do you want. Could you describe your case more details?
Also you can find examples on here: https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples
On Thu, Jan 5, 2017 at 10:20 AM, Shawn Du <shawn.du@neulion.com.cn <ma...@neulion.com.cn> > wrote:
Hi experts,
I want to compute the SQL returned data on the server side. Is there any examples?
Thanks
Shawn
Re: compute SQL returned data.
Posted by Nikolai Tikhonov <nt...@apache.org>.
Hi,
I'm not sure that undestrand what do you want. Could you describe your case
more details?
Also you can find examples on here:
https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples
On Thu, Jan 5, 2017 at 10:20 AM, Shawn Du <sh...@neulion.com.cn> wrote:
> Hi experts,
>
>
>
> I want to compute the SQL returned data on the server side. Is there any
> examples?
>
>
>
>
>
> Thanks
>
> Shawn
>