You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "Gordon Reid (Nine Mile)" <go...@ninemilefinancial.com> on 2018/09/10 07:00:46 UTC
Ignite Thin Client Continuous Query
Hi,
Is there any plan to support continuous query in the Ignite Thin client.
This would be very useful for us. Without it, we probably have to use some other technology for serialization and streaming updates to the desktop.
Thanks,
Gordon
This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Re: Ignite Thin Client Continuous Query
Posted by akurbanov <an...@gmail.com>.
Hello,
This feature has not been planned yet as far as I know, I didn't manage to
find JIRA's for this, but I think discussion on this feature may be started
on dev list.
Regards
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: Ignite Thin Client Continuous Query
Posted by vkulichenko <va...@gmail.com>.
Gordon,
Generally, having CQ on thin client would definitely be awesome. My only
point is that thin client has several technical limitations that would
introduce multiple "ifs" into the functionality. What exactly those ifs are,
and weather there is still value with all those ifs, is a big question for
me. An open question though, of course - by no means I'm trying to make a
claim that it doesn't make sense at all.
As for you use case, everything sounds reasonable to me. I would probably do
the following changes however:
- Move CQs to separate client node(s). That would separate the concerns, and
also simplify failover management.
- Use some other product to propagate updates from CQ to desktop apps (Kafka
maybe?). Basically, you need some sort of durable queue to deliver those
messages. You can definitely implement it from scratch, but this might be on
overkill.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: Ignite Thin Client Continuous Query
Posted by "Gordon Reid (Nine Mile)" <go...@ninemilefinancial.com>.
Hi Val,
It's a very simple, and I would say a very common use case. We want to send a filter to the grid, receive a snapshot, and then receive a continuous stream of updates that match that filter.
Consider a trading window, showing a grid of stocks. I want to subscribe to the continuously changing market prices for a specific list of stocks, and maybe I want to subscribe to trade events for another specific set. The universe of stocks is huge and not all users will want to see the same stocks, so server side filtering is key.
We are under time pressure, so for now we will probably end up building our own lightweight framework based on sockets and json for serialization between the java server side and the .net user app side. On the server side we will have to build a proxy service that will be responsible for managing subscriptions from the user apps, and then making those corresponding subscriptions onto the grid, using CQ. When it receives the results of the CQ it will need to serialize and push events back to the user apps.
The high level use case is that we have a java ignite cluster which implements our trading platform deployed into a data centre. And we have a C# .NET desktop app which currently hosts an ignite thick client. This user app needs to connect to clusters which might be in the metro area, or even in other countries. We have found for metro, it's okay, but not great (despite a huge amount of time tuning comms parameters), and for remote cities, its unusable. If we were .net to .net we could (for example) do this easily with WCF. The user app just needs to
- send RPC style commands (which we currently do using ignite service grid)
(eg. start / stop trading strategy)
- get cache snapshots based on some filter
(eg. show me all the orders on Microsoft for yesterday)
- subscribe to cache updates based on some filter
( eg. show me all the orders on Microsoft for today so far, placed by trading strategy X, and stream new orders or changes to existing orders as they occur)
Thanks,
Gordon.
-----Original Message-----
From: vkulichenko <va...@gmail.com>
Sent: Tuesday, September 11, 2018 10:47 AM
To: user@ignite.apache.org
Subject: RE: Ignite Thin Client Continuous Query
Gordon,
Yes, generally we do recommend using thin client for such applications.
However, it doesn't mean that you can't us client node in case it's really needed, although it might require some additional tuning.
Would you mind telling if you have any other technology in mind? I highly doubt that you will find anything that can provide functionality similar to CQ in Ignite, especially with the same delivery guarantees, while being based on a lightweight client. I believe you either will not succeed in finding such an alternative, or your use case does not require continuous queries in the first place. Can you give some more details on what you're trying to achieve? I might be able to suggest some other approach then.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Re: Ignite Thin Client Continuous Query
Posted by vkulichenko <va...@gmail.com>.
Gaurav,
Web Console receives updates from web agent which periodically polls the
cluster.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite Thin Client Continuous Query
Posted by Gaurav Bajaj <ga...@gmail.com>.
Guys,
Just wondering how does webconsole receievs updates from server
continuously?
Regards,
Gaurav
On 11-Sep-2018 10:31 PM, "Valentin Kulichenko" <
valentin.kulichenko@gmail.com> wrote:
> Igor,
>
> I just think that we're dealing with a trade off here, and that if we
> implement CQ for thin client, we will either end with a client that is not
> "thin" anymore, or semantics and guarantees of CQ would change so
> drastically that it would be a completely different feature. Either way,
> it's a big question weather there is a value in doing this.
>
> I'm open to discussion though. If you have any particular suggestions,
> let's discuss them on dev list.
>
> -Val
>
> On Tue, Sep 11, 2018 at 5:26 AM Igor Sapego <is...@apache.org> wrote:
>
>> Guys,
>>
>> Personally, I do not see any problems, why we can not implement
>> Continuous Queries for thin clients. This will require a decent amount
>> of work, and will not give such strong guaranties as thick clients
>> give (for example, on server crash thin client will get an exception and
>> will need to re-register listener once again), but to me it seems totally
>> implementable.
>>
>> Val,
>>
>> Why do you think that such features are unlikely to appear in thin
>> clients?
>>
>> Best Regards,
>> Igor
>>
>>
>> On Tue, Sep 11, 2018 at 3:07 PM Alexey Kuznetsov <ak...@apache.org>
>> wrote:
>>
>>>
>>> Gordon,
>>>
>>> How about to start several client nodes "near" to cluster and use them
>>> as "proxies" for your desktop GUI apps?
>>> You may write some code that will push data from client node to you GUI
>>> app.
>>> This will require some coding of course.
>>>
>>> --
>>> Alexey Kuznetsov
>>>
>>
RE: Ignite Thin Client Continuous Query
Posted by "Gordon Reid (Nine Mile)" <go...@ninemilefinancial.com>.
In my humble opinion there is a huge value here. We have these fantastic APIs between our cluster nodes, why should we have to go and use different APIs, and different serialization techniques in our end user apps? It’s totally acceptable that the reliability, and guaranteed delivery aspects are relaxed for the user app. The key is to use the same (or useful subset of) the cache interfaces.
From: Valentin Kulichenko <va...@gmail.com>
Sent: Wednesday, September 12, 2018 6:33 AM
To: user@ignite.apache.org
Cc: isapego@apache.org
Subject: Re: Ignite Thin Client Continuous Query
Igor,
I just think that we're dealing with a trade off here, and that if we implement CQ for thin client, we will either end with a client that is not "thin" anymore, or semantics and guarantees of CQ would change so drastically that it would be a completely different feature. Either way, it's a big question weather there is a value in doing this.
I'm open to discussion though. If you have any particular suggestions, let's discuss them on dev list.
-Val
On Tue, Sep 11, 2018 at 5:26 AM Igor Sapego <is...@apache.org>> wrote:
Guys,
Personally, I do not see any problems, why we can not implement
Continuous Queries for thin clients. This will require a decent amount
of work, and will not give such strong guaranties as thick clients
give (for example, on server crash thin client will get an exception and
will need to re-register listener once again), but to me it seems totally
implementable.
Val,
Why do you think that such features are unlikely to appear in thin clients?
Best Regards,
Igor
On Tue, Sep 11, 2018 at 3:07 PM Alexey Kuznetsov <ak...@apache.org>> wrote:
Gordon,
How about to start several client nodes "near" to cluster and use them as "proxies" for your desktop GUI apps?
You may write some code that will push data from client node to you GUI app.
This will require some coding of course.
--
Alexey Kuznetsov
This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Re: Ignite Thin Client Continuous Query
Posted by Valentin Kulichenko <va...@gmail.com>.
Igor,
I just think that we're dealing with a trade off here, and that if we
implement CQ for thin client, we will either end with a client that is not
"thin" anymore, or semantics and guarantees of CQ would change so
drastically that it would be a completely different feature. Either way,
it's a big question weather there is a value in doing this.
I'm open to discussion though. If you have any particular suggestions,
let's discuss them on dev list.
-Val
On Tue, Sep 11, 2018 at 5:26 AM Igor Sapego <is...@apache.org> wrote:
> Guys,
>
> Personally, I do not see any problems, why we can not implement
> Continuous Queries for thin clients. This will require a decent amount
> of work, and will not give such strong guaranties as thick clients
> give (for example, on server crash thin client will get an exception and
> will need to re-register listener once again), but to me it seems totally
> implementable.
>
> Val,
>
> Why do you think that such features are unlikely to appear in thin clients?
>
> Best Regards,
> Igor
>
>
> On Tue, Sep 11, 2018 at 3:07 PM Alexey Kuznetsov <ak...@apache.org>
> wrote:
>
>>
>> Gordon,
>>
>> How about to start several client nodes "near" to cluster and use them as
>> "proxies" for your desktop GUI apps?
>> You may write some code that will push data from client node to you GUI
>> app.
>> This will require some coding of course.
>>
>> --
>> Alexey Kuznetsov
>>
>
Re: Ignite Thin Client Continuous Query
Posted by Igor Sapego <is...@apache.org>.
Guys,
Personally, I do not see any problems, why we can not implement
Continuous Queries for thin clients. This will require a decent amount
of work, and will not give such strong guaranties as thick clients
give (for example, on server crash thin client will get an exception and
will need to re-register listener once again), but to me it seems totally
implementable.
Val,
Why do you think that such features are unlikely to appear in thin clients?
Best Regards,
Igor
On Tue, Sep 11, 2018 at 3:07 PM Alexey Kuznetsov <ak...@apache.org>
wrote:
>
> Gordon,
>
> How about to start several client nodes "near" to cluster and use them as
> "proxies" for your desktop GUI apps?
> You may write some code that will push data from client node to you GUI
> app.
> This will require some coding of course.
>
> --
> Alexey Kuznetsov
>
RE: Ignite Thin Client Continuous Query
Posted by "Gordon Reid (Nine Mile)" <go...@ninemilefinancial.com>.
Thanks Alexy, yes we have considered this approach. But I would normally consider this type of architecture an optimization, not a base requirement. It’s rather heavy and to me only makes sense when we have a large number of end users and we want to minimize bandwidth to the remote locations.
From: Alexey Kuznetsov <ak...@apache.org>
Sent: Tuesday, September 11, 2018 10:07 PM
To: user@ignite.apache.org
Subject: Re: Ignite Thin Client Continuous Query
Gordon,
How about to start several client nodes "near" to cluster and use them as "proxies" for your desktop GUI apps?
You may write some code that will push data from client node to you GUI app.
This will require some coding of course.
--
Alexey Kuznetsov
This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Re: Ignite Thin Client Continuous Query
Posted by Alexey Kuznetsov <ak...@apache.org>.
Gordon,
How about to start several client nodes "near" to cluster and use them as
"proxies" for your desktop GUI apps?
You may write some code that will push data from client node to you GUI app.
This will require some coding of course.
--
Alexey Kuznetsov
RE: Ignite Thin Client Continuous Query
Posted by vkulichenko <va...@gmail.com>.
Gordon,
Yes, generally we do recommend using thin client for such applications.
However, it doesn't mean that you can't us client node in case it's really
needed, although it might require some additional tuning.
Would you mind telling if you have any other technology in mind? I highly
doubt that you will find anything that can provide functionality similar to
CQ in Ignite, especially with the same delivery guarantees, while being
based on a lightweight client. I believe you either will not succeed in
finding such an alternative, or your use case does not require continuous
queries in the first place. Can you give some more details on what you're
trying to achieve? I might be able to suggest some other approach then.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: Ignite Thin Client Continuous Query
Posted by "Gordon Reid (Nine Mile)" <go...@ninemilefinancial.com>.
Thanks Val. We are currently using a client node in our desktop gui, but it performs very poorly when the latency to our server nodes is high. We also have other problems, such as when new client nodes join, the whole cluster will pause, which is unacceptable for an end user application. I raised a question about this a few weeks ago, and we were advised that client node is not intended for use in end user applications. Any sort of financial desktop application needs streaming data and event based updates. So it seems like ignite is of no use for this type of application. And now I wonder if there is much value for us to keep using ignite on the server side of our application, since we need totally different technology to enable streaming data and event based notifications on our client.
-----Original Message-----
From: vkulichenko <va...@gmail.com>
Sent: Tuesday, September 11, 2018 7:43 AM
To: user@ignite.apache.org
Subject: Re: Ignite Thin Client Continuous Query
Gordon,
Ignite thin client uses request-response model, which is not really suitable for functionality like this. I would never say never, but I think it's very unlikely that thin client would get any feature that imply pushing updates from server to client (this includes near caches, any type of listeners including continuous queries, etc.). If you have such a requirement, you should use client node instead.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Re: Ignite Thin Client Continuous Query
Posted by vkulichenko <va...@gmail.com>.
Gordon,
Ignite thin client uses request-response model, which is not really suitable
for functionality like this. I would never say never, but I think it's very
unlikely that thin client would get any feature that imply pushing updates
from server to client (this includes near caches, any type of listeners
including continuous queries, etc.). If you have such a requirement, you
should use client node instead.
-Val
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/