You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by luqmanahmad <lu...@gmail.com> on 2017/07/17 14:45:27 UTC
database table polling
Hi, We have a legacy system, 15 years old at-least, which we are working on
and trying to bring everything inside the grid - but in the meanwhile we
have to support our existing system as well. In the legacy system we have a
database polling mechanism ( please don't roll your eyes :D ) which is
constantly looking for a specific table for new rows which has got
references to other bits, and is quite expensive in terms of resources plus
there are many other problems as well. I was wondering if we have a cache
which is connected through cache store to this table - Is there any
functionality available in ignite to poll it directly for all the new rows.
This is something we want to add in our legacy system while working on our
new system.If such a functionality exist any help with respect to this would
be highly appreciated.Thanks,Luqman
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/database-table-polling-tp15006.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: database table polling
Posted by Luqman Ahmad <lu...@gmail.com>.
Jorn, its oracle but we are migrating to postgres for newer system as the
license is too expensive. But for now we need the solution for Oracle.
I althought thought of deploying java classes into oracle instance but the
end user are not comfortable with its already crowded.
On 17 Jul 2017 7:23 pm, "Jörn Franke" <jo...@gmail.com> wrote:
> Which database? Some databases can notify an application id they are
> updated. You could read these updates with a Java application and insert
> them in the ignite cache.
>
> On 17. Jul 2017, at 16:45, luqmanahmad <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=15029&i=0>> wrote:
>
> Hi, We have a legacy system, 15 years old at-least, which we are working
> on and trying to bring everything inside the grid - but in the meanwhile we
> have to support our existing system as well. In the legacy system we have a
> database polling mechanism ( please don't roll your eyes :D ) which is
> constantly looking for a specific table for new rows which has got
> references to other bits, and is quite expensive in terms of resources plus
> there are many other problems as well. I was wondering if we have a cache
> which is connected through cache store to this table - Is there any
> functionality available in ignite to poll it directly for all the new rows.
> This is something we want to add in our legacy system while working on our
> new system. If such a functionality exist any help with respect to this
> would be highly appreciated. Thanks, Luqman
> ------------------------------
> View this message in context: database table polling
> <http://apache-ignite-users.70518.x6.nabble.com/database-table-polling-tp15006.html>
> Sent from the Apache Ignite Users mailing list archive
> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/database-
> table-polling-tp15006p15029.html
> To unsubscribe from database table polling, click here
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=15006&code=bHVxbWFuYWhtZWRAZ21haWwuY29tfDE1MDA2fC04NjE4NDY2OTY=>
> .
> NAML
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
Re: database table polling
Posted by Jörn Franke <jo...@gmail.com>.
Which database? Some databases can notify an application id they are updated. You could read these updates with a Java application and insert them in the ignite cache.
> On 17. Jul 2017, at 16:45, luqmanahmad <lu...@gmail.com> wrote:
>
> Hi, We have a legacy system, 15 years old at-least, which we are working on and trying to bring everything inside the grid - but in the meanwhile we have to support our existing system as well. In the legacy system we have a database polling mechanism ( please don't roll your eyes :D ) which is constantly looking for a specific table for new rows which has got references to other bits, and is quite expensive in terms of resources plus there are many other problems as well. I was wondering if we have a cache which is connected through cache store to this table - Is there any functionality available in ignite to poll it directly for all the new rows. This is something we want to add in our legacy system while working on our new system. If such a functionality exist any help with respect to this would be highly appreciated. Thanks, Luqman
> View this message in context: database table polling
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: database table polling
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Luqman,
Please take a look at continuous queries.[1]
[1] https://apacheignite.readme.io/docs/continuous-queries
On Mon, Jul 17, 2017 at 5:45 PM, luqmanahmad <lu...@gmail.com> wrote:
> Hi, We have a legacy system, 15 years old at-least, which we are working
> on and trying to bring everything inside the grid - but in the meanwhile we
> have to support our existing system as well. In the legacy system we have a
> database polling mechanism ( please don't roll your eyes :D ) which is
> constantly looking for a specific table for new rows which has got
> references to other bits, and is quite expensive in terms of resources plus
> there are many other problems as well. I was wondering if we have a cache
> which is connected through cache store to this table - Is there any
> functionality available in ignite to poll it directly for all the new rows.
> This is something we want to add in our legacy system while working on our
> new system. If such a functionality exist any help with respect to this
> would be highly appreciated. Thanks, Luqman
> ------------------------------
> View this message in context: database table polling
> <http://apache-ignite-users.70518.x6.nabble.com/database-table-polling-tp15006.html>
> Sent from the Apache Ignite Users mailing list archive
> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>
--
Best regards,
Andrey V. Mashenkov
Re: database table polling
Posted by luqmanahmad <lu...@gmail.com>.
Val,
This might work and adding continuous queries on top, it will fly. So silly
I am using loadCache() every where in the new development but couldn't
think of it for this one.
Luqman
On 17 Jul 2017 23:18, "vkulichenko [via Apache Ignite Users]" <
ml+s70518n15037h72@n6.nabble.com> wrote:
Luqman,
Got it. This is not available out of the box. But I think you can just
create a cluster singleton service that will periodically call
IgniteCache#loadCache to poll the data. Will this work?
-Val
------------------------------
If you reply to this email, your message will be added to the discussion
below:
http://apache-ignite-users.70518.x6.nabble.com/database-
table-polling-tp15006p15037.html
To unsubscribe from database table polling, click here
<http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=15006&code=bHVxbWFuYWhtZWRAZ21haWwuY29tfDE1MDA2fC04NjE4NDY2OTY=>
.
NAML
<http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/database-table-polling-tp15006p15038.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: database table polling
Posted by vkulichenko <va...@gmail.com>.
Luqman,
Got it. This is not available out of the box. But I think you can just
create a cluster singleton service that will periodically call
IgniteCache#loadCache to poll the data. Will this work?
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/database-table-polling-tp15006p15037.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: database table polling
Posted by luqmanahmad <lu...@gmail.com>.
Hi Val,
In our legacy system database polling is achieved through a separate table,
let say P, in case employee is updated then a new row gets added to P table
with the primary key of the employee which is updated and then a background
process keeps polling that table for the update.
In new system it will be completely in the grid but for the time being we
have to optimise the legacy system as it is not very resource effective and
during the peak times it just collapse.
What I was thinking if I create a cache with store for this P table is
there any technique which can poll that table with some interval for the
newly added rows and by using bits from our new development we can make
this process improve through ignite
I know its not an ideal use case but just trying to find an optimise way to
achieve the better results for the legacy bits.
Thanks,
Luqman
On 17 Jul 2017 7:18 pm, "vkulichenko [via Apache Ignite Users]" <
ml+s70518n15027h62@n6.nabble.com> wrote:
Hi Luqman,
What exactly do you mean by "new rows"? Is the database updated directly
and you then need to refresh cache with these updates?
-Val
------------------------------
If you reply to this email, your message will be added to the discussion
below:
http://apache-ignite-users.70518.x6.nabble.com/database-
table-polling-tp15006p15027.html
To unsubscribe from database table polling, click here
<http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=15006&code=bHVxbWFuYWhtZWRAZ21haWwuY29tfDE1MDA2fC04NjE4NDY2OTY=>
.
NAML
<http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/database-table-polling-tp15006p15032.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: database table polling
Posted by vkulichenko <va...@gmail.com>.
Hi Luqman,
What exactly do you mean by "new rows"? Is the database updated directly and
you then need to refresh cache with these updates?
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/database-table-polling-tp15006p15027.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.