Re: What does mean caching database exactly ?

[vkulichenko <va...@gmail.com>]

Hi,

Can you please properly subscribe to the mailing list so that the community
can receive email notifications? Here is the instruction:
http://apache-ignite-users.70518.x6.nabble.com/mailing_list/MailingListOptions.jtp?forum=1


tomk wrote
> I consider caching database (postgresSQL). 
> However, I don't understand way of working ignite, although I looked for
> docs.
> Tell me please,
> Is it possible to cache into RAM table from postgres SQL and fastly answer
> to SQL queries ? The aim is make it faster to query postgres.
> I managed to establish connection with postgres (JDBC). (I separately
> created jdbc connection and create data source from that).  Now, I would
> like to cache into RAM memory entire table (using this JDBC connection) .
> Consequently, thanks to caching to memory RAM I am going to query postgres
> by SQL with low latencies.
> I am not sure if my way is good
> I am looking at  example: 
> https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples/datagrid/store/jdbc

If you run SQL in Ignite, it touches only data that is already in memory. So
you have to load the whole data set that you're going to work with into
Ignite cache prior to running the queries. But this should give you good
performance improvement.

The data can be loaded from the Postgres as well as any other storage. For
details refer to this page: https://apacheignite.readme.io/docs/data-loading

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/What-does-mean-caching-database-exactly-tp4043p4079.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: What does mean caching database exactly ?

[vkulichenko <va...@gmail.com>]

Hi,

Can you please subscribe to the mailing list as I asked earlier? You just
need to send couple of emails to do this. See details here:
http://apache-ignite-users.70518.x6.nabble.com/mailing_list/MailingListOptions.jtp?forum=1


tomk wrote
> I tell you what I am going to do. Then you can understand my problem.
> I have some app A that save data to postgres. Simply, it inserts row.  
> 
> Second app B is querying these data. The idea is app B can fast query data
> using cached created by Ignite. 
> 
> (0) Should server should be from java code ? 
> 
> So I can see different ways:
> 
> (1) Second app (B) is running in mode server. At begin, it loadCache (all
> data from database), then it may fastly queries data. 
> Here, is some doubt - does it be able to see newly inserted data by app A
> ? App A (in this scenario) insert data into postgres - it doesnt know
> about ignite.  
> From what I did read it is possible to create triggers in database, that
> by REST API of ignite force loading to cache newly inserted data.  I don't
> know if it is solution ok.  Maybe, it can see newly inserted data
> automatically ?
> 
> (2) First App (A, server mode) is using write-throught and read-throught.
> Then, newly inserted data (by A) are loaded into cache RAM and
> automatically to postgres. (It seems to be very nice).   Then clients (eg
> B, client mode) that will connect to ignite server (JDBC/REST) can SQL
> query data in RAM memory - so also newly inserted. 
> 
> (3) First App (A, server mode)) does only write-throught.  Second App (B)
> (client mode) loads entire data into cache (once time -  during starting).
> Then it can SQL query data in cache and be able to see newly inserted data
> by app A (thanks to write-throught).
> 
> The difference between (2) and (3) is small - who should loadCache data ?
> A or B?
> 
> 
> As you can see main aim: cache data from postgres  for query. New data are
> permanently inserted. 
> It seems that ideal solution is starting nodes in mode server by
> bin/ignite.sh. 
> Then App A do write-through on this sever node. When it comes App B it
> simply JDBC connect to this server. Also REST queries possible.

If DB is updated directly, Ignite is not aware about these changes. Such
awareness can be achieved with DB triggers, periodic polling, etc., but the
most effective approach is to use Ignite API to read and update the cache in
both apps. There is no much difference between approaches 2 and 3, it's up
to you who will load the data. Only make sure that all the required data is
in memory prior to running the queries.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/What-does-mean-caching-database-exactly-tp4043p4104.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: What does mean caching database exactly ?

[vkulichenko <va...@gmail.com>]

1. You can have a cluster consisting of several server nodes with one or more
caches storing some data. Then you can have multiple applications, each will
start a client node to connect to the same cluster and use this data. JDBC
and REST can also be used. So the answer is definitely YES, this is
possible.

2. You can do simple gets by key as well as SQL queries for more complicated
requests.

3. This link is not about read/write-through, but rather about bulk loading
of large amounts of data from DB into the cache. You should use one of two
approaches described there prior to executing SQL queries, because SQL does
not support read-through and queries only data that is already in memory.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/What-does-mean-caching-database-exactly-tp4043p4089.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.