You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Pavel Tupitsyn <pt...@apache.org> on 2017/05/26 17:06:21 UTC
New blog post: ADO.NET Ignite Cache Store
Short writeup with example code, inspired by user questions:
ADO.NET + Binary Mode Cache Store
https://ptupitsyn.github.io/Ado-Net-Cache-Store/
Re: New blog post: ADO.NET Ignite Cache Store
Posted by Denis Magda <dm...@apache.org>.
Great, Pavel! You rock!
Prachi, could you send the post to DZone and publish on Ignite’s page?
—
Denis
> On May 26, 2017, at 10:06 AM, Pavel Tupitsyn <pt...@apache.org> wrote:
>
> Short writeup with example code, inspired by user questions:
>
> ADO.NET <http://ado.net/> + Binary Mode Cache Store
> https://ptupitsyn.github.io/Ado-Net-Cache-Store/ <https://ptupitsyn.github.io/Ado-Net-Cache-Store/>
>
Re: New blog post: ADO.NET Ignite Cache Store
Posted by Denis Magda <dm...@apache.org>.
Great, Pavel! You rock!
Prachi, could you send the post to DZone and publish on Ignite’s page?
—
Denis
> On May 26, 2017, at 10:06 AM, Pavel Tupitsyn <pt...@apache.org> wrote:
>
> Short writeup with example code, inspired by user questions:
>
> ADO.NET <http://ado.net/> + Binary Mode Cache Store
> https://ptupitsyn.github.io/Ado-Net-Cache-Store/ <https://ptupitsyn.github.io/Ado-Net-Cache-Store/>
>
Re: New blog post: ADO.NET Ignite Cache Store
Posted by Pavel Tupitsyn <pt...@apache.org>.
Chetan,
Apache Ignite is free and open source, you can ask questions on user list
and get answers from community.
GridGain provides paid features and services (based on Apache Ignite), you
have to contact them if you have questions.
Pavel
On Fri, Jun 2, 2017 at 4:11 PM, Chetan D <cc...@gmail.com> wrote:
> Hi Pavel,
>
> I thought grid gain is built on Apache ignite so asked you.
> and when i google anything related to grid gain i get suggestion for
> Apache ignite,ignite.net.
>
> i am very much confused what is Apache ignite,ignite.net and grid gain.
>
> even when i google also i am not finding information which is useful.
>
> Your examples helped a lot and thank you for that.
>
> On Fri, Jun 2, 2017 at 6:21 PM, Pavel Tupitsyn <pt...@apache.org>
> wrote:
>
>> Hi Chetan, you'll have to contact GridGain company with these questions.
>>
>> On Fri, Jun 2, 2017 at 3:46 PM, Chetan D <cc...@gmail.com> wrote:
>>
>>> Hi Pavel,
>>>
>>> Your examples helped me understand things well in apache ignite but we
>>> are planning to use GRIDGAIN and i have the requirement which as follows.
>>>
>>> 1. Need data replication from DB to Cache
>>>
>>> 2. Do we need to write code to achieve this or some
>>> configuration(Read/write)
>>>
>>> 3. Need sample implementation with respect to above diagram using
>>> dot net.
>>>
>>> 4. As mentioned we have several cache modes such as replicated,
>>> partitioned, local, configuration and atomic write order mode.
>>>
>>> 5. Need detailed implementation of all the above with respect to
>>> dot net
>>>
>>> 6. Fail over handling and example
>>>
>>> 7. Need documents with respect to grid gain installation and
>>> usage.
>>>
>>> 8. Need examples on node singleton and cluster singleton
>>>
>>>
>>> These are all with respect to Grid gain.
>>>
>>> Just to brief adding data to cache should happen without writing code
>>> may be some configuration, need documentation with respect to this and an
>>> example with respect to dot net.
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, May 31, 2017 at 4:13 PM, Pavel Tupitsyn <pt...@apache.org>
>>> wrote:
>>>
>>>> You have to handle the deletion manually in some way or another:
>>>> delete entry from Ignite when deleting it from DB, or use some DB
>>>> change listener.
>>>>
>>>> On Wed, May 31, 2017 at 1:13 PM, Chetan D <cc...@gmail.com> wrote:
>>>>
>>>>> Hi Pavel,
>>>>>
>>>>> I was going through all methods and trying to implement cache using
>>>>> ado.net while doing that i came across some scenario which are as
>>>>> follows,
>>>>>
>>>>> 1.Data is added to cache and when trying to retrieve it using key
>>>>> first it checks if it is present in cache then gets it from cache if not
>>>>> then there will be a db call and fetches the data from DB and stores it in
>>>>> cache which again follows for the next time as well.
>>>>>
>>>>> 2. The second scenario is i have loaded cache with two values from DB
>>>>> taking your cars table as example.Once i have loaded cache then i went
>>>>> ahead and deleted a record from DB.
>>>>> so when i use a get() method by passing the key which i deleted in DB
>>>>> it actually shows me the value which because it is present in cache.
>>>>>
>>>>> so my question is when there is a record deleted in DB it should be
>>>>> deleted in cache. when user tries to retrieve any information which is not
>>>>> there in DB it should not be displayed from cache as well.
>>>>>
>>>>>
>>>>> Can you please guide me how to handle this.
>>>>>
>>>>> Regards
>>>>> Chetan D
>>>>>
>>>>>
>>>>>
>>>>> On Tue, May 30, 2017 at 5:36 PM, Pavel Tupitsyn <pt...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Basically, data streamer does the same thing as cache.Put() in a loop.
>>>>>> Data streamer is more efficient on large amounts of keys.
>>>>>>
>>>>>> 1) Yes, I guess there is no different than using tables
>>>>>> 2) Data streamer performs batched updates.
>>>>>> IDataStreamer.PerNodeBufferSize controls how many entries are
>>>>>> batched before being sent to remote nodes.
>>>>>> 3) Once the streaming is complete (IDataStreamer.Flush() or
>>>>>> IDataStreamer.Close(false)), all data is guaranteed to be in the cache. So
>>>>>> you can do cache.Get.
>>>>>>
>>>>>> On Tue, May 30, 2017 at 2:49 PM, Chetan D <cc...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Pavel,
>>>>>>>
>>>>>>> Thank you for the quick response.
>>>>>>> I tried using data streamer to get all the values in a table and
>>>>>>> store it in cache. while i was doing that i came across a situation where
>>>>>>> in i need to store data to cache from SQL VIEWS.
>>>>>>>
>>>>>>> 1.Is it possible to load cache from sql views?
>>>>>>> 2.with respect to data streamer, when i use AddData() method will it
>>>>>>> add data to cache which i specified while creating data streamer or the
>>>>>>> data will be stored in data streamer itself?
>>>>>>> 3. when i use data streamer can i use the same way such as
>>>>>>> cache.get() method or by using a foreach loop to get cache or is it
>>>>>>> different with respect to datastreamer?
>>>>>>>
>>>>>>>
>>>>>>> Regards
>>>>>>> Chetan D
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, May 29, 2017 at 6:45 PM, Pavel Tupitsyn <
>>>>>>> ptupitsyn@apache.org> wrote:
>>>>>>>
>>>>>>>> 1) ICache implements IEnumerable, so you can do `foreach (var entry
>>>>>>>> in cache)` or `cache.ToList()`.
>>>>>>>> 2) To bulk load the data into cache you can either use
>>>>>>>> IDataStreamer [1] or, in case with cache store, ICacheStore.LoadCache
>>>>>>>> method.
>>>>>>>> 3) No, you have to handle this manually.
>>>>>>>> 4) GetAll returns multiple cache entries by a set of known keys.
>>>>>>>>
>>>>>>>> [1] https://apacheignite-net.readme.io/docs/data-streamers
>>>>>>>>
>>>>>>>> On Mon, May 29, 2017 at 3:28 PM, Chetan D <cc...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Pavel,
>>>>>>>>>
>>>>>>>>> Thanks for the quick response. code helped me a lot.
>>>>>>>>> I have some general questions,
>>>>>>>>>
>>>>>>>>> 1.is it possible to get all data stored in cache without calling
>>>>>>>>> explicit get method if so how?
>>>>>>>>> 2.can i fetch all data from sql table and store it in cache
>>>>>>>>> without using any of the loops such as for each.
>>>>>>>>> 3.if i add any data in table will my cache gets updated
>>>>>>>>> automatically?
>>>>>>>>> 4.how can i make use of GetAll() method? will this fetch all cache
>>>>>>>>> data?
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> Chetan D
>>>>>>>>>
>>>>>>>>> On Fri, May 26, 2017 at 10:36 PM, Pavel Tupitsyn <
>>>>>>>>> ptupitsyn@apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> Short writeup with example code, inspired by user questions:
>>>>>>>>>>
>>>>>>>>>> ADO.NET + Binary Mode Cache Store
>>>>>>>>>> https://ptupitsyn.github.io/Ado-Net-Cache-Store/
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: New blog post: ADO.NET Ignite Cache Store
Posted by Pavel Tupitsyn <pt...@apache.org>.
You have to handle the deletion manually in some way or another:
delete entry from Ignite when deleting it from DB, or use some DB change
listener.
On Wed, May 31, 2017 at 1:13 PM, Chetan D <cc...@gmail.com> wrote:
> Hi Pavel,
>
> I was going through all methods and trying to implement cache using
> ado.net while doing that i came across some scenario which are as follows,
>
> 1.Data is added to cache and when trying to retrieve it using key first it
> checks if it is present in cache then gets it from cache if not then there
> will be a db call and fetches the data from DB and stores it in cache which
> again follows for the next time as well.
>
> 2. The second scenario is i have loaded cache with two values from DB
> taking your cars table as example.Once i have loaded cache then i went
> ahead and deleted a record from DB.
> so when i use a get() method by passing the key which i deleted in DB it
> actually shows me the value which because it is present in cache.
>
> so my question is when there is a record deleted in DB it should be
> deleted in cache. when user tries to retrieve any information which is not
> there in DB it should not be displayed from cache as well.
>
>
> Can you please guide me how to handle this.
>
> Regards
> Chetan D
>
>
>
> On Tue, May 30, 2017 at 5:36 PM, Pavel Tupitsyn <pt...@apache.org>
> wrote:
>
>> Basically, data streamer does the same thing as cache.Put() in a loop.
>> Data streamer is more efficient on large amounts of keys.
>>
>> 1) Yes, I guess there is no different than using tables
>> 2) Data streamer performs batched updates. IDataStreamer.PerNodeBufferSize
>> controls how many entries are batched before being sent to remote nodes.
>> 3) Once the streaming is complete (IDataStreamer.Flush() or
>> IDataStreamer.Close(false)), all data is guaranteed to be in the cache. So
>> you can do cache.Get.
>>
>> On Tue, May 30, 2017 at 2:49 PM, Chetan D <cc...@gmail.com> wrote:
>>
>>> Hi Pavel,
>>>
>>> Thank you for the quick response.
>>> I tried using data streamer to get all the values in a table and store
>>> it in cache. while i was doing that i came across a situation where in i
>>> need to store data to cache from SQL VIEWS.
>>>
>>> 1.Is it possible to load cache from sql views?
>>> 2.with respect to data streamer, when i use AddData() method will it add
>>> data to cache which i specified while creating data streamer or the data
>>> will be stored in data streamer itself?
>>> 3. when i use data streamer can i use the same way such as cache.get()
>>> method or by using a foreach loop to get cache or is it different with
>>> respect to datastreamer?
>>>
>>>
>>> Regards
>>> Chetan D
>>>
>>>
>>>
>>>
>>> On Mon, May 29, 2017 at 6:45 PM, Pavel Tupitsyn <pt...@apache.org>
>>> wrote:
>>>
>>>> 1) ICache implements IEnumerable, so you can do `foreach (var entry in
>>>> cache)` or `cache.ToList()`.
>>>> 2) To bulk load the data into cache you can either use IDataStreamer
>>>> [1] or, in case with cache store, ICacheStore.LoadCache method.
>>>> 3) No, you have to handle this manually.
>>>> 4) GetAll returns multiple cache entries by a set of known keys.
>>>>
>>>> [1] https://apacheignite-net.readme.io/docs/data-streamers
>>>>
>>>> On Mon, May 29, 2017 at 3:28 PM, Chetan D <cc...@gmail.com> wrote:
>>>>
>>>>> Hi Pavel,
>>>>>
>>>>> Thanks for the quick response. code helped me a lot.
>>>>> I have some general questions,
>>>>>
>>>>> 1.is it possible to get all data stored in cache without calling
>>>>> explicit get method if so how?
>>>>> 2.can i fetch all data from sql table and store it in cache without
>>>>> using any of the loops such as for each.
>>>>> 3.if i add any data in table will my cache gets updated automatically?
>>>>> 4.how can i make use of GetAll() method? will this fetch all cache
>>>>> data?
>>>>>
>>>>> Regards
>>>>> Chetan D
>>>>>
>>>>> On Fri, May 26, 2017 at 10:36 PM, Pavel Tupitsyn <ptupitsyn@apache.org
>>>>> > wrote:
>>>>>
>>>>>> Short writeup with example code, inspired by user questions:
>>>>>>
>>>>>> ADO.NET + Binary Mode Cache Store
>>>>>> https://ptupitsyn.github.io/Ado-Net-Cache-Store/
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: New blog post: ADO.NET Ignite Cache Store
Posted by Chetan D <cc...@gmail.com>.
Hi Pavel,
I was going through all methods and trying to implement cache using ado.net
while doing that i came across some scenario which are as follows,
1.Data is added to cache and when trying to retrieve it using key first it
checks if it is present in cache then gets it from cache if not then there
will be a db call and fetches the data from DB and stores it in cache which
again follows for the next time as well.
2. The second scenario is i have loaded cache with two values from DB
taking your cars table as example.Once i have loaded cache then i went
ahead and deleted a record from DB.
so when i use a get() method by passing the key which i deleted in DB it
actually shows me the value which because it is present in cache.
so my question is when there is a record deleted in DB it should be deleted
in cache. when user tries to retrieve any information which is not there in
DB it should not be displayed from cache as well.
Can you please guide me how to handle this.
Regards
Chetan D
On Tue, May 30, 2017 at 5:36 PM, Pavel Tupitsyn <pt...@apache.org>
wrote:
> Basically, data streamer does the same thing as cache.Put() in a loop.
> Data streamer is more efficient on large amounts of keys.
>
> 1) Yes, I guess there is no different than using tables
> 2) Data streamer performs batched updates. IDataStreamer.PerNodeBufferSize
> controls how many entries are batched before being sent to remote nodes.
> 3) Once the streaming is complete (IDataStreamer.Flush() or
> IDataStreamer.Close(false)), all data is guaranteed to be in the cache. So
> you can do cache.Get.
>
> On Tue, May 30, 2017 at 2:49 PM, Chetan D <cc...@gmail.com> wrote:
>
>> Hi Pavel,
>>
>> Thank you for the quick response.
>> I tried using data streamer to get all the values in a table and store it
>> in cache. while i was doing that i came across a situation where in i need
>> to store data to cache from SQL VIEWS.
>>
>> 1.Is it possible to load cache from sql views?
>> 2.with respect to data streamer, when i use AddData() method will it add
>> data to cache which i specified while creating data streamer or the data
>> will be stored in data streamer itself?
>> 3. when i use data streamer can i use the same way such as cache.get()
>> method or by using a foreach loop to get cache or is it different with
>> respect to datastreamer?
>>
>>
>> Regards
>> Chetan D
>>
>>
>>
>>
>> On Mon, May 29, 2017 at 6:45 PM, Pavel Tupitsyn <pt...@apache.org>
>> wrote:
>>
>>> 1) ICache implements IEnumerable, so you can do `foreach (var entry in
>>> cache)` or `cache.ToList()`.
>>> 2) To bulk load the data into cache you can either use IDataStreamer [1]
>>> or, in case with cache store, ICacheStore.LoadCache method.
>>> 3) No, you have to handle this manually.
>>> 4) GetAll returns multiple cache entries by a set of known keys.
>>>
>>> [1] https://apacheignite-net.readme.io/docs/data-streamers
>>>
>>> On Mon, May 29, 2017 at 3:28 PM, Chetan D <cc...@gmail.com> wrote:
>>>
>>>> Hi Pavel,
>>>>
>>>> Thanks for the quick response. code helped me a lot.
>>>> I have some general questions,
>>>>
>>>> 1.is it possible to get all data stored in cache without calling
>>>> explicit get method if so how?
>>>> 2.can i fetch all data from sql table and store it in cache without
>>>> using any of the loops such as for each.
>>>> 3.if i add any data in table will my cache gets updated automatically?
>>>> 4.how can i make use of GetAll() method? will this fetch all cache data?
>>>>
>>>> Regards
>>>> Chetan D
>>>>
>>>> On Fri, May 26, 2017 at 10:36 PM, Pavel Tupitsyn <pt...@apache.org>
>>>> wrote:
>>>>
>>>>> Short writeup with example code, inspired by user questions:
>>>>>
>>>>> ADO.NET + Binary Mode Cache Store
>>>>> https://ptupitsyn.github.io/Ado-Net-Cache-Store/
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: New blog post: ADO.NET Ignite Cache Store
Posted by Pavel Tupitsyn <pt...@apache.org>.
Basically, data streamer does the same thing as cache.Put() in a loop.
Data streamer is more efficient on large amounts of keys.
1) Yes, I guess there is no different than using tables
2) Data streamer performs batched updates. IDataStreamer.PerNodeBufferSize
controls how many entries are batched before being sent to remote nodes.
3) Once the streaming is complete (IDataStreamer.Flush() or
IDataStreamer.Close(false)), all data is guaranteed to be in the cache. So
you can do cache.Get.
On Tue, May 30, 2017 at 2:49 PM, Chetan D <cc...@gmail.com> wrote:
> Hi Pavel,
>
> Thank you for the quick response.
> I tried using data streamer to get all the values in a table and store it
> in cache. while i was doing that i came across a situation where in i need
> to store data to cache from SQL VIEWS.
>
> 1.Is it possible to load cache from sql views?
> 2.with respect to data streamer, when i use AddData() method will it add
> data to cache which i specified while creating data streamer or the data
> will be stored in data streamer itself?
> 3. when i use data streamer can i use the same way such as cache.get()
> method or by using a foreach loop to get cache or is it different with
> respect to datastreamer?
>
>
> Regards
> Chetan D
>
>
>
>
> On Mon, May 29, 2017 at 6:45 PM, Pavel Tupitsyn <pt...@apache.org>
> wrote:
>
>> 1) ICache implements IEnumerable, so you can do `foreach (var entry in
>> cache)` or `cache.ToList()`.
>> 2) To bulk load the data into cache you can either use IDataStreamer [1]
>> or, in case with cache store, ICacheStore.LoadCache method.
>> 3) No, you have to handle this manually.
>> 4) GetAll returns multiple cache entries by a set of known keys.
>>
>> [1] https://apacheignite-net.readme.io/docs/data-streamers
>>
>> On Mon, May 29, 2017 at 3:28 PM, Chetan D <cc...@gmail.com> wrote:
>>
>>> Hi Pavel,
>>>
>>> Thanks for the quick response. code helped me a lot.
>>> I have some general questions,
>>>
>>> 1.is it possible to get all data stored in cache without calling
>>> explicit get method if so how?
>>> 2.can i fetch all data from sql table and store it in cache without
>>> using any of the loops such as for each.
>>> 3.if i add any data in table will my cache gets updated automatically?
>>> 4.how can i make use of GetAll() method? will this fetch all cache data?
>>>
>>> Regards
>>> Chetan D
>>>
>>> On Fri, May 26, 2017 at 10:36 PM, Pavel Tupitsyn <pt...@apache.org>
>>> wrote:
>>>
>>>> Short writeup with example code, inspired by user questions:
>>>>
>>>> ADO.NET + Binary Mode Cache Store
>>>> https://ptupitsyn.github.io/Ado-Net-Cache-Store/
>>>>
>>>>
>>>
>>
>
Re: New blog post: ADO.NET Ignite Cache Store
Posted by Chetan D <cc...@gmail.com>.
Hi Pavel,
Thank you for the quick response.
I tried using data streamer to get all the values in a table and store it
in cache. while i was doing that i came across a situation where in i need
to store data to cache from SQL VIEWS.
1.Is it possible to load cache from sql views?
2.with respect to data streamer, when i use AddData() method will it add
data to cache which i specified while creating data streamer or the data
will be stored in data streamer itself?
3. when i use data streamer can i use the same way such as cache.get()
method or by using a foreach loop to get cache or is it different with
respect to datastreamer?
Regards
Chetan D
On Mon, May 29, 2017 at 6:45 PM, Pavel Tupitsyn <pt...@apache.org>
wrote:
> 1) ICache implements IEnumerable, so you can do `foreach (var entry in
> cache)` or `cache.ToList()`.
> 2) To bulk load the data into cache you can either use IDataStreamer [1]
> or, in case with cache store, ICacheStore.LoadCache method.
> 3) No, you have to handle this manually.
> 4) GetAll returns multiple cache entries by a set of known keys.
>
> [1] https://apacheignite-net.readme.io/docs/data-streamers
>
> On Mon, May 29, 2017 at 3:28 PM, Chetan D <cc...@gmail.com> wrote:
>
>> Hi Pavel,
>>
>> Thanks for the quick response. code helped me a lot.
>> I have some general questions,
>>
>> 1.is it possible to get all data stored in cache without calling
>> explicit get method if so how?
>> 2.can i fetch all data from sql table and store it in cache without using
>> any of the loops such as for each.
>> 3.if i add any data in table will my cache gets updated automatically?
>> 4.how can i make use of GetAll() method? will this fetch all cache data?
>>
>> Regards
>> Chetan D
>>
>> On Fri, May 26, 2017 at 10:36 PM, Pavel Tupitsyn <pt...@apache.org>
>> wrote:
>>
>>> Short writeup with example code, inspired by user questions:
>>>
>>> ADO.NET + Binary Mode Cache Store
>>> https://ptupitsyn.github.io/Ado-Net-Cache-Store/
>>>
>>>
>>
>
Re: New blog post: ADO.NET Ignite Cache Store
Posted by Pavel Tupitsyn <pt...@apache.org>.
+user
On Mon, May 29, 2017 at 4:15 PM, Pavel Tupitsyn <pt...@apache.org>
wrote:
> 1) ICache implements IEnumerable, so you can do `foreach (var entry in
> cache)` or `cache.ToList()`.
> 2) To bulk load the data into cache you can either use IDataStreamer [1]
> or, in case with cache store, ICacheStore.LoadCache method.
> 3) No, you have to handle this manually.
> 4) GetAll returns multiple cache entries by a set of known keys.
>
> [1] https://apacheignite-net.readme.io/docs/data-streamers
>
> On Mon, May 29, 2017 at 3:28 PM, Chetan D <cc...@gmail.com> wrote:
>
>> Hi Pavel,
>>
>> Thanks for the quick response. code helped me a lot.
>> I have some general questions,
>>
>> 1.is it possible to get all data stored in cache without calling
>> explicit get method if so how?
>> 2.can i fetch all data from sql table and store it in cache without using
>> any of the loops such as for each.
>> 3.if i add any data in table will my cache gets updated automatically?
>> 4.how can i make use of GetAll() method? will this fetch all cache data?
>>
>> Regards
>> Chetan D
>>
>> On Fri, May 26, 2017 at 10:36 PM, Pavel Tupitsyn <pt...@apache.org>
>> wrote:
>>
>>> Short writeup with example code, inspired by user questions:
>>>
>>> ADO.NET + Binary Mode Cache Store
>>> https://ptupitsyn.github.io/Ado-Net-Cache-Store/
>>>
>>>
>>
>