You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@openjpa.apache.org by Drewz <an...@elasticpath.com> on 2009/10/08 20:52:14 UTC
Re: How to write a L2 cache plug in that honors the DataCache
annotation?
Frederic,
How did your tangosol implementation go? I'm doing similar work, and thus
would be interested to see if this plugin was contributed back to OpenJPA.
We would definitely like to expand on any work already done on this.
Thanks,
Drew
Frederic_Bellier wrote:
>
> Hi Pinaki -
>
> Thank you for the sample plug in. It was very helpful and really help me
> get started in putting together a Tangosol plug in for kodo/openjpa.
> Right now we are only testing the plug in with openjpa - I will try to use
> it with kodo when the plug in will be finished.
>
> I spent quite some time going through the openjpa code. In particular
> everything that has to do with store manager - broker - datacache -
> configuration - etc...
> I am writting a plug in for Tangosol that has to be as efficient as I can
> make it - our focus here is code quality and performance - so this plug in
> as to be clean and efficient. So far it is going pretty well - basically
> it
> is working but not fully tested. But I have a few questions to make sure I
> did it right.
>
> I noticed that the store manager (DataCacheStoreManager) in openjpa
> handles
> the multiple cache concept very well (nice to see some good code/design -
> congratulation to whomever wrote it). So I am writing a DataCache plugin
> that will leverage the good work done in the StoreManager - that is to say
> allow the store manager to call the right specific cache from the
> DataCacheManager since it already looked up the proper cache name and in
> fact was coded to do this well - this code is in DataCacheStoreManager.
> updateCaches() method (again that is great code and it is exactly what I
> was hoping it will do).
> Your sample was leveraging the system wide datacache and do the lookup
> itself to find out which cache to use. So basically this was duplicating
> quite some work done already by the store manager and was not very
> efficient.
> Anyway - the new plugin has a new DataCacheManager - a new DataCache - a
> new SystemDataCache which basically is what you sent me and a new
> QueryCache.
>
> Here are my questions:
>
> Q: difference between datacache and systemDataCache?
> So I created a DataCache manager that manages multiple datacaches - one
> for
> each named cache - plus one systemWideDataCache and one
> systemWideQueryCache.
> I kept around the concept of the systemWideDataCache for now - becuase I
> can see the broker gets a reference to it. But I do not see it being used.
> Except by OpenJPAEntitymanagerFactory.getStoreCache().
> What is the exact concept behind the system wide datacache? Why does the
> broker gets a reference to it right when it is created?
> Is it just a simplified way to get a proxy to all the caches in case
> someone needs it and does not know which cache exactly they should get? It
> seems to just be fullfiling this role.
>
> Q: QueryCache scope?
> As for the QueryCache - my undertsanding is that this cache must be shared
> across multiple JVMs because otherwise (when data gets removed from the L2
> cache for example) the QueryCache could return wrong information. Is this
> correct?
>
> Code comment:
> The DataCacheStoreManager.transformToVersionSafePCDatas() line 261. This
> method should call either cache.containsAll() or cache.getAll(). The
> current implementation is not efficient becuase it makes one call to the
> cache for each element in the collection.
> The AbstractDataCache.putAllInternal() method suffers from the same
> issues.
> This is OK becuase I will overwrite this in the new plugin but I would
> think someone from the openjpa developer would want to fix that in there
> too. That way the base class could actually be used as a base class to
> write L2 cache plugins.
>
> There should be inside the openjpa.datacache namespace an
> AbstractDataCache
> class that has nothing to do with RemoteEvent and another one extending it
> that will also include the RemoteEvent methods. That way it will be easier
> to implement various L2 cache plugin on top of a clean class.
>
> OpenJPaID - The openjpaId class has a method to output a nice format
> key.toString() which is type+objectId. This method has been overwritten by
> children classes. Why is this? The overwritten method only outputs the
> objectId and therefore is not convenient when looking into the cache. Can
> this be reversed? Even the logging output is not useful with only the
> objectId (think about saving multiple object from a graph with same int
> value for PK - in such a case the output does not say which class was
> saved
> properly or not. The alternative is to use a nice String as the key for
> the
> cache but this is not efficient compare to using the hashcode of
> OpenJpaId.
>
> The EntitymanagerFactoryImpl.getStoreCache(String cacheName) . This method
> calls DataCacheManage.getDataCache(cacheName).
> I think it should call DataCachemanager.getDataCache(cacheName, true) to
> make sure if the cache does not exist it will get created. Unless of
> course
> this is the behavior specified by JPA spec. This is important becuase
> otherwise we get StoreCacheImpl reference with a DelegatingDataCache
> reference with a null cache reference inside. This generates a lot of bugs
> becuase it is not good practices to instanciate all the namedCache at
> startup since noone can know upfront whcih one will actually be used. The
> current DataCacheManagerImpl does not suffer from this issue becuase there
> is only one dataCache. But based on a DataCacheManager implementation with
> multiple named cache - which is what I am implementing now - then this
> does
> not work anymore.
> Moreover, I cannot chnage the behavior of getDataCache(String name)
> becuase
> the interface says:
> /**
> * Return the named data cache, or null if it does not exist.
> */
> public DataCache getDataCache(String name);
>
>
> Frederic.
>
> PS: Is openjpa interested in incorporating this Tangosol plugin code in
> the
> product when I am finished with it? It would better for us if we dont have
> to maintain it in the future. I would think openjpa would like to start
> including plugins for various L2 cache product and Coherence would be a
> good start.
> Moreover, this plugin is faster than the one that use to be in kodo -
> becuase I am removing all the issues that were in there.
>
>
>
>
>
> Pinaki Poddar
> <ppoddar@apache.o
> rg> To
> users@openjpa.apache.org
> cc
> 03/29/2008 11:11
> PM Subject
> Re: How to write a L2 cache plug in
> that honors the DataCache
> Please respond to annotation?
> users@openjpa.apa
> che.org
>
>
>
>
>
>
>
>
>
> Hi,
> The attached zip file contains the implementation of a OpenJPA DataCache
> plug-in for Coherence Distributed Caching Product. The plug-in honors the
> 'name' property value of @DataCache annotation in a persistent class X to
> cache instances of X to a corresponding named partition in Coherence. The
> plug-in
> is activated by the following configuration in META-INF/persistence.xml
>
> <property name="openjpa.DataCacheManager" value="coherence"/>
>
> The ProductDerivation magic wires up other appropriate plug-ins.
>
> To use it, add coherence-openjpa.jar to your classpath.
>
> This plug-in is somewhat differs in spirit from its counterpart
> TangosolCache (available in Kodo) in a sense that this one leaves *every*
> configuration of the cache to Coherence including how changes are
> communicated in a distributed environment i.e. OpenJPA effectively
> relinquishes its responsibility to broadcast changes via
> RemoteCommitProvider to other, possibly, remote L2 caches. This plug-in
> also
> recognizes that the instances of different classes may be cached to
> different Coherence partitions (Kodo version put all of them in one single
> named partition).
>
> The source code is provided (without any warranty or copyright). Usage is
> exemplified via JUnit Test cases and a simple persistence.xml
> configuration.
>
> The build script requires to be edited for your local environment.
>
> Good night & Good luck --
>
> Pinaki
> http://www.nabble.com/file/p16378911/coherence-openjpa.zip
> coherence-openjpa.zip
>
>
> --
> View this message in context:
> http://www.nabble.com/How-to-set-a-tangosol-cache-per-entity--tp16336199p16378911.html
>
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
>
>
>
>
--
View this message in context: http://n2.nabble.com/How-to-set-a-tangosol-cache-per-entity-tp210372p3790079.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: How to write a L2 cache plug in that honors the DataCache
annotation?
Posted by Kevin Sutter <kw...@gmail.com>.
FYI... Another example of a L2 Cache plugin for OpenJPA was done by
EhCache...
http://openjpa.apache.org/2009/07/28/ehcache-plugin-available-for-openjpa.html
http://ehcache.org/
http://ehcache.org/documentation/openjpa_provider.html
http://ehcache.sourceforge.net/download.html
On Thu, Oct 8, 2009 at 1:52 PM, Drewz <an...@elasticpath.com> wrote:
>
> Frederic,
>
> How did your tangosol implementation go? I'm doing similar work, and thus
> would be interested to see if this plugin was contributed back to OpenJPA.
> We would definitely like to expand on any work already done on this.
>
> Thanks,
>
> Drew
>
>
> Frederic_Bellier wrote:
> >
> > Hi Pinaki -
> >
> > Thank you for the sample plug in. It was very helpful and really help me
> > get started in putting together a Tangosol plug in for kodo/openjpa.
> > Right now we are only testing the plug in with openjpa - I will try to
> use
> > it with kodo when the plug in will be finished.
> >
> > I spent quite some time going through the openjpa code. In particular
> > everything that has to do with store manager - broker - datacache -
> > configuration - etc...
> > I am writting a plug in for Tangosol that has to be as efficient as I can
> > make it - our focus here is code quality and performance - so this plug
> in
> > as to be clean and efficient. So far it is going pretty well - basically
> > it
> > is working but not fully tested. But I have a few questions to make sure
> I
> > did it right.
> >
> > I noticed that the store manager (DataCacheStoreManager) in openjpa
> > handles
> > the multiple cache concept very well (nice to see some good code/design -
> > congratulation to whomever wrote it). So I am writing a DataCache plugin
> > that will leverage the good work done in the StoreManager - that is to
> say
> > allow the store manager to call the right specific cache from the
> > DataCacheManager since it already looked up the proper cache name and in
> > fact was coded to do this well - this code is in DataCacheStoreManager.
> > updateCaches() method (again that is great code and it is exactly what I
> > was hoping it will do).
> > Your sample was leveraging the system wide datacache and do the lookup
> > itself to find out which cache to use. So basically this was duplicating
> > quite some work done already by the store manager and was not very
> > efficient.
> > Anyway - the new plugin has a new DataCacheManager - a new DataCache - a
> > new SystemDataCache which basically is what you sent me and a new
> > QueryCache.
> >
> > Here are my questions:
> >
> > Q: difference between datacache and systemDataCache?
> > So I created a DataCache manager that manages multiple datacaches - one
> > for
> > each named cache - plus one systemWideDataCache and one
> > systemWideQueryCache.
> > I kept around the concept of the systemWideDataCache for now - becuase I
> > can see the broker gets a reference to it. But I do not see it being
> used.
> > Except by OpenJPAEntitymanagerFactory.getStoreCache().
> > What is the exact concept behind the system wide datacache? Why does the
> > broker gets a reference to it right when it is created?
> > Is it just a simplified way to get a proxy to all the caches in case
> > someone needs it and does not know which cache exactly they should get?
> It
> > seems to just be fullfiling this role.
> >
> > Q: QueryCache scope?
> > As for the QueryCache - my undertsanding is that this cache must be
> shared
> > across multiple JVMs because otherwise (when data gets removed from the
> L2
> > cache for example) the QueryCache could return wrong information. Is this
> > correct?
> >
> > Code comment:
> > The DataCacheStoreManager.transformToVersionSafePCDatas() line 261. This
> > method should call either cache.containsAll() or cache.getAll(). The
> > current implementation is not efficient becuase it makes one call to the
> > cache for each element in the collection.
> > The AbstractDataCache.putAllInternal() method suffers from the same
> > issues.
> > This is OK becuase I will overwrite this in the new plugin but I would
> > think someone from the openjpa developer would want to fix that in there
> > too. That way the base class could actually be used as a base class to
> > write L2 cache plugins.
> >
> > There should be inside the openjpa.datacache namespace an
> > AbstractDataCache
> > class that has nothing to do with RemoteEvent and another one extending
> it
> > that will also include the RemoteEvent methods. That way it will be
> easier
> > to implement various L2 cache plugin on top of a clean class.
> >
> > OpenJPaID - The openjpaId class has a method to output a nice format
> > key.toString() which is type+objectId. This method has been overwritten
> by
> > children classes. Why is this? The overwritten method only outputs the
> > objectId and therefore is not convenient when looking into the cache. Can
> > this be reversed? Even the logging output is not useful with only the
> > objectId (think about saving multiple object from a graph with same int
> > value for PK - in such a case the output does not say which class was
> > saved
> > properly or not. The alternative is to use a nice String as the key for
> > the
> > cache but this is not efficient compare to using the hashcode of
> > OpenJpaId.
> >
> > The EntitymanagerFactoryImpl.getStoreCache(String cacheName) . This
> method
> > calls DataCacheManage.getDataCache(cacheName).
> > I think it should call DataCachemanager.getDataCache(cacheName, true) to
> > make sure if the cache does not exist it will get created. Unless of
> > course
> > this is the behavior specified by JPA spec. This is important becuase
> > otherwise we get StoreCacheImpl reference with a DelegatingDataCache
> > reference with a null cache reference inside. This generates a lot of
> bugs
> > becuase it is not good practices to instanciate all the namedCache at
> > startup since noone can know upfront whcih one will actually be used. The
> > current DataCacheManagerImpl does not suffer from this issue becuase
> there
> > is only one dataCache. But based on a DataCacheManager implementation
> with
> > multiple named cache - which is what I am implementing now - then this
> > does
> > not work anymore.
> > Moreover, I cannot chnage the behavior of getDataCache(String name)
> > becuase
> > the interface says:
> > /**
> > * Return the named data cache, or null if it does not exist.
> > */
> > public DataCache getDataCache(String name);
> >
> >
> > Frederic.
> >
> > PS: Is openjpa interested in incorporating this Tangosol plugin code in
> > the
> > product when I am finished with it? It would better for us if we dont
> have
> > to maintain it in the future. I would think openjpa would like to start
> > including plugins for various L2 cache product and Coherence would be a
> > good start.
> > Moreover, this plugin is faster than the one that use to be in kodo -
> > becuase I am removing all the issues that were in there.
> >
> >
> >
> >
> >
> > Pinaki Poddar
> > <ppoddar@apache.o
> > rg>
> To
> > users@openjpa.apache.org
> >
> cc
> > 03/29/2008 11:11
> > PM
> Subject
> > Re: How to write a L2 cache plug
> in
> > that honors the DataCache
> > Please respond to annotation?
> > users@openjpa.apa
> > che.org
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Hi,
> > The attached zip file contains the implementation of a OpenJPA
> DataCache
> > plug-in for Coherence Distributed Caching Product. The plug-in honors the
> > 'name' property value of @DataCache annotation in a persistent class X to
> > cache instances of X to a corresponding named partition in Coherence. The
> > plug-in
> > is activated by the following configuration in META-INF/persistence.xml
> >
> > <property name="openjpa.DataCacheManager" value="coherence"/>
> >
> > The ProductDerivation magic wires up other appropriate plug-ins.
> >
> > To use it, add coherence-openjpa.jar to your classpath.
> >
> > This plug-in is somewhat differs in spirit from its counterpart
> > TangosolCache (available in Kodo) in a sense that this one leaves *every*
> > configuration of the cache to Coherence including how changes are
> > communicated in a distributed environment i.e. OpenJPA effectively
> > relinquishes its responsibility to broadcast changes via
> > RemoteCommitProvider to other, possibly, remote L2 caches. This plug-in
> > also
> > recognizes that the instances of different classes may be cached to
> > different Coherence partitions (Kodo version put all of them in one
> single
> > named partition).
> >
> > The source code is provided (without any warranty or copyright). Usage is
> > exemplified via JUnit Test cases and a simple persistence.xml
> > configuration.
> >
> > The build script requires to be edited for your local environment.
> >
> > Good night & Good luck --
> >
> > Pinaki
> > http://www.nabble.com/file/p16378911/coherence-openjpa.zip
> > coherence-openjpa.zip
> >
> >
> > --
> > View this message in context:
> >
> http://www.nabble.com/How-to-set-a-tangosol-cache-per-entity--tp16336199p16378911.html
> >
> > Sent from the OpenJPA Users mailing list archive at Nabble.com.
> >
> >
> >
> >
> >
>
> --
> View this message in context:
> http://n2.nabble.com/How-to-set-a-tangosol-cache-per-entity-tp210372p3790079.html
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
Re: How to write a L2 cache plug in that honors the DataCache
annotation?
Posted by Drewz <an...@elasticpath.com>.
Thanks Kevin and Pinaki for the responses. Plugin seems to be working with
any major problems so far, performance results are looking good.
If we make any changes or improvements, I'll try to follow up and post them
here.
Cheers,
Drew
fyi...
http://ppoddar.blogspot.com/
Drewz wrote:
>
> Frederic,
>
> How did your tangosol implementation go? I'm doing similar work, and thus
> would be interested to see if this plugin was contributed back to OpenJPA.
> We would definitely like to expand on any work already done on this.
>
> Thanks,
>
> Drew
>
>
>
--
View this message in context: http://n2.nabble.com/How-to-set-a-tangosol-cache-per-entity-tp210372p3876968.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: How to write a L2 cache plug in that honors the DataCache
annotation?
Posted by Pinaki Poddar <pp...@apache.org>.
fyi...
http://ppoddar.blogspot.com/
Frederic,
How did your tangosol implementation go? I'm doing similar work, and thus
would be interested to see if this plugin was contributed back to OpenJPA.
We would definitely like to expand on any work already done on this.
Thanks,
Drew
-----
Pinaki
--
View this message in context: http://n2.nabble.com/How-to-set-a-tangosol-cache-per-entity-tp210372p3876934.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.