You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@openjpa.apache.org by Fr...@capgroup.com on 2008/04/03 02:21:30 UTC
Re: How to write a L2 cache plug in that honors the DataCache annotation?
Hi Pinaki -
Thank you for the sample plug in. It was very helpful and really help me
get started in putting together a Tangosol plug in for kodo/openjpa.
Right now we are only testing the plug in with openjpa - I will try to use
it with kodo when the plug in will be finished.
I spent quite some time going through the openjpa code. In particular
everything that has to do with store manager - broker - datacache -
configuration - etc...
I am writting a plug in for Tangosol that has to be as efficient as I can
make it - our focus here is code quality and performance - so this plug in
as to be clean and efficient. So far it is going pretty well - basically it
is working but not fully tested. But I have a few questions to make sure I
did it right.
I noticed that the store manager (DataCacheStoreManager) in openjpa handles
the multiple cache concept very well (nice to see some good code/design -
congratulation to whomever wrote it). So I am writing a DataCache plugin
that will leverage the good work done in the StoreManager - that is to say
allow the store manager to call the right specific cache from the
DataCacheManager since it already looked up the proper cache name and in
fact was coded to do this well - this code is in DataCacheStoreManager.
updateCaches() method (again that is great code and it is exactly what I
was hoping it will do).
Your sample was leveraging the system wide datacache and do the lookup
itself to find out which cache to use. So basically this was duplicating
quite some work done already by the store manager and was not very
efficient.
Anyway - the new plugin has a new DataCacheManager - a new DataCache - a
new SystemDataCache which basically is what you sent me and a new
QueryCache.
Here are my questions:
Q: difference between datacache and systemDataCache?
So I created a DataCache manager that manages multiple datacaches - one for
each named cache - plus one systemWideDataCache and one
systemWideQueryCache.
I kept around the concept of the systemWideDataCache for now - becuase I
can see the broker gets a reference to it. But I do not see it being used.
Except by OpenJPAEntitymanagerFactory.getStoreCache().
What is the exact concept behind the system wide datacache? Why does the
broker gets a reference to it right when it is created?
Is it just a simplified way to get a proxy to all the caches in case
someone needs it and does not know which cache exactly they should get? It
seems to just be fullfiling this role.
Q: QueryCache scope?
As for the QueryCache - my undertsanding is that this cache must be shared
across multiple JVMs because otherwise (when data gets removed from the L2
cache for example) the QueryCache could return wrong information. Is this
correct?
Code comment:
The DataCacheStoreManager.transformToVersionSafePCDatas() line 261. This
method should call either cache.containsAll() or cache.getAll(). The
current implementation is not efficient becuase it makes one call to the
cache for each element in the collection.
The AbstractDataCache.putAllInternal() method suffers from the same issues.
This is OK becuase I will overwrite this in the new plugin but I would
think someone from the openjpa developer would want to fix that in there
too. That way the base class could actually be used as a base class to
write L2 cache plugins.
There should be inside the openjpa.datacache namespace an AbstractDataCache
class that has nothing to do with RemoteEvent and another one extending it
that will also include the RemoteEvent methods. That way it will be easier
to implement various L2 cache plugin on top of a clean class.
OpenJPaID - The openjpaId class has a method to output a nice format
key.toString() which is type+objectId. This method has been overwritten by
children classes. Why is this? The overwritten method only outputs the
objectId and therefore is not convenient when looking into the cache. Can
this be reversed? Even the logging output is not useful with only the
objectId (think about saving multiple object from a graph with same int
value for PK - in such a case the output does not say which class was saved
properly or not. The alternative is to use a nice String as the key for the
cache but this is not efficient compare to using the hashcode of OpenJpaId.
The EntitymanagerFactoryImpl.getStoreCache(String cacheName) . This method
calls DataCacheManage.getDataCache(cacheName).
I think it should call DataCachemanager.getDataCache(cacheName, true) to
make sure if the cache does not exist it will get created. Unless of course
this is the behavior specified by JPA spec. This is important becuase
otherwise we get StoreCacheImpl reference with a DelegatingDataCache
reference with a null cache reference inside. This generates a lot of bugs
becuase it is not good practices to instanciate all the namedCache at
startup since noone can know upfront whcih one will actually be used. The
current DataCacheManagerImpl does not suffer from this issue becuase there
is only one dataCache. But based on a DataCacheManager implementation with
multiple named cache - which is what I am implementing now - then this does
not work anymore.
Moreover, I cannot chnage the behavior of getDataCache(String name) becuase
the interface says:
/**
* Return the named data cache, or null if it does not exist.
*/
public DataCache getDataCache(String name);
Frederic.
PS: Is openjpa interested in incorporating this Tangosol plugin code in the
product when I am finished with it? It would better for us if we dont have
to maintain it in the future. I would think openjpa would like to start
including plugins for various L2 cache product and Coherence would be a
good start.
Moreover, this plugin is faster than the one that use to be in kodo -
becuase I am removing all the issues that were in there.
Pinaki Poddar
<ppoddar@apache.o
rg> To
users@openjpa.apache.org
cc
03/29/2008 11:11
PM Subject
Re: How to write a L2 cache plug in
that honors the DataCache
Please respond to annotation?
users@openjpa.apa
che.org
Hi,
The attached zip file contains the implementation of a OpenJPA DataCache
plug-in for Coherence Distributed Caching Product. The plug-in honors the
'name' property value of @DataCache annotation in a persistent class X to
cache instances of X to a corresponding named partition in Coherence. The
plug-in
is activated by the following configuration in META-INF/persistence.xml
<property name="openjpa.DataCacheManager" value="coherence"/>
The ProductDerivation magic wires up other appropriate plug-ins.
To use it, add coherence-openjpa.jar to your classpath.
This plug-in is somewhat differs in spirit from its counterpart
TangosolCache (available in Kodo) in a sense that this one leaves *every*
configuration of the cache to Coherence including how changes are
communicated in a distributed environment i.e. OpenJPA effectively
relinquishes its responsibility to broadcast changes via
RemoteCommitProvider to other, possibly, remote L2 caches. This plug-in
also
recognizes that the instances of different classes may be cached to
different Coherence partitions (Kodo version put all of them in one single
named partition).
The source code is provided (without any warranty or copyright). Usage is
exemplified via JUnit Test cases and a simple persistence.xml
configuration.
The build script requires to be edited for your local environment.
Good night & Good luck --
Pinaki
http://www.nabble.com/file/p16378911/coherence-openjpa.zip
coherence-openjpa.zip
--
View this message in context:
http://www.nabble.com/How-to-set-a-tangosol-cache-per-entity--tp16336199p16378911.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: How to write a L2 cache plug in that honors the DataCache
annotation?
Posted by Kevin Sutter <kw...@gmail.com>.
FYI... Another example of a L2 Cache plugin for OpenJPA was done by
EhCache...
http://openjpa.apache.org/2009/07/28/ehcache-plugin-available-for-openjpa.html
http://ehcache.org/
http://ehcache.org/documentation/openjpa_provider.html
http://ehcache.sourceforge.net/download.html
On Thu, Oct 8, 2009 at 1:52 PM, Drewz <an...@elasticpath.com> wrote:
>
> Frederic,
>
> How did your tangosol implementation go? I'm doing similar work, and thus
> would be interested to see if this plugin was contributed back to OpenJPA.
> We would definitely like to expand on any work already done on this.
>
> Thanks,
>
> Drew
>
>
> Frederic_Bellier wrote:
> >
> > Hi Pinaki -
> >
> > Thank you for the sample plug in. It was very helpful and really help me
> > get started in putting together a Tangosol plug in for kodo/openjpa.
> > Right now we are only testing the plug in with openjpa - I will try to
> use
> > it with kodo when the plug in will be finished.
> >
> > I spent quite some time going through the openjpa code. In particular
> > everything that has to do with store manager - broker - datacache -
> > configuration - etc...
> > I am writting a plug in for Tangosol that has to be as efficient as I can
> > make it - our focus here is code quality and performance - so this plug
> in
> > as to be clean and efficient. So far it is going pretty well - basically
> > it
> > is working but not fully tested. But I have a few questions to make sure
> I
> > did it right.
> >
> > I noticed that the store manager (DataCacheStoreManager) in openjpa
> > handles
> > the multiple cache concept very well (nice to see some good code/design -
> > congratulation to whomever wrote it). So I am writing a DataCache plugin
> > that will leverage the good work done in the StoreManager - that is to
> say
> > allow the store manager to call the right specific cache from the
> > DataCacheManager since it already looked up the proper cache name and in
> > fact was coded to do this well - this code is in DataCacheStoreManager.
> > updateCaches() method (again that is great code and it is exactly what I
> > was hoping it will do).
> > Your sample was leveraging the system wide datacache and do the lookup
> > itself to find out which cache to use. So basically this was duplicating
> > quite some work done already by the store manager and was not very
> > efficient.
> > Anyway - the new plugin has a new DataCacheManager - a new DataCache - a
> > new SystemDataCache which basically is what you sent me and a new
> > QueryCache.
> >
> > Here are my questions:
> >
> > Q: difference between datacache and systemDataCache?
> > So I created a DataCache manager that manages multiple datacaches - one
> > for
> > each named cache - plus one systemWideDataCache and one
> > systemWideQueryCache.
> > I kept around the concept of the systemWideDataCache for now - becuase I
> > can see the broker gets a reference to it. But I do not see it being
> used.
> > Except by OpenJPAEntitymanagerFactory.getStoreCache().
> > What is the exact concept behind the system wide datacache? Why does the
> > broker gets a reference to it right when it is created?
> > Is it just a simplified way to get a proxy to all the caches in case
> > someone needs it and does not know which cache exactly they should get?
> It
> > seems to just be fullfiling this role.
> >
> > Q: QueryCache scope?
> > As for the QueryCache - my undertsanding is that this cache must be
> shared
> > across multiple JVMs because otherwise (when data gets removed from the
> L2
> > cache for example) the QueryCache could return wrong information. Is this
> > correct?
> >
> > Code comment:
> > The DataCacheStoreManager.transformToVersionSafePCDatas() line 261. This
> > method should call either cache.containsAll() or cache.getAll(). The
> > current implementation is not efficient becuase it makes one call to the
> > cache for each element in the collection.
> > The AbstractDataCache.putAllInternal() method suffers from the same
> > issues.
> > This is OK becuase I will overwrite this in the new plugin but I would
> > think someone from the openjpa developer would want to fix that in there
> > too. That way the base class could actually be used as a base class to
> > write L2 cache plugins.
> >
> > There should be inside the openjpa.datacache namespace an
> > AbstractDataCache
> > class that has nothing to do with RemoteEvent and another one extending
> it
> > that will also include the RemoteEvent methods. That way it will be
> easier
> > to implement various L2 cache plugin on top of a clean class.
> >
> > OpenJPaID - The openjpaId class has a method to output a nice format
> > key.toString() which is type+objectId. This method has been overwritten
> by
> > children classes. Why is this? The overwritten method only outputs the
> > objectId and therefore is not convenient when looking into the cache. Can
> > this be reversed? Even the logging output is not useful with only the
> > objectId (think about saving multiple object from a graph with same int
> > value for PK - in such a case the output does not say which class was
> > saved
> > properly or not. The alternative is to use a nice String as the key for
> > the
> > cache but this is not efficient compare to using the hashcode of
> > OpenJpaId.
> >
> > The EntitymanagerFactoryImpl.getStoreCache(String cacheName) . This
> method
> > calls DataCacheManage.getDataCache(cacheName).
> > I think it should call DataCachemanager.getDataCache(cacheName, true) to
> > make sure if the cache does not exist it will get created. Unless of
> > course
> > this is the behavior specified by JPA spec. This is important becuase
> > otherwise we get StoreCacheImpl reference with a DelegatingDataCache
> > reference with a null cache reference inside. This generates a lot of
> bugs
> > becuase it is not good practices to instanciate all the namedCache at
> > startup since noone can know upfront whcih one will actually be used. The
> > current DataCacheManagerImpl does not suffer from this issue becuase
> there
> > is only one dataCache. But based on a DataCacheManager implementation
> with
> > multiple named cache - which is what I am implementing now - then this
> > does
> > not work anymore.
> > Moreover, I cannot chnage the behavior of getDataCache(String name)
> > becuase
> > the interface says:
> > /**
> > * Return the named data cache, or null if it does not exist.
> > */
> > public DataCache getDataCache(String name);
> >
> >
> > Frederic.
> >
> > PS: Is openjpa interested in incorporating this Tangosol plugin code in
> > the
> > product when I am finished with it? It would better for us if we dont
> have
> > to maintain it in the future. I would think openjpa would like to start
> > including plugins for various L2 cache product and Coherence would be a
> > good start.
> > Moreover, this plugin is faster than the one that use to be in kodo -
> > becuase I am removing all the issues that were in there.
> >
> >
> >
> >
> >
> > Pinaki Poddar
> > <ppoddar@apache.o
> > rg>
> To
> > users@openjpa.apache.org
> >
> cc
> > 03/29/2008 11:11
> > PM
> Subject
> > Re: How to write a L2 cache plug
> in
> > that honors the DataCache
> > Please respond to annotation?
> > users@openjpa.apa
> > che.org
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Hi,
> > The attached zip file contains the implementation of a OpenJPA
> DataCache
> > plug-in for Coherence Distributed Caching Product. The plug-in honors the
> > 'name' property value of @DataCache annotation in a persistent class X to
> > cache instances of X to a corresponding named partition in Coherence. The
> > plug-in
> > is activated by the following configuration in META-INF/persistence.xml
> >
> > <property name="openjpa.DataCacheManager" value="coherence"/>
> >
> > The ProductDerivation magic wires up other appropriate plug-ins.
> >
> > To use it, add coherence-openjpa.jar to your classpath.
> >
> > This plug-in is somewhat differs in spirit from its counterpart
> > TangosolCache (available in Kodo) in a sense that this one leaves *every*
> > configuration of the cache to Coherence including how changes are
> > communicated in a distributed environment i.e. OpenJPA effectively
> > relinquishes its responsibility to broadcast changes via
> > RemoteCommitProvider to other, possibly, remote L2 caches. This plug-in
> > also
> > recognizes that the instances of different classes may be cached to
> > different Coherence partitions (Kodo version put all of them in one
> single
> > named partition).
> >
> > The source code is provided (without any warranty or copyright). Usage is
> > exemplified via JUnit Test cases and a simple persistence.xml
> > configuration.
> >
> > The build script requires to be edited for your local environment.
> >
> > Good night & Good luck --
> >
> > Pinaki
> > http://www.nabble.com/file/p16378911/coherence-openjpa.zip
> > coherence-openjpa.zip
> >
> >
> > --
> > View this message in context:
> >
> http://www.nabble.com/How-to-set-a-tangosol-cache-per-entity--tp16336199p16378911.html
> >
> > Sent from the OpenJPA Users mailing list archive at Nabble.com.
> >
> >
> >
> >
> >
>
> --
> View this message in context:
> http://n2.nabble.com/How-to-set-a-tangosol-cache-per-entity-tp210372p3790079.html
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
Re: How to write a L2 cache plug in that honors the DataCache
annotation?
Posted by Drewz <an...@elasticpath.com>.
Thanks Kevin and Pinaki for the responses. Plugin seems to be working with
any major problems so far, performance results are looking good.
If we make any changes or improvements, I'll try to follow up and post them
here.
Cheers,
Drew
fyi...
http://ppoddar.blogspot.com/
Drewz wrote:
>
> Frederic,
>
> How did your tangosol implementation go? I'm doing similar work, and thus
> would be interested to see if this plugin was contributed back to OpenJPA.
> We would definitely like to expand on any work already done on this.
>
> Thanks,
>
> Drew
>
>
>
--
View this message in context: http://n2.nabble.com/How-to-set-a-tangosol-cache-per-entity-tp210372p3876968.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: How to write a L2 cache plug in that honors the DataCache
annotation?
Posted by Pinaki Poddar <pp...@apache.org>.
fyi...
http://ppoddar.blogspot.com/
Frederic,
How did your tangosol implementation go? I'm doing similar work, and thus
would be interested to see if this plugin was contributed back to OpenJPA.
We would definitely like to expand on any work already done on this.
Thanks,
Drew
-----
Pinaki
--
View this message in context: http://n2.nabble.com/How-to-set-a-tangosol-cache-per-entity-tp210372p3876934.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: How to write a L2 cache plug in that honors the DataCache
annotation?
Posted by Drewz <an...@elasticpath.com>.
Frederic,
How did your tangosol implementation go? I'm doing similar work, and thus
would be interested to see if this plugin was contributed back to OpenJPA.
We would definitely like to expand on any work already done on this.
Thanks,
Drew
Frederic_Bellier wrote:
>
> Hi Pinaki -
>
> Thank you for the sample plug in. It was very helpful and really help me
> get started in putting together a Tangosol plug in for kodo/openjpa.
> Right now we are only testing the plug in with openjpa - I will try to use
> it with kodo when the plug in will be finished.
>
> I spent quite some time going through the openjpa code. In particular
> everything that has to do with store manager - broker - datacache -
> configuration - etc...
> I am writting a plug in for Tangosol that has to be as efficient as I can
> make it - our focus here is code quality and performance - so this plug in
> as to be clean and efficient. So far it is going pretty well - basically
> it
> is working but not fully tested. But I have a few questions to make sure I
> did it right.
>
> I noticed that the store manager (DataCacheStoreManager) in openjpa
> handles
> the multiple cache concept very well (nice to see some good code/design -
> congratulation to whomever wrote it). So I am writing a DataCache plugin
> that will leverage the good work done in the StoreManager - that is to say
> allow the store manager to call the right specific cache from the
> DataCacheManager since it already looked up the proper cache name and in
> fact was coded to do this well - this code is in DataCacheStoreManager.
> updateCaches() method (again that is great code and it is exactly what I
> was hoping it will do).
> Your sample was leveraging the system wide datacache and do the lookup
> itself to find out which cache to use. So basically this was duplicating
> quite some work done already by the store manager and was not very
> efficient.
> Anyway - the new plugin has a new DataCacheManager - a new DataCache - a
> new SystemDataCache which basically is what you sent me and a new
> QueryCache.
>
> Here are my questions:
>
> Q: difference between datacache and systemDataCache?
> So I created a DataCache manager that manages multiple datacaches - one
> for
> each named cache - plus one systemWideDataCache and one
> systemWideQueryCache.
> I kept around the concept of the systemWideDataCache for now - becuase I
> can see the broker gets a reference to it. But I do not see it being used.
> Except by OpenJPAEntitymanagerFactory.getStoreCache().
> What is the exact concept behind the system wide datacache? Why does the
> broker gets a reference to it right when it is created?
> Is it just a simplified way to get a proxy to all the caches in case
> someone needs it and does not know which cache exactly they should get? It
> seems to just be fullfiling this role.
>
> Q: QueryCache scope?
> As for the QueryCache - my undertsanding is that this cache must be shared
> across multiple JVMs because otherwise (when data gets removed from the L2
> cache for example) the QueryCache could return wrong information. Is this
> correct?
>
> Code comment:
> The DataCacheStoreManager.transformToVersionSafePCDatas() line 261. This
> method should call either cache.containsAll() or cache.getAll(). The
> current implementation is not efficient becuase it makes one call to the
> cache for each element in the collection.
> The AbstractDataCache.putAllInternal() method suffers from the same
> issues.
> This is OK becuase I will overwrite this in the new plugin but I would
> think someone from the openjpa developer would want to fix that in there
> too. That way the base class could actually be used as a base class to
> write L2 cache plugins.
>
> There should be inside the openjpa.datacache namespace an
> AbstractDataCache
> class that has nothing to do with RemoteEvent and another one extending it
> that will also include the RemoteEvent methods. That way it will be easier
> to implement various L2 cache plugin on top of a clean class.
>
> OpenJPaID - The openjpaId class has a method to output a nice format
> key.toString() which is type+objectId. This method has been overwritten by
> children classes. Why is this? The overwritten method only outputs the
> objectId and therefore is not convenient when looking into the cache. Can
> this be reversed? Even the logging output is not useful with only the
> objectId (think about saving multiple object from a graph with same int
> value for PK - in such a case the output does not say which class was
> saved
> properly or not. The alternative is to use a nice String as the key for
> the
> cache but this is not efficient compare to using the hashcode of
> OpenJpaId.
>
> The EntitymanagerFactoryImpl.getStoreCache(String cacheName) . This method
> calls DataCacheManage.getDataCache(cacheName).
> I think it should call DataCachemanager.getDataCache(cacheName, true) to
> make sure if the cache does not exist it will get created. Unless of
> course
> this is the behavior specified by JPA spec. This is important becuase
> otherwise we get StoreCacheImpl reference with a DelegatingDataCache
> reference with a null cache reference inside. This generates a lot of bugs
> becuase it is not good practices to instanciate all the namedCache at
> startup since noone can know upfront whcih one will actually be used. The
> current DataCacheManagerImpl does not suffer from this issue becuase there
> is only one dataCache. But based on a DataCacheManager implementation with
> multiple named cache - which is what I am implementing now - then this
> does
> not work anymore.
> Moreover, I cannot chnage the behavior of getDataCache(String name)
> becuase
> the interface says:
> /**
> * Return the named data cache, or null if it does not exist.
> */
> public DataCache getDataCache(String name);
>
>
> Frederic.
>
> PS: Is openjpa interested in incorporating this Tangosol plugin code in
> the
> product when I am finished with it? It would better for us if we dont have
> to maintain it in the future. I would think openjpa would like to start
> including plugins for various L2 cache product and Coherence would be a
> good start.
> Moreover, this plugin is faster than the one that use to be in kodo -
> becuase I am removing all the issues that were in there.
>
>
>
>
>
> Pinaki Poddar
> <ppoddar@apache.o
> rg> To
> users@openjpa.apache.org
> cc
> 03/29/2008 11:11
> PM Subject
> Re: How to write a L2 cache plug in
> that honors the DataCache
> Please respond to annotation?
> users@openjpa.apa
> che.org
>
>
>
>
>
>
>
>
>
> Hi,
> The attached zip file contains the implementation of a OpenJPA DataCache
> plug-in for Coherence Distributed Caching Product. The plug-in honors the
> 'name' property value of @DataCache annotation in a persistent class X to
> cache instances of X to a corresponding named partition in Coherence. The
> plug-in
> is activated by the following configuration in META-INF/persistence.xml
>
> <property name="openjpa.DataCacheManager" value="coherence"/>
>
> The ProductDerivation magic wires up other appropriate plug-ins.
>
> To use it, add coherence-openjpa.jar to your classpath.
>
> This plug-in is somewhat differs in spirit from its counterpart
> TangosolCache (available in Kodo) in a sense that this one leaves *every*
> configuration of the cache to Coherence including how changes are
> communicated in a distributed environment i.e. OpenJPA effectively
> relinquishes its responsibility to broadcast changes via
> RemoteCommitProvider to other, possibly, remote L2 caches. This plug-in
> also
> recognizes that the instances of different classes may be cached to
> different Coherence partitions (Kodo version put all of them in one single
> named partition).
>
> The source code is provided (without any warranty or copyright). Usage is
> exemplified via JUnit Test cases and a simple persistence.xml
> configuration.
>
> The build script requires to be edited for your local environment.
>
> Good night & Good luck --
>
> Pinaki
> http://www.nabble.com/file/p16378911/coherence-openjpa.zip
> coherence-openjpa.zip
>
>
> --
> View this message in context:
> http://www.nabble.com/How-to-set-a-tangosol-cache-per-entity--tp16336199p16378911.html
>
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
>
>
>
>
--
View this message in context: http://n2.nabble.com/How-to-set-a-tangosol-cache-per-entity-tp210372p3790079.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.