You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Olivier NOUGUIER <ol...@gmail.com> on 2017/03/28 17:47:23 UTC
ClientCache / PDX best practice
Hi,
In a reactive extension to connect Apache Geode with akka-stream in
alpakka community project [0], I don't know how to use ClientCache in an
efficient way.
Given:
- 2 Pojos: Animal and Person
- 2 Regions for each
Should I use 2 ClientCaches with 2 custom PDXSerializer or only one
ClientCache with a more generic PDXSerializer ?
Another way to ask, if ClientCache is expensive to instanciate.
Thanks in advance.
PS: I've hesitate to post this on the user list...
[0]: https://github.com/cheleb/alpakka/tree/geode/geode
Re: ClientCache / PDX best practice
Posted by Jacob Barrett <jb...@pivotal.io>.
On Tue, Mar 28, 2017 at 11:43 AM John Blum <jb...@pivotal.io> wrote:
> that you use, say, the *Composite* Design Pattern
> <https://en.wikipedia.org/wiki/Composite_pattern> [1] to "compose" a
> collection of PdxSerializer implementations that are specific to each type
> of POJO. This is even preferable over having each POJO implement Geode's
> PdxSerializable interface which affords you similar fine grained control.
>
This needs to be a first class feature of Geode already!
Re: ClientCache / PDX best practice
Posted by Olivier NOUGUIER <ol...@gmail.com>.
Thank you all for the answers !
@Udo to give you an idea of what I is about you can take a look at java
test [0] or even simpler scala [1].
I've made many test, Composite PDX serializer, Custom and even a magical
one generated at compilation time [2] with Shapeless [3] (scala only
feature).
But as you mention, the only one PDXSerializer per ClientCache confused me
a little, a then I was tempted to use a ClientCache/PDXSerializer for each
modelObject / region.
Thanks a lot, the POC works quite well, I tell you more soon hopefully.
Cheers,
Olivier
[0]:
https://github.com/cheleb/alpakka/blob/geode/geode/src/test/java/akka/stream/alpakka/geode/javadsl/GeodeFlowTestCase.java#L73
[1]:
https://github.com/cheleb/alpakka/blob/geode/geode/src/test/scala/akka/stream/alpakka/geode/scaladsl/GeodeSinkSpec.scala#L38
[2]:
https://github.com/cheleb/alpakka/blob/geode/geode/src/main/scala/akka/stream/alpakka/geode/internal/ShapelessPdxSerializer.scala
[3]: https://github.com/milessabin/shapeless
Le mar. 28 mars 2017 à 20:49, John Blum <jb...@pivotal.io> a écrit :
> I should also add that using the *Composite* Design Pattern is preferable
> over a single, "bloated" PdxSerializer implementation (which I see all too
> often in user applications) handling multiple domain object types (<sigh>),
> particularly when the ReflectionBasedAutoSerializer is not, or cannot be
> used.
>
> Food for thought.
>
> On Tue, Mar 28, 2017 at 11:43 AM, John Blum <jb...@pivotal.io> wrote:
>
> > +1 to what Udo stated; additionally..
> >
> > It is also advisable, if you required more fine grained control over the
> > serialization of your POJOs (i.e. necessarily more than the
> > ReflectionBasedAutoSerializer
> > <
> http://geode.apache.org/releases/latest/javadoc/org/apache/geode/pdx/ReflectionBasedAutoSerializer.html
> >
> > [2]), that you use, say, the *Composite* Design Pattern
> > <https://en.wikipedia.org/wiki/Composite_pattern> [1] to "compose" a
> > collection of PdxSerializer implementations that are specific to each
> > type of POJO. This is even preferable over having each POJO implement
> > Geode's PdxSerializable interface which affords you similar fine grained
> > control.
> >
> > It is unfortunate that you cannot register more than 1 PdxSerializer per
> > cache instance for different domain object, but essentially, you can
> > achieve a similar effect using the Composite Design Pattern. Here is 1
> > example...
> >
> > https://github.com/spring-projects/spring-data-gemfire/
> > blob/master/src/test/java/org/springframework/data/gemfire/function/
> > ClientCacheFunctionExecutionWithPdxIntegrationTest.java#L312-L348
> >
> > This "*ComposablePdxSerializer*" is composed of 2 PdxSerializer
> > implementations, 1 for each domain object...
> >
> > // PersonPdxSerializer
> >
> > https://github.com/spring-projects/spring-data-gemfire/
> > blob/master/src/test/java/org/springframework/data/gemfire/function/
> > ClientCacheFunctionExecutionWithPdxIntegrationTest.java#L408-L430
> >
> > // AddressPdxSerializer
> >
> > https://github.com/spring-projects/spring-data-gemfire/
> > blob/master/src/test/java/org/springframework/data/gemfire/function/
> > ClientCacheFunctionExecutionWithPdxIntegrationTest.java#L382-L407
> >
> > And then the "*ComposablePdxSerializer*" (and individual, POJO based
> > PdxSerializer implementations) are configured and registered
> > accordingly...
> >
> > https://github.com/spring-projects/spring-data-gemfire/
> > blob/master/src/test/resources/org/springframework/data/gemfire/function/
> > ClientCacheFunctionExecutionWithPdxIntegrationTest-server-
> > context.xml#L28-L42
> >
> > Of course, this will all shown in *Spring Data Geode* (XML)
> > configuration, but a similar configuration can be achieved using the
> > equivalent Geode config where *Spring* (*Data Geode*) is not used.
> >
> > -j
> >
> >
> >
> > [1] https://en.wikipedia.org/wiki/Composite_pattern
> > [2]
> http://geode.apache.org/releases/latest/javadoc/org/apache/geode/pdx/
> > ReflectionBasedAutoSerializer.html
> >
> >
> > On Tue, Mar 28, 2017 at 11:20 AM, Udo Kohlmeyer <uk...@pivotal.io>
> > wrote:
> >
> >> Hi there Olivier.
> >>
> >> It is hard to say what you are trying to do... but to answer your
> >> question... The typical approach is to start a clientcache which spans
> >> lifetime of the client application. It is not something that I would
> want
> >> to start/stop as the mood strikes.
> >>
> >> As for generic PdxSerializer vs Pojo specific serializer, I would error
> >> on the side of Pojo specific serializer, as the serialization logic is
> >> defined on the pojo. BUT if you don't want to "taint" your pojo with
> >> serialization code, you could use a generic PDXSerializer. In this case
> you
> >> can write your own or use the provided ReflectionBasedAutoSerializer.
> [0]
> >>
> >> So to *hopefully* answer your question. 1 clientcache, with either 1
> >> generic PdxSerializer (custom or ReflectionAutoSerializer) or 2 pojo
> >> specific PdxSerializable implementations.
> >>
> >> --Udo
> >>
> >> [0] - https://geode.apache.org/docs/guide/developing/data_serializ
> >> ation/gemfire_pdx_serialization.html
> >>
> >>
> >>
> >>
> >> On 3/28/17 10:47, Olivier NOUGUIER wrote:
> >>
> >>> Hi,
> >>> In a reactive extension to connect Apache Geode with akka-stream in
> >>> alpakka community project [0], I don't know how to use ClientCache in
> an
> >>> efficient way.
> >>>
> >>> Given:
> >>>
> >>> - 2 Pojos: Animal and Person
> >>> - 2 Regions for each
> >>>
> >>> Should I use 2 ClientCaches with 2 custom PDXSerializer or only one
> >>> ClientCache with a more generic PDXSerializer ?
> >>>
> >>> Another way to ask, if ClientCache is expensive to instanciate.
> >>>
> >>> Thanks in advance.
> >>>
> >>> PS: I've hesitate to post this on the user list...
> >>>
> >>>
> >>> [0]: https://github.com/cheleb/alpakka/tree/geode/geode
> >>>
> >>>
> >>
> >
> >
> > --
> > -John
> > john.blum10101 (skype)
> >
>
>
>
> --
> -John
> john.blum10101 (skype)
>
Re: ClientCache / PDX best practice
Posted by John Blum <jb...@pivotal.io>.
I should also add that using the *Composite* Design Pattern is preferable
over a single, "bloated" PdxSerializer implementation (which I see all too
often in user applications) handling multiple domain object types (<sigh>),
particularly when the ReflectionBasedAutoSerializer is not, or cannot be
used.
Food for thought.
On Tue, Mar 28, 2017 at 11:43 AM, John Blum <jb...@pivotal.io> wrote:
> +1 to what Udo stated; additionally..
>
> It is also advisable, if you required more fine grained control over the
> serialization of your POJOs (i.e. necessarily more than the
> ReflectionBasedAutoSerializer
> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/pdx/ReflectionBasedAutoSerializer.html>
> [2]), that you use, say, the *Composite* Design Pattern
> <https://en.wikipedia.org/wiki/Composite_pattern> [1] to "compose" a
> collection of PdxSerializer implementations that are specific to each
> type of POJO. This is even preferable over having each POJO implement
> Geode's PdxSerializable interface which affords you similar fine grained
> control.
>
> It is unfortunate that you cannot register more than 1 PdxSerializer per
> cache instance for different domain object, but essentially, you can
> achieve a similar effect using the Composite Design Pattern. Here is 1
> example...
>
> https://github.com/spring-projects/spring-data-gemfire/
> blob/master/src/test/java/org/springframework/data/gemfire/function/
> ClientCacheFunctionExecutionWithPdxIntegrationTest.java#L312-L348
>
> This "*ComposablePdxSerializer*" is composed of 2 PdxSerializer
> implementations, 1 for each domain object...
>
> // PersonPdxSerializer
>
> https://github.com/spring-projects/spring-data-gemfire/
> blob/master/src/test/java/org/springframework/data/gemfire/function/
> ClientCacheFunctionExecutionWithPdxIntegrationTest.java#L408-L430
>
> // AddressPdxSerializer
>
> https://github.com/spring-projects/spring-data-gemfire/
> blob/master/src/test/java/org/springframework/data/gemfire/function/
> ClientCacheFunctionExecutionWithPdxIntegrationTest.java#L382-L407
>
> And then the "*ComposablePdxSerializer*" (and individual, POJO based
> PdxSerializer implementations) are configured and registered
> accordingly...
>
> https://github.com/spring-projects/spring-data-gemfire/
> blob/master/src/test/resources/org/springframework/data/gemfire/function/
> ClientCacheFunctionExecutionWithPdxIntegrationTest-server-
> context.xml#L28-L42
>
> Of course, this will all shown in *Spring Data Geode* (XML)
> configuration, but a similar configuration can be achieved using the
> equivalent Geode config where *Spring* (*Data Geode*) is not used.
>
> -j
>
>
>
> [1] https://en.wikipedia.org/wiki/Composite_pattern
> [2] http://geode.apache.org/releases/latest/javadoc/org/apache/geode/pdx/
> ReflectionBasedAutoSerializer.html
>
>
> On Tue, Mar 28, 2017 at 11:20 AM, Udo Kohlmeyer <uk...@pivotal.io>
> wrote:
>
>> Hi there Olivier.
>>
>> It is hard to say what you are trying to do... but to answer your
>> question... The typical approach is to start a clientcache which spans
>> lifetime of the client application. It is not something that I would want
>> to start/stop as the mood strikes.
>>
>> As for generic PdxSerializer vs Pojo specific serializer, I would error
>> on the side of Pojo specific serializer, as the serialization logic is
>> defined on the pojo. BUT if you don't want to "taint" your pojo with
>> serialization code, you could use a generic PDXSerializer. In this case you
>> can write your own or use the provided ReflectionBasedAutoSerializer. [0]
>>
>> So to *hopefully* answer your question. 1 clientcache, with either 1
>> generic PdxSerializer (custom or ReflectionAutoSerializer) or 2 pojo
>> specific PdxSerializable implementations.
>>
>> --Udo
>>
>> [0] - https://geode.apache.org/docs/guide/developing/data_serializ
>> ation/gemfire_pdx_serialization.html
>>
>>
>>
>>
>> On 3/28/17 10:47, Olivier NOUGUIER wrote:
>>
>>> Hi,
>>> In a reactive extension to connect Apache Geode with akka-stream in
>>> alpakka community project [0], I don't know how to use ClientCache in an
>>> efficient way.
>>>
>>> Given:
>>>
>>> - 2 Pojos: Animal and Person
>>> - 2 Regions for each
>>>
>>> Should I use 2 ClientCaches with 2 custom PDXSerializer or only one
>>> ClientCache with a more generic PDXSerializer ?
>>>
>>> Another way to ask, if ClientCache is expensive to instanciate.
>>>
>>> Thanks in advance.
>>>
>>> PS: I've hesitate to post this on the user list...
>>>
>>>
>>> [0]: https://github.com/cheleb/alpakka/tree/geode/geode
>>>
>>>
>>
>
>
> --
> -John
> john.blum10101 (skype)
>
--
-John
john.blum10101 (skype)
Re: ClientCache / PDX best practice
Posted by John Blum <jb...@pivotal.io>.
+1 to what Udo stated; additionally..
It is also advisable, if you required more fine grained control over the
serialization of your POJOs (i.e. necessarily more than the
ReflectionBasedAutoSerializer
<http://geode.apache.org/releases/latest/javadoc/org/apache/geode/pdx/ReflectionBasedAutoSerializer.html>
[2]),
that you use, say, the *Composite* Design Pattern
<https://en.wikipedia.org/wiki/Composite_pattern> [1] to "compose" a
collection of PdxSerializer implementations that are specific to each type
of POJO. This is even preferable over having each POJO implement Geode's
PdxSerializable interface which affords you similar fine grained control.
It is unfortunate that you cannot register more than 1 PdxSerializer per
cache instance for different domain object, but essentially, you can
achieve a similar effect using the Composite Design Pattern. Here is 1
example...
https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/function/ClientCacheFunctionExecutionWithPdxIntegrationTest.java#L312-L348
This "*ComposablePdxSerializer*" is composed of 2 PdxSerializer
implementations, 1 for each domain object...
// PersonPdxSerializer
https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/function/ClientCacheFunctionExecutionWithPdxIntegrationTest.java#L408-L430
// AddressPdxSerializer
https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/function/ClientCacheFunctionExecutionWithPdxIntegrationTest.java#L382-L407
And then the "*ComposablePdxSerializer*" (and individual, POJO based
PdxSerializer implementations) are configured and registered accordingly...
https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/resources/org/springframework/data/gemfire/function/ClientCacheFunctionExecutionWithPdxIntegrationTest-server-context.xml#L28-L42
Of course, this will all shown in *Spring Data Geode* (XML) configuration,
but a similar configuration can be achieved using the equivalent Geode
config where *Spring* (*Data Geode*) is not used.
-j
[1] https://en.wikipedia.org/wiki/Composite_pattern
[2]
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/pdx/ReflectionBasedAutoSerializer.html
On Tue, Mar 28, 2017 at 11:20 AM, Udo Kohlmeyer <uk...@pivotal.io>
wrote:
> Hi there Olivier.
>
> It is hard to say what you are trying to do... but to answer your
> question... The typical approach is to start a clientcache which spans
> lifetime of the client application. It is not something that I would want
> to start/stop as the mood strikes.
>
> As for generic PdxSerializer vs Pojo specific serializer, I would error on
> the side of Pojo specific serializer, as the serialization logic is defined
> on the pojo. BUT if you don't want to "taint" your pojo with serialization
> code, you could use a generic PDXSerializer. In this case you can write
> your own or use the provided ReflectionBasedAutoSerializer. [0]
>
> So to *hopefully* answer your question. 1 clientcache, with either 1
> generic PdxSerializer (custom or ReflectionAutoSerializer) or 2 pojo
> specific PdxSerializable implementations.
>
> --Udo
>
> [0] - https://geode.apache.org/docs/guide/developing/data_serializ
> ation/gemfire_pdx_serialization.html
>
>
>
>
> On 3/28/17 10:47, Olivier NOUGUIER wrote:
>
>> Hi,
>> In a reactive extension to connect Apache Geode with akka-stream in
>> alpakka community project [0], I don't know how to use ClientCache in an
>> efficient way.
>>
>> Given:
>>
>> - 2 Pojos: Animal and Person
>> - 2 Regions for each
>>
>> Should I use 2 ClientCaches with 2 custom PDXSerializer or only one
>> ClientCache with a more generic PDXSerializer ?
>>
>> Another way to ask, if ClientCache is expensive to instanciate.
>>
>> Thanks in advance.
>>
>> PS: I've hesitate to post this on the user list...
>>
>>
>> [0]: https://github.com/cheleb/alpakka/tree/geode/geode
>>
>>
>
--
-John
john.blum10101 (skype)
Re: ClientCache / PDX best practice
Posted by Udo Kohlmeyer <uk...@pivotal.io>.
Hi there Olivier.
It is hard to say what you are trying to do... but to answer your
question... The typical approach is to start a clientcache which spans
lifetime of the client application. It is not something that I would
want to start/stop as the mood strikes.
As for generic PdxSerializer vs Pojo specific serializer, I would error
on the side of Pojo specific serializer, as the serialization logic is
defined on the pojo. BUT if you don't want to "taint" your pojo with
serialization code, you could use a generic PDXSerializer. In this case
you can write your own or use the provided
ReflectionBasedAutoSerializer. [0]
So to *hopefully* answer your question. 1 clientcache, with either 1
generic PdxSerializer (custom or ReflectionAutoSerializer) or 2 pojo
specific PdxSerializable implementations.
--Udo
[0] -
https://geode.apache.org/docs/guide/developing/data_serialization/gemfire_pdx_serialization.html
On 3/28/17 10:47, Olivier NOUGUIER wrote:
> Hi,
> In a reactive extension to connect Apache Geode with akka-stream in
> alpakka community project [0], I don't know how to use ClientCache in an
> efficient way.
>
> Given:
>
> - 2 Pojos: Animal and Person
> - 2 Regions for each
>
> Should I use 2 ClientCaches with 2 custom PDXSerializer or only one
> ClientCache with a more generic PDXSerializer ?
>
> Another way to ask, if ClientCache is expensive to instanciate.
>
> Thanks in advance.
>
> PS: I've hesitate to post this on the user list...
>
>
> [0]: https://github.com/cheleb/alpakka/tree/geode/geode
>
Re: ClientCache / PDX best practice
Posted by Michael Stolz <ms...@pivotal.io>.
I think you may be confusing ClientCache with Region.
The Region is the abstraction similar to Named Cache in several other
projects or Table in a database.
--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: +1-631-835-4771
On Tue, Mar 28, 2017 at 10:47 AM, Olivier NOUGUIER <
olivier.nouguier@gmail.com> wrote:
> Hi,
> In a reactive extension to connect Apache Geode with akka-stream in
> alpakka community project [0], I don't know how to use ClientCache in an
> efficient way.
>
> Given:
>
> - 2 Pojos: Animal and Person
> - 2 Regions for each
>
> Should I use 2 ClientCaches with 2 custom PDXSerializer or only one
> ClientCache with a more generic PDXSerializer ?
>
> Another way to ask, if ClientCache is expensive to instanciate.
>
> Thanks in advance.
>
> PS: I've hesitate to post this on the user list...
>
>
> [0]: https://github.com/cheleb/alpakka/tree/geode/geode
>